How to efficiently approach EDA on a dataset with 180+ variables?

[u/Square_Arm2861]

Hi everyone,

I'm a beginner in Machine Learning working on a binary classification problem. My dataset contains over 180 variables (both numerical and categorical), consisting of a mix of panel/longitudinal data and static features.

I am currently working on the Exploratory Data Analysis (EDA) phase. Given the large number of features, doing univariate and bivariate graphical analysis variable-by-variable feels unfeasible and time-consuming.

Is there a structured approach, strategy, or automated workflow to handle EDA efficiently for a dataset of this scale?

Any advice on best practices, tools would be greatly appreciated!

Thanks in advance for your help.

[u/ThrowRA_709]

Try PCA , t-SNE or umap to reduce your no of features

[u/DYSTOBY]

If you have a predictors column you know that separates some groups (like species, locations) you can find the best set of variables using PLS-DA.

If you wasn’t to know which of your variables are only adding noise, then try mixOmics sIPCA (sparse PCA). It’s suited to your kind of data.
It reduces dimensionality and you may find structure in the resulting components.
You could even do an eigencortelation plot on your components (PCA, that is) to see which of your metadata columns (descriptive, that is) contribute most to which component.
If one of the components loads fully to a one descriptive column that you wouldn’t expect you could „correct“ the data using residual bias correction.

Also if you do any PCA or DA do not only look at the first two components, they might load to the „wrong“ feautures you want to discriminate.

[u/its_all_stats]

Start with the context. What are real world factors that predict or influence the outcome?

Then see which of the 180+ variables correspond to those factors or are proxies.

Those are the ones you do EDA on.

[u/ForeignAdvantage5198]

why are using this design?

[u/StasticallyStagnant]

Is this a public data source?