Exploratory Data Analysis (EDA) Dashboard

Complete Step-by-Step EDA Guide

A comprehensive, hands-on approach to Exploratory Data Analysis using a customer loan application dataset

🩺 The Medical Analogy

Think of EDA as the "doctor's check-up" for your data. Before any treatment (modeling), you need a thorough diagnosis to understand what you're working with, identify any problems, and determine the right course of action.

Our Patient: A dataset of 10,000 customer loan applications.
Our Goal: Understand the factors that influence loan default (loan_status: 0 = Paid, 1 = Defaulted).

The Mindset & Tools

Before we start, remember: EDA is iterative and creative. You will constantly ask "Why?" and follow hunches.

Tools: Python (Pandas, NumPy, Matplotlib, Seaborn, SciPy) or R (tidyverse, ggplot2) are standard.

# Typical import statements for EDA in Python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

Data Acquisition & Profiling (The First Glance)

Goal: Understand the basic structure and integrity of your dataset.

What to Do:

Load the Data: df = pd.read_csv('loan_data.csv')
Peek at the Data:
- df.head() - First 5 rows
- df.tail() - Last 5 rows (check if data was appended)
- df.sample(5) - Random 5 rows
Basic Information: df.info()
Shows number of rows, columns, column names, data types, and non-null values.
Descriptive Statistics: df.describe(include='all')

Example Output for Loan Data:

df.shape: (10000, 15)
income has 9,800 non-null values (200 missing)
age min=18, max=82, mean=38
loan_amount 75th percentile is $15,000 but max is $500,000 (potential outlier!)

Check: ✅ Data loaded? ✅ Basic structure understood? ✅ Missing values and potential outliers identified?

Univariate Analysis (Understanding Individuals)

Goal: Analyze each variable in isolation to understand its distribution, central tendency, and spread.

For Numerical Variables (e.g., age, income, loan_amount):

Visualization: Histograms, Kernel Density Plots (KDE), and Boxplots.

# Create a 1x3 grid of plots
fig, axes = plt.subplots(1, 3, figsize=(15, 5))
df['income'].plot.hist(bins=30, ax=axes[0], title='Histogram of Income')
df['income'].plot.kde(ax=axes[1], title='KDE of Income')
df['income'].plot.box(ax=axes[2], title='Boxplot of Income')
plt.show()

What to Look For:

Shape: Normal, skewed left/right, bimodal?
Central Tendency: Mean vs. Median
Spread: Standard deviation, IQR
Example: income is right-skewed. Most cluster around $50k, but some have very high incomes.

For Categorical Variables (e.g., marital_status, loan_purpose):

Visualization: Bar charts (count of each category).

status_counts = df['loan_status'].value_counts()
plt.figure(figsize=(6,4))
sns.barplot(x=status_counts.index, y=status_counts.values)
plt.title('Count of Loan Status')
plt.show()

What to Look For:

Cardinality: Number of unique categories
Class Imbalance: 8,500 "Paid" vs 1,500 "Defaulted"
Rare Categories: Categories with few samples

Check: ✅ Distribution of each variable understood? ✅ Skewness and outliers noted? ✅ Target variable imbalance identified?

Bivariate/Multivariate Analysis (Understanding Relationships)

Goal: Uncover relationships and interactions between variables, especially with the target (loan_status).

A. Numerical vs. Numerical (e.g., income vs. loan_amount):

# Scatter plot
plt.scatter(df['income'], df['loan_amount'], alpha=0.5)
plt.xlabel('Annual Income')
plt.ylabel('Loan Amount')
plt.show()

# Correlation matrix
corr_matrix = df[['income', 'loan_amount', 'age', 'credit_score']].corr()
sns.heatmap(corr_matrix, annot=True, cmap='coolwarm', center=0)
plt.show()

B. Categorical vs. Categorical (e.g., marital_status vs. loan_status):

crosstab = pd.crosstab(df['marital_status'], df['loan_status'], normalize='index') * 100
crosstab.plot(kind='bar', stacked=True)
plt.title('Loan Status by Marital Status')
plt.ylabel('Percentage (%)')
plt.show()

Insight: 20% of "Divorced" applicants default vs. only 10% of "Married" applicants.

C. Numerical vs. Categorical (e.g., income vs. loan_status):

sns.boxplot(x='loan_status', y='income', data=df)
plt.title('Income Distribution by Loan Status')
plt.show()

Insight: Median income for defaulted loans is lower than for paid loans.

Check: ✅ Key relationships with target identified? ✅ Strong inter-feature correlations noted?

Handling Missing Values

Goal: Develop a strategy for dealing with incomplete data.

Strategies:

1. Delete:

Drop rows if missingness is very low (<5%) and random. Drop columns if mostly missing (>50%) and not critical.

2. Impute (Fill):

Numerical: Mean/Median (use median for skewed data)
Categorical: Mode or create "Missing" category

# Numerical imputation
median_income = df['income'].median()
df['income'].fillna(median_income, inplace=True)

# Categorical imputation
df['marital_status'].fillna('Missing', inplace=True)

3. Advanced:

Use algorithms (K-Nearest Neighbors, Regression) to predict missing values.

Check: ✅ Missing value strategy defined and executed?

Handling Outliers

Goal: Decide if extreme values are errors or natural, and treat them accordingly.

How to Find Them:

Boxplot Method: IQR method (Q1 - 1.5*IQR, Q3 + 1.5*IQR)
Z-score Method: Values with |Z-score| > 3

How to Treat Them:

Keep: If real and important (legitimate large loans)
Cap/Winsorize: Replace with 5th/95th percentile values
Remove: If clear errors (age = 150)

# Capping outliers
p95 = df['loan_amount'].quantile(0.95)
df['loan_amount_capped'] = np.where(df['loan_amount'] > p95, p95, df['loan_amount'])

Check: ✅ Outliers investigated and treated based on domain knowledge?

Addressing Imbalanced Data

Problem: loan_status has 85% "Paid" and 15% "Defaulted". A model that always predicts "Paid" would be 85% accurate but useless.

Solutions:

Resampling:

Oversampling: Create synthetic examples (SMOTE)
Undersampling: Remove majority class examples

Algorithmic:

Use models that handle imbalance (Random Forest, XGBoost)
Adjust parameters: class_weight='balanced', scale_pos_weight

Evaluation:

Never use accuracy as a metric. Use Precision, Recall, F1-Score, and AUC-ROC instead.

Check: ✅ Strategy for handling imbalance chosen for modeling?

Feature Engineering (The "Art" of EDA)

Goal: Create new, more informative features from existing ones.

Examples:

Create debt_to_income_ratio from existing_debt and income
Bin numerical age into categorical groups ['Young', 'Adult', 'Senior']
Extract day_of_week from application_date timestamp

These new features are often more predictive than the raw data.

✅ Final Checklist & Recap (P-U-B-M-O-I-V-F)

Before you leave EDA and move to modeling, ask:

Profiling: Do I know my data's shape, types, and basic stats?

Univariate: Do I know the distribution of every single column?

Bivariate: Do I know the key relationships between features and the target?

Missing Values: Have I dealt with all missing data appropriately?

Outliers: Have I identified and treated outliers?

Imbalance: Have I addressed the imbalance in my target variable?

Visualization: Have I used plots to tell a clear story about the data?

Feature Engineering: Have I created any new, useful features?

By following this guide, you won't just be cleaning data; you'll be building a deep, intuitive understanding of the problem you're trying to solve, which is the true power of EDA.

Exploratory Data Analysis

Data Profiling

Statistical Analysis

Data Cleaning

Visualization

Key EDA Components