Learning in Public: Machine Learning Zoomcamp - Week 3
Machine Learning for Probability Prediction
Churn Prediction Project
Predict monthly customer churn for a telecom company using a set of present and past customers.
Classification: Binary
We look at one customer and predict one of two possible outcomes.
1 is the positive class: The customer churned. 0 is the negative class: The customer stayed.
The Dataset
Telco Customer Churn data from Kaggle https://www.kaggle.com/datasets/blastchar/telco-customer-churn
Data Preparation
- Download the data
- Read the data into the environment
- Replace all spaces in data with underscores
- For numbers encoded as strings, force them to be numbers
- Check for null values, and replace them with zeros
- Convert "yes" and "no" in churn column to 1 and 0 respectively
Validation
Our validation framework splits the source data into:
- 60% training data
- 20% validation data
- 20% test data
Exploratory Data Analysis
- Look for missing values
- Make a determination on the distribution of the target variable
- Peruse the numerical and categorial columns
value_count() on the churn column gives us a quick churn rate.
Numerical and Categorical Values
This data in numerical values are just numbers.
The data in categorical valuesare categories, such as "yes" and "no", the categories of which contract the customer is on, etc.
Feature Importance: Churn Rate and Risk Ratio
We want to identify which features affect our target variable.
We do it for categorical variables by comparing the churn rate of each group with the global churn rate, first as a difference and then as a ratio.
Risk Ratio is the group likelihood to churn divided by global likelihood to churn.
Feature Importance: Mutual Information
Mutual Information allows us to compare whole variables with each other. It is a single number per variable that measures how important it is.
Feature Importance: Correlation
Mutual information measures the importance of categorical variables.
For numerical variables we use the correlation coefficient instead. It tells us how strongly each numerical feature is related to churn.
OHE - One-Hot Encoding
Here we encode categorical features as numbers so that they can be used by a machine learning model.
One-hot encoding replaces one categorical column with several binary columns: One per category.
Each new column says whether the row has that category (1) or not (0).
Logistic Regression
Whereas linear regression gave us a price prediction (zero to infinity), logistic regression gives us a binary classification (zero or one).
From the binary classification we can calculate the probability (between zero (0%) and one (100%))
At this stage we make use of the sigmoid function. Maps a broad range of real numbers to a range between 0 and 1. -10,000 maps to 0, 0 maps to 0.5, and 10,000 maps to 1. The upshot is to turn a raw score into a probability.
Training Logistic Regression with SciKit-Learn
Finally we train a logistic regression model on the encoded training data.
Then we apply the model to the validation set, and measure its accuracy objectively.
In practice, "training a model" is calling a training function on the input data.
Final Thoughts
I fell so far behind in the homework for both the ML Zoomcamp and the AI Dev Tools Zoomcamp. But then I thought, "Can I really stop? Even if I get no 'credit', this is too much fun to quit."
Not only am I learning how actual predictions are made with data, I am also learning super cool tools like LATEX. Again, LATEX is one of those things that I thought "was for the cool kids, not for me". Part of the beauty is that when working in Jupyter notebooks on GitHub Codespaces, LATEX and its math markup is available without even asking.