Level: Moderate
Language: Python
Requirements:
- HDP + CDSW
- Spark 2.3
Author: Ian Brooks
Follow: LinkedIn - Ian Brooks PhD
Orginal Fork From: CDSW Demo
This Github repo is designed to be optmized for Cloudera Data Science Workbench (CDSW), but it is not required. The PySpark code can be used with Apache Spark, and the code examples will run with the included dataset.
In this project, there are 5 different supervised classifer models designed for telco customer churn. The first four classsifer models user are: Random Forest, Gradient Boost Tree, Suport Vector Machines, and Multilayer Perception. The most sucessful model is a Stacked Ensemble Model.
- In CSDW, download the project using the git url for here
- Open a new session, and execute the setup.sh file
- In Experiments, run the following scripts
- dsforteko_pyspark.py - vanilla random forest churn model
- gbt_churn_pyspark.py - gradient boost tree churn model with normamlized variables, hyperturning, and crossvalidation
- mlp_churn_pyspark.py - multilayer perceptron churn model with normamlized variables, hyperturning, and crossvalidatio
- rf_churn_pyspark.py - random forest churn model with normamlized variables, hyperturning, and crossvalidation
- svm_churn_pyspark.py - support vection machine churn model with normamlized variables, hyperturning, and crossvalidation
- Once all experiments have completed, the stacked ensemble classifer model be built, run the following script to build the stacked model.
- stacked_churn_pyspark.py - stacked ensemble model trained on the prediction of random forest, gradient boost tree, and support vector machine
- Once the stacked model has been built, the stacked model can be deployed using the following script.
- predict_churn_stackedMLP_pyspark.py