This research paper was written in summer 2020 within the Master's seminar Data Mining in Marketing: Data Driven Customer Analytics with Machine Learning. In partial fulfillment of the requirements of the seminar, I predicted and interpreted cross-selling purchase probabilities using XGBoost and SHAP values in R. Read the full paper here: Boosting Gradient Boosting Interpretability: Predicting and Interpreting Cross-Selling Purchase Probabilities of a Large German Savings Bank.
The availability of new data sources and marketing analytics enables companies to create more value for their customers, leverage customer experiences, and to increase customer loyalty. New metrics and more powerful analytical methods are needed to be more efficient and effective in data-driven marketing (Wedel & Kannan 2016). Besides descriptive metrics, machine learning methods have become popular due to their high predictive performance. A disadvantage is, however, that the interpretability of complex machine learning models is questionable. Therefore, eliminating the tradeoff between a model’s accuracy and a model’s interpretability has gained attention from many researchers (Ribeiro et al. 2016, Lundberg & Lee 2017, Chen et al. 2018, Lipton 2018).
In this paper, I use a data set from a large German savings bank to predict cross-selling purchase probabilities and decisions in the customer base. The goals of this paper are (1) to accurately predict whether an already existing customer will open a checking account and (2) to explore which effect the features have on the prediction to enhance the interpretability of the model. The paper leverages one of the leading gradient boosting algorithms, namely XGBoost, to reach these goals. It has been used with great success in many machine learning and data mining challenges (Chen & Guestrin 2016). Since boosted trees lack interpretability (Friedman 2001), this paper implements SHapley Additive exPlanations (SHAP) values to lower the tradeoff between the model’s accuracy and the model’s interpretability.
The predictive accuracy of a hyperparameter-tuned XGBoost model proved to be superior compared to a benchmark logit model. Hence, one should use a more complex model when accuracy is the primary goal. A disadvantage, however, is that more complex models are more computationally expensive than simple models. Generally, SHAP values enable its users to critically examine complex models and understand how dependent variables were predicted. Through this method, users gain further knowledge about the importance, extent, and direction of feature variables on the target variable. Although causal statements cannot be made through this approach, it still helps users gain trust in the model, find ways to improve the model and get a new understanding of the data. When using ordinary feature impact tools, this would not be feasible to such an extent. Also, the trust of the analysis through SHAP values is increased because underlying trends of the data set in this paper have been amongst others discovered in the research of RFM-models (Bauer 1988, Miglautsch 2000):
Customers that have recently acquired another product have a higher predictive value of opening a checking account than customers who have not. Also, the more active customers are (as measured by logins), the higher is the prediction value. Another important trend found in the data is that younger customers exhibit a higher prediction probability than older customers. Interestingly, there is an uptick in the prediction probability for customers that enter their retirement period. Also, checking account ads always lead to a positive effect on the prediction, although varying. When appearing with younger or recent customers, there is a negative interaction effect on the prediction.
My main conclusion is that XGBoost models should be used in practice for predicting cross-selling purchase probabilities and decisions. One of the most significant disadvantages - lack of interpretability - can be mitigated with SHAP values that greatly expand the transparency, explainability, interpretability of complex tree-based models.