A competing risks model based on latent Dirichlet Allocation for predicting churn reasons

Abstract Due to low switching costs and stiff competition, customer relationship management has become a central component in the marketing strategy of telecommunication service providers. Since the costs of acquiring a new customer are five times higher than the costs of maintaining an existing customer, telecommunication service providers are eager to reduce the churn rate. A solid understanding of customer churn behavior can help to address this problem. Reducing the churn rate can translate into significant revenue gains and might provide the edge to outperform the competitor. In this paper, we predict the propensity to churn for customers of a Dutch telecommunication service provider by employing a duration model. While predicting churn, we simultaneously predict the reason for which the customer churns, using a competing risks model. Since the telecommunication service provider has valuable textual data based on transcripts of calls between customers and the customer service center, we incorporate topics extracted from this textual data as variables in our models, by employing Latent Dirichlet Allocation (LDA). We compare four models and find that the models that have incorporated topic variables usually yield the best churn forecasts. Also, the investigated models beat the considered benchmark model, which is the model currently deployed at the telecommunication service provider.