Problem

The challenge was to enhance the accuracy of existing customer propensity models by using a large amount of newly available input variables and state-of-the-art machine learning (ML) techniques. Another challenge was to optimize the MLOps workflows to streamline model development and deployment. Creating scalable processes within Amazon Web Services (AWS) that can handle huge volumes of data is also a challenge.

The Before State

• Improve the accuracy of existing customer propensity models by leveraging large volumes of newly available input variables & state-of-the-art ML techniques
• How to optimally configure MLOps workflows that streamline model development & deployment
• Creating scalable processes within AWS that can handle huge volumes of data

What KPI Delivered

• Trained ML classifiers (XGBoost) that demonstrate a significant increase in accuracy
• Implemented custom SHAP explainers to diagnose the model and analyze the key drivers
• Integrated and leveraged MLOps components such as the feature store, model registry, and task orchestration
• Created customized, scalable Airflow operators within the AWS Sagemaker environment

The After State

• Reduced Airflow directed acyclic graph (DAG) failures due to high prediction volumes by 40%

• Improved feature understanding and model explainability

• Lowered the overall model development and deployment time by 70%