Designing Machine Learning Systems

Machine learning systems are both complex and unique. They are complex because they consist of many different components and involve many different stakeholders. They are unique because they are data-dependent, and data varies wildly from one use case to the next.

This book takes a holistic approach to designing machine learning systems that are reliable, scalable, maintainable, and adaptive to changing environments and business requirements. It considers each design decision — e.g. how to create training data, what features to include, how to deploy, what to monitor, how often to retrain your model — in the context of how it can help the system as a whole achieve its objectives. The iterative framework laid out in this book is illustrated using actual case studies and backed by ample references.

Examples of the scenarios that this book will be able to help you tackle.

You have been given a business problem and a lot of raw data. You want to engineer this data and choose the right metrics to solve this problem.

Your initial models perform well in offline experiments and you want to deploy them.

You have little feedback on how your models are performing after your models are deployed, and you want to figure out a way to quickly detect, debug, and address any issue your models might run into in production.

The process of developing, evaluating, deploying, and updating models for your team has been mostly manual, slow, and error-prone. You want to automate and improve this process.

Each machine learning use case in your organization has been deployed using its own workflow, and you want to lay down the foundation (e.g. model store, feature store, monitoring tools) that can be shared and reused across use cases.

You're worried that there might be biases in your machine learning systems and you want to make your systems responsible!

Read less