Knowledge Graph — Coursera Notes › Academic disciplines › Information Technology / Computer Science › Software Development › DevOps › CI/CD

CI/CD pipelines

concept · part of CI/CD

CI/CD pipelines are automated processes for building, testing, and deploying code. In machine learning, they automate retraining models, running validation tests, and deploying models to production. Benefits include reduced manual errors, faster time to production, consistent deployment process, and improved collaboration.

Continuous delivery (CD) extends CI by automatically deploying code changes to a staging or production environment after passing tests. In ML, CD can automate the deployment of retrained models to a serving infrastructure, ensuring that validated models are quickly and reliably available for inference.

Continuous deployment (CD) automates deploying validated code or models to production after passing all tests, making new model versions available without manual intervention. This enables rapid response to changing data and delivers accurate predictions to end-users.

A CI/CD pipeline for ML generally consists of four stages: source control (version-controlled repository for code, config files, and model artifacts), building (create virtual environment, install dependencies, set up libraries like Scikit-Learn or TensorFlow), testing (automated unit tests and model performance tests for accuracy, precision, recall), and deployment (deploy model to production, e.g., Azure Kubernetes Service or Azure Container Instances).

GitHub Actions is a popular automation tool that allows developers to create workflows directly within GitHub repositories. It can be used to build and manage CI/CD pipelines for ML models, integrating seamlessly with version control.

Jenkins is an open-source automation server that can be configured to implement CI/CD pipelines for various projects, including ML. It is highly customizable and integrates with different stages of the ML lifecycle.

Best practices include: automate testing at each stage (code quality, data quality, model performance); version control datasets, model artifacts, and code; monitor model performance post-deployment and set triggers for retraining if accuracy drops.

An online retail company uses a recommendation engine that must be frequently updated with latest customer behavior data. A CI/CD pipeline automates retraining, validating, and redeploying the model as new data arrives, ensuring recommendations are based on the most recent data, improving customer experience and sales.

This is the text view of an interactive 3D knowledge graph — open this page with JavaScript enabled to explore it visually.