MLOps

Modern workflow orchestration for data and ML pipelines

Prefect is an open-source workflow orchestration platform for data engineering and ML pipelines. It provides observability, scheduling, and retry logic for Python workflows with a hybrid execution model.

Open-source MLOps platform for the full ML lifecycle

MLflow is an open-source platform for managing the complete machine learning lifecycle including experiment tracking, model packaging, model registry, and deployment. Widely adopted across data science and ML engineering teams.

Git for data — version control for ML data, models, and pipelines

DVC (Data Version Control) is an open-source ML experiment and data versioning tool built on top of Git. It enables reproducible ML pipelines, tracks datasets and models alongside code, and integrates with all major cloud storage providers.

The industry standard for programmatic workflow orchestration

Apache Airflow is an open-source platform for authoring, scheduling, and monitoring data and ML pipelines as directed acyclic graphs (DAGs). It is the most widely deployed workflow orchestration tool for data engineering teams.