ml-flow

2110446 Data Science and Data Engineering (2022/2) - Project

End-to-end ML pipeline on Azure Machine Learning for heart disease prediction. Features 4-step automated workflow (data prep, training, evaluation, registration), MLflow experiment tracking, and managed endpoint deployment. Built with Azure ML SDK v2, scikit-learn, and auto-scaling compute clusters.

An end-to-end machine learning project implementing MLflow for comprehensive experiment tracking and model lifecycle management. This project demonstrates a complete ML pipeline using Random Forest Classification, showcasing MLflow's powerful capabilities in managing machine learning workflows.

These are my personal notes regarding Machine Learning DevOps stuff

A ML pipeline predicting NYC taxi trip duration from 1.46M trips (MAE 171 s). Data contracts (pandera), SQL cleaning (DuckDB), geospatial features, LightGBM with MLflow and SHAP, time-based evaluation, PySpark cross-check, Prefect scheduling, 91 tests in CI, and a live FastAPI demo in Docker on AWS Lambda.

🔀 ML-Dataflows using Prefect (e.g.: Structured Hierarchical RAG)

End-to-end data engineering platform on Databricks: medallion lakehouse, ML detractor scoring, LLM survey analysis, orchestration

Agentic data-quality engine for Indian healthcare, built at Hack-Nation's Global AI Hackathon (Databricks for Good). Scores 10K facility records against contradiction rules and answers planner questions with a LangGraph agent over Mosaic AI Vector Search, with row-level citations.

Use ML-FLOW and TensorFlow2.0(Keras) to record all the experiments on the Fashion MNIST dataset.

a machine learning project from data ingestion from s3 bucket to model deployment with ml-flow tracking

Carbon-aware energy forecasting and decision ranking for cleaner, cost-efficient workload scheduling.