Skip to content

Latest commit

 

History

265 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Vedanth Baliga — Data Engineering Portfolio

Hello World! I'm Vedanth.

This is a complete portfolio of the projects I have designed with a major focus on implementing various data engineering tech and cloud services.

Feel Free to Connect with me 🤠

LinkedIn | GitHub

Quick Links

Here is an index of projects with the tech, domain and link.

Real Time Streaming and Analytics

Project Tech
Realtime Vehicle Fleet Monitoring Analysis Redpanda,Apache Flink,Clickhouse,Metabase,Docker,Kubernetes
InvestIQ Metrics AWS EC2,Docker,Airflow,ReapidAPI,AWS Lambda,AWS S3,AWS CloudWatch,AWS Redshift,Prometheus,Grafana,PowerBI

Databricks End to End Products

Project Description Tech
Deal2Delivery (Live Demo) Unified SAP + Salesforce demand intelligence platform on a Databricks Lakehouse - combines churn prediction and demand forecasting ML models with a Genie AI/BI space for internal analytics and a Next.js app for business stakeholders. 🏆 2nd Place Winner, Databricks DAIS 2026 Virtual Hackathon Databricks (Serverless), Delta Lake, Unity Catalog, Lakeflow DLT, Databricks Asset Bundles, XGBoost, Genie AI/BI, Next.js, Vercel
Dataflix (Live Demo) Netflix-themed, Databricks-native AI content-strategy platform - real TMDB catalog data plus synthetic licensing/engagement/marketing data, four tuned Genie spaces, a document-AI pipeline for contract and script ingestion, and a Multi-Agent Supervisor that orchestrates it all behind a chat interface, casting recommender, and auto-generated compliance memos. Unity Catalog, Delta Lake, Databricks Jobs, Genie Spaces, Agent Bricks Multi-Agent Supervisor, MCP, Lakebase, Next.js, Vercel

Small Data Engineering And Analytics Projects

Project Tech
Pt 1 : Streaming ETL with Airflow Orchestrator and Postgres DB Near Realtime Streaming,SQL Database, Kafka, Cricket!
Pt 2 : Serving Postgres Data using FastAPI Backend API Development, Streaming Data
Pt 3 : Frontend Web App Powered by Streaming Data with Streamlit Frontend Application Development,MVC Architecture
Pt 4 : Realtime Ingestion and Transformation with Apache Druid as OLAP Database Data Ingestion and Big Data Processing
User Mingle : Kafka Driven User Profile Streaming Airflow, Zookeeper, Cassandra, Schema Registry, Spark
Grand Prix Data Odyssey Azure Databricks,Spark SQL,Postman,Blob Storage,Unity Catalog,ADF,Azure Devops,Synapse Studio, Delta Lake,PowerBI
Medal Metrics: Tokyo Olympics Data Alchemy ADF,Azure Data Lake Gen2,Blob Storage,Databricks,Synapse Analytics,PowerBI
Airflow with Postgres as OLTP Database [Batch Processing] Beautiful Soup, Apache Airflow, Postgres, Docker

Releases

Packages

Contributors

Languages