Kartik Aneja

AI/ML Platform Engineer in Boston. I build production AI systems, streaming data platforms, and the MLOps glue between them.

Boston, MA · AI/ML Platform Engineer · 8+ years

I build the platforms that make AI products work in production.

Shipped Kafka pipelines at 10M+ events/day, low-latency inference for 1M+ daily orders, and TB-scale data platforms across UKG, Dunzo, and Warpspd. Now open-sourcing the AI platform tools — eval, observability, NL→SQL, and a modern data stack — I keep rebuilding at every job.

See what I’m shippingDownload résuméBook a 30-min chat
Previously at
UKGStaplesDunzoWarpspdDezervMake Us Fly
LLM platform
Eval harnesses, LLM observability, RAG + prompt-CI — built on top of OpenAI + Anthropic.
Data platforms
Streaming ETL, lakehouse-shaped pipelines, feature engineering, dbt-driven analytics.
ML infrastructure
Production inference, model deployment & monitoring, MLOps — and the glue between them.
Production APIsRAG systemsStreaming ETLDeployment toolingData platform design
One-minute summary

I’m an AI/ML Platform Engineer who likes shipping the unglamorous parts: the APIs, feature pipelines, evaluation harnesses, and deployment paths that make AI products reliable once people actually use them.

What I ship

  • RAG platforms and LLM evaluation harnesses that survive iteration.
  • Drop-in observability — latency, cost, hallucination patterns.
  • Modern data stacks with clean bronze / silver / gold boundaries.

What teams get

  • Cleaner handoffs from prototype to production.
  • Systems that are easier to debug, deploy, and operate.
  • Practical collaboration across engineering and product.

Currently building

All projects →

Four open-source tools shipped in the last week. All MIT-licensed, self-hostable, and demo-able locally with one docker compose up.

evalstack — run-level diff page with judge histograms and top regressions/improvementsAlpha · Demo liveLive ↗
LLM evaluation · MLOps

evalstack

Open-source LLM evaluation framework — Python SDK + CI plugin, LLM-as-judge rubrics, regression detection, side-by-side run + event diff. Gates prompt/model changes in CI before they ship. Braintrust alternative.

PythonFastAPINext.jsLLM-as-judgeCI/CDMLOps
Open live demoView on GitHub
dataask — MRR by plan question answered with generated SQL, EXPLAIN cost preview, and result tableAlpha · Demo live
NL→SQL · RAG analytics

dataask

Natural-language analytics for founders + PMs — schema-aware, RAG-grounded, sqlglot-safety-gated SQL generation against DuckDB. EXPLAIN-based cost preview, single-page chat UI, key-free demo mode.

PythonFastAPIDuckDBOpenAIRAGsqlglot
View on GitHub
tracelens dashboard — latency percentiles, total cost, error rate, per-model breakdown, recent tracesAlpha · Demo live
LLM observability · monitoring

tracelens

Production LLM monitoring for inference in the wild — a drop-in @traced decorator captures latency / tokens / cost / errors. FastAPI collector, pluggable storage (SQLite or ClickHouse), p50/p95/p99 dashboard. Auto-detects OpenAI + Anthropic response shapes.

PythonFastAPIClickHouseObservabilityMonitoringfail-soft
View on GitHub
lakehouseit — terminal walkthrough of the full make demo pipeline runMVP · Working end-to-end
Modern data stack · data platform

lakehouseit

End-to-end open-source data platform template — the feature-engineering backbone under ML/AI systems. Postgres → Debezium / snapshot CDC → Iceberg-style Parquet → dbt (bronze/silver/gold) → DuckDB queries. Single make demo runs the full path.

PostgresKafkaDebeziumIcebergdbtDuckDB
View on GitHub

Quick links