Kartik Aneja

AI/ML Platform Engineer in Boston. I build production AI systems, streaming data platforms, and the MLOps glue between them.

Open-source tools for the AI/ML platform layer.

Four projects shipped in the last week, each targeting a real ML platform / MLOps gap: LLM evaluation in CI, RAG-grounded NL→SQL, production LLM observability & monitoring, and a self-hostable data platform for feature engineering. All MIT-licensed, all demo-able locally.

evalstack — run-level diff page with judge histograms and top regressions/improvementsAlpha · Demo liveLive ↗
LLM evaluation · MLOps

evalstack

Open-source LLM evaluation framework — Python SDK + CI plugin, LLM-as-judge rubrics, regression detection, side-by-side run + event diff. Gates prompt/model changes in CI before they ship. Braintrust alternative.

PythonFastAPINext.jsLLM-as-judgeCI/CDMLOps
dataask — MRR by plan question answered with generated SQL, EXPLAIN cost preview, and result tableAlpha · Demo live
NL→SQL · RAG analytics

dataask

Natural-language analytics for founders + PMs — schema-aware, RAG-grounded, sqlglot-safety-gated SQL generation against DuckDB. EXPLAIN-based cost preview, single-page chat UI, key-free demo mode.

PythonFastAPIDuckDBOpenAIRAGsqlglot
tracelens dashboard — latency percentiles, total cost, error rate, per-model breakdown, recent tracesAlpha · Demo live
LLM observability · monitoring

tracelens

Production LLM monitoring for inference in the wild — a drop-in @traced decorator captures latency / tokens / cost / errors. FastAPI collector, pluggable storage (SQLite or ClickHouse), p50/p95/p99 dashboard. Auto-detects OpenAI + Anthropic response shapes.

PythonFastAPIClickHouseObservabilityMonitoringfail-soft
lakehouseit — terminal walkthrough of the full make demo pipeline runMVP · Working end-to-end
Modern data stack · data platform

lakehouseit

End-to-end open-source data platform template — the feature-engineering backbone under ML/AI systems. Postgres → Debezium / snapshot CDC → Iceberg-style Parquet → dbt (bronze/silver/gold) → DuckDB queries. Single make demo runs the full path.

PostgresKafkaDebeziumIcebergdbtDuckDB
Want a walk-through?Each repo includes a screenshot, a quickstart, an architecture doc, and a roadmap. Hit the GitHub links above, or book time and I’ll demo the running stack live.