LIVE
8× K8S THROUGHPUT  ·  WEEKS → UNDER AN HOUR  ·  TENS OF MILLIONS OF DOMAINS  ·  p95 MINUTES → SECONDS  ·  7+ YEARS AT SCALE  ·  THE LAB: A CALCULATOR MADE OF 1,536 RESISTORS  ·  REMOTE-FIRST · IST (UTC+5:30)  ·  
Data Platform · AI · SWE II @ Demandbase The Lab · 1,536 resistors

Mohit Khare

I turn the open web into data AI systems can trust — crawling and extraction at scale, and the retrieval and evaluation layers on top.

Software Engineer II at Demandbase. 7+ years across data platform, backend, and ML systems — distributed crawlers on Kubernetes, warehouse-native data pipelines, and natural-language query engines with the eval harnesses that keep them honest. Open to senior AI-engineering and data-platform roles, remote-first from India.

Remote-first · India · IST (UTC+5:30)
01 / Selected work

Impact, with the numbers attached

Six problems I owned end to end. Each one started ambiguous and shipped with a measurable result.

01 Data platform · Iceberg · Spark
weeks under an hour
multi-week cycledaily refresh
two months of runszero failures
8 repositories · 5 platform teamsSpark · Iceberg · StarRocks

Warehouse-native data re-architecture

A customer-facing dataset reached users through a transactional database, then a data lake, then a warehouse, then the serving store — a chain that took weeks end to end, and inherited that latency for every fix and every new request. I took the path out of the transactional database entirely and landed the data warehouse-native in Iceberg, normalized and tenant-partitioned, with the serving store following from there. The refresh now runs daily in well under an hour, every downstream consumer was migrated, and the legacy path was decommissioned rather than left dormant. Two consecutive months of production runs, no failures.

02 Kubernetes · Crawling at scale
8× throughput
serverlessKubernetes
8× throughputvs serverless
tens of millions of domains per run~80% lower cost per domain

Web-scale crawling on Kubernetes

Moved a rendering crawler off a serverless architecture that couldn't hold throughput onto Kubernetes: 8× the domain-processing rate, tens of millions of domains inside a single multi-day run, at roughly a fifth of the cost per domain. Headless browser rendering, concurrency control, retry and backoff behaviour, and compute standardized so other teams could run the same pattern.

03 AI/LLM · Retrieval · Evals
minutes under 30s p95
three generationsmeasured pivots
~60% prompt reductionschema linking
CPU-only, no GPU spendeval-gated CI

Natural-language query engine + evals

An engine that answers natural-language questions over B2B business data. LLM-first generation with schema linking to keep prompts small, vector value-grounding over sentence embeddings for vocabulary resolution, and MCP tool servers so the engine is callable from agents and IDEs. I built it three times: each rewrite was triggered by a benchmark, not a preference — a formal query corpus and SLOs set in advance showed where the architecture actually failed. p95 went from minutes to inside a 30-second target; broad fan-out queries dropped roughly 7×.

The durable part is the evaluation harness: it scores whether the returned data is correct, not whether the generated query parsed, and it gates CI so regressions can't ship. Negative results are written down — vector grounding closes category-to-member gaps but fails on proper nouns — so the next design inherits the finding rather than rediscovering it.

04 Distributed systems · Crawling · Event-driven
0 hundreds of pods
prototypeproduction service
KEDA from zeroand back
daily crawl, tens of thousands of domainsAho-Corasick · Redis Streams · DLQ

Daily competitive-signal detection service

Took a prototype detection script and turned it into a daily production service. Async crawler with headless rendering and SSRF guarding, Aho-Corasick matching over a large signature set, Redis Streams with a dead-letter queue for durable work distribution, and KEDA autoscaling from zero to hundreds of pods and back so it costs nothing between runs. Added a stable-detection window to kill false negatives from single-day crawl failures, reconciled every result against CRM records, and synced downstream through platform events. I ran a full pre-production review of my own code before go-live and closed every finding, critical down to minor, with the fixes independently re-verified.

05 AI/ML · Spark · GPU
94% runtime cut
56h3h
94% fasterruntime
overnight batch → same-day75% lower compute

GPU-accelerated NER pipeline

Overhauled an AI-driven named-entity-recognition pipeline (spaCy on Spark, GPU-accelerated). Runtime collapsed from 56 hours to 3 while cutting compute 75% — turning an overnight batch into a same-day job.

06 Backend · API performance
1,000% search throughput
100%20%
80% lower latencyAPI p50
Redis keys: billions → thousands

Scaled e-commerce APIs at Fynd

Boosted search processing 1,000% and price-cache performance 200%, cut API latency 80%, shrank Redis keys from billions to thousands, and built APIs handling 50M daily updates.

02 / Experience

Seven years, three companies, one throughline

From OCR support and release automation to owning AI/ML data platforms — a steady climb toward harder, higher-leverage systems.

Demandbase
May 2025 – Present · Hyderabad
Current

Software Engineer II — Data Platform & AI

  • Re-architected a customer-facing dataset off a transactional database onto a warehouse-native Iceberg + StarRocks design. Refresh latency to customers went from weeks to under an hour, running daily; two months of consecutive production runs with zero failures. Delivered across 8 repositories and five platform teams' surfaces, with every downstream consumer migrated and the legacy path decommissioned rather than left dormant.
  • Built a natural-language query engine over B2B business data: LLM-first schema linking (~60% prompt reduction), SQL and Cypher generation, vector-based value grounding over sentence embeddings, and MCP tool servers exposing it to agents and IDEs. Three architecture generations, each pivot driven by a measured benchmark; p95 latency from minutes to inside a 30-second SLO, CPU-only with no GPU spend.
  • Built the evaluation layer behind it — a harness scoring data-level correctness rather than query validity, versioned corpora, and eval-gated CI. Documented negative results (vector grounding closes category-to-member gaps but fails on proper nouns) so later designs inherit the finding.
  • Took a competitive-signal detection crawler from prototype to production service: daily crawls across tens of thousands of domains, Aho-Corasick matching over a large signature set, headless rendering with SSRF guarding, Redis Streams with a dead-letter queue, KEDA autoscaling from zero to hundreds of pods, and CRM sync via platform events. Ran the full pre-production review myself and closed every finding before go-live.
  • Modelled data as agent-consumable business objects and semantic-layer cubes, which is what made the dataset usable by a production LLM chat agent — from partially answerable to passing its full golden-dataset suite multi-turn.
  • Reliability and security: first-line on-call triage, written RCAs for production incidents, credential-exposure remediation, and CVE response.
  • Led the complete overhaul of the AI-driven NER pipeline (spaCy on Spark + GPU): 56h → 3h runtime, 75% lower compute.
  • Owned and stabilized a customer-facing technographics dataset, cutting data-upgrade time 90% (4 days → 30 min).
  • An internal applied-AI platform that other teams build against. Recognized company-wide at an internal AI showcase.
Demandbase
Aug 2023 – May 2025 · Hyderabad

Software Engineer

  • Migrated a rendering crawler from serverless to Kubernetes — 8× throughput, tens of millions of domains per run, roughly a fifth of the cost per domain.
  • Replaced a paid geocoding API with Pelias for location tagging — 95% cheaper per unit of work, no loss in quality.
  • Standardized Kubernetes compute across teams, ran internal K8s training, and mentored teammates.
  • Won Most Impressive Technical Implementation at a company AI hackathon for an outreach assistant that surfaces high-potential leads.
Fynd · Shopsense Retail
Sep 2021 – Aug 2023 · Bengaluru / Mumbai

Software Engineer → Senior Software Engineer

  • Cut API response time 80% and shrank Redis keys from billions to thousands.
  • Lifted system performance 200% with a system-wide price cache and boosted search processing 1,000%.
  • Added pincode support for 1.5M products and designed APIs handling up to 50M daily updates.
Odessa
Dec 2018 – Sep 2021 · Bengaluru

Intern → Software Engineer → Senior Software Engineer

  • Automated regression testing and the release process — delivery time from 1 week to 5 hours.
  • Built and supported APIs for Comments and Documents; handled vulnerabilities and OCR document support.
  • Hypercare member for critical client defects.
03 / Projects & open source

Things I build outside the backlog

Open-source tooling for the messy realities of AI-generated code, an analog calculator you can run in the browser, and internal applied-AI work recognized company-wide.

The Analog Calculator

The Lab · EXP-001 · Simulation-validated

One 1-layer Transformer trained on single-digit +, −, ×, ÷, rebuilt as three 512-resistor crossbars (ngspice-validated), then composed by a digital controller into a full multi-digit calculator. Type any numbers and press =; every single-digit step runs through the real circuit, the way a CPU builds big math from small.

PyTorch QAT ngspice crossbar MAC

PyCodeGate

PyPI · OSS

A static-analysis CLI built for the specific failure modes of AI-generated Python. 40+ rules targeting AI-prone bugs, framework-aware checks for FastAPI, Django, and Flask, and SARIF output that drops straight into CI/CD.

Python Static analysis SARIF CI/CD

Internal applied-AI platform

Recognized · Showcase

An internal applied-AI platform that other teams build against. Recognized company-wide at an internal AI showcase.

AI/ML Applied AI Internal platform

DBX

Hackathon · Winner

An AI-powered outreach assistant that uses ML to surface high-potential leads and automate personalized outreach — Most Impressive Technical Implementation at a company AI hackathon.

ML Automation GTM tooling
04 / Toolkit

What I reach for

Open to senior AI-engineering and data-platform roles — remote-first from India.

Languages & core
Python Scala SQL Apache Spark Kafka
Cloud & infra
AWS Kubernetes Docker Terraform EMR
Data & search
OpenSearch Apache Iceberg StarRocks PostgreSQL Redis
AI / LLM systems
LLM orchestration RAG Sentence embeddings Vector search NL-to-SQL MCP Eval harnesses Claude / Bedrock OpenAI API spaCy · NER
Crawling & extraction
Distributed crawling Headless rendering Concurrency & backoff SSRF hardening robots / politeness Proxy rotation DNS resolution Aho-Corasick Entity resolution Dead-letter queues KEDA
Engineering practice
System design Microservices API performance Cost optimization
05 / Contact

Let's build something that scales

Open to senior AI-engineering and data-platform roles, remote-first from India. Happy to walk through any of the work in detail. Email is fastest.

Email me Resume LinkedIn GitHub