Eshan Pandey

Platform & backend engineer. 3 years running production infrastructure on AWS and GCP. I migrate databases, move clouds, and keep live video streaming up. Go and Ruby on Rails, Kubernetes underneath.

$ curl -s localhost:8080/v1/whoami | jq

{
  "name": "Eshan Pandey",
  "role": "Software Engineer, Platform & Backend",
  "company": "Learnyst",
  "location": "Bangalore, IN",
  "experience_years": 3,
  "languages": ["go", "ruby", "javascript", "python", "sql"],
  "clouds": ["gcp", "aws", "linode"],
  "on_call": true,
  "focus": ["db migrations", "multi-cloud", "kubernetes", "webrtc", "rag"]
}
HTTP/2 200 · content-type: application/json · x-response-time: 12ms

$ curl -s localhost:9090/metrics | grep eshan_

# HELP eshan_deploy_time_reduction_percent CI/CD on AWS ECS + CodePipeline
# TYPE eshan_deploy_time_reduction_percent gauge
eshan_deploy_time_reduction_percent{pipeline="ecs"}          60
eshan_incident_response_reduction_percent{src="cloudwatch"} 40
eshan_cloud_migration_window_days{from="aws",to="gcp"}  10
eshan_db_migrations_total{from="mysql",to="alloydb"}  1
eshan_cutover_tables_verified{gate="fingerprint"}   365
eshan_cutover_rows_lost                              0
eshan_db_cutover_window_hours{db="core"}           3
eshan_gas_reduction_percent{contract="cryptoballot"}    25
eshan_awards_total{name="bravo",year="2025"}          1

$ cat docs/adr/mysql-to-alloydb.md # core app DB, planned and cut over solo

# ADR: move the core database from MySQL to AlloyDB
status  accepted · shipped · 0 rows lost
owner   eshan (benchmark → schema → CDC → validation → cutover)

## why
The roadmap needed AI search: vector embeddings for natural-language course
discovery. AlloyDB is Google's PostgreSQL-compatible database with vector search
built in, next to the app data. Getting there meant leaving MySQL.

## evidence first
Benchmarked both on identical 4 vCPU / 32 GB instances with the real Rails app:
~1.5M requests. Reads were a tie. Under a 2,000-signup burst AlloyDB had
0 failed signups (MySQL: 0.5%) and 39% better p95 write latency.

## decision
Heterogeneous migration with a blue-green cluster:
  · stream every change MySQL → AlloyDB with CDC while prod keeps running
  · build a second GKE cluster running dark against AlloyDB
  · at cutover: freeze, reconcile, fingerprint every table, flip DNS
  · MySQL is only ever read, so rollback stays a DNS flip away

## consequences
+ AlloyDB embeddings and RAG course discovery are unblocked (see wip/)
+ better write tail latency and reliability under bursts
- two engines, different types and SQL semantics, synced live
- MySQL-isms hide in app code (so I built a scanner, see tools/)

$ stat pipeline/cdc # how both databases stayed in step

› click a stage to inspect it

$ ./cutover --window 02:00-05:00 IST # replay the real cutover night

02:00 freeze: both clusters scaled to zero, zero writes verified (not assumed)
02:1x CDC drained, max ids identical on hot tables
02:24 schema migrations on AlloyDB (106s)
02:36 reconcile 365 tables, 4 parallel workers (86 → 55 min)
03:32 fingerprint gate: 365/365 MATCH
03:33 reset 367 sequences, ANALYZE full DB
03:54 DNS flip to the new cluster, IPv4 + IPv6 verified
04:1x 91% of traffic on AlloyDB. done inside the window

$ ls /var/log/postmortems/ # bugs that threw no errors. click to open.

DATA The script that deleted live rows while reporting success
symptom   a reconcile run finished "ALL DONE", yet ~10,400 rows were missing on the target
cause     the MySQL client defaulted to autocommit off + REPEATABLE READ, so the first
          SELECT pinned one snapshot for a multi-hour run. rows inserted mid-run were
          invisible to the script but already replicated by CDC, so they looked like
          "extras" and were deleted. the safety re-check read the same stale snapshot.
proof     every missing row's created_at sat in one 30-min band starting the second
          the script started
fix       autocommit on. a re-verify that shares the snapshot is not a safety check.
outcome   caught before cutover. cutover night: 0 rows lost
DATA cancel vs drain: tens of millions of rows gone in one click
symptom   10 streaming jobs silently ran pinned at 2 workers. relaunching them fixed
          throughput, but ~183 tables came back short, some by 20M+ rows
cause     "--max-workers" alone does not enable autoscaling. and "cancel" drops data
          that was acked but not yet written. "drain" flushes it.
twist     draining the 4 biggest jobs forced a flush their 15 GB workers couldn't
          hold: OOM crash loop. moved them to 32 GB highmem workers.
fix       full re-backfill of all 183 tables, verified with exact COUNT(*) on both sides
lesson    drain, never cancel. estimated row counts are not evidence.
DATA Rows that vanished with no error, no dead-letter, no trace
symptom   the same ~18k rows of one table failed to land on 4 backfills in a row.
          then a sweep found 24 tables stuck the same way.
cause     the streaming job's per-key dedup state was poisoned by earlier churn. it
          silently dropped any row whose sort key didn't beat the stored state.
proof     replayed one "lost" row by hand inside a rolled-back transaction: postgres
          accepted it. a fresh-state job then recovered the whole table.
lesson    recover with a fresh-state job, never by re-feeding the same one
DATA Counts matched, checksums matched, data was 3.5 months old
symptom   found on cutover night: 177 records in the dead-letter queue were updates
          whose row already existed. right id, stale content.
why hidden row count matches, SUM(id) matches, fingerprint says MATCH, id-based
          reconcile never touches it. one row was 3.5 months stale.
fix       whole-table force-upsert of the 5 affected tables, inside the window
lesson    every check that compares ids is blind to content drift
INFRA Connection storm with nothing in the logs
symptom   background workers hit "server closed the connection unexpectedly"
not it    not the worker, not connection lifetimes, not the database
cause     the connection pooler had its own autoscaler flapping 4→3→4 pods in two
          minutes. each scale-down sent SIGTERM, which the pooler treats as
          "exit now", killing every client on that pod. killed processes log nothing.
fix       preStop sends SIGINT (its graceful shutdown), 60s grace period,
          10-minute scale-down stabilization
INFRA Deploys failing: "IP space exhausted" with half the cluster idle
symptom   a rolling deploy couldn't schedule pods, and the autoscaler couldn't add nodes
math      one /19 pod range shared by six clusters. at 110 max pods per node, each
          node reserves a /24 (256 IPs) → a hard ceiling of 32 nodes, total.
          real usage: ~14 pods per node. 12× over-reserved.
fix       new node pools at 32 max pods (/26 per node) on a fresh range: 4× denser
          per node, room for 1,024 nodes. rehearsed in staging with zero downtime.
lesson    max-pods is immutable on a pool. 33 costs the same as 64. 32 is the sweet spot.
MEMORY A worker that ate 1.3 GB while doing nothing
symptom   messaging worker memory climbed 459 MB → 1,350+ MB and never came back
digging   live heap dumps via rbtrace: 135,970 APM Layer objects retained at idle.
          the APM agent stored each job's trace in a thread-local that was never
          cleared, so 10 threads each kept the previous job alive.
fix       APM off for background jobs + jemalloc decay tuned to 1s
result    idle RSS 1,350 MB (growing) → ~730 MB (stable), spike-and-release pattern

$ quiz silent-bugs # MySQL ran this for years. what did it do?

where("exam_date >= #{Time.zone.today}")   # note: no quotes around the date
pick an answer

$ journalctl -u career.service --reverse | grep -E '.*'

Learnyst

Software EngineerMay 2024 → present · Bangalore
Go · Rails · Node.js (Fastify, BullMQ) · GKE · Dataflow · DataStream · Pub/Sub · AlloyDB · OpenSearch · ECS · Lambda · CloudWatch · Linode · Helm · LiveKit/WebRTC · Postgres · MySQL · Kafka · Redis · Docker
  • [MIGR]Independently planned and executed the MySQL → AlloyDB (PostgreSQL) migration of the core application database, from schema design through a blue-green production cutover: 365/365 tables verified, zero data loss.
  • [MIGR]Built a real-time CDC pipeline with DataStream, Dataflow and Pub/Sub to keep data in sync during the move with minimal downtime.
  • [ARCH]Made the backend vendor-agnostic with the Strategy Pattern: authenticator and cloud provider became configuration, not hardcoded dependencies. That groundwork made the multi-cloud migration possible.
  • [MIGR]Migrated OpenSearch and AWS Lambda workloads in the team push that moved critical infrastructure from AWS to GCP in a 10-day window.
  • [SHIP]Build features on Superlive, the live class streaming platform, including automatic session recording in the Go-based LiveKit WebRTC media server. Primary point of contact for the area. The stack includes a Node.js/Fastify resource manager that schedules Linode VMs through BullMQ.
  • [MIGR]Moved backend services and async workers from instances to GKE, deployed with Helm.
  • [ARCH]Designing natural-language course discovery: RAG over embeddings stored in AlloyDB, resolving open-ended queries into courses, articles and assessments.
  • [SHIP]Built a Coding Challenge module with secure sandboxed code execution for interactive programming assessments.
  • [PAGE]On-call for production: API downtime, database issues, failed deploys. Wired CloudWatch monitoring into Sidekiq workloads, cutting incident response time by 40%.
  • [SHIP]Automated CI/CD on AWS ECS + CodePipeline, cutting deploy time by 60%.

EXL Service

Graduate Engineer TraineeOct 2023 → Mar 2024 · Remote
Python · Jenkins · data engineering pipelines
  • [INFO]Technical rotation in Python-based data engineering and Jenkins deployment automation.

LCEPL

Software Development InternApr 2022 → Oct 2022 · Pune
Flask · JavaScript · CSS
  • [SHIP]Built an automated email response engine in Flask, improving response latency by 50%.
  • [SHIP]Front-end work in JavaScript and CSS that lifted user engagement metrics by 25%.

$ kubectl describe system superlive # live class streaming, I own the feature area

› click a component to inspect it

# design rules I work to
1 the product backend never talks to video infrastructure. it asks the
  resource manager, which owns placement, capacity, VMs and recording.
2 recording is intent vs observed state. the host sets intent, a reconciler
  converges reality to it, so a crashed recorder heals itself. a human stop
  flips intent, which is how "stopped on purpose" differs from "crashed".
3 a self-hosted server dies mid-class → users reconnect onto LiveKit Cloud
  automatically, and recording resumes as cloud egress.

$ CLOUD_PROVIDER=gcp rails runner 'p CloudProvider.current' # strategy pattern, flip it

# illustrative sketch of the pattern, not production code
class CloudProvider
  STRATEGIES = { aws: AwsStrategy, gcp: GcpStrategy }

  def self.current
    STRATEGIES.fetch(ENV["CLOUD_PROVIDER"].to_sym).new
  end
end

=> #<GcpStrategy>
# same code path, different vendor. this is what made the AWS → GCP move possible.

$ k6 run puma-ladder.js # tuning Rails concurrency on GKE. pick a config.


# why: Ruby's GVL lets one thread run Ruby at a time per process. threads
# beyond what one worker can feed just queue behind the lock. workers add real
# parallelism, threads add overlap on I/O. measure, don't guess.

$ ls ~/bin # tools I built to make the big moves safe

pg_compat_scan

Static scanner for MySQL-isms that break, or silently change behaviour, on PostgreSQL. Pure Ruby, no gems, no DB, no Rails boot. Sorts every finding into actionable, review or analytics-only.

$ ruby pg_compat_scan.rb --fail-on-actionable   # CI gate
$ ruby test_pg_compat_scan.rb                   # 52 self-tests

cutover fingerprint gate

Per-table fingerprints on both databases, compared before any traffic moves. Parallel reconcile on idle cores cut the in-window run from 86 to 55 minutes.

365/365 MATCH  IDSET_DIFF=0  → go

live alerts framework (in review)

Error alerting for the live-class stack: environment tags, email per error type (not per occurrence, so nobody mutes it), and error-tracker faults pulled into the same incidents.

scope live servers · resource manager · rails · composer

$ git log --oneline -1 feature/course-discovery

(WIP) design natural-language course discovery with RAG over AlloyDB

# a learner types "how to build gundam" and gets courses from that school,
# each with a one-line reason grounded in lesson transcripts.
rule 1 retrieval is always scoped to one school
rule 2 generated text may only restate what was retrieved. no world knowledge.
rule 3 below a relevance floor, return nothing. nearest-neighbour search on a
       small corpus always finds something; confident wrong answers are worse.

› click a stage to inspect it

$ pagerctl simulate # a replay of the kinds of incidents I handle. pick one.

pager idle. no active incidents. trigger one above.

$ helm get values eshan -o yaml

languages: [go, ruby, javascript, python, sql, solidity]
frameworks: [rails, node, fastify, bullmq, graphql, rest, rspec, django, flask]
cloud:
  gcp: [gke, dataflow, datastream, pubsub, alloydb]
  aws: [ecs, lambda, cloudwatch, codepipeline]
  other: [linode]
platform: [kubernetes, helm, docker, jenkins, ci/cd]
data: [postgresql, mysql, alloydb, redis, opensearch, kafka, pubsub, sqs, cdc]
realtime: [webrtc, livekit, sfu, live-streaming, recording-pipelines]
ai: [rag, semantic-search, embeddings, vector-search, llm-integration, claude-api]
practices: [distributed-systems, system-design, microservices, incident-response]
tools: [git, postman, sidekiq, jira]

$ kubectl get projects -n side -o wide

NAMESTACKSTATUSNOTES
videosummarizerPython, Django, PostgreSQL, Gemini APIRunningExtracts and summarizes key insights from video transcripts with Gemini Pro. JWT auth, Postgres-backed user data.
cryptoballotSolidity, Ethereum, Web3.jsRunningDecentralized voting protocol on smart contracts for tamper-proof results. Refactor cut gas use by 25%.

$ tail /var/log/eshan/events.log

2025 AWARD  Bravo Award for the critical AWS → GCP migration, delivered in 10 days
2023 EDU    B.Tech, Information Technology, Bharati Vidyapeeth University, Pune · GPA 8.9/10 (2019–2023)

$ ps aux | grep off-hours # when the pager is quiet

USER    PID  %CPU  STAT  COMMAND
eshan   1101  97.0  R     skyrim --dragonborn --one-more-quest
eshan   1007  64.0  R     james-bond --licence-to-kill
eshan   2048  42.0  S     read --books

$ cat ~/.contact

email   pandeyeshan13@gmail.com  
github  github.com/eshanpandey
leetcode leetcode.com/u/03_Eshan
based   Bangalore, India

$ ./eshan-cli # try it: type help

eshan-cli v1.0 · type help for commands