CV · Mengjie Zhao

Build and operate realtime model inference serving infrastructure for ad ranking at high QPS under strict per-request latency budgets, spanning CPU and GPU inference paths.
Shipped dual-fire serving to A/B new PyTorch GPU inference stacks against production CPU champions at request granularity without regressing the served path.
Engineered the GPU serving path: server-side dynamic batching to maximize occupancy, admission control to concentrate scarce GPU capacity, runtime model-version resolution, and request routing with asymmetric resolution; emitted counterfactual scores for model performance evaluation.
Led inference-serving initiatives end-to-end, from system design to serving-side integration, scaling to 200k QPS total under a 50 ms round-trip budget.

Build retrieval and ranking infrastructure that selects and scores ad candidates at ~10M scale within tight funnel latency budgets.
Co-designed infra collapsing retrieval (~1M→~3K) and first-pass ranking (~3K→~400) into a single multi-task Two-Tower scoring layer over the entire active creative corpus.
Designed the embedding-serving pipeline: member embeddings inferred online per request, creative embeddings generated nearline and fanned out across thousands of shards, with a backfill-then-activate rollout so a new model version never serves until embeddings have propagated to every indexer node.
Delivered over-20% onsite CTR lift on the first end-to-end experiment.

Mengjie Zhao