Research

Two projects, one question about where the bottleneck is

Systems evaluation on CPU-constrained serving. The first project measured a three-way architectural trade-off and found a 59× network I/O gap; the second was built to test what that gap meant.

Research · Methodology + tooling · 2026

Batch2go: isolating batching effects by placement

Treating batching placement as an isolable architectural variable, with a single-flight Go proxy that decomposes batch cycle time into compute and transport.

Primary artifact · GitHub

matthewhoung/batch2go

The proxy, the three controlled conditions, the run harness and the matplotlib analysis pipeline. C1, C2 and C3 are byte-frozen at a release tag; every result cell carries a completion marker so a run can be resumed idempotently, and each session writes a machine-readable manifest.

Go 1.25+MIT3,960 runs10 OTel metrics

Read the code

Research · Systems evaluation · 2026

A reproducible evaluation framework for ML serving architectures under CPU constraints

A controlled comparison of monolithic, microservices and Triton serving on CPU-constrained hardware. The finding is a three-way trade-off, not a ranking.

Primary artifact · GitHub

matthewhoung/inference-arena

The evaluation framework, all three serving implementations, the load harness, the observability stack, and the analysis scripts that drew every figure on this page.

Python 3.11+88% coverageMIT63-test matrix

Read the code