Benchmarking

The parts of the service that run on every provide and require, splitting a spec, merging snippets into a bundle, checking compatibility, are micro-benchmarked with Criterion.rs. This page is for a contributor touching that code; an operator sizing an instance needs the Scaling section instead.

Running the benchmarks#

cargo bench

This compiles the project in release mode and runs every benchmark in benches/openapi_bench.rs. Criterion prints the statistics to the terminal and writes an HTML report to target/criterion/report/index.html. One bench binary exists, so cargo bench --bench openapi_bench is the same run by name.

What is measured#

The OpenAPI benchmarks take the service's own api.yaml as their input, so the size of the document grows with the contract; the AsyncAPI and proto benchmarks use small inline fixtures.

Benchmark	Function	Input
`split_openapi`	Splitting an OpenAPI document into one snippet per endpoint.	`api.yaml`
`merge_endpoint_yamls`	Merging endpoint snippets back into one document with shared components, what a bundle require answers.	The snippets of `api.yaml`
`split_asyncapi`	Splitting an AsyncAPI document into one snippet per channel.	Inline fixture
`split_proto`	Splitting a proto file into one snippet per method.	Inline fixture
`normalize_path`	Normalising four endpoint paths (collapsing slashes, unifying path variables), the step that makes `sanshain.yaml` paths match the spec.	Four inline paths
`generate_diff`	A unified diff between two endpoint snippets, what the spec viewer shows.	Two snippets of `api.yaml`
`check_backward_compatibility`	The compatibility check between two OpenAPI documents that decides whether a GA needs a major bump.	`api.yaml` against itself

Last measured results#

Measured on 24 June 2026 against version 1.4.0 of the service on a developer laptop. The seven benchmarks still exist in 2.6.0 (verified on 18 October 2026; the bench file changed once since 1.4.0, for its error handling, not for what it measures), but the numbers were not re-measured and api.yaml has grown since, so treat them as an order of magnitude: the whole spec-processing path stays below a millisecond.

Benchmark	Time
`split_openapi`	~900 µs
`merge_endpoint_yamls`	~850 µs
`split_asyncapi`	~119 µs
`split_proto`	~8 µs
`normalize_path`	~2.5 µs
`generate_diff`	~22 µs
`check_backward_compatibility`	~913 µs

Sub-millisecond processing means the version rules and the compatibility check never dominate a provide; the round trip to the database and the network do.

Comparing before and after a change#

Run the benchmarks once before touching the code, so Criterion stores a baseline, and again after. The second run reports the change of every benchmark against the stored baseline as a percentage with its confidence interval, and names it an improvement or a regression when the difference is significant.

cargo bench --bench openapi_bench   # baseline
# … change the code …
cargo bench --bench openapi_bench   # compared against the baseline

Checked against v2.6.0.