Proxy Benchmarking: Measure Latency and Useful Success
Compare access profiles using the same targets, bounded concurrency and separate transport, HTTP and extraction outcomes.
KAIROXO
A proxy benchmark should measure whether your workload obtains usable results within its budget. An open port, a successful ping and an HTTP response are useful diagnostics, but none alone establishes that a collection job succeeded. Without a clear denominator and content check, a comparison can reward a route that returns errors quickly.
Start with the business unit you actually need: a validated page, product record or permitted search response. Then measure the network stages that contribute to it. This separates a transport issue from a parser issue and makes the results actionable. The goal is a reproducible decision for your workload, not a universal ranking detached from targets and request context.
Write a fixed test contract
Choose a small representative set of authorized destinations and expected fields. Keep methods, headers, payload sizes, language and session assumptions consistent across candidates. Specify concurrency, timeouts, test duration and retry policy before starting. If one candidate gets extra retries or a different response format, its totals cannot be compared directly with the others.
Use a controlled probe to establish basic routing, then a target-specific sample to establish usefulness. A proxy can pass the first and fail the second. Keep warm-connection and new-connection observations identifiable because connection setup affects latency. If a stable session is required, test that requirement explicitly instead of treating every request as interchangeable.
Record three distinct outcomes
At the transport layer, record whether the connection and request completed within the configured boundaries. At the HTTP layer, record the status and relevant response category. At the content layer, check the expected fields, representation and minimum quality. Keep a reason for rejection so an empty page, wrong market and parser failure do not become the same undifferentiated error.
Maintain both logical-job counts and network-attempt counts. A job that succeeds after a retry consumed more work than a first-attempt success. Record retry count and end-to-end job duration, including waiting and backoff. Avoid exposing credential-bearing URLs or full response bodies in metrics; use a profile identifier and a controlled target category instead.
Use metrics with clear denominators
Metric | Definition | Interpretation |
|---|---|---|
Transport completion | Completed attempts divided by attempts | Describes network-stage completion |
HTTP acceptance | Accepted final HTTP outcomes divided by logical jobs | Uses your documented status policy |
Useful success | Validated final results divided by logical jobs | Reflects the actual collection goal |
End-to-end duration | Queue, attempts, processing and retry delays | Measures the job users wait for |
Cost per usable result | Total relevant cost divided by usable results | Includes failed and repeated work |
For a purely illustrative test of 100 logical jobs, suppose 80 finish with an accepted HTTP status and 72 pass content validation. The two success rates are 80% and 72%, not the same metric. If retries caused 115 network attempts, record that extra work separately. These numbers demonstrate accounting and are not measured performance claims for any provider.
Define what enters the cost numerator: transfer charges, request charges and any compute included in your decision. Use the same definition for all candidates. A low unit price is not automatically a low cost per usable result, but a more expensive route is not automatically justified either. The sample must show a relevant difference for the workload.
Read latency distributions honestly
Report a distribution such as median and tail percentiles rather than only an average. Averages can hide a small group of very slow jobs. Keep sample size and measurement window beside every percentile. Separate connection setup, response progress and complete job duration where the client exposes those observations, and distinguish queue waiting from remote network time.
Do not silently exclude failed or timed-out work from the story. A timeout gives you the elapsed cutoff, not the unknown time a successful response would have required. Show timeout counts and the configured boundary beside latency measurements. A fast successful subset with many failed jobs is not evidence that the overall service meets the workload's latency goal.
Run small, repeatable comparisons
Start at low concurrency and increase gradually within the destination's permitted limits. Repeat comparable windows to reduce the effect of a single transient event, and alternate candidates when possible so time-of-day does not decide the result. Stop if the test creates excessive load or the target communicates a limit. A benchmark should not become an uncontrolled stress test.
After a run, inspect representative failures before changing the network choice. Wrong currency fields may require parser or request-context repair, while pool waits may require local concurrency tuning. Save the non-secret configuration and test manifest so another operator can reproduce the comparison. Publish conclusions only as far as the sample supports, with any missing targets or incomplete checks clearly identified.