Node Fetch Proxy: Use an Undici Dispatcher Correctly

Implementation steps

1. 1

   Identify the client and supported runtime

   Record the Node release, package version and actual import. Native fetch is part of the runtime; importing fetch from undici selects the installed package's implementation. Importing from node-fetch selects a different library that uses its documented agent interface. Do not treat an agent and a dispatcher as synonyms.

   For the example below, install an Undici release compatible with your Node version and use fetch and ProxyAgent from the same package. Keep the lockfile with your application. Some Node versions support environment-based proxy configuration, but its availability and activation rules are version-specific. Explicit dispatchers make the route easier to review without changing every HTTP call in the process.

2. 2

   Give the dispatcher a clear owner

   A dispatcher owns reusable transport resources. For one small standalone job, create it at job startup and close it in finally. In a long-running worker, reuse one dispatcher across the requests that intentionally share the same proxy profile, and close it during orderly shutdown. Creating one per URL throws away pooling advantages.

   import { fetch, ProxyAgent } from 'undici'; export async function readThroughProxy(url, settings) {  const dispatcher = new ProxyAgent({    uri: settings.proxyOrigin,    token: 'Basic ' + Buffer.from(settings.username + ':' + settings.password).toString('base64'),  });  try {    const response = await fetch(url, {      dispatcher, redirect: 'manual', signal: AbortSignal.timeout(30_000),    });    if (response.status !== 200) {      await response.body?.cancel();      throw new Error('unexpected status ' + response.status);    }    const chunks = [];    let bytes = 0;    for await (const chunk of response.body) {      bytes += chunk.byteLength;      if (bytes > 2_000_000) throw new Error('decoded response exceeds budget');      chunks.push(chunk);    }    return Buffer.concat(chunks);  } finally {    await dispatcher.close();  }}

   The example receives settings from its caller rather than embedding credentials. The caller can supply proxyOrigin, username and password from its existing configuration system. Basic authentication is encoded for transport, not encrypted by base64. Keep credentials out of URLs written to logs, and use the gateway protocol documented for the selected service.

3. 3

   Keep proxy and destination authentication separate

   The ProxyAgent token configures authentication to the proxy. It should not become an Authorization header on the destination request. A website may have its own authentication requirements, but those require separate authority and scope. For public collection, do not import browser credentials merely to make a failed request succeed.

   The example keeps redirects manual, applies an overall abort deadline and accepts only status 200. That policy is intentionally narrow. A real job may handle other responses, but it should name them explicitly. A 407 suggests a proxy authentication issue; a 401 or 403 may belong to a different layer. Retrying every code identically hides that distinction and can waste a request budget.

4. 4

   Verify routing and consume the response

   Begin with a controlled endpoint that reports its observed peer, then request one approved target page. Check both the requested and final destination, response type and a small known field. A successful TCP tunnel alone does not prove that the returned document is the expected page. A proxy check endpoint also cannot promise identical behavior at every destination.

   Read or cancel each body so connections can be reused cleanly. The example limits exposed body bytes and closes the dispatcher in all cases. It does not implement disk streaming, a retry queue or a full extraction pipeline. If your job needs those, add them around the transport with their own resource limits and test fixtures.