Published insights

Web data infrastructure notes from KAIROXO

Field notes on reliable access, automation workflows, public data collection, and the network layer behind AI agents.

aiohttp Proxy: Sessions, Timeouts and Controlled Concurrency

Route asynchronous Python requests through an explicit proxy, keep sessions reusable and distinguish pool waiting from network timeouts.

Normalize public schedules without turning missing hours into closures; preserve timezone, overnight intervals and dated exceptions.

Treat compression as a transport feature, measure the bytes your client exposes and set limits before decoding large responses.

Choose export formats by field shape, stream processing and spreadsheet use; document nulls, identifiers and schema versions.

Keep native fetch, node-fetch and Undici configuration separate, scope the proxy dispatcher and release connections after a job.

Separate in-stock, preorder and unknown states, retain variant and location context, and verify changes before sending alerts.

An IP address and a cookie jar represent different kinds of state. Keep each collection task's locale and browser context explicit.

Resolve relative destinations, limit hops and keep proxy credentials separate from the origin request while recording useful redirect evidence.

Build a documentation corpus that identifies release versions, preserves code blocks and rejects retrieval from the wrong API generation.