JSON-LD Extraction: Select Entities and Resolve Page Graphs
Parse multiple JSON-LD blocks, identify the intended entity and preserve relationships instead of collecting unrelated names and values.
KAIROXO
JSON-LD extraction gives a collector access to structured descriptions that a website publishes inside script elements. It can reduce dependence on visual CSS classes, but it does not remove the need to identify the correct entity. A page can describe its publisher, breadcrumbs, the main product and several recommendations in separate or shared blocks. The first object is not automatically the record you want.
For an authorized directory collection, imagine one page describing an organization and its upcoming events. We want the organization name and the events attached to that organization, not every name or date appearing anywhere in the markup. The extraction must preserve relationships as well as values.
Locate scripts in the representation you actually fetched
Look for script elements whose type is application/ld+json. Compare the initial HTML with the rendered DOM when necessary: JavaScript may insert additional blocks after loading. If the required data is already in the initial response, a browser is not needed merely because the format contains the word JavaScript. Navigation requirements may still justify browser use separately.
Keep the original block and source URL alongside extracted fields during validation. Publisher markup can be stale or describe a default selection rather than the visible selected option. The Web Scraper guide stresses checking the intended entity instead of selecting a block solely because its type looks promising. Structured syntax improves accessibility, not truthfulness.
Implementation steps
Parse each block independently
Use a JSON parser on the script's raw text content. Do not evaluate it as JavaScript, fetch remote instructions embedded in its strings, or use a regular expression to parse arbitrary nesting. A malformed block should produce a recorded parse error without necessarily discarding other valid blocks on the same page. Keep a bound on input size and nesting for untrusted pages.
The top-level value may be an object or an array. An object can contain an @graph array, and entities can also be nested under ordinary properties. Preserve the distinction between a list of entities and a list of property values. Flattening every object into one row loses the ownership of fields such as name, URL and startDate.
Select entities with more than a type check
An @type value may be a string or a list. After finding a candidate type, compare its URL, @id and stable source identifier with the target page and collection contract. A related organization or recommended event may have the same type as the main entity. Report ambiguity instead of silently choosing the first match when several candidates remain.
For the directory example, identify the main organization first, then collect only events that belong to it according to the published relationships. Do not join arrays by position: a missing value in one array can pair the wrong organization with the wrong date. Keep a record-level source pointer showing which script and entity supplied each field.
Resolve local references without inventing data
Build a bounded index of @id values found in the page. A property containing only an @id may refer to an object defined elsewhere in another block. Resolve relative identifiers against the effective page URL where the format requires it, but preserve the original value for diagnostics. Handle repeated definitions deliberately rather than overwriting conflicting values unnoticed.
Track visited identifiers during traversal to avoid cycles. A reference to an unknown object is an unresolved reference, not permission to fabricate its missing name or follow arbitrary external URLs. Full JSON-LD expansion has context-processing rules; a lightweight page-local extractor should state its supported subset instead of claiming complete standards conformance.
Preserve value types and missingness
Dates, numeric-looking strings and language-tagged values need explicit handling. Retain an event's timezone when supplied; do not convert a date-only value into an invented midnight timestamp. Keep absent values distinct from null, an empty string or zero according to the destination contract. A display-ready sentence is not always a machine-ready numeric field.
Compare selected fields with visible page content in the same language and state. If the markup describes a default option while the page shows a selected alternative, the discrepancy matters. Extraction and publishing validation are different operations: this guide reads a source's JSON-LD, while KAIROXO's BlogPosting markup describes its own article page.
Test several graph shapes
Use saved fixtures containing multiple script blocks, a top-level array, an @graph, references across blocks and a reference cycle. Add two same-type entities so tests prove that selection follows identity, not document order. Include malformed JSON next to valid JSON, and assert that the error remains visible while valid data is retained.
Also test missing required fields and multilingual text without forcing one language into another. Review character encoding before parsing if raw text is already corrupted. A JSON parser can accept a perfectly valid string containing incorrectly decoded characters, so parsing success alone cannot validate the transport-to-text boundary.