Contributing

Several organisations share this tool. Keep reports comparable: a rule must mean the same thing everywhere, and every rule must point to the SDMx text that justifies it. The target is a direct SDMx 2.1 -> SDMx 3.1 move.

Set-up

uv sync --extra web   # the web extra is needed for tests/test_web.py
uv run pytest

Develop against cached data, so that you do not hammer production services:

uv run sdmx-readiness scan -e spc                                # once, online
uv run sdmx-readiness scan -e spc --from-cache reports/spc/<date> --out /tmp/x   # then offline

Adding an endpoint

  1. Add a block to endpoints.toml. The fields are documented at the top of the file.

  2. Run uv run sdmx-readiness scan -e <name>.

  3. Check fetch_log.csv. Rows marked unsupported or error are classes that were not assessed.

    • 2.1 queries should all be ok or empty.
    • 3.1 queries answer unsupported (406) until the server offers SDMX-ML 3.1.

    If a server uses a status code that we interpret wrongly, fix client.outcome_for and name the server in the comment.

  4. Commit the endpoint block. Committing a report is optional; never commit raw/.

Adding or changing a rule

  1. Find the source, and its strength. Quote the exact text and give its location. Then choose the basis (definitions in sdmx_readiness/rules/engine.py and docs/METHODOLOGY.md §6):

    • SCHEMA if the SDMX-ML 3.1 XSD enforces or states it. Compare tags v2.1 and v3.1.0 of sdmx-twg/sdmx-ml.
    • STANDARD for a MUST/SHOULD in a 3.1 document that the XSD does not enforce.
    • GUIDELINE for non-normative guidance, a MAY, or a consequence you derive.
    • OBSERVED for endpoint behaviour.

  2. Write the rule in the module for its category under sdmx_readiness/rules/:

    @rule(
    id="DSD-011",               # stable: never renumber or reuse a published id
    title="...",                # one line, says what is wrong
    severity=WARNING,           # BLOCKER / WARNING / INFO
    basis=SCHEMA,               # SCHEMA / STANDARD / GUIDELINE / OBSERVED
    change=MECHANICAL,          # mechanical / decision / remodel / none
    category=CATEGORY_DSD,
    applies_to="...",           # which artefacts are examined
    spec="SDMX-ML 3.1 SDMXStructureDataStructure.xsd ...: '<exact quote>'",
    rationale="why it matters for a 2.1 -> 3.1 move",
    fix="what the owner should do",
)
def dsd_something(ctx: Context):
    for a in ctx.primary:
        ...
        yield Finding("DSD-011", WARNING, a, "message naming the offending value")

    Rules read only Artefact / Reference / Inventory, never XML. If you need a new fact from the XML, add it to Artefact.props in parse.py (for DSDs: _dsd_facts) and document the key there.

  3. Test it in tests/test_rules.py or tests/test_structures.py, with one case that fires and the nearest case that must not. The fixture builders are in tests/sdmx_fixtures.py.

  4. Regenerate the docs:

    uv run sdmx-readiness rules --markdown > docs/RULES.md
uv run sdmx-readiness rules --review docs/rules_review.xlsx --evidence spc=reports/spc/<date>/findings.csv ...

  5. Re-run on cached scans and read the new findings yourself before merging.

  6. Get it validated. Add the rule to the table in docs/RULES_REVIEW.md §4. If it rests on an undocumented mapping or a judgement call, add an open question in §5.

Rule id prefixes:

Prefix Category
VER versioning
REF references
MOD artefact-level 2.1 -> 3.1 model changes
DSD DSD structure and modelling guidelines
MSD MSD structure
XF 2.1 vs 3.1 output of the same endpoint

Conventions