Output

What a run writes into output/lczkit/<run_id>/.

units.parquet is the archival record, units_viz.parquet the rounded table a map renders from, and manifest.json everything needed to read either of them and to reproduce the run - including the CRS, which is derived from the extent rather than configured and so is not in config.

units.gpkg is the same unit table in a format whose reader is unconditional. It is written by default and is not the archival record; see OutputConfig.gis_format.

A run directory is written to output/lczkit/<run_id>/ and contains:

Artefact	What it is
units.parquet	the archival table with geometry attached, in the run's projected coordinate system
units.gpkg	the same table as a GeoPackage, written beside it and never instead
units_viz.parquet	the display-ready attribute table the map site reads
manifest.json	the full serialised config, source versions and every report

Why both Parquet and GeoPackage

Every run's GeoParquet is valid 1.0.0 and carries the extent's coordinate reference system with an EPSG authority code. But the Parquet driver in GDAL — the format layer under most geographic software — is an optional build component, so a copy of QGIS built without it opens a correct file as a plain table and reports "this layer has no CRS". GeoPackage is SQLite and is in GDAL's core, so it opens everywhere.

The manifest also carries crs and crs_wkt. The CRS is derived from the extent via estimate_utm_crs(), so it appears in no config and no argument — without these two fields a run directory could state its own CRS only through the file format the reader who needs telling cannot open.

lczkit.output.writer

Writing a run: units.parquet, units_viz.parquet and manifest.json.

Three files, into output/lczkit/<run_id>/ and nowhere else. A run never writes outside its own directory, and the viz table exists so that the map site is a pure transform of run outputs rather than a second analysis.

The split between the two tables is deliberate. units.parquet is the archival record: full precision, geometry attached, every column any stage produced. units_viz.parquet is what a map renders from: no geometry (the tiles carry it), floats rounded to three significant figures, and the seventeen distances stored as scaled integers. The rounding is not cosmetic - at full float64 precision the distance vector alone triples the size of what a browser has to parse, and no choropleth or sidebar reads past the third digit.

MORPHOMETRICS_FILE module-attribute

MORPHOMETRICS_FILE = 'morphometrics.parquet'

The morphometrics output: tessellation cells and their attributes, keyed by their own unit_id. Never merged into units.parquet; it is descriptive, not classifier input.

GPKG_LAYER module-attribute

GPKG_LAYER = 'units'

The GeoPackage copy of the unit table, written when output.gis_format asks for it.

Not a replacement for units.parquet, which stays the archival record and stays what the site build reads. It exists because GeoParquet's reader is conditional in a way its format is not: GDAL's Parquet driver is an optional build component, so a correct file with an EPSG code in its geo metadata can still land in a GIS as a table with no CRS. See OutputConfig.gis_format for the measurement and the cost.

RunOutputs dataclass

RunOutputs(run_dir: Path, units: Path, units_viz: Path, manifest_path: Path, manifest: RunManifest, units_gpkg: Path | None = None, layers: dict[str, Path] = dict(), morphometrics: Path | None = None)

What a run wrote, and the manifest describing it.

units_gpkg class-attribute instance-attribute

units_gpkg: Path | None = None

The GeoPackage copy of the unit table. None where output.gis_format is "none".

layers class-attribute instance-attribute

layers: dict[str, Path] = field(default_factory=dict)

Context layers persisted under layers/, by name. Empty unless the caller asked for them.

morphometrics class-attribute instance-attribute

morphometrics: Path | None = None

morphometrics.parquet. None unless the caller passed a morphometrics table.

write_run

write_run(settings: Settings, units: GeoDataFrame, parameters: DataFrame, classification: DataFrame, classifier: PrototypeClassifier, *, extras: DataFrame | None = None, cleaning: CleaningReport | None = None, extent: ExtentRecord | None = None, units_report: PatchReport | None = None, height_fill: HeightFillReport | None = None, height_source_availability: SourceAvailability | None = None, tag_availability: TagAvailability | None = None, smoothing: SmoothingReport | None = None, validation: AgreementReport | None = None, layers: Mapping[str, GeoDataFrame] | None = None, morphometrics: GeoDataFrame | None = None, morphometrics_report: MorphometricsReport | None = None) -> RunOutputs

Write one run's three files into settings.run_dir, returning their paths.

extras is anything else keyed on unit_id that belongs in the output - the height provenance and the land-cover fractions, typically. It is joined verbatim, so a column appearing in two inputs is an error rather than a silent overwrite.

layers persists the run's context geometry - the cleaned streets, water, land use and buildings - under layers/<name>.parquet. It exists so that the site build is a pure transform of run outputs: without it the only way to draw a basemap or extrude buildings would be to re-read input/ at site-build time, which would make the site depend on data the run does not carry and could not be rebuilt from an archived run directory. Names outside CONTEXT_LAYERS are rejected rather than written.

morphometrics, if given, is written as morphometrics.parquet — its own artefact, keyed by its own unit_id scheme, never joined into attributes/table. See MORPHOMETRICS_FILE.

Nothing outside run_dir is touched, and nothing under input/ is read.

Source code in src/lczkit/output/writer.py

def write_run(
    settings: Settings,
    units: gpd.GeoDataFrame,
    parameters: pd.DataFrame,
    classification: pd.DataFrame,
    classifier: PrototypeClassifier,
    *,
    extras: pd.DataFrame | None = None,
    cleaning: CleaningReport | None = None,
    extent: ExtentRecord | None = None,
    units_report: PatchReport | None = None,
    height_fill: HeightFillReport | None = None,
    height_source_availability: SourceAvailability | None = None,
    tag_availability: TagAvailability | None = None,
    smoothing: SmoothingReport | None = None,
    validation: AgreementReport | None = None,
    layers: Mapping[str, gpd.GeoDataFrame] | None = None,
    morphometrics: gpd.GeoDataFrame | None = None,
    morphometrics_report: MorphometricsReport | None = None,
) -> RunOutputs:
    """Write one run's three files into `settings.run_dir`, returning their paths.

    `extras` is anything else keyed on `unit_id` that belongs in the output - the height
    provenance and the land-cover fractions, typically. It is joined verbatim, so a
    column appearing in two inputs is an error rather than a silent overwrite.

    `layers` persists the run's context geometry - the cleaned streets, water, land use and
    buildings - under `layers/<name>.parquet`. It exists so that **the site build is a pure
    transform of run outputs**: without it the only way to draw a basemap
    or extrude buildings would be to re-read `input/` at site-build time, which would make the site
    depend on data the run does not carry and could not be rebuilt from an archived run directory.
    Names outside `CONTEXT_LAYERS` are rejected rather than written.

    `morphometrics`, if given, is written as `morphometrics.parquet` — its own artefact, keyed by
    its own `unit_id` scheme, never joined into `attributes`/`table`. See `MORPHOMETRICS_FILE`.

    Nothing outside `run_dir` is touched, and nothing under `input/` is read.
    """
    check_units(units)
    unknown = sorted(set(layers or {}) - set(CONTEXT_LAYERS))
    if unknown:
        raise ValueError(
            f"unknown context layers {', '.join(unknown)}; "
            f"write_run persists only {', '.join(CONTEXT_LAYERS)}"
        )
    for name, frame in (("parameters", parameters), ("classification", classification)):
        if not frame.index.equals(units.index):
            raise ValueError(f"{name} index does not match units; both must be the same unit_id")

    attributes = _join(parameters, classification, extras)
    table = gpd.GeoDataFrame(
        attributes.join(units[["geometry"]]), geometry="geometry", crs=units.crs
    )

    continuous = [
        column
        for column in attributes.columns
        if column not in _NOT_CONTINUOUS
        and is_numeric_dtype(attributes[column])
        and not is_bool_dtype(attributes[column])
    ]
    run_dir = settings.run_dir
    run_dir.mkdir(parents=True, exist_ok=True)
    units_path = run_dir / UNITS_FILE
    viz_path = run_dir / VIZ_FILE
    manifest_path = run_dir / MANIFEST_FILE

    written = _write_layers(run_dir, layers)
    gpkg_path = None if settings.output.gis_format == "none" else write_gpkg(run_dir, table)
    morphometrics_path = None
    if morphometrics is not None:
        morphometrics_path = run_dir / MORPHOMETRICS_FILE
        morphometrics.to_parquet(morphometrics_path)
    manifest = build_manifest(
        settings,
        classifier,
        breaks=breaks_for(attributes, continuous, settings.output.break_count),
        classification_summary=classification_summary(classification),
        cleaning=cleaning,
        extent=extent,
        units=units_report,
        height_fill=height_fill,
        # From the joined table rather than from a separate argument: the dispersion is a statistic
        # *of what this run wrote*, and reading the same frame is what guarantees it describes the
        # units on disk rather than an intermediate that no longer matches them.
        height_dispersion=dispersion_report(attributes),
        height_source_availability=height_source_availability,
        tag_availability=tag_availability,
        smoothing=smoothing,
        validation=validation,
        morphometrics=morphometrics_report,
        crs=units.crs,
        outputs=[
            UNITS_FILE,
            VIZ_FILE,
            MANIFEST_FILE,
            *([] if gpkg_path is None else [GPKG_FILE]),
            *([] if morphometrics_path is None else [MORPHOMETRICS_FILE]),
            *(str(path.relative_to(run_dir)) for path in written.values()),
        ],
    )

    table.to_parquet(units_path)
    viz_table(attributes, settings).to_parquet(viz_path)
    manifest_path.write_text(manifest.model_dump_json(indent=2) + "\n", encoding="utf-8")

    return RunOutputs(
        run_dir=run_dir,
        units=units_path,
        units_viz=viz_path,
        manifest_path=manifest_path,
        manifest=manifest,
        units_gpkg=gpkg_path,
        layers=written,
        morphometrics=morphometrics_path,
    )

write_gpkg

write_gpkg(run_dir: Path, table: GeoDataFrame) -> Path

Write the unit table as units.gpkg, returning its path.

unit_id is reset into a column first. A GeoPackage has no index, so leaving it as one would drop the join key every other artefact in the run is keyed on — the single thing that would make this copy useless rather than merely redundant.

Called before the manifest is built, so a failure raises rather than leaving a manifest advertising a file that is not there.

Source code in src/lczkit/output/writer.py

def write_gpkg(run_dir: Path, table: gpd.GeoDataFrame) -> Path:
    """Write the unit table as `units.gpkg`, returning its path.

    `unit_id` is reset into a column first. A GeoPackage has no index, so leaving it as one would
    drop the join key every other artefact in the run is keyed on — the single thing that would
    make this copy useless rather than merely redundant.

    Called before the manifest is built, so a failure raises rather than leaving a manifest
    advertising a file that is not there.
    """
    path = run_dir / GPKG_FILE
    path.unlink(missing_ok=True)
    table.reset_index().to_file(path, driver="GPKG", layer=GPKG_LAYER)
    return path

classification_summary

classification_summary(classification: DataFrame) -> dict[str, Any]

What the classifier did to this city, for the manifest.

The LCZ 10 firing count is the reason this exists. A rule that never fires is indistinguishable, from the output alone, from one that was never configured, and the concern about LCZ 10 is precisely that it can go silently unemitted — so the count belongs in every run's manifest rather than in whatever investigation happens to look for it.

Source code in src/lczkit/output/writer.py

def classification_summary(classification: pd.DataFrame) -> dict[str, Any]:
    """What the classifier did to this city, for the manifest.

    The LCZ 10 firing count is the reason this exists. A rule that never fires is
    indistinguishable, from the output alone, from one that was never configured, and the concern
    about LCZ 10 is precisely that it can go silently unemitted — so the count belongs in
    every run's manifest rather than in whatever investigation happens to look for it.
    """
    labels = classification["lcz_primary"]
    routes = classification["label_route"]
    uniqueness = classification["uniqueness"].dropna()
    counted = labels.dropna().astype("int64").value_counts().sort_index()
    return {
        "n_units": int(len(classification)),
        "n_unlabelled": int(labels.isna().sum()),
        "labels": {str(code): int(count) for code, count in counted.to_dict().items()},
        "label_route": {str(route): int(count) for route, count in routes.value_counts().items()},
        "lcz10_rule_applied": int(classification["lcz10_rule_applied"].sum()),
        "median_uniqueness": float(uniqueness.median()) if not uniqueness.empty else None,
        "median_n_params_used": float(classification["n_params_used"].median()),
    }

viz_table

viz_table(attributes: DataFrame, settings: Settings) -> DataFrame

The rounded, integer-scaled attribute table the map site renders from.

Exposed separately from write_run so the transformation is testable without a filesystem, and so a caller can inspect exactly what a viewer will see.

Source code in src/lczkit/output/writer.py

def viz_table(attributes: pd.DataFrame, settings: Settings) -> pd.DataFrame:
    """The rounded, integer-scaled attribute table the map site renders from.

    Exposed separately from `write_run` so the transformation is testable without a filesystem,
    and so a caller can inspect exactly what a viewer will see.
    """
    digits = settings.output.viz_significant_figures
    scale = settings.output.viz_distance_scale

    result = attributes.copy()
    for column in result.columns:
        if column in DISTANCE_COLUMNS:
            result[column] = _scaled_int(result[column], scale)
        elif is_numeric_dtype(result[column]) and not is_bool_dtype(result[column]):
            if pd.api.types.is_float_dtype(result[column]):
                result[column] = _round_significant(result[column], digits)
    result.index.name = "unit_id"
    return result

Manifest

lczkit.output.manifest

The run manifest: everything needed to read a run's output and to reproduce it.

One JSON file carrying the serialised config, the pinned Overture release, the Earth Engine assets, package versions, the cleaning and height reports, the parameter registry with units and references, and the known limitations, so a run can be read without the code in hand.

TRACKED_PACKAGES module-attribute

TRACKED_PACKAGES: tuple[str, ...] = ('lczkit', 'geopandas', 'shapely', 'pandas', 'numpy', 'pyarrow', 'pyogrio', 'momepy', 'libpysal', 'neatnet', 'geoplanar', 'duckdb', 'exactextract', 'rasterio', 'pydantic', 'earthengine-api')

Packages whose version changes could change a run's numbers. Every one of them performs a geometric or zonal computation whose result this package reports as a measurement.

RunManifest

Bases: BaseModel

One run, described completely enough to be read and repeated.

created_utc instance-attribute

created_utc: str

ISO 8601, UTC, second resolution.

config instance-attribute

config: dict[str, Any]

Settings serialised verbatim.

overture_release instance-attribute

overture_release: str | None

The pinned release the vector layers came from, never "latest".

earth_engine_assets class-attribute instance-attribute

earth_engine_assets: dict[str, dict[str, Any]] = Field(default_factory=dict)

Per land-cover dataset, the collection ID, asset type, band, date range and scale.

parameters class-attribute instance-attribute

parameters: list[dict[str, str]] = Field(default_factory=list)

The parameter registry: every emitted column with its unit, description and source.

not_computed class-attribute instance-attribute

not_computed: dict[str, str] = Field(default_factory=dict)

Stewart & Oke properties this package does not compute, and why.

limitations class-attribute instance-attribute

limitations: dict[str, str] = Field(default_factory=dict)

Known limitations of specific parameters, in the parameters' own terms.

unused_lcz_properties class-attribute instance-attribute

unused_lcz_properties: dict[str, str] = Field(default_factory=dict)

Stewart & Oke properties absent from the distance metric (five of the ten).

unapplied_weights class-attribute instance-attribute

unapplied_weights: list[dict[str, Any]] = Field(default_factory=list)

Published weights for properties this package does not compute: 4.5 of Bernard et al.'s 21.5 under bernard2024_partial.

classification class-attribute instance-attribute

classification: dict[str, Any] = Field(default_factory=dict)

Active weights, normalisation, the full prototype table, every threshold, and which classes could not be assigned.

classification_summary class-attribute instance-attribute

classification_summary: dict[str, Any] = Field(default_factory=dict)

What the classifier did: the label distribution, units per route, and how often the LCZ 10 rule fired, so a rule that never fires is visible from the output.

breaks class-attribute instance-attribute

breaks: list[VariableBreaks] = Field(default_factory=list)

Precomputed classification breaks. The map site reads these and never recomputes a quantile.

extent class-attribute instance-attribute

extent: ExtentRecord | None = None

The ground this run covered and the locator that chose it. Not part of config, since the extent is an argument to run_pipeline. lczkit export backfills older runs from the units' bounds under kind="recovered".

units class-attribute instance-attribute

units: PatchReport | None = None

What the patch merge produced, where units.strategy is "patch" (the request itself is in config).

height_dispersion class-attribute instance-attribute

height_dispersion: DispersionReport | None = None

Within-unit height spread per tier. Hr is a geometric mean, so an areal product that compresses spread biases it upward. See lczkit.heights.dispersion.

tag_availability class-attribute instance-attribute

tag_availability: TagAvailability | None = None

Overture attribute availability by upstream dataset. Read every sem_* column against it: a semantic fraction of 0.0 in an untagged city is not evidence of absence.

smoothing class-attribute instance-attribute

smoothing: SmoothingReport | None = None

What the modal filter did. Written even when the filter is off.

validation class-attribute instance-attribute

validation: AgreementReport | None = None

Agreement against the Demuzere global map: a comparator, not ground truth. Populated only by a caller who validates; run_pipeline does not.

validation_ground_truth class-attribute instance-attribute

validation_ground_truth: AgreementReport | None = None

Agreement against hand-labelled LCZ polygons (So2Sat LCZ42 / DFC2017) where they exist. This is the primary validation figure; validation is secondary.

reference_ceiling class-attribute instance-attribute

reference_ceiling: AgreementReport | None = None

Agreement between the Demuzere map and the labelled polygons on the same units: context for validation, though not a strict bound (a run can beat it).

morphometrics class-attribute instance-attribute

morphometrics: MorphometricsReport | None = None

What the morphometrics stage produced, describing morphometrics.parquet. None where the stage did not run.

morphometrics_raster class-attribute instance-attribute

morphometrics_raster: dict[str, Any] | None = None

The morphometrics raster, if one was written: RasterExportReport.as_manifest(). A dict because refresh_raster patches it into an already-written manifest's JSON.

crs class-attribute instance-attribute

crs: str | None = None

The CRS every geometry in this run is written in, e.g. "EPSG:32618". Derived from the extent, so it is in no config. None where there is no authority code; see crs_wkt.

crs_wkt class-attribute instance-attribute

crs_wkt: str | None = None

The same CRS as WKT2, so it is recoverable when no authority code applies.

outputs class-attribute instance-attribute

outputs: list[str] = Field(default_factory=list)

Files written into the run directory, by name.

package_versions

package_versions() -> dict[str, str]

Resolved version of every tracked package.

An absent one is recorded as such rather than omitted, because "not installed" is itself a fact about the run - earthengine-api missing means the Earth Engine path could not have been used.

Source code in src/lczkit/output/manifest.py

def package_versions() -> dict[str, str]:
    """Resolved version of every tracked package.

    An absent one is recorded as such rather than omitted, because "not installed" is itself a
    fact about the run - `earthengine-api` missing means the Earth Engine path could not have
    been used.
    """
    versions: dict[str, str] = {}
    for name in TRACKED_PACKAGES:
        try:
            versions[name] = importlib.metadata.version(name)
        except importlib.metadata.PackageNotFoundError:
            versions[name] = "not installed"
    return versions

build_manifest

build_manifest(settings: Settings, classifier: PrototypeClassifier, *, breaks: list[VariableBreaks] | None = None, classification_summary: dict[str, Any] | None = None, cleaning: CleaningReport | None = None, extent: ExtentRecord | None = None, units: PatchReport | None = None, height_fill: HeightFillReport | None = None, height_dispersion: DispersionReport | None = None, height_source_availability: SourceAvailability | None = None, tag_availability: TagAvailability | None = None, smoothing: SmoothingReport | None = None, validation: AgreementReport | None = None, validation_ground_truth: AgreementReport | None = None, reference_ceiling: AgreementReport | None = None, morphometrics: MorphometricsReport | None = None, morphometrics_raster: dict[str, Any] | None = None, crs: CRS | None = None, outputs: list[str] | None = None) -> RunManifest

Assemble the manifest for one run. Reports a stage did not produce are left None.

Source code in src/lczkit/output/manifest.py

def build_manifest(
    settings: Settings,
    classifier: PrototypeClassifier,
    *,
    breaks: list[VariableBreaks] | None = None,
    classification_summary: dict[str, Any] | None = None,
    cleaning: CleaningReport | None = None,
    extent: ExtentRecord | None = None,
    units: PatchReport | None = None,
    height_fill: HeightFillReport | None = None,
    height_dispersion: DispersionReport | None = None,
    height_source_availability: SourceAvailability | None = None,
    tag_availability: TagAvailability | None = None,
    smoothing: SmoothingReport | None = None,
    validation: AgreementReport | None = None,
    validation_ground_truth: AgreementReport | None = None,
    reference_ceiling: AgreementReport | None = None,
    morphometrics: MorphometricsReport | None = None,
    morphometrics_raster: dict[str, Any] | None = None,
    crs: CRS | None = None,
    outputs: list[str] | None = None,
) -> RunManifest:
    """Assemble the manifest for one run. Reports a stage did not produce are left `None`."""
    epsg = None if crs is None else crs.to_epsg()
    return RunManifest(
        run_id=settings.run_id,
        created_utc=datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%SZ"),
        config=settings.model_dump(mode="json"),
        versions=package_versions(),
        overture_release=settings.overture.release,
        earth_engine_assets={
            dataset.name: dataset.gee.model_dump(mode="json")
            for dataset in settings.land_cover.datasets
        },
        parameters=[
            {
                "name": parameter.name,
                "label": parameter.label,
                "unit": parameter.unit,
                "description": parameter.description,
                "reference": parameter.reference,
            }
            # The semantic specs come from the configured groups, not from a static
            # list, so a group added in config documents itself here rather than
            # appearing in the output with no unit and no reference.
            for parameter in (
                *PARAMETERS,
                *semantic_specs(settings.ucp.semantic_groups),
                # Only documented when the stage actually ran — otherwise a run that never
                # touched morphometrics would claim 107+ parameters it does not carry.
                *(MORPHOMETRICS_PARAMETERS if morphometrics is not None else ()),
                *(
                    contextual_specs(settings.morphometrics.contextual_quantiles)
                    if morphometrics is not None and morphometrics.contextual_enabled
                    else ()
                ),
            )
        ],
        not_computed=dict(NOT_COMPUTED),
        limitations=dict(LIMITATIONS),
        unused_lcz_properties=dict(UNUSED_PROPERTIES),
        unapplied_weights=[
            {"property": name, "weight": weight, "reason": reason}
            for name, weight, reason in UNAPPLIED_BERNARD_WEIGHTS
        ]
        if classifier.weights.name == BERNARD2024.name
        else [],
        classification=classifier.describe(),
        classification_summary=classification_summary or {},
        legend=legend(),
        breaks=breaks or [],
        extent=extent,
        cleaning=cleaning,
        units=units,
        height_fill=height_fill,
        height_dispersion=height_dispersion,
        height_source_availability=height_source_availability,
        tag_availability=tag_availability,
        smoothing=smoothing,
        validation=validation,
        validation_ground_truth=validation_ground_truth,
        reference_ceiling=reference_ceiling,
        morphometrics=morphometrics,
        morphometrics_raster=morphometrics_raster,
        crs=None if epsg is None else f"EPSG:{epsg}",
        crs_wkt=None if crs is None else crs.to_wkt(),
        outputs=outputs or [],
    )

GIS export

lczkit.output.gis

Making an already-written run openable in a GIS, without re-running it.

write_run emits units.gpkg and records the CRS in the manifest, but every run written before that did neither — and a metropolitan run is ten minutes, so re-running one to change how it is packaged would be the wrong trade. This converts a run directory in place.

It reads only what the run already wrote and adds only what was missing. No parameter is recomputed, no geometry is moved, and nothing that exists is overwritten except a units.gpkg this function itself produced. The manifest gains crs, crs_wkt, an extent and a mention in outputs; every other field is left exactly as the run wrote it, because a manifest is the record of what was measured and this is not a re-measurement.

The extent is reconstructed from the units' own bounds and tagged kind="recovered" rather than presented as what the run was asked for. A reconstruction is bounded by the units that were written, which is not the same rectangle as the window that was requested — a grid overhangs its bbox by up to a cell on each side — and there is nothing on disk that says which city was named. Recording it under a distinct kind is what stops a later reader treating the two as equivalent.

GisExport dataclass

GisExport(run_dir: Path, units_gpkg: Path, crs: str | None, n_units: int, extent: ExtentRecord | None, manifest_updated: bool)

What export_gis found and what it wrote.

crs instance-attribute

crs: str | None

The authority code, or None where the CRS carries none — see RunManifest.crs.

extent instance-attribute

extent: ExtentRecord | None

The window recovered from the units' bounds, or None where the manifest already had one.

Never overwrites a recorded extent: a run that stated where it was knows better than a reconstruction from the units it happened to write.

manifest_updated instance-attribute

manifest_updated: bool

False where the manifest already carried the CRS and the extent, or where there is no manifest.

export_gis

export_gis(run_dir: Path) -> GisExport

Write units.gpkg beside an existing run's units.parquet and record its CRS.

Raises FileNotFoundError naming the path when the run has no units.parquet, because the alternative is a pyarrow error several frames down that does not say which directory was wrong — and the likeliest mistake here is pointing at output/lczkit/ rather than at one run.

Source code in src/lczkit/output/gis.py

def export_gis(run_dir: Path) -> GisExport:
    """Write `units.gpkg` beside an existing run's `units.parquet` and record its CRS.

    Raises `FileNotFoundError` naming the path when the run has no `units.parquet`, because the
    alternative is a `pyarrow` error several frames down that does not say which directory was
    wrong — and the likeliest mistake here is pointing at `output/lczkit/` rather than at one run.
    """
    units_path = run_dir / UNITS_FILE
    if not units_path.exists():
        raise FileNotFoundError(f"no {UNITS_FILE} in {run_dir}; this is not a run directory")

    table = gpd.read_parquet(units_path)
    gpkg_path = write_gpkg(run_dir, table)
    epsg = None if table.crs is None else table.crs.to_epsg()
    crs = None if epsg is None else f"EPSG:{epsg}"
    extent = _recover_extent(table)
    updated, recorded = _backfill_manifest(run_dir, crs, table.crs, extent)
    return GisExport(
        run_dir=run_dir,
        units_gpkg=gpkg_path,
        crs=crs,
        n_units=len(table),
        extent=recorded,
        manifest_updated=updated,
    )

Classification breaks

Precomputed here and written into the manifest, so the map site is a pure transform of run outputs. The site build must never recompute a parameter or a quantile.

lczkit.output.breaks

Classification breaks, precomputed once at run time so the map site never recomputes them.

The static site is a pure transform of run outputs: it never recomputes a parameter or a quantile. Breaks are the only quantity a choropleth needs beyond the values themselves, so computing them here is what makes that possible — and it also means two viewers of the same run see the same class boundaries, which they would not if each rendering derived its own.

Quantiles, not natural breaks. Jenks would mean a mapclassify dependency to save a one-line call. The method name is recorded alongside the values so a consumer is never left inferring it.

VariableBreaks

Bases: BaseModel

Break points for one continuous variable, plus what they were computed from.

method instance-attribute

method: str

Currently always "quantile". Stated rather than assumed.

k instance-attribute

k: int

Number of classes the breaks divide the values into.

breaks instance-attribute

breaks: list[float]

The k + 1 boundaries, ascending, from the minimum to the maximum. Fewer than k + 1 distinct values collapse to however many distinct boundaries exist - a variable that is constant over a run gets a single-element list, which is the truthful answer.

n_valid instance-attribute

n_valid: int

Non-null values the breaks were computed over.

quantile_breaks

quantile_breaks(values: Series, k: int) -> VariableBreaks

k-quantile breaks over the non-null values of values.

An all-null or empty variable returns no breaks rather than raising: a run over a small extent can legitimately produce a parameter nothing measured, and a missing choropleth is a better outcome than a failed run.

Source code in src/lczkit/output/breaks.py

def quantile_breaks(values: pd.Series, k: int) -> VariableBreaks:
    """`k`-quantile breaks over the non-null values of `values`.

    An all-null or empty variable returns no breaks rather than raising: a run over a small extent
    can legitimately produce a parameter nothing measured, and a missing choropleth is a better
    outcome than a failed run.
    """
    if k < 2:
        raise ValueError(f"k must be at least 2, got {k}")
    clean = pd.to_numeric(values, errors="coerce").dropna()
    name = str(values.name)
    if clean.empty:
        return VariableBreaks(
            column=name, method="quantile", k=k, breaks=[], n_valid=0, minimum=None, maximum=None
        )

    array = clean.to_numpy(dtype="float64")
    edges = np.nanquantile(array, np.linspace(0.0, 1.0, k + 1))
    # Duplicate edges are real - a variable that is zero over most of a city has several
    # coincident quantiles - and a renderer given [0, 0, 0, 0.4] would draw three empty classes.
    unique = np.unique(edges)
    return VariableBreaks(
        column=name,
        method="quantile",
        k=k,
        breaks=[float(edge) for edge in unique],
        n_valid=int(clean.size),
        minimum=float(array.min()),
        maximum=float(array.max()),
    )

breaks_for

breaks_for(frame: DataFrame, columns: Iterable[str], k: int) -> list[VariableBreaks]

Breaks for each named column of frame, in the order given.

Source code in src/lczkit/output/breaks.py

def breaks_for(frame: pd.DataFrame, columns: Iterable[str], k: int) -> list[VariableBreaks]:
    """Breaks for each named column of `frame`, in the order given."""
    return [quantile_breaks(frame[column], k) for column in columns]

Extent

What ground a run covered, and how that ground was chosen. Derived rather than configured: the extent is an argument to run_pipeline and reaches no Settings field, so it needs a manifest slot of its own, as the run CRS does.

lczkit.output.extent

What ground a run covered, and how that ground was chosen.

A run directory could not say where it was. Checked across every manifest written before this module existed: no bbox, no place name, no extent of any kind. The reason is structural rather than an oversight — the extent is an argument to run_pipeline, so it is in no Settings field, no preset and no command-line default, and Settings.model_dump() is what the manifest serialises. It is the same shape as the run CRS, and the same rule closes it: a derived property has to be recorded somewhere the derivation is not.

It matters more now that --city reaches 5 558 urban regions rather than 28. Two runs of "Berlin" can legitimately mean the GUPPD urban region or the densest 30 km window of its So2Sat labels; those are different ground, and a bbox alone does not say which was asked for or why.

ExtentKind module-attribute

ExtentKind = Literal['bbox', 'guppd', 'so2sat_window', 'recovered']

How an extent was arrived at.

recovered is reserved for lczkit export, which reconstructs an extent from an archived run's own geometry. It is deliberately a distinct value rather than a best guess at the original: a reconstruction is bounded by the units that were written, not by what was requested, and the two differ wherever the unit grid overhangs or falls short of the window.

ExtentRecord

Bases: BaseModel

The window a run covered, with the locator that produced it.

bbox instance-attribute

bbox: tuple[float, float, float, float]

The window actually run, in lon/lat degrees, after any extent_km shrink.

name class-attribute instance-attribute

name: str | None = None

The place as the gazetteer spells it — "São Paulo", not the query that found it.

query class-attribute instance-attribute

query: str | None = None

What the caller typed, kept beside name because they differ under normalisation and the query is what has to be retyped to reproduce the run.

smod_id class-attribute instance-attribute

smod_id: str | None = None

GUPPD's own identifier for the region. Unambiguous where the name is not — 149 of the 5 558 names are shared — so this is what identifies the extent when the record is read back.

city_key class-attribute instance-attribute

city_key: str | None = None

The lczkit.cities registry key, where the extent came from a So2Sat window.

side_km class-attribute instance-attribute

side_km: float | None = None

Side of the So2Sat search window, in kilometres. 30 for every recorded sweep.

extent_km class-attribute instance-attribute

extent_km: float | None = None

The --extent-km shrink applied, if any. source_bbox is what it was applied to.

source_bbox class-attribute instance-attribute

source_bbox: tuple[float, float, float, float] | None = None

The window before the shrink, so a trimmed trial run still records the whole region it was trimmed from.

area_km2 class-attribute instance-attribute

area_km2: float = Field(default=0.0)

Area of bbox, precomputed so a reader needs no geodesy to size the run.

label property

label: str

A short name for this extent, for progress output and the run's own log line.

model_post_init

model_post_init(_context: object) -> None

Fill area_km2 from bbox, so the two cannot disagree.

Source code in src/lczkit/output/extent.py

def model_post_init(self, _context: object) -> None:
    """Fill `area_km2` from `bbox`, so the two cannot disagree."""
    object.__setattr__(self, "area_km2", bbox_area_km2(tuple(self.bbox)))  # type: ignore[arg-type]

shrunk

shrunk(bbox: BBox, extent_km: float) -> ExtentRecord

The same locator over a concentric extent_km window of it.

The locator is preserved rather than replaced by a bare bbox: a 3 km trial over Cambridge is still a run about Cambridge, and losing that on the way into the manifest is how a directory full of trial runs becomes unreadable.

Source code in src/lczkit/output/extent.py

def shrunk(self, bbox: BBox, extent_km: float) -> ExtentRecord:
    """The same locator over a concentric `extent_km` window of it.

    The locator is preserved rather than replaced by a bare bbox: a 3 km trial over Cambridge
    is still a run about Cambridge, and losing that on the way into the manifest is how a
    directory full of trial runs becomes unreadable.
    """
    return self.model_copy(
        update={
            "bbox": tuple(bbox),
            "source_bbox": tuple(self.source_bbox or self.bbox),
            "extent_km": extent_km,
            "area_km2": bbox_area_km2(bbox),
        }
    )

bbox_area_km2

bbox_area_km2(bbox: BBox) -> float

Roughly how much ground a lon/lat window covers.

A cosine-corrected rectangle, matching lczkit.places.Place.area_km2, because both feed the same decision — whether this is a minutes run or an hours one — and a projected area would disagree with the figure lczkit cities printed for the same place.

Source code in src/lczkit/output/extent.py

def bbox_area_km2(bbox: BBox) -> float:
    """Roughly how much ground a lon/lat window covers.

    A cosine-corrected rectangle, matching `lczkit.places.Place.area_km2`, because both feed the
    same decision — whether this is a minutes run or an hours one — and a projected area would
    disagree with the figure `lczkit cities` printed for the same place.
    """
    west, south, east, north = bbox
    mid = math.radians((south + north) / 2.0)
    return (east - west) * 111.32 * math.cos(mid) * (north - south) * 110.57