Sources¶
Ingestion. Everything here writes into input/<Source>/ and nowhere else, and nothing here ever
modifies or deletes a file that is already there — that directory is shared with other projects.
There is no separate cache directory: a cache hit is just a file that is already on disk, with
the cache key expressed as the filename.
lczkit.sources.overture
¶
VectorSource backed by DuckDB spatial+httpfs reads of Overture's S3 GeoParquet.
Cached locally, keyed on (release, bbox, layer, query), under
settings.source_dir(settings.overture.source_dir_name). A cache hit never touches DuckDB or
the network — the file being present on disk is the cache.
The query component of that key matters: a cached file's contents depend on which columns
were selected and which rows were filtered, not just on which layer was asked for. Keying on
the layer alone would let a change to a layer's column set silently return a stale frame that
is missing columns the caller now requires.
OvertureSource
¶
OvertureSource(settings: Settings)
Reads the five Overture layers this package ingests, from a pinned release.
buildings, streets, rail, water and land_use.
Pin the release, bind the cache directory, and open a DuckDB spatial connection.
Refuses a release of None rather than falling back to "latest": every manifest
records the release string, and a run against a moving target is not reproducible. The
connection is in-memory — nothing is written outside input/<source_dir_name>/.
Source code in src/lczkit/sources/overture.py
buildings
¶
buildings(bbox: BBox) -> GeoDataFrame
Building footprints intersecting bbox.
Columns: id, height, num_floors, subtype, class, sources.
height and num_floors are nullable and frequently null — that is expected, not an
error. Overture's conflation is winner-takes-all at the geometry level, and height is
parsed only from OSM tags, so footprints won by a machine-learning source carry no
height at all. The height cascade owns that problem; nothing in ingestion or cleaning may
treat a null height as a failure.
subtype/class carry usage type (residential / commercial / industrial) and sources
carries per-feature dataset provenance. Both are retained through cleaning — class is
the only route to LCZ 10, and sources drives the source-availability diagnostic.
Source code in src/lczkit/sources/overture.py
streets
¶
streets(bbox: BBox) -> GeoDataFrame
Road segments intersecting bbox.
subtype = 'road', excluding class = 'service'.
rail
¶
rail(bbox: BBox) -> GeoDataFrame
Rail segments intersecting bbox: subtype = 'rail', no class filter.
Rail is a barrier type for EnclosureUnits, and unlike streets' class != 'service' it
takes no sub-filtering.
Source code in src/lczkit/sources/overture.py
water
¶
water(bbox: BBox) -> tuple[GeoDataFrame, GeoDataFrame]
(waterlines, waterbodies) intersecting bbox.
Excludes underground/aboveground features (Overture's level field, nonzero) and
subtypes human_made, reservoir, spring, wastewater. Note that Overture's own
WaterSubtype value is human_made, with an underscore rather than a hyphen.
Source code in src/lczkit/sources/overture.py
land_use
¶
land_use(bbox: BBox) -> GeoDataFrame
Land-use polygons intersecting bbox. Columns: id, subtype, class.
Functional semantics only — this layer exists to supply the industrial share of a unit's
area, which is industrial_fraction and which the LCZ 8/10 rule reads.
It is not a barrier for spatial-unit generation and not a land-cover source;
rasters own land cover.
Non-polygon features are dropped here rather than in SQL, so the cached file stays the
raw query result — the same split water() performs.
Source code in src/lczkit/sources/overture.py
bbox_key
¶
bbox_key(bbox: BBox) -> str
A stable, filesystem-safe, human-inspectable cache key for a bbox.
Six decimal places (~11cm) — a fixed-precision string, not a hash, so a cache directory shared with other projects stays browsable.
Source code in src/lczkit/sources/overture.py
Areal height products¶
Fetchers for the rasters that tiers 2–4 of the height cascade read. None of these implements
HeightSource — they resolve a path, and ArealRasterTier reads it. Keeping fetch and read
apart is what lets the tier stay offline and testable, and what lets a user who has already
placed a product by hand skip these entirely.
lczkit.sources.height_products
¶
Placing the areal height products that the cascade's tiers 2-4 read.
A caller may place each product as a COG under input/GOB25D/, input/WSF3D/ or input/GHSL/
by hand. Across several cities and three products that is a lot of windows, so these fetchers do
the placing instead, and they own those three
directories the way OvertureSource owns input/Overture_Maps/ — which is the rule the
departure is made under rather than against.
Every fetcher is cache-first: a file already on disk is the answer, and nothing existing is
rewritten, moved or removed. Downloads land on a .partial sibling and are renamed only once
complete, so an interrupted fetch can never be mistaken for a cache hit.
None of these classes implements HeightSource. They resolve a path; ArealRasterTier reads
it. Keeping fetch and read apart is what lets the tier stay offline and testable, and what lets
a user who has placed a product by hand skip these entirely.
MOLLWEIDE_TILE_M
module-attribute
¶
Side of one GHS-BUILT-H R2023A tile, in metres. Tiles are 10000x10000 cells at 100 m.
MOLLWEIDE_ORIGIN_Y
module-attribute
¶
Upper-left corner of the R2023A nominal tile grid, in ESRI:54009 metres.
Not a guess: read off tile R4_C19, bounds (-41000, 5000000, 959000, 6000000), and checked
against five more tiles across four continents.
Tiles are cropped to their valid data extent, so a tile's own bounds are a subset of its
nominal square. R14_C20 is 4000x3000 cells rather than 10000x10000, and R14_C19 is not
published at all — Mollweide puts both partly outside the world ellipse. That is why
_verify_tile_position checks containment rather than equality, and why a missing tile is
treated as "this product has no data here" rather than as an error. An origin wrong by one
whole tile would otherwise return heights from the wrong continent in silence.
HeightProductSource
¶
Bases: Protocol
Resolves one areal height product to a local path, fetching it if it is not there.
bbox is lon/lat, matching every other ingestion boundary in the package. Implementations
return a path readable by lczkit.heights.raster.zonal_mean.
None means the product has no coverage over bbox and the tier should be left out of the
cascade — a shorter cascade with an honest height_completeness, not a failure. Only a
regional product can answer that way; the two global ones raise instead, because for them an
absent tile is a defect rather than a fact about the world.
Wsf3dSource
¶
Wsf3dSource(settings: Settings, config: Wsf3dConfig | None = None)
Tier 3: WSF-3D V02 building height, one global file.
DLR publishes the global product as a tiled GeoTIFF with overviews, so there is nothing to
clip: zonal_mean's covering_window reads a city-sized window straight out of the 2.1 GB
file. That is why ensure ignores its bbox — the same path answers for every city, and a
per-window clip would be a second copy of data already on disk.
Values are int16 decimetres with nodata -32767 (DLR, README_BuildingHeight.txt), which is
why ArealTierConfig for this tier carries scale=0.1.
Bind the fetcher to input/<source_dir_name>/, the directory it owns.
config defaults to settings.height_products.wsf3d; passing one explicitly is how a
test points the fetcher at a fixture without a Settings carrying the real product.
Source code in src/lczkit/sources/height_products.py
name
property
¶
The height_source tag this product writes onto every building it resolves.
ensure
¶
ensure(bbox: BBox) -> Path
Local path to the global WSF-3D file, downloading it once if it is not there.
bbox is ignored, and that is the point: one 2.1 GB tiled GeoTIFF answers for every
city, so there is nothing to clip and a per-window copy would duplicate data already on
disk. It never returns None — WSF-3D is global, so an absent file is a failed download
rather than a fact about the world.
Source code in src/lczkit/sources/height_products.py
GhslBuiltHSource
¶
GhslBuiltHSource(settings: Settings, config: GhslProductConfig | None = None)
Tier 4: GHS-BUILT-H ANBH R2023A, resolved to the Mollweide tiles covering a bbox.
ANBH is BUVOL / BUSURF — building volume over built-up surface, so it is the mean height
of the built fabric in a cell rather than a height smeared across open ground. That is the
right quantity to hand a building, and the reason this reads ANBH and not the gross AGBH
published alongside it. Float32 metres, nodata 255 (GHSL Data Package 2023, p. 36).
Tiles are 1000 km square, so one tile covers a city window in the ordinary case and the fetcher returns it untouched. A window straddling a tile boundary gets a small merged clip instead — bbox-keyed, alongside the tiles it came from.
Bind the fetcher to input/<source_dir_name>/, the directory it owns.
config defaults to settings.height_products.ghsl; passing one explicitly is how a
test points the fetcher at a fixture without a Settings carrying the real product.
Source code in src/lczkit/sources/height_products.py
name
property
¶
The height_source tag this product writes onto every building it resolves.
tiles_for
¶
tiles_for(bbox: BBox) -> list[tuple[int, int]]
The (row, col) tiles covering bbox, in a stable order.
The bbox is reprojected corner-wise and then densified along its edges: Mollweide is not a rectangle-preserving projection, so a lon/lat box maps to a curved quadrilateral whose extreme x can lie partway along an edge rather than at a corner. Sampling the edges is cheap insurance against dropping a tile the window genuinely touches.
Source code in src/lczkit/sources/height_products.py
tile_name
¶
The R2023A name of one nominal tile, e.g. R4_C19, from its 1-based grid position.
Read off config.tile_template rather than formatted here, so the naming scheme stays a
configured property of the product and not a constant in this module.
Source code in src/lczkit/sources/height_products.py
ensure
¶
ensure(bbox: BBox) -> Path
Local path to a raster covering bbox, fetching only the tiles that are missing.
One tile in the ordinary case, returned untouched. A window straddling a tile boundary gets a bbox-keyed merged clip instead, bounded to the window so that answering a 30 km question does not write 800 MB of two whole 1000 km tiles.
Raises rather than returning None when the nominal grid names no published tile: GHS-
BUILT-H is global, so that is a defect in this module's tiling constants rather than an
absence of coverage.
Source code in src/lczkit/sources/height_products.py
OpenBuildings25dSource
¶
OpenBuildings25dSource(settings: Settings, config: OpenBuildings25dConfig | None = None)
Tier 2: Google Open Buildings 2.5D Temporal, exported per window from Earth Engine.
The only fine-resolution tier, and the only one distributed exclusively through Earth Engine
— there is no public bucket. building_height is metres above terrain in [0, 100] at an
effective 4 m (the rasters are served on a 0.5 m grid), annual 2016-2023, covering Africa,
South and South-East Asia, Latin America and the Caribbean. Nothing outside those regions.
Exported in a grid of sub-windows because a 900 km2 city at 4 m is 56 Mpx, over Earth Engine's per-request caps on both pixel count and payload size. The grid is sized from the configured caps rather than fixed, so the failure mode is a smaller request rather than an opaque server error.
Absence is a real answer here: a window with no imagery returns no path, and the caller leaves the tier out of that city's cascade rather than failing the city.
Bind the fetcher to input/<source_dir_name>/ and to an Earth Engine project.
config defaults to settings.height_products.gob25d. The project is read once here
rather than at download time, so a missing GEE_PROJECT_NAME surfaces as one clear
error from ensure instead of an Earth Engine authentication failure.
Source code in src/lczkit/sources/height_products.py
name
property
¶
The height_source tag this product writes onto every building it resolves.
path_for
¶
path_for(bbox: BBox) -> Path
Where this window's export lives, keyed on tier, year and bbox.
The cache key is the filename: a cache hit is just a file that is already there. The year is in it because the collection is annual, and two years over one city are different rasters rather than the same one refetched.
Source code in src/lczkit/sources/height_products.py
sub_windows
¶
bbox split into a grid small enough for one Earth Engine download each.
Sized from the pixel cap, which binds before the byte cap for a float32 band: the grid is the smallest square one for which every cell is under both.
Source code in src/lczkit/sources/height_products.py
ensure
¶
ensure(bbox: BBox) -> Path | None
Local path to a raster covering bbox, exporting it from Earth Engine if absent.
The only tier that reaches the network at read time, and the only one that can honestly
answer None: Open Buildings 2.5D covers Africa, South and South-East Asia, Latin
America and the Caribbean, so an empty collection over Berlin is coverage the product
does not claim rather than a failure. The caller drops the tier from that city's cascade.
The export goes out as a grid of sub-windows sized by sub_windows, staged under a
hidden sibling directory and merged on success, so an interrupted export leaves no file
that a later run could mistake for a cache hit.
Source code in src/lczkit/sources/height_products.py
resolve_areal_tiers
¶
resolve_areal_tiers(settings: Settings, bbox: BBox, config: HeightConfig | None = None) -> tuple[HeightConfig, dict[str, str | None]]
Place every enabled areal tier's product for bbox and return a ready HeightConfig.
The missing half of build_cascade, which reads filename and never sets it. Without this
the only route from a configured tier to a raster on disk was a private helper in one
experiment script, so the package's own default cascade could not actually run — a shipped
default nothing exercises is a claim, not a behaviour.
Tiers run in config.areal_tiers order. A tier with enabled=False is left out, and so is
one whose product has no coverage here — Open Buildings stops at Europe — but for different
reasons that stay separable: the first shows as enabled=False in the serialised config, the
second as a None in the returned record. A tier configured to read a file that is not there
is neither, and still raises in build_cascade.
confidence is deliberately not filled in. It is an ordinal ranking of measurement quality
with no published value behind it, exactly like the two Overture confidences, and a default
cascade that invented one would write a quality claim nobody chose into every manifest. Set
it on the config and build_cascade accepts it; leave it unset and build_cascade says so.
Source code in src/lczkit/sources/height_products.py
Land cover¶
lczkit.sources.worldcover
¶
Placing the ESA WorldCover window a run's land cover is reduced over.
LocalRasterSource reads a COG the caller placed, which is what happens for a run whose
LandCoverDatasetConfig.filename is set. This module is the other half, for a run that names a
bbox and expects the land cover to follow — the same split sources.height_products makes for
tiers 2-4.
Why this is a module and not four lines at a call site. WorldCover ships on a 3-degree grid.
A 30 km window usually lands inside one tile and sometimes spans two or four, and the failure when
it spans two is not an error: clip_raster windows with from_bounds and read(window=...)
returns a smaller array, while LocalRasterSource.fractions turns uncovered units into
all-NaN. Two individually-correct behaviours compose into a map missing a quarter of its land
cover with nothing saying so — and land cover is the sole classifier for LCZ A-G, so a window
that spans two WorldCover tiles and reads only one loses a whole edge of the map.
clip_worldcover therefore reopens what it wrote and raises, naming the short side.
Where it writes. Into the run directory, never into input/. A clip keyed to one run's bbox
is not a source cache, and input/ is shared with other projects.
WORLDCOVER_TILE_DEG
module-attribute
¶
ESA WorldCover v200 ships on a 3-degree grid named for each tile's lower-left corner. Read as
remote COGs over range requests, the way scripts/build_landcover_fixture.py reads them.
worldcover_tiles
¶
worldcover_tiles(bbox: BBox) -> list[str]
Every ESA WorldCover v200 tile URL covering bbox.
A 30 km window is about a quarter of a degree and usually lands inside one 3-degree tile, but nothing makes it do so — a city near a tile corner needs two or four. Returning the list and mosaicking is the only version of this that is correct everywhere, and a single-tile guess would fail as a band of nodata down one side of the map rather than as an error.
Source code in src/lczkit/sources/worldcover.py
clip_worldcover
¶
clip_worldcover(bbox: BBox, destination: Path) -> Path
Mosaic whichever WorldCover tiles bbox spans and write the window into the run dir.