aiprojektgaia/gaia_buildings_50e

Model Card for Building Segmentation (FPN / ResNet34)

Binary semantic segmentation model that extracts buildings from aerial imagery. Produces a per-pixel likelihood map, not a thresholded mask.

Trained for GAIA, a self-hosted system that turns aerial imagery into vector map data for Swedish municipalities.

Model Details

Model Description

An FPN (Feature Pyramid Network) segmentation model with a ResNet34 encoder, trained to score each RGB pixel by how likely it is to be building. Built with segmentation_models_pytorch and trained via PyTorch Lightning.

• Developed by: GAIA
• Funded by: Smart Built Environment
• Model type: Semantic segmentation (binary), CNN encoder–decoder (FPN + ResNet34)
• Language(s) (NLP): N/A (vision model)
• License: Apache 2.0
• Finetuned from model: ResNet34 encoder. The training wrapper obtains its normalisation statistics via smp.encoders.get_preprocessing_params("resnet34"), which returns ImageNet values — consistent with an ImageNet-pretrained encoder. Confirm against ckpt["hyper_parameters"].

Model Sources

• Repository: AIProjektGAIA
• Consumed by: the segmentation module of gaia-engine, via ResUNetInferencer

Naming note. GAIA registers this checkpoint under "model type": "resunet" in segmentation/parameters.json. That string selects the loader, not the architecture — the loader passes arch straight from the checkpoint hyperparameters into smp.create_model(). The architecture is whatever the checkpoint declares (FPN), despite the resunet label.

Uses

Direct Use

Building extraction from nadir aerial RGB orthophotos.

The model is resolution-sensitive and expects imagery at approximately 0.05 m/pixel. The GAIA inference wrapper rescales input to that target before inference (resizeImageToTargetResolution), and additionally crops dimensions to a multiple of 32 pixels, as the FPN decoder requires.

In production GAIA requests tiles from a WMS at 0.08 m/pixel and the wrapper upsamples them to 0.05 m/pixel. This is handled explicitly rather than being a silent mismatch, but upsampling cannot recover detail that was never captured — imagery natively at or near 0.05 m/pixel should be expected to perform better.

Output is a single-channel uint8 raster, 0–255, where the value is the sigmoid-scaled likelihood that the pixel is building. It is not a binary mask.

Downstream Use

Within GAIA the likelihood raster feeds a fixed pipeline:

1. Threshold — an operator-chosen confidence level (default 75%) zeroes sub-threshold pixels.
2. Vectorise — buildings use a dedicated vectoriser rather than the generic raster tracer, fitting rectangular primitives (growing_rectangles) so outlines have straight walls and square corners instead of the ragged edges generic tracing produces.
3. Export — polygons are written to a GeoPackage and loaded as QGIS layers.

Keeping the model's output continuous is deliberate: it lets a run be re-thresholded without re-running inference.

Plausible uses beyond GAIA include map updating, building inventory and change detection — none of which have been evaluated here.

Out-of-Scope Use

• Ground-level, oblique, or street-view imagery. Trained on nadir aerial imagery only.
• Building height or 3D reconstruction. This is a 2D binary segmentation model with no elevation output.
• Non-RGB inputs. Fixed at 3 input channels; no NIR, elevation or SAR.
• Authoritative cadastral, property or taxation records. GAIA treats output as a proposal requiring human review, not as a source of truth. See the footprint caveat below.
• Population estimation, damage assessment or disaster response without independent validation — these are consequential uses and nothing here has been evaluated for them.
• Imagery substantially outside the training distribution — different geography, sensor, resolution or season.

Bias, Risks, and Limitations

It segments roofs, not footprints. Nadir imagery sees the roof. For tall buildings, or anywhere the orthorectification is imperfect, the visible roof is displaced from the true ground footprint and overhanging eaves extend beyond it. Output approximates a footprint and should not be treated as a survey-grade one.

Geographic and architectural bias. The model was trained for Swedish municipal use. Roof materials, colours, building density, forms and the sun angles typical of northern latitudes all vary by region; performance elsewhere is unknown and should not be assumed. Dense informal settlements and irregular non-rectangular structures are common failure modes for this model family, and the rectangle-fitting vectoriser downstream compounds that by assuming angular geometry.

Adjoining buildings merge. Terraced housing and connected structures often segment as one region, because the boundary between them is a roof detail rather than a change in material.

Confusable surfaces. Flat grey roofs against adjacent asphalt or paving are a recurring source of both false positives and missed detections. Shadow and overhanging tree canopy remove parts of otherwise clear buildings.

Resolution sensitivity. The model targets 0.05 m/pixel. Imagery at materially different ground sampling distance is rescaled, which degrades quality in both directions.

Unquantified. No evaluation metrics are published with this checkpoint (see Evaluation). Its accuracy on any given imagery is therefore unverified.

Recommendations

• Validate on a held-out sample of your own imagery before operational use.
• Treat output as a proposal for human review, especially for planning, regulatory, property or financial decisions.
• Do not use output as a legal footprint. If a true ground footprint is needed, reconcile against cadastral data or a source with height information.
• Tune the confidence threshold on your own data — the 75% default is a starting point, not a calibrated value.
• Prefer imagery near 0.05 m/pixel where you have a choice.

How to Get Started with the Model

Normalisation is applied inside the training wrapper, not by the bare smp model. Loading the raw state dict as below and feeding it unnormalised images will produce poor results. Either reproduce the wrapper (shown second) or normalise with ImageNet statistics yourself.

import torch
import segmentation_models_pytorch as smp

# weights_only=True is a security control: without it, torch.load unpickles
# arbitrary Python objects and a crafted checkpoint can execute code.
ckpt = torch.load("buildings_100e.ckpt", map_location="cpu", weights_only=True)
hparams = ckpt["hyper_parameters"]

model = smp.create_model(
    arch=hparams["arch"],
    encoder_name=hparams["encoder_name"],
    in_channels=hparams["in_channels"],
    classes=hparams["out_classes"],
)

state_dict = {
    k.replace("model.", "", 1): v
    for k, v in ckpt["state_dict"].items()
    if k.startswith("model.")
}
model.load_state_dict(state_dict)
model.eval()

Reproducing the wrapper, which matches how GAIA runs the model:

import cv2, numpy as np, torch
import pytorch_lightning as pl
import segmentation_models_pytorch as smp

class BuildingSegModel(pl.LightningModule):
    def __init__(self, arch, encoder_name, in_channels, out_classes, **kwargs):
        super().__init__()
        self.save_hyperparameters()
        self.model = smp.create_model(
            arch, encoder_name=encoder_name,
            in_channels=in_channels, classes=out_classes, **kwargs,
        )
        params = smp.encoders.get_preprocessing_params(encoder_name)
        self.register_buffer("std", torch.tensor(params["std"]).view(1, 3, 1, 1))
        self.register_buffer("mean", torch.tensor(params["mean"]).view(1, 3, 1, 1))

    def forward(self, image):
        return self.model((image - self.mean) / self.std)

ckpt = torch.load("buildings_100e.ckpt", map_location="cpu", weights_only=True)
model = BuildingSegModel(**ckpt["hyper_parameters"])
model.load_state_dict(ckpt["state_dict"])
model.eval()

TARGET_RES = 0.05  # m/pixel

def predict(image: np.ndarray, image_resolution: float) -> np.ndarray:
    """RGB HxWx3 uint8 -> HxW uint8 likelihood (0-255)."""
    h, w = image.shape[:2]
    scale = image_resolution / TARGET_RES
    # FPN requires spatial dimensions divisible by 32
    nh, nw = int(h * scale) // 32 * 32, int(w * scale) // 32 * 32
    resized = cv2.resize(image, (nw, nh))

    tensor = torch.tensor(np.array([np.moveaxis(resized, 2, 0)]))
    with torch.no_grad():
        logits = model(tensor)

    pred = logits.sigmoid()[0].numpy().squeeze()
    pred = cv2.resize(pred, (w, h), interpolation=cv2.INTER_NEAREST)
    return np.uint8(255 * pred)

Threshold the result to obtain a binary mask, e.g. pred >= 191 for 75% confidence.

Training Details

Training Data

[More Information Needed]

Not documented in the released artefacts. Required for a complete card: dataset provenance and licence, imagery source and date, ground sampling distance, geographic coverage, annotation guidelines (specifically whether roof outline or ground footprint was annotated, and how attached structures, garages, sheds and canopies were treated), dataset size, and the train/validation/test split with any geographic separation between them.

Check the licence of the source imagery before relying on the Apache-2.0 designation. Models derived from licensed orthophotography may carry obligations from the imagery provider.

Training Procedure

Preprocessing

• Input rescaled to a target of 0.05 m/pixel and cropped to dimensions divisible by 32.
• Per-channel normalisation using ImageNet statistics from smp.encoders.get_preprocessing_params("resnet34"), applied inside the model wrapper.
• Augmentation: [More Information Needed]

Training Hyperparameters

• Architecture: FPN
• Encoder: ResNet34
• Input channels: 3 (RGB)
• Output classes: 1 (binary, sigmoid)
• Epochs: 50 per checkpoint metadata; the filename says 100e — confirm which is correct and correct whichever is wrong
• Steps: 11,250
• Optimizer: AdamW
• LR schedule: Cosine annealing, 2e-4 → 1e-5
• Loss function: [More Information Needed]
• Batch size / tile size: [More Information Needed]
• Training regime: [More Information Needed]

The epoch discrepancy is worth resolving before this card is relied on: 11,250 steps implies roughly 225 batches per epoch at 50 epochs, or about 113 at 100. Either figure is a useful signal of dataset size once the batch size is known. For comparison, the companion roads model ran 3,500 steps over 50 epochs, which suggests a substantially larger training set here — worth confirming rather than assuming.

Speeds, Sizes, Times

• Parameters: ~23.2M
• Checkpoint size: ~278 MB (Lightning .ckpt, includes optimizer state)
• Training time: [More Information Needed]

The published checkpoint retains optimizer state, which roughly triples its size. Stripping it to weights only would reduce the download substantially for inference-only users, at the cost of resumable training.

Evaluation

Testing Data, Factors & Metrics

Testing Data

[More Information Needed]

Factors

Evaluation should be disaggregated by at least: urban / suburban / rural setting; building size and density; roof material and colour; detached versus attached structures; occlusion by shadow or canopy; and imagery source, season and resolution.

Metrics

[More Information Needed]

Recommended for building segmentation: IoU and F1 at the operating threshold, precision/recall across thresholds, and an instance-level measure — per-building detection rate and a count of merged instances. Pixel IoU hides the failure that matters most downstream, where two adjoining buildings become one polygon.

Results

[More Information Needed]

Summary

No quantitative evaluation is published with this checkpoint. Its performance on any given imagery is unverified, and it should be validated before operational use.

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: [More Information Needed]
• Hours used: [More Information Needed]
• Cloud Provider: [More Information Needed]
• Compute Region: [More Information Needed]
• Carbon Emitted: [More Information Needed]

Technical Specifications

Model Architecture and Objective

FPN decoder over a ResNet34 encoder; single-channel output with a sigmoid, trained for binary building segmentation. ~23.2M parameters. Spatial dimensions must be divisible by 32.

Compute Infrastructure

[More Information Needed]

Software

• PyTorch
• PyTorch Lightning 2.5.0
• segmentation_models_pytorch
• OpenCV (inference-time rescaling)

Citation

[More Information Needed]

Glossary

• FPN: Feature Pyramid Network — a decoder that merges multi-scale feature maps, well suited to objects appearing at varying sizes.
• IoU: Intersection over Union — overlap between predicted and ground-truth masks.
• GSD / m/pixel: Ground sampling distance — the ground area a pixel covers. 0.05 m/pixel means one pixel spans 5 cm.
• Nadir imagery: Captured looking straight down, as opposed to oblique.
• Orthophoto: Aerial imagery geometrically corrected so scale is uniform and it can be used as a map.
• Footprint: The outline of a building where it meets the ground — which is not necessarily what is visible from directly above.

More Information

The model is one component of GAIA, which handles tiling, thresholding and vectorisation around it. A companion model covers roads. See the GAIA repositories for the surrounding pipeline.

Model Card Authors

[More Information Needed]

Model Card Contact

[More Information Needed]