DORA

Useful for

Use cases llm-inferred

Check whether an LLM agent can produce building-damage counts and areas from pre- and post-event imagery before relying on it for an emergency-response job · Use the raster and vector tool chain (e.g. ras.vectorize, vec.intersect) as a reference for building footprint and damage-area pipelines · Stress-test agent tool selection and argument grounding on multi-step GIS workflows (average 3.35 to 11.96 steps per task)

Audience llm-inferred

GIS analysts · AI/ML engineers evaluating geospatial agents · emergency-management and disaster-response analysts · flood risk and resilience consultants (indirect use)

Limitations

Constraints llm-inferred

Tasks, source layers and perception checkpoints are not yet released ('coming soon'), so the benchmark data cannot be downloaded · Hugging Face dataset is gated: access form plus a Read token required · Perception tools want a CUDA GPU with 12 GB or more; CPU fallback is slow · Agent runs need third-party LLM API keys; this is an evaluation benchmark and tool library, not a hosted data API · Protocols and data are global disaster-response standards (UNOSAT, Copernicus EMS, FEMA, UN OCHA); no UK-specific coverage · Research code with maintenance risk; v1.1 corrected specs and changed scores, so earlier published numbers may shift · Docs are inconsistent: the tool table lists seg.building_damage but the example trajectory calls seg.building_damage_opt_sar · Pinned environment (Python 3.11, PyTorch 2.11 cu128 wheels) installed from source with pip install -e .

Example tool calls docs-verified

{
  "tool": "ras.vectorize",
  "arguments": {
    "mask_path": "la_palma_volcano_00000516_building_damage.tif",
    "classes": [
      3
    ],
    "pixel_size_m": 0.8
  }
}