OpenAQ_obs

{
  "kind": "points",
  "source": {
    "url_template": "https://openaq-data-archive.s3.amazonaws.com/records/csv.gz/locationid={location_id}/year={date:%Y}/month={date:%m}/location-{location_id}-{date:%Y%m%d}.csv.gz"
  },
  "temporal": {
    "start": "2016-01-01T00:00:00Z",
    "end": "2099-12-31T23:59:59Z",
    "file_period": "P1D",
    "frequency": "PT1H",
    "records_per_file": "auto",
    "time_variable": "datetime"
  },
  "reference": {
    "citation": "OpenAQ. OpenAQ Data Archive. Open-source global air quality data aggregation platform. https://openaq.org/",
    "url": "https://openaq.org/"
  },
  "metadata": {
    "tags": [
      "air_quality",
      "observations",
      "monitoring_stations",
      "hourly",
      "point_observations"
    ],
    "native_crs": {
      "projection": "longlat",
      "datum": "WGS84",
      "proj4": "+proj=longlat +datum=WGS84 +no_defs",
      "note": "Geographic (unprojected) coordinate reference of the station coordinates. Recorded here for provenance because this points loader omits the formal GDD `grid` block (the native point set is resolved at runtime via station discovery). The regridding model (openaq.esm) selects the longlat IDENTITY reprojection by convention from this projection and bins the stations into target cells with the cell-averaging regridder (ESD regridding/point_cell_average_regrid.esm, esd-47z.4). Migrated from the loader's former DataLoaderSpatial block (crs '+proj=longlat +datum=WGS84 +no_defs', grid_type 'unstructured', staggering lon/lat 'center')."
    },
    "unit_conversion_note": "Raw CSV 'value' is reported in µg/m³ (the per-record source unit is carried in the CSV 'unit' column). The runtime scales each variable to its declared `units` (kg/m³) via unit_scale/to_unit at load time (1 µg/m³ = 1×10⁻⁹ kg/m³). The pre-migration explicit numeric `unit_conversion: 1e-9` factor is dropped: under the ESS DataLoaderVariable schema `unit_conversion` is oneOf[number, Expression] and Expression itself admits a number literal, so a bare numeric factor is ambiguous and rejected at load; the conversion is instead expressed by the target `units` declaration (matching the canonical ESS points-loader fixture tests/fixtures/data_loaders/openaq.esm, which declares target units and relies on runtime unit_scale with no explicit factor).",
    "earthscidata_source": "src/openaq.jl::OpenAQFileSet",
    "s3_mirror": "https://openaq-data-archive.s3.amazonaws.com",
    "api_endpoint": "https://api.openaq.org/v3",
    "station_discovery": "runtime (OpenAQ API v3 /locations endpoint — requires OPENAQ_API_KEY env var or api_key constructor argument; results cached to $EARTHSCIDATADIR/openaq_stations/{parameter}_{bbox}.json; no API key needed for S3 data download)",
    "file_format": "gzip-compressed CSV (.csv.gz)",
    "csv_columns": "location_id,sensor_id,location,datetime,lat,lon,parameter,unit,value",
    "datetime_format": "ISO8601 with timezone offset (e.g. 2024-01-15T12:00:00+00:00); runtime converts all timestamps to UTC before aggregation",
    "valid_parameters": [
      "pm25",
      "pm10",
      "o3",
      "no2",
      "so2",
      "co",
      "bc",
      "pm1",
      "no",
      "nox"
    ],
    "former_regridding": "The pre-migration data_loaders.regridding block declared fill_value 0.0 and extrapolation 'clamp'. fill_value migrates to the model regrid blocks' `missing_value` 0.0 (cell_average kernel); extrapolation 'clamp' has no analogue in the cell-averaging kernel (which fills empty cells with missing_value rather than extrapolating) and is dropped (RFC pure-io-data-loaders §5.2).",
    "location_id_note": "The {location_id} URL substitution is a custom (non-standard) template variable resolved at runtime from the station list returned by station discovery. Each station in the discovered list produces one URL per day in the simulation time span.",
    "data_availability": "Coverage varies by region and time period. Data starting ~2016 in many regions. Missing S3 files (HTTP 404) are treated as no-data (no error); the corresponding station-day contributes zero readings to its grid cell.",
    "merged_from": "components/earthsci_data/openaq_loader.esm",
    "merge_note": "Merged from the standalone pure-I/O file components/earthsci_data/openaq_loader.esm, which is deleted by the same commit. The 0.x `X.esm` + `X_loader.esm` split existed only because a loader was a COMPONENT and a {\"ref\"} target had to contain exactly one component. At esm 1.0.0 a data source is NOT a component, not a coupling endpoint and not a scoped-name path root, and `data_sources` no longer counts toward a file's top-level system count — so one file may hold a model AND the data_sources that model consumes and still be mountable by {\"ref\"}. This registry entry is byte-identical to the one the deleted file declared (it was already mirrored here so this document would validate), and the deleted file's own top-level metadata — name, description, tags, created, authors, references — is preserved verbatim in `merged_file_metadata` so no provenance, citation or migration note is lost.",
    "merged_file_metadata": {
      "name": "OpenAQ_loader",
      "description": "Pure-I/O data loader for OpenAQ ground-based air quality monitoring observations. Provides hourly-averaged ambient concentrations (PM2.5, PM10, O3, NO2, SO2, CO, BC, PM1, NO, NOx) from irregular monitoring stations worldwide, sourced from the OpenAQ AWS S3 archive of daily gzip-compressed per-station CSV files.\n\nThis is the loader half of the openaq loader+regridding-model split (RFC pure-io-data-loaders §4.1/§7; bead esm-3nc.5). It is PURE I/O: it declares its source (url_template), the file variables it reads (with units and unit conversions), and its temporal cadence. It performs NO point-to-grid aggregation and NO regridding — those are the per-variable concern of the regridding model (components/earthsci_data/openaq.esm), which references this file as a subsystem via {\"ref\": \"./openaq_loader.esm\"} and bins the stations into target cells with the cell-averaging regridder.\n\nThis is a `kind: \"points\"` loader: measurements come from irregular stations whose locations are NOT a fixed lattice but are discovered at runtime via the OpenAQ REST API v3 /locations endpoint (bbox-filtered, parameter-filtered; requires OPENAQ_API_KEY; results cached to disk). Because the native point set is resolved at runtime, the loader carries NO GDD `grid` block (per the DataLoader schema: \"any loader whose native grid is resolved at runtime may omit it\"); the stations' geographic (longlat/WGS84) coordinate reference is recorded in this loader's free-form metadata for provenance and to drive the model's identity reprojection by convention.\n\nThe removed co-located blocks: the old DataLoaderSpatial block (crs string / grid_type: unstructured / staggering) is gone — its geographic crs is preserved here in metadata.native_crs, and the unstructured point geometry is resolved at runtime; the old per-loader `regridding` block (fill_value / extrapolation) is gone — regridding is declared per-variable on the model (openaq.esm `regrid`, method `cell_average` with `missing_value`). Because a `temporal` block is present, this loader seeds DISCRETE cadence for its variables (ess-v9a.5): the runtime refreshes the hourly station buffers on schedule.\n\nMigrated from EarthSciData.jl@830435a065f27e99f10bba91107ca1730322cc6b:src/openaq.jl:1-674 (OpenAQFileSet, OpenAQ constructor). Split out of the pre-migration components/earthsci_data/openaq.esm (esm-mfi7).",
      "tags": [
        "earthsci_data",
        "air_quality",
        "observations",
        "monitoring_stations",
        "hourly",
        "data_loader",
        "pure-io",
        "point_observations",
        "longlat"
      ],
      "created": "2026-06-25T00:00:00Z",
      "authors": [
        "EarthSciData.jl contributors",
        "EarthSciModels migration (esm-3nc.5)"
      ],
      "references": [
        {
          "citation": "OpenAQ (2024). OpenAQ Data Archive. Open-source global air quality data aggregation platform. https://openaq.org/",
          "url": "https://openaq.org/"
        },
        {
          "citation": "Source: EarthSciData.jl@830435a065f27e99f10bba91107ca1730322cc6b:src/openaq.jl:1-674 (OpenAQFileSet, OpenAQ constructor). Test: test/openaq_test.jl. Docs: docs/src/openaq.md."
        }
      ]
    }
  }
}