netcdf_reader

Python

class NetCDFReader
NetCDFReader.formats() -> List[str]
NetCDFReader.extensions() -> List[str]
NetCDFReader.open(blob_path: Any) -> Any
NetCDFReader.dim_length(path: Any, dim: str) -> Optional[int]
NetCDFReader.read_native(handle: Any, variables: Optional[Sequence[str]] = None, select: Optional[Any] = None, records: Optional[Any] = None, **_: Any) -> NativeDataset

The active netcdf reader, backed by xarray (netCDF4 engine).

CF-decodes per spec/conformance.md §3, identically to the oracle (conformance/verify.py): opens with decode_times=False and mask_and_scale=True, so scale_factor/add_offset are applied in float64, _FillValue/missing_value cells become NaN, and the time axis is returned raw (its stored integers) with units+calendar carried in attrs for ESS. Data variables land in variables; dimension coordinates (latitude/longitude/time) land in coords.

variables is a projection: an unrequested variable is never read. select is a DECODE-TIME orthogonal selection — the reader still gets the whole blob (one URL, one cache key, nothing about the fetch changes) but materialises only the requested hyperslab. Hence supports_selection = True while store_backed stays absent/False, which is how a caller tells a decode-scoped selection from the zarr reader’s fetch-scoped one. See read_native for the axis rules.

records is RECORD PUSHDOWN, the other half of the refused-time-axis rule: a select refuses a time axis because record selection belongs to whoever owns the cadence, and records is how that owner says which records it wants. See read_native for its semantics and dim_length for the metadata read that makes its indices computable out of process.

Members

• open — Return the blob path as the handle; the dataset is opened in read_native under a with block so nothing leaks.
• dim_length — The length of dimension dim in the blob at path, or None.
• read_native — Decode handle into a NativeDataset .

Source

Julia

struct NetCDFReader <: Reader
NetCDFReader()

NetCDFReader()

The netcdf format reader, backed by NCDatasets. CF-decodes per spec/conformance.md §3: applies scale_factor/add_offset (math in float64), maps _FillValue/missing_value cells to NaN, and returns the time axis raw (the stored integers/floats) with units+calendar carried in attrs — calendar→wall-clock decoding is ESS’s job, never the reader’s.

NCDatasets exposes arrays in column-major (reversed) dimension order; this reader permutes each array back to file order so field.dims and size(field.data) match the on-disk layout (and the Python/xarray track).

TEXT variables (char arrays, NetCDF-4 NC_STRING) are string fields like any other field — projectable, selectable, never skipped (spec/conformance.md §3). NCDatasets returns a char array as a raw Char array of the full on-disk shape, which is not what the bytes mean: the classic-NetCDF convention is that the last dimension is the STRING LENGTH exactly when nothing else claims it as an axis (it has no coordinate variable of its own and every variable using it is a char using it last). So char label(n, strlen) decodes to n strings on dims == ["n"], a 1-D char label(strlen) on a private dimension to a SCALAR string (dims == []), while char label(n) beside a float value(n) stays n one-character strings. Trailing NUL padding is stripped and only trailing NULs — a trailing SPACE is data — and an all-NUL cell is the EMPTY string, never "". This is exactly xarray’s CharacterArrayCoder/conventions.stackable rule, so the three tracks return the same strings for the same bytes.

reader_kwargs: variables=[...] restricts the returned data variables and is a real PROJECTION PUSHDOWN — an unrequested variable is never decoded, rather than decoded and thrown away. Coordinate fields are always returned (they are the native grid the arrays live on). An empty list, like nothing, means every data variable (spec/conformance.md §3), never none; a requested name absent from the blob is an error listing what is present.

select=Dict("axes"=>[...]) is a DECODE-TIME orthogonal selection: the reader still fetches the whole blob (one URL, one cache key — nothing about the fetch or the cache changes), but materialises only the requested hyperslab, reading only the intersecting HDF5 chunks off local disk. supports_selection is true while store_backed stays false, and that pair is how a caller tells a DECODE-scoped selection from the zarr reader’s FETCH-scoped one.

The vocabulary is the zarr reader’s, unchanged and 0-based: each axis is "all", Dict("indices"=>[...]) (an explicit, possibly non-contiguous, ordered index list, returned in the order given) or Dict("slice"=>[start, stop, step?]) (half-open, step defaults to 1). Zero-based is chosen over the AST side’s 1-based convention on purpose: it is ONE select spelling across every reader and both sibling tracks, so a caller that translates for zarr does not translate differently here. Applied thus:

• the axes are positional over FILE-order dims ([time, lev, lat, lon] for a GEOS-FP A3dyn variable, i.e. the order field.dims reports) of every array whose rank equals the axis count — the zarr rule. Rank is the DECODED field’s, so a char label(n, strlen) counts as rank 1: a consumed string length is not an axis, no selector may bind to it, and a selection naming one is an error rather than a silent truncation of every string;
• that induces a dimension-name → selector map, which is then applied by NAME to every other array and to the coordinate fields, so a windowed variable can never come back beside a full-length lon/lat. Two arrays of matching rank that disagree about a dimension are an error, not a coin toss;
• a select whose time axis is anything but "all" is REFUSED: record selection belongs to the Provider, which owns the cadence (records_per_sample), not to the reader (spec/conformance.md §3);
• an axis count matching no array in the blob is an error rather than a silently-ignored selection;
• every resolved index is bounds-checked (0 <= i < dim_len) for a slice exactly as for an indices list: an over-long or negative [start, stop) is an error, never a silent clamp and never a negative wrap-around;
• a dimension is never dropped — a one-index axis comes back at length 1 — and an axis may legally select NOTHING (Dict("indices"=>[]), or an empty half-open Dict("slice"=>[1, 1])), giving a zero-length axis kept in dims rather than an error.

records=Dict("dim"=>"time", "indices"=>[3,4]) is RECORD PUSHDOWN, the other half of the refused-time-axis rule above. select refuses a time axis because record selection belongs to whoever owns the cadence; records is how that owner — the [Provider], which has already turned a cadence tick into this file’s own record index — says which records it wants, in the file’s own 0-based numbering. The reader is told the records, never the cadence: it does no mod1, knows nothing of times or records_per_sample, and simply materialises the named records of dim (and of dim’s own coordinate) in the order given. Duplicates are legal — the end-of-data bracket [last, last] is one — and an index outside 0:len-1, or a dim the blob does not have, is an error rather than a wrapped read. records and a select may be combined; they must not both name dim.

indices may also be a CALLABLE len -> indices, evaluated against dim’s own length inside this reader’s open. That is the in-process spelling only (the wire form is the list), and it exists for one measured reason: the record a cadence owner wants is a function of the file’s length along dim, so with the list form alone the caller has to open the blob once to learn that length and the reader opens it again to read — and on a GEOS-FP A1 file that second open (16.9 ms) costs MORE than the records it saves (21.4 ms whole → 18.5 ms selected). With the callable there is exactly ONE open, and the mod1 still runs in the caller’s own closure. A blob with no dim at all does not call the resolver and makes no selection, which is what a caller that had probed would have found.

Source

Rust

pub struct NetcdfReader;
impl NetcdfReader { pub fn new() -> Self }

The active netcdf reader: pure-Rust NetCDF-3 + NetCDF-4/HDF5 decode.

Source · Rust docs