native_field

Stable type · Native arrays and readers

Tier: stable · Kind: type · Topic: Native arrays and readers

Python	Julia	Rust
`NativeField`	`NativeField`	`NativeField`

Python

class NativeField
NativeField(data: Any, dims: Sequence[str], attrs: Optional[Mapping[str, Any]] = None)
NativeField.shape: Tuple[int, ...]  # property
NativeField.dtype: Any  # property

One native-grid array as a reader decodes it.

Parameters

data (Any) — The values. Numeric fields are a numpy.ndarray whose axes correspond, in order, to dims (on-disk dimension names, file order — e.g. ("time", "latitude", "longitude")); text columns are a plain list of str.
dims (Sequence[str]) — Ordered dimension names. Row-major (C order) flattening of data follows this order, matching the corpus encoding.
attrs (Optional[Mapping[str, Any]]) — Decode-relevant metadata the reader must NOT act on but ESS needs — notably a CF time axis’s units/calendar (calendar decoding to wall-clock instants is ESS’s job, never the reader’s).
Per, for, integer

Members

shape — Length of each dimension, in dims order.
dtype — The numpy dtype for a numeric field, or None for a string field.

Source

Example (provider)

source = DataSource(
    "stations",
    "csv",  # the format-registry key that selects the reader
    (FIXTURES / "stations.csv").as_uri(),
    variables=["value", "location_id"],
    reader_kwargs={"numeric_columns": ["latitude", "longitude", "value"]},
)
data = Provider(source, cache).materialize()
assert isinstance(data, NativeDataset)
assert data.variable_names() == ["location_id", "value"]
assert data["value"].data.dtype == np.float64
assert list(data["value"].data) == [152.3, 168.7, 98.1, 110.4]
# Digit-only text columns stay strings: nothing is inferred.
assert data["location_id"].data == ["1", "1", "2", "2"]

From docs/api/examples/python/provider.py

Julia

struct NativeField
NativeField(data::AbstractArray, dims::Vector{String}, attrs::Dict{String, Any})
NativeField(data, dims, attrs)
NativeField(data::AbstractArray, dims::AbstractVector)

NativeField(data, dims, attrs)

One native-grid array exactly as a reader decodes it: data is a Julia array whose axes correspond, in order, to dims (the on-disk dimension names, file order — e.g. ["time","latitude","longitude"]). Numeric fields are Float64 with NaN for masked/_FillValue cells; raw integer reads (e.g. an undecoded time axis) keep their stored integer type; text columns are String. attrs carries decode-relevant metadata the reader must NOT act on but ESS needs — notably a time axis’s units/calendar (calendar decoding is ESS’s job).

Source

Example (provider)

url = "file://" * abspath(joinpath(FIXTURES, "stations.csv"))
p = const_provider(cache, url; format = "csv", source_loader = "stations",
                   variables = ["value", "location_id"],
                   reader_kwargs = (numeric_columns = ["latitude", "longitude", "value"],))
@assert is_const(p) && refresh_times(p) == Float64[]  # a CONST provider never refreshes
data = materialize(p)
@assert data isa NativeDataset
@assert Set(variable_names(data)) == Set(["location_id", "value"])
@assert data["value"].data == [152.3, 168.7, 98.1, 110.4]
# Digit-only text columns stay strings: nothing is inferred.
@assert data["location_id"].data == ["1", "1", "2", "2"]

From docs/api/examples/julia/provider.jl

Rust

pub struct NativeField {
    pub dtype: DType,
    pub dims: Vec<String>,
    pub shape: Vec<usize>,
    pub data: ArrayData,
    pub fill_value: Option<f64>,
}

impl NativeField { pub fn select_leading(&self, i: usize) -> Result<NativeField> }

One native-grid array, keyed by its on-disk file_variable name and validated against spec/schemas/native-field.schema.json. The native grid is the loader’s own grid — regrid is ESD/C4’s job, not the reader’s.

Source · Rust docs