Version: dev

ff10_reader

Stable type · Native arrays and readers

Tier: stable · Kind: type · Topic: Native arrays and readers

PythonJuliaRust
FF10ReaderFF10ReaderFf10Reader

Python

class FF10Reader
FF10Reader.formats() -> List[str]
FF10Reader.extensions() -> List[str]
FF10Reader.open(blob_path: Any) -> Any
FF10Reader.read_native(handle: Any, variables: Optional[Sequence[str]] = None, select: Optional[Any] = None, *, member: Optional[str] = None, members: Optional[Sequence[str]] = None, member_glob: Optional[str] = None, skip_header_row: bool = False, kind: str = 'point', numeric_columns: Optional[Sequence[str]] = None, delimiter: str = ',', comment: str = '#', **_: Any) -> NativeDataset

The active ff10 reader — the RAW long-format FF10 point table.

Unlike CSVReader (which only skips empty lines and does not apply a fixed positional schema), this reader (a) skips the leading # comment header block (#FORMAT=…, #COUNTRY, …), (b) applies the fixed 77-column FF10_POINT_COLUMNS schema positionally — FF10 data rows carry no clean header row, so the names come from the schema constant exactly as Emissions.jl supplies them — and (c) does RFC-4180 quote handling (via csv) so a free-text FACILITY_NAME may embed the delimiter.

Each of the 77 columns becomes one NativeField on a single index dim (one index per data row); there are no coordinates (LONGITUDE / LATITUDE are ordinary variables — a points table has no gridded axis). The 42 numeric columns parse to float64 (blank -> NaN); the other 35 stay str (blank -> "").

READER-ONLY (Risk R3): no pollutant pivot (POLID stays a data column, rows are not reshaped), no unit conversion (STKHGT/STKDIAM stay feet, STKTEMP °F, STKFLOW ft³/s, STKVEL ft/s, ANN_VALUE tons/yr), no FIPS/SCC normalization, no EGU/pollutant filter — those move downstream into the .esm.

reader_kwargs: member="path/in/zip" extracts a named member from a .zip blob (the whole zip stays the content-addressed cached blob; the member is reader config so it never enters the cache key). members (an explicit list of member names) and/or member_glob (an fnmatch-style pattern — *, ?, [...], case-sensitive, matched against the full member path, e.g. *egu*) select MULTIPLE members: the selection is the union of the explicit list and the glob matches, deduplicated, and the members are read and their rows concatenated in ascending lexicographic (byte) order of member name. A members name absent from the archive, and a member_glob matching zero members, are errors; directory placeholder entries (names ending in /) are never selected; member (singular) is mutually exclusive with members/member_glob. Like member, none of these enter the cache key. skip_header_row=True handles the EPA 2016fd-style column-header line: after comment (#) and blank lines are dropped, the first remaining line of each selected input (each selected zip member, or the bare file) must be a header row — its first delimiter-separated field, compared case-insensitively, must be country_cd — and exactly that one line is skipped per member; if the first field is anything else the reader errors (the option asserts a header row and never silently drops a data row). kind="point" selects the schema (only point ships). numeric_columns, delimiter, comment override the defaults.

Members

  • read_native — Decode an FF10 point blob into a points NativeDataset .

Source

Julia

struct FF10Reader <: Reader
FF10Reader()
FF10Reader()

The ff10 format reader — the RAW long-format FF10 point table (SMOKE / Emissions.jl FF10_POINT) as a points [NativeDataset] in native units.

Unlike [CSVReader] (which only skips empty lines and splits naively), this reader (a) skips the leading # comment header block (#FORMAT=…, #COUNTRY, …), (b) applies the fixed 77-column [FF10_POINT_COLUMNS] schema — FF10 data rows carry no clean header row, so the names come from the schema constant exactly as Emissions.jl supplies them — and (c) does RFC-4180 quote handling so a free-text FACILITY_NAME may embed the delimiter ("Autauga Plant, Unit 1").

Each of the 77 columns becomes one [NativeField] on a single index dim (one index per data row); there are no coordinates (LONGITUDE/LATITUDE are ordinary variables — a points table has no gridded axis). The 42 numeric columns parse to Float64 (blank → NaN); the other 35 (IDs/codes/free-text) stay String (blank → "").

READER-ONLY (Risk R3): NO pollutant pivot (POLID stays a data column, rows are not reshaped), NO unit conversion (STKHGT/STKDIAM stay feet, STKTEMP °F, STKFLOW ft³/s, STKVEL ft/s, ANN_VALUE tons/yr), NO FIPS/SCC normalization, NO EGU/pollutant filter — those transforms move DOWNSTREAM into the .esm.

reader_kwargs: member="path/in/zip" extracts a named member from a .zip blob (the whole zip stays the cached content-addressed blob; the member is reader config so it never enters the cache key). members=[…] (an explicit list of member names) and/or member_glob="*egu*" (fnmatch-style — *, ?, [...], case-sensitive, matched against the full member path) select MULTIPLE members: the selection is the union of the explicit list and the glob matches, deduplicated, and the members are read and their rows concatenated in ascending lexicographic (byte) order of member name. An explicit name absent from the archive, and a glob matching zero members, are errors; directory placeholder entries (names ending in /) are never selected; member (singular) is mutually exclusive with members/member_glob. None of these enter the cache key. skip_header_row=true handles the EPA 2016fd-style column-header line: after comment (#) and blank lines are dropped, the first remaining line of each selected input (each selected zip member, or the bare file) must be a header row — its first delimiter-separated field, compared case-insensitively, must be country_cd — and exactly that one line is skipped per member; if the first field is anything else the reader errors (the option asserts a header row and never silently drops a data row). kind="point" selects the schema (only point ships). numeric_columns, delimiter, comment override the defaults; variables=[…] restricts the returned columns (default = all 77).

Source

Rust

pub struct Ff10Reader {
    /* private fields */
}

impl Ff10Reader { pub fn new() -> Self }

impl Ff10Reader { pub fn member(self, member: impl Into<String>) -> Self }

impl Ff10Reader { pub fn members<I, S>(self, members: I) -> Self where I: IntoIterator<Item = S>, S: Into<String> }

impl Ff10Reader { pub fn member_glob(self, glob: impl Into<String>) -> Self }

impl Ff10Reader { pub fn skip_header_row(self, yes: bool) -> Self }

impl Ff10Reader { pub fn kind_point(self) -> Self }

The active ff10 reader. member/members/member_glob/ skip_header_row/kind/numeric are carried on the instance (see the module docs on the reader-kwarg asymmetry).

Source · Rust docs