ff10_reader
Stable type · Native arrays and readers
Tier: stable · Kind: type · Topic: Native arrays and readers
| Python | Julia | Rust |
|---|---|---|
FF10Reader | FF10Reader | Ff10Reader |
Python
class FF10Reader
FF10Reader.formats() -> List[str]
FF10Reader.extensions() -> List[str]
FF10Reader.open(blob_path: Any) -> Any
FF10Reader.read_native(handle: Any, variables: Optional[Sequence[str]] = None, select: Optional[Any] = None, *, member: Optional[str] = None, members: Optional[Sequence[str]] = None, member_glob: Optional[str] = None, skip_header_row: bool = False, kind: str = 'point', numeric_columns: Optional[Sequence[str]] = None, delimiter: str = ',', comment: str = '#', **_: Any) -> NativeDataset
The active ff10 reader — the RAW long-format FF10 point table.
Unlike CSVReader
(which only skips empty lines and does not apply a
fixed positional schema), this reader (a) skips the leading # comment
header block (#FORMAT=…, #COUNTRY, …), (b) applies the fixed 77-column
FF10_POINT_COLUMNS schema positionally — FF10 data rows carry no clean
header row, so the names come from the schema constant exactly as Emissions.jl
supplies them — and (c) does RFC-4180 quote handling (via csv) so a
free-text FACILITY_NAME may embed the delimiter.
Each of the 77 columns becomes one NativeField
on a single index
dim (one index per data row); there are no coordinates (LONGITUDE /
LATITUDE are ordinary variables — a points table has no gridded axis). The
42 numeric columns parse to float64 (blank -> NaN); the other 35 stay
str (blank -> "").
READER-ONLY (Risk R3): no pollutant pivot (POLID stays a data column, rows are
not reshaped), no unit conversion (STKHGT/STKDIAM stay feet, STKTEMP °F,
STKFLOW ft³/s, STKVEL ft/s, ANN_VALUE tons/yr), no FIPS/SCC normalization, no
EGU/pollutant filter — those move downstream into the .esm.
reader_kwargs: member="path/in/zip" extracts a named member from a
.zip blob (the whole zip stays the content-addressed cached blob; the
member is reader config so it never enters the cache key). members (an
explicit list of member names) and/or member_glob (an fnmatch-style
pattern — *, ?, [...], case-sensitive, matched against the full
member path, e.g. *egu*) select MULTIPLE members: the selection is the
union of the explicit list and the glob matches, deduplicated, and the
members are read and their rows concatenated in ascending lexicographic
(byte) order of member name. A members name absent from the archive, and
a member_glob matching zero members, are errors; directory placeholder
entries (names ending in /) are never selected; member (singular)
is mutually exclusive with members/member_glob. Like member,
none of these enter the cache key. skip_header_row=True handles the EPA
2016fd-style column-header line: after comment (#) and blank lines are
dropped, the first remaining line of each selected input (each selected zip
member, or the bare file) must be a header row — its first
delimiter-separated field, compared case-insensitively, must be
country_cd — and exactly that one line is skipped per member; if the
first field is anything else the reader errors (the option asserts a header
row and never silently drops a data row). kind="point" selects the
schema (only point ships). numeric_columns, delimiter, comment
override the defaults.
Members
read_native— Decode an FF10 point blob into apointsNativeDataset.
Julia
struct FF10Reader <: Reader
FF10Reader()
FF10Reader()
The ff10 format reader — the RAW long-format FF10 point table (SMOKE /
Emissions.jl FF10_POINT) as a points [NativeDataset] in native units.
Unlike [CSVReader] (which only skips empty lines and splits naively), this
reader (a) skips the leading # comment header block (#FORMAT=…, #COUNTRY,
…), (b) applies the fixed 77-column [FF10_POINT_COLUMNS] schema — FF10 data
rows carry no clean header row, so the names come from the schema constant
exactly as Emissions.jl supplies them — and (c) does RFC-4180 quote handling so a
free-text FACILITY_NAME may embed the delimiter ("Autauga Plant, Unit 1").
Each of the 77 columns becomes one [NativeField] on a single index dim (one
index per data row); there are no coordinates (LONGITUDE/LATITUDE are ordinary
variables — a points table has no gridded axis). The 42 numeric columns parse to
Float64 (blank → NaN); the other 35 (IDs/codes/free-text) stay String
(blank → "").
READER-ONLY (Risk R3): NO pollutant pivot (POLID stays a data column, rows are
not reshaped), NO unit conversion (STKHGT/STKDIAM stay feet, STKTEMP °F,
STKFLOW ft³/s, STKVEL ft/s, ANN_VALUE tons/yr), NO FIPS/SCC normalization,
NO EGU/pollutant filter — those transforms move DOWNSTREAM into the .esm.
reader_kwargs: member="path/in/zip" extracts a named member from a .zip
blob (the whole zip stays the cached content-addressed blob; the member is reader
config so it never enters the cache key). members=[…] (an explicit list of
member names) and/or member_glob="*egu*" (fnmatch-style — *, ?, [...],
case-sensitive, matched against the full member path) select MULTIPLE members:
the selection is the union of the explicit list and the glob matches,
deduplicated, and the members are read and their rows concatenated in ascending
lexicographic (byte) order of member name. An explicit name absent from the
archive, and a glob matching zero members, are errors; directory placeholder
entries (names ending in /) are never selected; member (singular) is
mutually exclusive with members/member_glob. None of these enter the cache
key. skip_header_row=true handles the EPA 2016fd-style column-header line:
after comment (#) and blank lines are dropped, the first remaining line of
each selected input (each selected zip member, or the bare file) must be a
header row — its first delimiter-separated field, compared case-insensitively,
must be country_cd — and exactly that one line is skipped per member; if the
first field is anything else the reader errors (the option asserts a header row
and never silently drops a data row). kind="point" selects the schema (only
point ships). numeric_columns, delimiter, comment override the defaults;
variables=[…] restricts the returned columns (default = all 77).
Rust
pub struct Ff10Reader {
/* private fields */
}
impl Ff10Reader { pub fn new() -> Self }
impl Ff10Reader { pub fn member(self, member: impl Into<String>) -> Self }
impl Ff10Reader { pub fn members<I, S>(self, members: I) -> Self where I: IntoIterator<Item = S>, S: Into<String> }
impl Ff10Reader { pub fn member_glob(self, glob: impl Into<String>) -> Self }
impl Ff10Reader { pub fn skip_header_row(self, yes: bool) -> Self }
impl Ff10Reader { pub fn kind_point(self) -> Self }
The active ff10 reader. member/members/member_glob/
skip_header_row/kind/numeric are carried on the instance (see the
module docs on the reader-kwarg asymmetry).