Config

The config module loads YAML pipeline configurations that define what data to read, how to aggregate it, and where to write output. See configs/atl06.yaml for the default configuration.

Loading

zagg.config.load_config

load_config(path: str) -> PipelineConfig

Load a YAML config file and return a validated PipelineConfig.

Parameters:

• path (str) –

  Path to YAML file.

Returns:

• PipelineConfig –

zagg.config.load_config_from_dict

load_config_from_dict(d: dict) -> PipelineConfig

Build a PipelineConfig from a plain dict (e.g. Lambda JSON payload).

Float-NaN fill_value declarations are canonicalized to the string "NaN" here (issue #448) -- this is the single funnel every config passes through (load_config, default_config, the client, and the Lambda worker's event["config"]). The scope is deliberate: rewriting every float NaN in the tree would silently mangle authored values. A NaN somewhere else (a filter value, a variable attrs entry) is a config ERROR, not something to canonicalize, and :func:validate_config refuses it -- see :func:_validate_json_floats.

Parameters:

• d (dict) –

  Dictionary with keys data_source, aggregation, output.

Returns:

• PipelineConfig –

zagg.config.default_config

default_config(name: str = 'atl06', *, validate: bool = True) -> PipelineConfig

Load a built-in YAML config shipped with the package.

Parameters:

• name (str, default: 'atl06' ) –

  Config name (without .yaml extension). Default "atl06".

• validate (bool, default: True ) –

  Run validate_config (default). Internal callers loading a packaged config as a structural fallback — e.g. HealpixGrid's implicit config default, where the aggregation block is irrelevant — skip it so template-authoring warnings don't fire on configs the user never wrote (packaged configs are validated by the test suite).

Returns:

• PipelineConfig –

Raises:

• FileNotFoundError –

  If the named config does not exist.

Validation

zagg.config.validate_config

validate_config(config: PipelineConfig) -> None

Cross-validate a PipelineConfig.

Parameters:

• config (PipelineConfig) –

Raises:

• ValueError –

  On any validation failure.

Function Resolution

zagg.config.resolve_function

resolve_function(name: str) -> Callable

Resolve a function name to a callable.

Resolution rules: - "len" or "count" -> builtin len - No dot (e.g. "min") -> np.<name> - Dotted path (e.g. "np.quantile") -> importlib resolution

Parameters:

• name (str) –

  Function name or dotted path.

Returns:

• Callable –

Raises:

• ValueError –

  If the name cannot be resolved to a callable.

zagg.config.evaluate_expression

evaluate_expression(expression: str, columns: dict[str, ndarray]) -> float

Evaluate an expression string in a restricted namespace.

Parameters:

• expression (str) –

  Python expression using numpy and column variables.

• columns (dict[str, ndarray]) –

  Mapping of column names to arrays.

Returns:

• float –

Accessors

zagg.config.get_agg_fields

get_agg_fields(config: PipelineConfig) -> dict

Return aggregation variable metadata keyed by variable name.

Parameters:

• config (PipelineConfig) –

Returns:

• dict –

  {name: {function/expression, source, params, dtype, fill_value, ...}}. A field may also declare a non-scalar output (issue #29) via kind (scalar default, or vector) and trailing_shape; use :func:get_output_signature to read the normalized declaration.

zagg.config.get_coords

get_coords(config: PipelineConfig) -> list[str]

Return coordinate column names from the aggregation config.

Parameters:

• config (PipelineConfig) –

Returns:

• list[str] –

zagg.config.get_data_vars

get_data_vars(config: PipelineConfig) -> list[str]

Return data variable column names from the aggregation config.

Parameters:

• config (PipelineConfig) –

Returns:

• list[str] –

zagg.config.get_child_order

get_child_order(config: PipelineConfig) -> int

Return child_order from the output grid config.

Parameters:

• config (PipelineConfig) –

Returns:

• int –

Raises:

• ValueError –

  If child_order is not set in the config.

zagg.config.get_store_path

get_store_path(config: PipelineConfig) -> str | None

Return the store path from the output config, or None.

Parameters:

• config (PipelineConfig) –

Returns:

• str or None –

Types

zagg.config.PipelineConfig dataclass

Full pipeline configuration.

Parameters:

• data_source (DataSourceDict, default: dict() ) –

  Reader, groups, coordinates, variables, quality filter.

• aggregation (dict, default: dict() ) –

  Coordinate and variable aggregation definitions.

• output (dict, default: dict() ) –

  Grid spec, store path, and indexing details.

• catalog (str or None, default: None ) –

  Optional path to granule catalog JSON.

• bounds (dict or None, default: None ) –

  Optional temporal/spatial bounds for filtering.

• pipeline (dict, default: (lambda: {'type': 'spatial'})() ) –

  Pipeline kind selector (issue #12). {"type": "spatial"} (default) runs the point-cloud->grid aggregation path; "temporal" / "event" route to the event-streaming engines added in later phases. Absent pipeline key in YAML defaults to spatial for backward compatibility with every existing config.

• worker (dict or None, default: None ) –

  Optional Lambda worker-size selector (issue #235): {"memory": 2048|4096|8192, "extra_disk": bool}. The runner resolves it to a pre-provisioned function-name suffix (-<memory>, plus -disk when extra_disk is true) on the lambda backend; an explicit function_name kwarg wins over it. Absent block -> the unsuffixed default function, byte-identical prior behavior. Ignored by the local backend.