ths_disagg_import
Console script for extracting disaggregation realizations from OpenQuake to parquet dataset format.
Either for a given General Task or a single HDF5 file (as used in AWS batch jobs).
Each input HDF5 must contain exactly one site, one IMT and one POE (i.e. a single-row
sitecol and iml_disagg of the form {<imt>: [<iml>]}). Inputs with more than one of
any of these are rejected with a ValueError.
ths_disagg_import¶
Console script for extracting NSHM disaggregation results to parquet dataset format.
Either for a given General Task or a single HDF5 file (as used in runzi AWS batch jobs).
Each input HDF5 must contain exactly one site, one IMT and one POE; inputs with more than one of any of these are rejected.
Usage:
ths_disagg_import [OPTIONS] COMMAND [ARGS]...
Options:
--help Show this message and exit.
extract¶
Extract disaggregation results for the given GT_ID, writing to OUTPUT in parquet format.
Each subtask's HDF5 must contain exactly one site, one IMT and one POE; any HDF5 with more than one of any of these will cause the import to fail with a ValueError.
Arguments:
GT_ID: is an NSHM General task id containing HazardAutomation Tasks
COMPATIBLE_CALC_ID: FK of the compatible calculation.
Usage:
ths_disagg_import extract [OPTIONS] GT_ID COMPATIBLE_CALC_ID
Options:
-W, --work_folder TEXT defaults to current directory
-O, --output TEXT local or S3 target
-P, --probability [_86_PCT_IN_50YRS|_63_PCT_IN_50YRS|_39_PCT_IN_50YRS|_18_PCT_IN_50YRS|_10_PCT_IN_50YRS|_5_PCT_IN_50YRS|_2_PCT_IN_50YRS|_1_PCT_IN_50YRS|_05_PCT_IN_50YRS]
ProbabilityEnum name identifying the target
hazard level. [required]
-K, --kind TEXT Disaggregation kind to extract (must be in
oqparam['disagg_outputs']).
-M, --hazard-model-id TEXT NSHM hazard model identifier e.g.
'NSHM_v1.0.4'. [required]
-A, --target-aggr TEXT Aggregate of the hazard curve the disagg
targets e.g. 'mean', '0.5'. [required]
-v, --verbose
-d, --dry-run
-CID, --partition-by-calc-id
-f64, --use-64bit
-ff, --skip-until-id TEXT
--debug turn on debug logging
--help Show this message and exit.
producers¶
Prepare and validate Producer Configs for a given GT_ID.
GT_ID is an NSHM General task id containing HazardAutomation Tasks\n compatible_calc_fk is the unique key of the compatible_calc
Usage:
ths_disagg_import producers [OPTIONS] GT_ID COMPATIBLE_CALC_FK
Options:
-W, --work_folder TEXT defaults to current directory
-U, --update overwrite existing producer record.
-v, --verbose
--help Show this message and exit.
store-disagg¶
Extract openquake disaggregation results from HDF5_PATH writing to OUTPUT in parquet format.
HDF5_PATH must contain exactly one site, one IMT and one POE; inputs with more than one of any of these are rejected with a ValueError.
Arguments:
HDF5_PATH: path to the disaggregation HDF5 file (exactly one site, one IMT and one POE).
CONFIG_PATH: path to the oq_config.json file.
COMPATIBLE_CALC_ID: FK of the compatible calculation.
HAZARD_CALC_ID: FK of the hazard calculation.
ECR_DIGEST: AWS ECR SHA256 digest of the hazard docker image.
e.g. sha256:db023d95e7ec6707fe3484c7b3c1f8fd4d1c134d5a6d7ec5e939700b625293d9
OUTPUT: path to the output file OR S3 URI.
Usage:
ths_disagg_import store-disagg [OPTIONS] HDF5_PATH CONFIG_PATH
COMPATIBLE_CALC_ID HAZARD_CALC_ID ECR_DIGEST
OUTPUT
Options:
-P, --probability [_86_PCT_IN_50YRS|_63_PCT_IN_50YRS|_39_PCT_IN_50YRS|_18_PCT_IN_50YRS|_10_PCT_IN_50YRS|_5_PCT_IN_50YRS|_2_PCT_IN_50YRS|_1_PCT_IN_50YRS|_05_PCT_IN_50YRS]
ProbabilityEnum name identifying the target
hazard level. [required]
-K, --kind TEXT Disaggregation kind to extract (must be in
oqparam['disagg_outputs']).
-M, --hazard-model-id TEXT NSHM hazard model identifier e.g.
'NSHM_v1.0.4'. [required]
-A, --target-aggr TEXT Aggregate of the hazard curve the disagg
targets e.g. 'mean', '0.5'. [required]
-f64, --use-64bit
--help Show this message and exit.