7. GridPath RA Toolkit

The GridPath RA Toolkit package generates GridPath scenario inputs for resource-adequacy-style stochastic modeling: temporal scenarios built from weather draws, load profiles, variable generation profiles, weather-dependent generator derates, hydro conditions, and thermal-outage availability iterations — each in synchronous (historical weather years used directly) or Monte Carlo (draws assembled from weather bins) flavors where applicable. It works off its own raw data database (created from ra_toolkit/raw_data_db_schema.sql and loaded with the RA Toolkit’s load_raw_data step), separate from the GridPath Data Toolkit’s raw database, and its steps run from their own settings CSV via the gridpath_run_ra_toolkit command (see open_data_toolkit for the Data Toolkit’s counterpart).

Not to be confused with the GridPath RA Toolkit datasets, developed to support the 2026 Western US resource adequacy study — those are input data (one possible source for this package’s raw tables, provided by the user via the load_raw_data manifest), not this software package.

7.1. Using the GridPath RA Toolkit

RA Toolkit steps run from their own gridpath_run_ra_toolkit command and settings CSV — the RA Toolkit counterpart to the GridPath Data Toolkit’s gridpath_run_data_toolkit (see GridPath Data Toolkit); steps are resolved from the gridpath.ra_toolkit_steps entry-point group. The RA Toolkit works off its own raw data database, separate from the Data Toolkit’s: create it with the create_database step pointing db_schema at ra_toolkit/raw_data_db_schema.sql, then load the RA-related raw data (system load, unit-level variable generation and availability profiles, hydro conditions, unit availability parameters, and the user-defined weather bins and unit mappings) with the RA Toolkit’s load_raw_data step before running the steps below.

Run GridPath RA Toolkit steps per a settings CSV (the gridpath_run_ra_toolkit command). The steps are resolved by name from the ‘gridpath.ra_toolkit_steps’ entry-point group (declared in pyproject.toml), not from a hardcoded import list — see gridpath.step_runner for the generic engine. The Data Toolkit’s steps live in their own entry-point group and are run with the Data Toolkit’s own gridpath_run_data_toolkit command.

Load data into the GridPath RA Toolkit raw data database (a separate database from the GridPath Data Toolkit’s raw database, created from the ra_toolkit/raw_data_db_schema.sql schema). See the documentation of each GridPath RA Toolkit module for data prerequisites. Use the files_to_import.csv file to tell GridPath which CSV files should be loaded into which database table.

7.1.1. What this step does

This module is a generic bulk loader for raw CSV data into the RA Toolkit raw data database. It reads a file named files_to_import.csv located in the directory given by --csv_location. Each row of that file describes one CSV file: an import flag (whether the file should be loaded), the CSV filename (relative to --csv_location), and the database table the file should be loaded into.

The loader iterates over the CSV file rows and, for each row whose import flag is True, reads the corresponding CSV from --csv_location and appends its contents to the named database table (existing rows are preserved; data is inserted with if_exists="append"). Rows whose import flag is False are skipped.

This generic loader is used to populate the raw_data tables (e.g., the historical load, VER, and availability profiles and their unit mappings, hydro operating characteristics) that the RA Toolkit steps depend on.

7.1.2. Usage

>>> python -m ra_toolkit.load_raw_data --database PATH/TO/DATABASE --csv_location PATH/TO/CSV/DIRECTORY

7.1.3. Settings

  • database

  • csv_location

The --csv_location directory must contain a files_to_import.csv manifest with columns for the import flag, the CSV filename, and the destination database table, in that order.

Note that the RA Toolkit steps take their project names from user-provided mapping tables (e.g. raw_data_var_project_units, user_defined_load_zone_units) rather than from the Data Toolkit’s fleet-selection settings. When combining RA Toolkit inputs with Data-Toolkit-generated project inputs in one scenario, those mappings must use the same project and load zone names the Data Toolkit run generated — including the effects of its aggregation and load-zone-level settings.

7.1.4. Temporal Inputs

Monte Carlo Weather Iteration Draws

The Monte Carlo approach employed in the GridPath RA Toolkit study synthesizes multiple years of plausible hourly load, wind availability, solar availability, and temperature-driven thermal derate data over which the system operations can be simulated. Synthetic days are built by combining load, wind, solar, and temperature derate shapes from different but similar days in the historical record. For a detailed description of the methodology, see Appendix B of the report available at https://gridlab.org/wp-content/uploads/2022/10/GridLab_RA-Toolkit-Report-10-12-22.pdf.

Methodology

This module produces, for each Monte Carlo iteration, a full synthetic study-year sequence of weather day bins. A weather day bin is a categorical label (e.g., one of five quintiles per month/day_type) that indicates the historical day’s weather (e.g., based on maximum or average temperature). The bins themselves are produced upstream by the user and are stored in user_defined_weather_bins; this module only resamples them into new synthetic chronologies. The downstream create_monte_carlo_weather_draw_profiles step then maps each drawn bin to an actual historical day’s load/wind/solar/derate shapes.

First-order Markov bin chain

The synthetic bin sequence is generated as a first-order Markov chain over the historical record, in order to preserve realistic day-to-day persistence of weather (e.g., heat waves and cold snaps span multiple days):

  1. The first day of the year (Jan 1) is seeded by drawing uniformly at random from all historical January bins (see starting_weather_bin).

  2. For each subsequent calendar day, given the prior day’s bin b, we look across the historical record for every day in the current month whose bin equals b, take the bin of the day that immediately followed each such day, and draw uniformly at random from that set of “following-day” bins. That draw becomes the current day’s bin and the prior bin for the next step. In effect, step 2 samples from the empirically estimated transition probability P(bin_today | bin_yesterday) for the relevant month.

Reproducibility (seeding)

By default no seed is set (weather_draws_seed defaults to None), so each run produces a different random ensemble of synthetic weather years. To get reproducible draws, pass --weather_draws_seed <int>. The seed actually used is recorded in aux_weather_draws_info alongside the draws.

Other notes

Modeled distributions may deviate from historical averages depending on within-month weather drift. In shoulder seasons, day-to-day directional trends can cause modeled averages to deviate from historical means by ~1–3% (skewing lower in warming months and higher in cooling months).

Usage
>>> gridpath_run_ra_toolkit --single_step create_monte_carlo_weather_draws --settings_csv PATH/TO/SETTINGS/CSV
Input prerequisites
This module assumes the following raw input database tables have been populated:
  • user_defined_weather_bins

Settings
  • database

  • weather_bins_id

  • weather_draws_id

  • weather_draws_seed

  • n_iterations

  • study_year

  • timeseries_iteration_draw_initial_seed

Temporal Scenarios

This is a very basic module that copies over the base CSVs created by the user and calls the temporal iterations method to create the iterations.csv file if needed. The location of the base CSVs and the iterations description CSV are specified in a settings file you can point to with the --csv_path argument.

Usage
>>> gridpath_run_ra_toolkit --single_step create_temporal_scenarios --settings_csv PATH/TO/SETTINGS/CSV
Settings
  • csv_path

7.1.5. Load Inputs

Sync Loads

Create GridPath sync load profile inputs.

Usage
>>> gridpath_run_ra_toolkit --single_step create_sync_load_input_csvs --settings_csv PATH/TO/SETTINGS/CSV
Input prerequisites
This module assumes the following raw input database tables have been populated:
  • raw_data_system_load

  • user_defined_load_zone_units

The load_zone values in user_defined_load_zone_units must use the same zone vocabulary as the rest of the scenario’s inputs — i.e., match the load_zone_level (BA, EIA930 region, or interconnect) used for the eia930_load_zone and project-level steps.

Settings
  • database

  • output_directory

  • load_scenario_id

  • load_scenario_name

  • stage_id

  • study_year

  • overwrite

Timepoint IDs are the hour of the historical year (1 through 8760, or 8784 in a leap year) offset by study_year * 10000, so they start at 1 by default or at YYYY0001 when a study year is provided – the same convention the Monte Carlo steps use. Each historical year becomes a weather iteration.

Monte Carlo Loads

Create GridPath Monte Carlo load profile inputs. Before running this module, you will need to create weather draws with the create_monte_carlo_draws module (see Monte Carlo Weather Iteration Draws).

What this step does

This module reads the synthetic per-iteration load implied by the weather draws produced upstream and writes it out as the GridPath load input CSVs that the model actually consumes. The synthetic per-iteration load is constructed by joining the drawn synthetic weather days (aux_weather_iterations, indexed by weather_bins_id and weather_draws_id) back to the historical hourly load in raw_data_system_load. The resulting CSVs are tagged with the given load / load-components / load-levels scenario ids and names and are written to --output_directory.

Methodology

The module produces up to three CSVs, each of which can be suppressed with a corresponding --skip_* flag:

  • Load scenario CSV (create_load_scenario_csv): written to --output_directory as <load_scenario_id>_<load_scenario_name>.csv. It is a one-row mapping that ties the load_scenario_id to its load_components_scenario_id and load_levels_scenario_id.

  • Load components CSV (create_load_components_scenario_csv): written to the load_components subdirectory as <load_components_scenario_id>_<load_components_scenario_name>.csv. It lists one load_component (named via --load_component, default all) per load_zone found in user_defined_load_zone_units.

  • Load levels CSV (create_load_levels_csv): written to the load_levels subdirectory as <load_levels_scenario_id>_<load_levels_scenario_name>.csv. This file holds the actual hourly load_mw per load_zone, weather_iteration, stage_id, and timepoint.

The load levels are built one synthetic weather iteration at a time. For every drawn day in aux_weather_iterations (matched on weather_bins_id and weather_draws_id), the module identifies the corresponding historical calendar day and, for each load_zone, sums the hourly load from raw_data_system_load across that zone’s constituent unit rows, each scaled by its unit_weight from user_defined_load_zone_units. Timepoint IDs are derived from the draw number and hour_of_day and are offset by --study_year (so they start at 1 by default, or at YYYY0001 when a study year is provided). Rows for successive draws are appended to the single load-levels CSV for the scenario.

These CSVs are the files the GridPath model consumes for load; they are not loaded back into the database by this step.

Overwrite behavior

By default each output file is created only if it does not already exist. The three *_overwrite flags allow the corresponding scenario CSVs to be regenerated in place when they already exist:

  • --load_scenario_overwrite for the load scenario CSV,

  • --load_components_overwrite for the load components CSV, and

  • --load_levels_overwrite for the load levels CSV.

For the load levels CSV, overwrite resets the file (writes a fresh header) only on the first draw of the run and then appends the remaining draws, so the file is rebuilt for the full ensemble rather than truncated mid-write.

Usage
>>> gridpath_run_ra_toolkit --single_step create_monte_carlo_load_input_csvs --settings_csv PATH/TO/SETTINGS/CSV
Input prerequisites
This module assumes the following raw input database tables have been populated:
  • aux_weather_iterations (see the create_monte_carlo_draws step for how to create synthetic weather years and populate this table)

  • raw_data_system_load

  • user_defined_load_zone_units

You must run create_monte_carlo_draws before running this module to populate the database with the raw data and the synthetic weather draws.

Settings
  • database

  • weather_bins_id

  • weather_draws_id

  • output_directory

  • load_scenario_id

  • load_scenario_name

  • load_components_scenario_id

  • load_components_scenario_name

  • load_levels_scenario_id

  • load_levels_scenario_name

  • stage_id

  • study_year

  • load_component

  • load_scenario_overwrite

  • load_components_overwrite

  • load_levels_overwrite

  • skip_load_scenario

  • skip_load_components

  • skip_load_levels

7.1.6. Project Inputs

Machinery shared by the project steps that generate weather- and hydro-dependent inputs — variable generation profiles, generator weather derates, hydro budgets — in either of the two flavors GridPath supports:

  • synchronous (ra_toolkit.project.stochastic.sync_gen_common) — one profile per historical weather year, used directly;

  • Monte Carlo (ra_toolkit.project.stochastic.monte_carlo_gen_common) — draws assembled from weather bins by the temporal weather-draw steps.

ra_toolkit.project.stochastic.iterations writes the per-project iterations CSV that tells GridPath which draws a project’s data varies over.

This is a separate pipeline from the EIA860(M) fleet selection in open_data_toolkit.project.fleet: the two share no code, and a step draws on one or the other, never both. Steps here take their project list from the profiles’ own raw data rather than from the fleet filters.

Variable Gen Profiles (Sync)

Create GridPath sync variable generation profile inputs.

Usage
>>> gridpath_run_ra_toolkit --single_step create_sync_var_gen_input_csvs --settings_csv PATH/TO/SETTINGS/CSV
Input prerequisites
This module assumes the following raw input database tables have been populated:
  • raw_data_var_profiles

  • raw_data_var_project_units

Settings
  • database

  • output_directory

  • variable_generator_profile_scenario_id

  • variable_generator_profile_scenario_name

  • stage_id

  • study_year

  • overwrite

  • n_parallel_projects

Timepoint IDs are the hour of the historical year (1 through 8760, or 8784 in a leap year) offset by study_year * 10000, so they start at 1 by default or at YYYY0001 when a study year is provided – the same convention the Monte Carlo steps use. Each historical year becomes a weather iteration.

Variable Gen Profiles (Monte Carlo)

Create GridPath Monte Carlo variable generation profile inputs. Before running this module,you will need to create weather draws with the create_monte_carlo_draws module (see Monte Carlo Weather Iteration Draws).

What this step does

This is the variable energy resource (VER) counterpart to the load-CSV step. It reads the synthetic per-iteration variable generation profiles – assembled from raw_data_var_profiles (the raw hourly unit-level cap_factor data) and raw_data_var_project_units (the project-to-unit mapping and per-unit weights), resampled according to the weather draws stored in aux_weather_iterations – and writes them out as GridPath variable-generator profile input CSVs in --output_directory, tagged with the given --variable_generator_profile_scenario_id and --variable_generator_profile_scenario_name. These CSVs are the files the GridPath model consumes for variable generation.

Methodology

For each project, the per-unit cap_factor values from raw_data_var_profiles are multiplied by their unit_weight and summed to produce a single project-level cap_factor time series. The weather draws in aux_weather_iterations (selected by --weather_bins_id and --weather_draws_id) determine, for each Monte Carlo weather_iteration and draw_number, which historical day’s data to pull, and the draw number is used to compute the timepoint ID. One output CSV is written per project, named {project}-{scenario_id}-{scenario_name}.csv, with an accompanying iterations CSV written to an iterations subdirectory of --output_directory.

--n_parallel_projects N processes up to N projects concurrently (via a multiprocessing pool over the project pool) to speed things up. --overwrite deletes any existing CSVs with the matching project/scenario filename before writing; without it, output is appended to existing files.

Usage
>>> gridpath_run_ra_toolkit --single_step create_monte_carlo_var_gen_input_csvs --settings_csv PATH/TO/SETTINGS/CSV
Input prerequisites
This module assumes the following raw input database tables have been populated:
  • raw_data_var_profiles

  • raw_data_var_project_units

  • aux_weather_iterations (see the create_monte_carlo_draws step for how to create synthetic weather years and populate this table)

You must run create_monte_carlo_draw_profiles before running this module to populate the database with the raw data and the synthetic weather draws.

Settings
  • database

  • output_directory

  • variable_generator_profile_scenario_id

  • variable_generator_profile_scenario_name

  • overwrite

  • n_parallel_projects

  • weather_bins_id

  • weather_draws_id

Weather Derates (Sync)

Create GridPath sync weather iteration availability inputs.

Usage
>>> gridpath_run_ra_toolkit --single_step create_sync_gen_weather_derate_input_csvs --settings_csv PATH/TO/SETTINGS/CSV
Input prerequisites
This module assumes the following raw input database tables have been populated:
  • raw_data_availability_profiles

  • raw_data_unit_availability_params

Settings
  • database

  • output_directory

  • exogenous_availability_weather_scenario_id

  • exogenous_availability_weather_scenario_name

  • stage_id

  • study_year

  • overwrite

  • n_parallel_projects

  • print_ones

Timepoint IDs are the hour of the historical year (1 through 8760, or 8784 in a leap year) offset by study_year * 10000, so they start at 1 by default or at YYYY0001 when a study year is provided – the same convention the Monte Carlo steps use. Each historical year becomes a weather iteration.

Weather Derates (Monte Carlo)

Create GridPath Monte Carlo weather iteration availability inputs. Before running this module, you will need to create weather draws with the create_monte_carlo_draws module (see Monte Carlo Weather Iteration Draws).

Usage
>>> gridpath_run_ra_toolkit --single_step create_monte_carlo_weather_derate_input_csvs --settings_csv PATH/TO/SETTINGS/CSV
Input prerequisites
This module assumes the following raw input database tables have been populated:
  • raw_data_unit_availability_params

  • raw_data_availability_profiles

  • aux_weather_iterations (see the create_monte_carlo_draws step for how to create synthetic weather years and populate this table)

Settings
  • database

  • output_directory

  • exogenous_availability_weather_scenario_id

  • exogenous_availability_weather_scenario_name

  • overwrite

  • n_parallel_projects

  • weather_bins_id

  • weather_draws_id

Availability Iteration Inputs

Run unit outage simulation and create availability iteration inputs.

What this step does

This module runs a Monte Carlo unit-outage simulation and writes the resulting exogenous availability (derate) input CSVs. Using the per-unit availability parameters loaded from --outage_params_input_csv (into the raw_data_unit_availability_params table) – forced-outage rates (unit_for), mean time to repair (unit_mttr), the number of units (n_units), the unit weight, and the per-unit outage model (unit_fo_model) – it simulates --n_iterations independent outage timelines for each project, drawing random forced and (under the sequential model) repair/maintenance transitions.

The outage model is selected per unit via the unit_fo_model column and may be one of:

  • Derate – a static derate 1 - unit_for applied in every timepoint.

  • MC_independent – each timepoint’s outage state is drawn independently from a uniform distribution against the forced-outage rate.

  • MC_sequential – a sequential (exponential) failure/repair process driven by the forced-outage rate and unit_mttr (the implied mean time to failure is mttr * (1 / for - 1)), preserving outage persistence across timepoints.

  • historical_year – instead of simulating, a random historical year is sampled for the unit from --historical_availability_csv and that year’s hourly derate series is used directly. (This is can be used for units whose availability is taken from a historical record rather than simulated; the choice is driven by the unit’s unit_fo_model value, not by project type.)

For each project the per-unit availability adjustments are combined using each unit’s unit_weight to form a weighted project-level derate. Hybrid-storage projects (hybrid_stor set) additionally get a separately simulated derate for the storage component. By default only rows whose derate differs from 1 are written (as default availability in GridPath is 1); pass --print_ones to retain all rows.

Output is written to --output_directory as one CSV per project, named <project>-<project_availability_scenario_id>-<project_availability_scenario_name>.csv. --n_parallel_projects parallelizes the simulation across projects and --overwrite replaces existing files (otherwise existing files are appended to). --sort re-sorts each output file at the end. These outage iterations are intended to align with the weather/hydro iterations to form complete Monte Carlo draws.

Reproducibility (seeding)

By default seeding is OFF: --user_provided_seeding is not set, so the outage simulation is fully random and non-reproducible from run to run. When seeding is off, all of the seeding flags below are ignored – the seed arguments are replaced with None before the simulation runs, and NumPy’s global RNG is never explicitly seeded.

To get reproducible outages, set --user_provided_seeding together with a --starting_project_iteration_seed <int> (defaults to 0). With seeding on:

  • Per-project, non-overlapping seed ranges. Each project is assigned a starting seed of starting_project_iteration_seed + project_idx * n_iterations. Within a project the per-iteration seed starts at that value and is incremented by 1 for each of the n_iterations iterations, so the seed ranges of distinct projects do not overlap.

  • Per-unit seeds within an iteration. For a given project iteration, the per-iteration seed is used to seed NumPy’s RNG, which then draws one integer seed per unit via np.random.randint(1, max_integer_for_unit_outage_seeding, size=n_units_in_project). Each unit’s outage timeline is then simulated from its own seed. --max_integer_for_unit_outage_seeding defaults to 1000000.

  • Hybrid-storage offset. For hybrid-storage projects, the storage component is simulated with a seed offset from the generator component’s unit seed by --hybrid_storage_seed_increment (defaults to 1000).

Every project / unit / iteration still draws its own independent random outage timeline, but the whole simulation reproduces exactly when re-run with the same seed settings. Again, these flags are ignored unless --user_provided_seeding is set. Caution advised when seeding.

Usage
>>> gridpath_run_ra_toolkit --single_step create_availability_iteration_input_csvs --settings_csv PATH/TO/SETTINGS/CSV
Input prerequisites
This module assumes the following raw input database table has been populated:
  • raw_data_unit_availability_params

This table can be populated ahead of time, or loaded at run time by passing --outage_params_input_csv. Units that use the historical_year outage model additionally read their derate series from the CSV passed via --historical_availability_csv.

Settings
  • database

  • outage_params_input_csv

  • historical_availability_csv

  • stage_id

  • n_iterations

  • study_year

  • project_availability_scenario_id

  • project_availability_scenario_name

  • output_directory

  • overwrite

  • sort

  • print_ones

  • n_parallel_projects

  • user_provided_seeding

  • starting_project_iteration_seed

  • max_integer_for_unit_outage_seeding

  • hybrid_storage_seed_increment

Hydro Gen Inputs

Create hydro iteration input CSVs from year/month data.

Usage
>>> gridpath_run_ra_toolkit --single_step create_hydro_iteration_input_csvs --settings_csv PATH/TO/SETTINGS/CSV
Input prerequisites
This module assumes the following raw input database tables have been populated:
  • raw_data_project_hydro_opchars_by_year_month

  • raw_data_hydro_years

  • user_defined_balancing_type_horizons

Settings
  • database

  • output_directory

  • hydro_operational_chars_scenario_id

  • hydro_operational_chars_scenario_name

  • overwrite

  • n_parallel_projects

What this step does

This module builds GridPath hydro operational-characteristics input CSVs from the year/month hydro data loaded earlier (raw_data_project_hydro_opchars_by_year_month, raw_data_hydro_years, and the user-defined balancing-type horizons in user_defined_balancing_type_horizons). For each hydro iteration it derives the per-horizon hydro operating parameters – the average, minimum, and maximum power fractions – and writes them to --output_directory under the given hydro_operational_chars_scenario_id and hydro_operational_chars_scenario_name. --n_parallel_projects N runs up to N projects at once, and --overwrite replaces existing CSVs.

Methodology

The distinct projects to process are read from raw_data_project_hydro_opchars_by_year_month, and one CSV is written per project, named <project>-<scenario_id>-<scenario_name>.csv in --output_directory. Projects are processed in a multiprocessing pool sized by --n_parallel_projects (defaults to 1).

Hydro iterations and balancing-type horizons

The set of hydro years is read from raw_data_hydro_years and each year is treated as one hydro iteration (written into the hydro_iteration column). The set of (balancing_type, horizon) pairs is read from user_defined_balancing_type_horizons; if --hydro_balancing_type is supplied, the pairs are filtered to that single balancing type (e.g. day, week, month), otherwise all balancing types are included. For every combination of hydro year and balancing-type horizon, one output row is produced.

Deriving per-horizon power fractions

The average_power_fraction, min_power_fraction, and max_power_fraction for each horizon are computed by month-weighting the raw monthly opchar values. For a given balancing-type horizon, the module reads its hour_ending_of_year_start and hour_ending_of_year_end from user_defined_balancing_type_horizons and walks each hour of the year in that range, mapping the hour to a calendar month (via a pandas.Timestamp anchored at January 1 of the hydro year) and counting the number of hours that fall in each month. These hour counts become the per-month weights for the horizon.

For each month touched by the horizon, the module looks up the project’s average_power_fraction, min_power_fraction, and max_power_fraction for that hydro year and month in raw_data_project_hydro_opchars_by_year_month, multiplies each by the month’s hour-count weight, sums across months, and divides by the total number of hours in the horizon. The result is an hours-weighted average of the monthly fractions for each of the three parameters, written as a single row keyed by balancing_type_project and horizon (with weather_iteration set to 0, i.e. no weather iteration). Note we take the weighted averages of the mins and maxes, not the mins of the mins or the maxes of the maxes.

Writing and overwriting output

Rows are appended to the project’s CSV as they are generated, with the header written only when the file does not yet exist. When --overwrite is set, any existing CSV for the project is deleted before processing begins so it is rebuilt from scratch; without --overwrite, new rows are appended to any existing file.

An iterations metadata CSV is also written per project to an iterations/ subdirectory of --output_directory (same filename), recording that hydro operational characteristics vary by hydro iteration but not by weather iteration (varies_by_weather_iteration = 0, varies_by_hydro_iteration = 1). Without --overwrite, an existing iterations CSV is left in place.

If the corresponding --*_input_csv paths are provided, the raw-data tables (raw_data_project_hydro_opchars_by_year_month, raw_data_hydro_years, user_defined_balancing_type_horizons) are loaded from those CSVs before the inputs are built; otherwise the data is assumed to already be present in the database.