7. GridPath RA Toolkit
The GridPath RA Toolkit package generates GridPath scenario inputs for
resource-adequacy-style stochastic modeling: temporal scenarios built from
weather draws, load profiles, variable generation profiles,
weather-dependent generator derates, hydro conditions, and thermal-outage
availability iterations — each in synchronous (historical weather years
used directly) or Monte Carlo (draws assembled from weather bins) flavors
where applicable. It works off its own raw data database (created from
ra_toolkit/raw_data_db_schema.sql and loaded with the RA Toolkit’s
load_raw_data step), separate from the GridPath Data Toolkit’s raw
database, and its steps run from their own settings CSV via the
gridpath_run_ra_toolkit command (see open_data_toolkit for the Data
Toolkit’s counterpart).
Not to be confused with the GridPath RA Toolkit datasets, developed to
support the 2026 Western US resource adequacy study — those are input
data (one possible source for this package’s raw tables, provided by the
user via the load_raw_data manifest), not this software package.
7.1. Using the GridPath RA Toolkit
RA Toolkit steps run from their own gridpath_run_ra_toolkit command
and settings CSV — the RA Toolkit counterpart to the GridPath Data
Toolkit’s gridpath_run_data_toolkit (see GridPath Data Toolkit); steps
are resolved from the gridpath.ra_toolkit_steps entry-point group. The
RA Toolkit works off its own raw data database, separate from the Data
Toolkit’s: create it with the create_database step pointing
db_schema at ra_toolkit/raw_data_db_schema.sql, then load the
RA-related raw data (system load, unit-level variable generation and
availability profiles, hydro conditions, unit availability parameters, and
the user-defined weather bins and unit mappings) with the RA Toolkit’s
load_raw_data step before running the steps below.
Run GridPath RA Toolkit steps per a settings CSV (the
gridpath_run_ra_toolkit command). The steps are resolved by name from
the ‘gridpath.ra_toolkit_steps’ entry-point group (declared in
pyproject.toml), not from a hardcoded import list — see
gridpath.step_runner for the generic engine. The Data Toolkit’s
steps live in their own entry-point group and are run with the Data
Toolkit’s own gridpath_run_data_toolkit command.
Load data into the GridPath RA Toolkit raw data database (a separate
database from the GridPath Data Toolkit’s raw database, created from the
ra_toolkit/raw_data_db_schema.sql schema). See the documentation of
each GridPath RA Toolkit module for data prerequisites. Use the
files_to_import.csv file to tell GridPath which CSV files should be
loaded into which database table.
7.1.1. What this step does
This module is a generic bulk loader for raw CSV data into the RA Toolkit
raw data database. It reads a file named files_to_import.csv located in
the directory given by --csv_location. Each row of that file describes
one CSV file: an import flag (whether the file should be loaded), the CSV
filename (relative to --csv_location), and the database table the file
should be loaded into.
The loader iterates over the CSV file rows and, for each row whose import
flag is True, reads the corresponding CSV from --csv_location and
appends its contents to the named database table (existing rows are
preserved; data is inserted with if_exists="append"). Rows whose import
flag is False are skipped.
This generic loader is used to populate the raw_data tables (e.g., the
historical load, VER, and availability profiles and their unit mappings,
hydro operating characteristics) that the RA Toolkit steps depend on.
7.1.2. Usage
>>> python -m ra_toolkit.load_raw_data --database PATH/TO/DATABASE --csv_location PATH/TO/CSV/DIRECTORY
7.1.3. Settings
database
csv_location
The --csv_location directory must contain a files_to_import.csv
manifest with columns for the import flag, the CSV filename, and the
destination database table, in that order.
Note that the RA Toolkit steps take their project names from user-provided
mapping tables (e.g. raw_data_var_project_units,
user_defined_load_zone_units) rather than from the Data Toolkit’s
fleet-selection settings. When combining RA Toolkit inputs with
Data-Toolkit-generated project inputs in one scenario, those mappings must
use the same project and load zone names the Data Toolkit run generated —
including the effects of its aggregation and load-zone-level settings.
7.1.4. Temporal Inputs
Monte Carlo Weather Iteration Draws
The Monte Carlo approach employed in the GridPath RA Toolkit study synthesizes multiple years of plausible hourly load, wind availability, solar availability, and temperature-driven thermal derate data over which the system operations can be simulated. Synthetic days are built by combining load, wind, solar, and temperature derate shapes from different but similar days in the historical record. For a detailed description of the methodology, see Appendix B of the report available at https://gridlab.org/wp-content/uploads/2022/10/GridLab_RA-Toolkit-Report-10-12-22.pdf.
Methodology
This module produces, for each Monte Carlo iteration, a full synthetic
study-year sequence of weather day bins. A weather day bin is a categorical
label (e.g., one of five quintiles per month/day_type) that indicates the
historical day’s weather (e.g., based on maximum or average temperature). The
bins themselves are produced upstream by the user and are stored in
user_defined_weather_bins; this module only resamples them into
new synthetic chronologies. The downstream
create_monte_carlo_weather_draw_profiles step then maps each drawn bin to
an actual historical day’s load/wind/solar/derate shapes.
First-order Markov bin chain
The synthetic bin sequence is generated as a first-order Markov chain over the historical record, in order to preserve realistic day-to-day persistence of weather (e.g., heat waves and cold snaps span multiple days):
The first day of the year (Jan 1) is seeded by drawing uniformly at random from all historical January bins (see
starting_weather_bin).For each subsequent calendar day, given the prior day’s bin
b, we look across the historical record for every day in the current month whose bin equalsb, take the bin of the day that immediately followed each such day, and draw uniformly at random from that set of “following-day” bins. That draw becomes the current day’s bin and the prior bin for the next step. In effect, step 2 samples from the empirically estimated transition probabilityP(bin_today | bin_yesterday)for the relevant month.
Reproducibility (seeding)
By default no seed is set (weather_draws_seed defaults to None), so each
run produces a different random ensemble of synthetic weather years. To get
reproducible draws, pass --weather_draws_seed <int>. The seed actually used
is recorded in aux_weather_draws_info alongside the draws.
Other notes
Modeled distributions may deviate from historical averages depending on within-month weather drift. In shoulder seasons, day-to-day directional trends can cause modeled averages to deviate from historical means by ~1–3% (skewing lower in warming months and higher in cooling months).
Usage
>>> gridpath_run_ra_toolkit --single_step create_monte_carlo_weather_draws --settings_csv PATH/TO/SETTINGS/CSV
Input prerequisites
- This module assumes the following raw input database tables have been populated:
user_defined_weather_bins
Settings
database
weather_bins_id
weather_draws_id
weather_draws_seed
n_iterations
study_year
timeseries_iteration_draw_initial_seed
Temporal Scenarios
This is a very basic module that copies over the base CSVs created by the user
and calls the temporal iterations method to create the iterations.csv file if
needed. The location of the base CSVs and the iterations description CSV are
specified in a settings file you can point to with the --csv_path argument.
Usage
>>> gridpath_run_ra_toolkit --single_step create_temporal_scenarios --settings_csv PATH/TO/SETTINGS/CSV
Settings
csv_path
7.1.5. Load Inputs
Sync Loads
Create GridPath sync load profile inputs.
Usage
>>> gridpath_run_ra_toolkit --single_step create_sync_load_input_csvs --settings_csv PATH/TO/SETTINGS/CSV
Input prerequisites
- This module assumes the following raw input database tables have been populated:
raw_data_system_load
user_defined_load_zone_units
The load_zone values in user_defined_load_zone_units must use the same zone vocabulary as the rest of the scenario’s inputs — i.e., match the load_zone_level (BA, EIA930 region, or interconnect) used for the eia930_load_zone and project-level steps.
Settings
database
output_directory
load_scenario_id
load_scenario_name
stage_id
study_year
overwrite
Timepoint IDs are the hour of the historical year (1 through 8760, or 8784 in
a leap year) offset by study_year * 10000, so they start at 1 by default
or at YYYY0001 when a study year is provided – the same convention the
Monte Carlo steps use. Each historical year becomes a weather iteration.
Monte Carlo Loads
Create GridPath Monte Carlo load profile inputs. Before running this module,
you will need to create weather draws with the create_monte_carlo_draws
module (see Monte Carlo Weather Iteration Draws).
What this step does
This module reads the synthetic per-iteration load implied by the weather
draws produced upstream and writes it out as the GridPath load input CSVs that
the model actually consumes. The synthetic per-iteration load is constructed
by joining the drawn synthetic weather days (aux_weather_iterations,
indexed by weather_bins_id and weather_draws_id) back to the historical
hourly load in raw_data_system_load. The resulting CSVs are tagged with
the given load / load-components / load-levels scenario ids and names and are
written to --output_directory.
Methodology
The module produces up to three CSVs, each of which can be suppressed with a
corresponding --skip_* flag:
Load scenario CSV (
create_load_scenario_csv): written to--output_directoryas<load_scenario_id>_<load_scenario_name>.csv. It is a one-row mapping that ties theload_scenario_idto itsload_components_scenario_idandload_levels_scenario_id.Load components CSV (
create_load_components_scenario_csv): written to theload_componentssubdirectory as<load_components_scenario_id>_<load_components_scenario_name>.csv. It lists oneload_component(named via--load_component, defaultall) perload_zonefound inuser_defined_load_zone_units.Load levels CSV (
create_load_levels_csv): written to theload_levelssubdirectory as<load_levels_scenario_id>_<load_levels_scenario_name>.csv. This file holds the actual hourlyload_mwperload_zone,weather_iteration,stage_id, andtimepoint.
The load levels are built one synthetic weather iteration at a time. For every
drawn day in aux_weather_iterations (matched on weather_bins_id and
weather_draws_id), the module identifies the corresponding historical
calendar day and, for each load_zone, sums the hourly load from
raw_data_system_load across that zone’s constituent unit rows, each
scaled by its unit_weight from user_defined_load_zone_units. Timepoint
IDs are derived from the draw number and hour_of_day and are offset by
--study_year (so they start at 1 by default, or at YYYY0001 when a
study year is provided). Rows for successive draws are appended to the single
load-levels CSV for the scenario.
These CSVs are the files the GridPath model consumes for load; they are not loaded back into the database by this step.
Overwrite behavior
By default each output file is created only if it does not already exist. The
three *_overwrite flags allow the corresponding scenario CSVs to be
regenerated in place when they already exist:
--load_scenario_overwritefor the load scenario CSV,
--load_components_overwritefor the load components CSV, and
--load_levels_overwritefor the load levels CSV.
For the load levels CSV, overwrite resets the file (writes a fresh header) only on the first draw of the run and then appends the remaining draws, so the file is rebuilt for the full ensemble rather than truncated mid-write.
Usage
>>> gridpath_run_ra_toolkit --single_step create_monte_carlo_load_input_csvs --settings_csv PATH/TO/SETTINGS/CSV
Input prerequisites
- This module assumes the following raw input database tables have been populated:
aux_weather_iterations (see the
create_monte_carlo_drawsstep for how to create synthetic weather years and populate this table)raw_data_system_load
user_defined_load_zone_units
You must run create_monte_carlo_draws before running this module to populate the database with the raw data and the synthetic weather draws.
Settings
database
weather_bins_id
weather_draws_id
output_directory
load_scenario_id
load_scenario_name
load_components_scenario_id
load_components_scenario_name
load_levels_scenario_id
load_levels_scenario_name
stage_id
study_year
load_component
load_scenario_overwrite
load_components_overwrite
load_levels_overwrite
skip_load_scenario
skip_load_components
skip_load_levels
7.1.6. Project Inputs
Machinery shared by the project steps that generate weather- and hydro-dependent inputs — variable generation profiles, generator weather derates, hydro budgets — in either of the two flavors GridPath supports:
synchronous (
ra_toolkit.project.stochastic.sync_gen_common) — one profile per historical weather year, used directly;Monte Carlo (
ra_toolkit.project.stochastic.monte_carlo_gen_common) — draws assembled from weather bins by the temporal weather-draw steps.
ra_toolkit.project.stochastic.iterations writes the per-project
iterations CSV that tells GridPath which draws a project’s data varies
over.
This is a separate pipeline from the EIA860(M) fleet selection in
open_data_toolkit.project.fleet: the two share no code, and a step draws
on one or the other, never both. Steps here take their project list from
the profiles’ own raw data rather than from the fleet filters.
Variable Gen Profiles (Sync)
Create GridPath sync variable generation profile inputs.
Usage
>>> gridpath_run_ra_toolkit --single_step create_sync_var_gen_input_csvs --settings_csv PATH/TO/SETTINGS/CSV
Input prerequisites
- This module assumes the following raw input database tables have been populated:
raw_data_var_profiles
raw_data_var_project_units
Settings
database
output_directory
variable_generator_profile_scenario_id
variable_generator_profile_scenario_name
stage_id
study_year
overwrite
n_parallel_projects
Timepoint IDs are the hour of the historical year (1 through 8760, or 8784 in
a leap year) offset by study_year * 10000, so they start at 1 by default
or at YYYY0001 when a study year is provided – the same convention the
Monte Carlo steps use. Each historical year becomes a weather iteration.
Variable Gen Profiles (Monte Carlo)
Create GridPath Monte Carlo variable generation profile inputs. Before running
this module,you will need to create weather draws with the
create_monte_carlo_draws module (see Monte Carlo Weather Iteration Draws).
What this step does
This is the variable energy resource (VER) counterpart to the load-CSV step. It
reads the synthetic per-iteration variable generation profiles – assembled from
raw_data_var_profiles (the raw hourly unit-level cap_factor data) and
raw_data_var_project_units (the project-to-unit mapping and per-unit
weights), resampled according to the weather draws stored in
aux_weather_iterations – and writes them out as GridPath variable-generator
profile input CSVs in --output_directory, tagged with the given
--variable_generator_profile_scenario_id and
--variable_generator_profile_scenario_name. These CSVs are the files the
GridPath model consumes for variable generation.
Methodology
For each project, the per-unit cap_factor values from raw_data_var_profiles
are multiplied by their unit_weight and summed to produce a single
project-level cap_factor time series. The weather draws in
aux_weather_iterations (selected by --weather_bins_id and
--weather_draws_id) determine, for each Monte Carlo weather_iteration and
draw_number, which historical day’s data to pull, and the draw number is used
to compute the timepoint ID. One output CSV is written per project, named
{project}-{scenario_id}-{scenario_name}.csv, with an accompanying iterations
CSV written to an iterations subdirectory of --output_directory.
--n_parallel_projects N processes up to N projects concurrently (via a
multiprocessing pool over the project pool) to speed things up. --overwrite
deletes any existing CSVs with the matching project/scenario filename before
writing; without it, output is appended to existing files.
Usage
>>> gridpath_run_ra_toolkit --single_step create_monte_carlo_var_gen_input_csvs --settings_csv PATH/TO/SETTINGS/CSV
Input prerequisites
- This module assumes the following raw input database tables have been populated:
raw_data_var_profiles
raw_data_var_project_units
aux_weather_iterations (see the
create_monte_carlo_drawsstep for how to create synthetic weather years and populate this table)
You must run create_monte_carlo_draw_profiles before running this module to populate the database with the raw data and the synthetic weather draws.
Settings
database
output_directory
variable_generator_profile_scenario_id
variable_generator_profile_scenario_name
overwrite
n_parallel_projects
weather_bins_id
weather_draws_id
Weather Derates (Sync)
Create GridPath sync weather iteration availability inputs.
Usage
>>> gridpath_run_ra_toolkit --single_step create_sync_gen_weather_derate_input_csvs --settings_csv PATH/TO/SETTINGS/CSV
Input prerequisites
- This module assumes the following raw input database tables have been populated:
raw_data_availability_profiles
raw_data_unit_availability_params
Settings
database
output_directory
exogenous_availability_weather_scenario_id
exogenous_availability_weather_scenario_name
stage_id
study_year
overwrite
n_parallel_projects
print_ones
Timepoint IDs are the hour of the historical year (1 through 8760, or 8784 in
a leap year) offset by study_year * 10000, so they start at 1 by default
or at YYYY0001 when a study year is provided – the same convention the
Monte Carlo steps use. Each historical year becomes a weather iteration.
Weather Derates (Monte Carlo)
Create GridPath Monte Carlo weather iteration availability inputs. Before
running this module, you will need to create weather draws with the
create_monte_carlo_draws module (see Monte Carlo Weather Iteration Draws).
Usage
>>> gridpath_run_ra_toolkit --single_step create_monte_carlo_weather_derate_input_csvs --settings_csv PATH/TO/SETTINGS/CSV
Input prerequisites
- This module assumes the following raw input database tables have been populated:
raw_data_unit_availability_params
raw_data_availability_profiles
aux_weather_iterations (see the
create_monte_carlo_drawsstep for how to create synthetic weather years and populate this table)
Settings
database
output_directory
exogenous_availability_weather_scenario_id
exogenous_availability_weather_scenario_name
overwrite
n_parallel_projects
weather_bins_id
weather_draws_id
Availability Iteration Inputs
Run unit outage simulation and create availability iteration inputs.
What this step does
This module runs a Monte Carlo unit-outage simulation and writes the resulting
exogenous availability (derate) input CSVs. Using the per-unit availability
parameters loaded from --outage_params_input_csv (into the
raw_data_unit_availability_params table) – forced-outage rates
(unit_for), mean time to repair (unit_mttr), the number of units
(n_units), the unit weight, and the per-unit outage model
(unit_fo_model) – it simulates --n_iterations independent outage
timelines for each project, drawing random forced and (under the sequential
model) repair/maintenance transitions.
The outage model is selected per unit via the unit_fo_model column and may
be one of:
Derate– a static derate1 - unit_forapplied in every timepoint.
MC_independent– each timepoint’s outage state is drawn independently from a uniform distribution against the forced-outage rate.
MC_sequential– a sequential (exponential) failure/repair process driven by the forced-outage rate andunit_mttr(the implied mean time to failure ismttr * (1 / for - 1)), preserving outage persistence across timepoints.
historical_year– instead of simulating, a random historical year is sampled for the unit from--historical_availability_csvand that year’s hourly derate series is used directly. (This is can be used for units whose availability is taken from a historical record rather than simulated; the choice is driven by the unit’sunit_fo_modelvalue, not by project type.)
For each project the per-unit availability adjustments are combined using each
unit’s unit_weight to form a weighted project-level derate. Hybrid-storage
projects (hybrid_stor set) additionally get a separately simulated derate
for the storage component. By default only rows whose derate differs from 1 are
written (as default availability in GridPath is 1); pass --print_ones to
retain all rows.
Output is written to --output_directory as one CSV per project, named
<project>-<project_availability_scenario_id>-<project_availability_scenario_name>.csv.
--n_parallel_projects parallelizes the simulation across projects and
--overwrite replaces existing files (otherwise existing files are appended
to). --sort re-sorts each output file at the end. These outage iterations
are intended to align with the weather/hydro iterations to form complete Monte
Carlo draws.
Reproducibility (seeding)
By default seeding is OFF: --user_provided_seeding is not set, so the
outage simulation is fully random and non-reproducible from run to run. When
seeding is off, all of the seeding flags below are ignored – the seed
arguments are replaced with None before the simulation runs, and NumPy’s
global RNG is never explicitly seeded.
To get reproducible outages, set --user_provided_seeding together with a
--starting_project_iteration_seed <int> (defaults to 0). With seeding
on:
Per-project, non-overlapping seed ranges. Each project is assigned a starting seed of
starting_project_iteration_seed + project_idx * n_iterations. Within a project the per-iteration seed starts at that value and is incremented by 1 for each of then_iterationsiterations, so the seed ranges of distinct projects do not overlap.Per-unit seeds within an iteration. For a given project iteration, the per-iteration seed is used to seed NumPy’s RNG, which then draws one integer seed per unit via
np.random.randint(1, max_integer_for_unit_outage_seeding, size=n_units_in_project). Each unit’s outage timeline is then simulated from its own seed.--max_integer_for_unit_outage_seedingdefaults to1000000.Hybrid-storage offset. For hybrid-storage projects, the storage component is simulated with a seed offset from the generator component’s unit seed by
--hybrid_storage_seed_increment(defaults to1000).
Every project / unit / iteration still draws its own independent random outage
timeline, but the whole simulation reproduces exactly when re-run with the same
seed settings. Again, these flags are ignored unless --user_provided_seeding
is set. Caution advised when seeding.
Usage
>>> gridpath_run_ra_toolkit --single_step create_availability_iteration_input_csvs --settings_csv PATH/TO/SETTINGS/CSV
Input prerequisites
- This module assumes the following raw input database table has been populated:
raw_data_unit_availability_params
This table can be populated ahead of time, or loaded at run time by passing
--outage_params_input_csv. Units that use the historical_year outage
model additionally read their derate series from the CSV passed via
--historical_availability_csv.
Settings
database
outage_params_input_csv
historical_availability_csv
stage_id
n_iterations
study_year
project_availability_scenario_id
project_availability_scenario_name
output_directory
overwrite
sort
print_ones
n_parallel_projects
user_provided_seeding
starting_project_iteration_seed
max_integer_for_unit_outage_seeding
hybrid_storage_seed_increment
Hydro Gen Inputs
Create hydro iteration input CSVs from year/month data.
Usage
>>> gridpath_run_ra_toolkit --single_step create_hydro_iteration_input_csvs --settings_csv PATH/TO/SETTINGS/CSV
Input prerequisites
- This module assumes the following raw input database tables have been populated:
raw_data_project_hydro_opchars_by_year_month
raw_data_hydro_years
user_defined_balancing_type_horizons
Settings
database
output_directory
hydro_operational_chars_scenario_id
hydro_operational_chars_scenario_name
overwrite
n_parallel_projects
What this step does
This module builds GridPath hydro operational-characteristics input CSVs from
the year/month hydro data loaded earlier
(raw_data_project_hydro_opchars_by_year_month, raw_data_hydro_years,
and the user-defined balancing-type horizons in
user_defined_balancing_type_horizons). For each hydro iteration it derives
the per-horizon hydro operating parameters – the average, minimum, and maximum
power fractions – and writes them to --output_directory under the given
hydro_operational_chars_scenario_id and
hydro_operational_chars_scenario_name. --n_parallel_projects N runs up
to N projects at once, and --overwrite replaces existing CSVs.
Methodology
The distinct projects to process are read from
raw_data_project_hydro_opchars_by_year_month, and one CSV is written per
project, named <project>-<scenario_id>-<scenario_name>.csv in
--output_directory. Projects are processed in a multiprocessing pool sized
by --n_parallel_projects (defaults to 1).
Hydro iterations and balancing-type horizons
The set of hydro years is read from raw_data_hydro_years and each year is
treated as one hydro iteration (written into the hydro_iteration column).
The set of (balancing_type, horizon) pairs is read from
user_defined_balancing_type_horizons; if --hydro_balancing_type is
supplied, the pairs are filtered to that single balancing type (e.g. day,
week, month), otherwise all balancing types are included. For every
combination of hydro year and balancing-type horizon, one output row is
produced.
Deriving per-horizon power fractions
The average_power_fraction, min_power_fraction, and
max_power_fraction for each horizon are computed by month-weighting the raw
monthly opchar values. For a given balancing-type horizon, the module reads its
hour_ending_of_year_start and hour_ending_of_year_end from
user_defined_balancing_type_horizons and walks each hour of the year in that
range, mapping the hour to a calendar month (via a pandas.Timestamp anchored
at January 1 of the hydro year) and counting the number of hours that fall in
each month. These hour counts become the per-month weights for the horizon.
For each month touched by the horizon, the module looks up the project’s
average_power_fraction, min_power_fraction, and max_power_fraction
for that hydro year and month in
raw_data_project_hydro_opchars_by_year_month, multiplies each by the month’s
hour-count weight, sums across months, and divides by the total number of hours
in the horizon. The result is an hours-weighted average of the monthly
fractions for each of the three parameters, written as a single row keyed by
balancing_type_project and horizon (with weather_iteration set to
0, i.e. no weather iteration). Note we take the weighted averages of the
mins and maxes, not the mins of the mins or the maxes of the maxes.
Writing and overwriting output
Rows are appended to the project’s CSV as they are generated, with the header
written only when the file does not yet exist. When --overwrite is set, any
existing CSV for the project is deleted before processing begins so it is
rebuilt from scratch; without --overwrite, new rows are appended to any
existing file.
An iterations metadata CSV is also written per project to an iterations/
subdirectory of --output_directory (same filename), recording that hydro
operational characteristics vary by hydro iteration but not by weather
iteration (varies_by_weather_iteration = 0, varies_by_hydro_iteration
= 1). Without --overwrite, an existing iterations CSV is left in place.
If the corresponding --*_input_csv paths are provided, the raw-data tables
(raw_data_project_hydro_opchars_by_year_month, raw_data_hydro_years,
user_defined_balancing_type_horizons) are loaded from those CSVs before the
inputs are built; otherwise the data is assumed to already be present in the
database.