8. GridPath Architecture

This section describes GridPath’s modular architecture, i.e. how the modules are linked and work together (as opposed to their formulation or the functionality they add).

8.1. GridPath Modules

8.1.1. gridpath.auxiliary.module_list

This module contains:

1. the list of all GridPath modules;

2. the modules included in each optional feature;

3. the ‘cross-feature’ modules;

4. the method for determining the user-requested features for the scenarios;

5. the method for loading modules.

all_modules_list()

Returns:

list of all GridPath modules in order they are loaded

This is the list of all GridPath modules in the order they would be loaded if all optional features were selected.

cross_feature_modules_list()

Returns:

dictionary with a tuple of features as keys and a list of modules to be included if all those features are selected as values

Some modules depend on more than one feature, i.e. they are included only if multiple features are selected. These relationships are described in the ‘cross_modules’ dictionary here.

determine_modules(features=None, scenario_directory=None, multi_stage=None)

Parameters:

• features – List of requested features. Optional input; if not specified, function will try to load ‘features.csv’ file to determine the requested features.

• scenario_directory – the scenario directory, where we will look for the list of requested features. Optional input; if not specified, function will look for the ‘features’ input parameter

• multi_stage – Boolean. Optional input that determines whether the modules that fix variables are used (yes if True, no if False); if not specified, this function will check the scenario_directory to determine whether there are stage subdirectories (if there are not, the ‘fix variables’ modules are removed).

Returns:

the list of modules – a subset of all GridPath modules – needed for a scenario. These are the module names, not the actual modules.

This method determines which modules are needed for a scenario based on the features specified for the scenario. The features can be either directly specified as a list or by providing the directory where a ‘features.csv’ file lists the requested features.

We start with the list of all GridPath modules from all_modules_list() as the list of modules to use in the scenario. We then iterate over all optional features, which we get from the keys of the optional_modules_list() method above; if the feature is in the list of user-requested features, we do nothing; if it is not, we remove all of the feature’s modules from the list of modules to use. Similarly, for the cross feature modules, which we get from the cross_feature_module_list() method, we check if all features they depend on are included and, if not, remove those modules from the list of modules to use.

load_modules(modules_to_use)

Parameters:

modules_to_use – a list of the names of the modules to use

Returns:

list of imported modules (Python <class ‘module’> objects)

Load the requested modules and return them as a list of Python module objects.

optional_modules_list()

Returns:

dictionary with the optional feature names as keys and a list of the modules included in each feature as values

These are all of GridPath’s optional modules grouped by features (features as the dictionary keys). Each of these modules belongs to only one feature.

8.1.2. gridpath.auxiliary.dynamic_components

This module creates the DynamicComponents class, which contains the lists and dictionaries of the names of dynamic optimization components. These are components that are populated by GridPath modules based on the selected features and the scenario input data.

class DynamicComponents

Here we initialize the class object and its components that will contain the dynamic model components, i.e. lists and dictionary with the names of the optimization components that are populated based on whether features are selected (i.e. certain modules are called) and based on the scenario input data.

8.2. Running GridPath

8.2.1. Running a Scenario

gridpath.run_scenario

This script runs a GridPath scenario. It assumes that scenario inputs have already been written.

The main() function of this script can also be called with the gridpath_run command when GridPath is installed.

create_abstract_model(model, dynamic_components, loaded_modules, scenario_directory, weather_iteration, hydro_iteration, availability_iteration, subproblem, stage)

Parameters:

• model – the Pyomo AbstractModel object

• dynamic_components – the populated dynamic model components class

• loaded_modules – list of the required modules as Python objects

• scenario_directory –

• subproblem –

• stage –

To create the abstract model, we iterate over all required modules and call their add_model_components method to add components to the Pyomo AbstractModel. Some modules’ add_model_components method also require the dynamic component class as an argument for any dynamic components to be added to the model.

create_problem_instance(model, loaded_data)

Parameters:

• model – the AbstractModel Pyomo object with components added

• loaded_data – the DataPortal object with the data loaded in and linked to the relevant model components

Returns:

the compiled problem instance

Compile the problem based on the abstract model formulation and the data loaded into the model components.

fix_variables(instance, dynamic_components, scenario_directory, weather_iteration, hydro_iteration, availability_iteration, subproblem, stage, loaded_modules)

Parameters:

• instance – the compiled problem instance

• dynamic_components – the dynamic component class

• scenario_directory – str

• subproblem – str

• stage – str

• loaded_modules – list of imported GridPath modules as Python objects

Returns:

the problem instance with the relevant variables fixed

Iterate over the required GridPath modules and fix variables by calling the modules’ fix_variables, if applicable. Return the modified problem instance with the relevant variables fixed.

load_scenario_data(model, dynamic_components, loaded_modules, scenario_directory, weather_iteration, hydro_iteration, availability_iteration, subproblem, stage)

Parameters:

• model – the Pyomo abstract model object with components added

• dynamic_components – the dynamic components class

• loaded_modules – list of the imported GridPath modules as Python objects

• scenario_directory – the main scenario directory

• subproblem – the horizon subproblem

• stage – the stage subproblem

Returns:

the DataPortal object populated with the input data

Iterate over all required GridPath modules and call their load_model_data method in order to load input data into the relevant model components. Return the resulting DataPortal object with the data loaded in.

main(args=None)

This is the ‘main’ method that runs a scenario. It takes in and parses the script arguments, determines the scenario structure (i.e. whether it is a single optimization or has subproblems), and runs the scenario. This method also returns the objective function value(s).

parse_arguments(args)

Parameters:

args – the script arguments specified by the user

Returns:

the parsed known argument values (<class ‘argparse.Namespace’> Python object)

Parse the known arguments.

run_optimization_for_subproblem(scenario_directory, weather_iteration_directory, hydro_iteration_directory, availability_iteration_directory, subproblem_directory, stage_directories, multi_stage, parsed_arguments, objective_values)

Check if there are stages in the subproblem; if not solve subproblem; if, yes, solve each stage sequentially

run_scenario(scenario_directory, scenario_structure, parsed_arguments)

Check the scenario structure, iterate over all subproblems if they exist, and run the subproblem optimization.

The objective function is returned, but it’s only really used if we are in ‘testing’ mode.

Parameters:

• scenario_directory – scenario directory path

• scenario_structure – the subproblem structure object

• parsed_arguments –

Returns:

the objective function value (NPV); only used in ‘testing’ mode.

solve(instance, parsed_arguments)

Parameters:

• instance – the compiled problem instance

• parsed_arguments – the user-defined arguments (parsed)

Returns:

the problem results

Send the compiled problem instance to the solver and solve.

8.2.2. Database Access

gridpath.get_scenario_inputs

This script iterates over all modules required for a GridPath scenario and calls their write_model_inputs() method, which queries the GridPath database and writes the .tab input files to the scenario directory.

The main() function of this script can also be called with the gridpath_get_inputs command when GridPath is installed.

gridpath.import_scenario_results

This script iterates over all modules required for a GridPath scenario and calls their import_results_into_database() method, which loads the scenario results files into their respective database table.

The main()_ function of this script can also be called with the gridpath_import_results command when GridPath is installed.

gridpath.process_results

This script iterates over all modules required for a GridPath scenario and calls their process_results() method, which makes updates to database tables.

The main() function of this script can also be called with the gridpath_process_results command when GridPath is installed.

8.2.3. Running End-to-End

gridpath.run_end_to_end

This script calls the __main__ functions of get_scenario_inputs.py, run scenario.py, import_scenario_results.py, and process_results.py to run a scenario end-to-end, i.e. get the scenario inputs from the database, solve the scenario problem, import the results the database and perform any necessary results-processing.

The main() function of this script can also be called with the gridpath_process_results command when GridPath is installed.

8.2.4. Input Validation

gridpath.validate_inputs

This script iterates over all modules required for a GridPath scenario and calls their validate_inputs() method, which performs various validations of the input data and scenario setup.