API documentation for the config module

The config module is used to read in the various configuration files, validate their contents, and then configure a ready to run instance of the virtual ecosystem model. The basic details of how this system is used can be found here.

The validation of configuration documents is done using JSONSchema documents associated with the different model components. See the schema module for details.

Classes:

Config([cfg_paths, cfg_strings, ...])

Configuration loading and validation.

Functions:

config_merge(dest, source[, conflicts, path])

Recursively merge two dictionaries detecting duplicated key definitions.

class virtual_ecosystem.core.config.Config(cfg_paths: str | Path | Sequence[str | Path] = [], cfg_strings: str | list[str] = [], override_params: dict[str, Any] = {}, auto: bool = True)

Configuration loading and validation.

The Config class is used to generate a validated configuration for a Virtual Ecosystem simulation. The cfg_paths attribute is used to provide paths to TOML configuration files or directories containing sets of files to be used. The provided paths are then run through the follow steps to resolve and load the configuration data.

• The collect_config_paths() method is used to collect the actual TOML files to be used to build the configuration from the provided paths.

• The load_config_toml() method is then used to load the parsed contents of each resolved file into the toml_contents attribute.

Alternatively, configuration data may be passed as a string or list of strings using the cfg_strings argument. These strings must contain TOML formatted data, which is parsed and added to the toml_contents attribute.

Whichever approach is used, the next two steps are then applied to the provided TOML data:

• The build_config() method is used to merge the loaded configuration across files and check that configuration settings are uniquely defined.

• The validate_config() method validates the compiled configuration against the appropriate configuration schema for the core module and any models included in the configuration. This validation will also fill in any missing configuration settings with defined defaults.

By default, creating a Config instance automatically runs these steps across the provided cfg_paths, but the auto argument can be used to turn off automatic validation.

The export_config() method can be used to export the compiled and validated configuration as a single TOML file.

If the core.data_output_options.save_merged_config option is set to true a merged config file will be automatically generated, unless auto is set to false.

Parameters:

• cfg_paths – A string, Path or list of strings or Paths giving configuration file or directory paths.

• cfg_strings – A string or list of strings containing TOML formatted configuration data.

• override_params – Extra parameters provided by the user.

• auto – A boolean flag setting whether the configuration data is automatically loaded and validated

Methods:

build_config()	Build a combined configuration from the loaded files.
build_schema()	Build a schema to validate the model configuration.
collect_config_paths()	Collect TOML config files from provided paths.
export_config(outfile)	Exports a validated and merged configuration as a single file.
load_config_toml()	Load the contents of resolved configuration files.
load_config_toml_string()	Load the contents of a config provided as a string.
override_config(override_params)	Override any parameters desired.
resolve_config_file_paths()	Resolve the locations of configured file paths.
validate_config()	Validate the model configuration.

Attributes:

cfg_paths	The configuration file paths, normalised from the cfg_paths argument.
cfg_strings	A list of strings containing TOML content, provided by the cfg_strings argument.
config_errors	Configuration errors, as a list of tuples of key path and error details.
from_cfg_strings	A boolean flag indicating whether paths or strings were used to create the instance.
merge_conflicts	A list of configuration keys duplicated across configuration files.
merged_schema	The merged schema for the core and modules present in the configuration.
model_classes	A dictionary of the model classes specified in the configuration, keyed by model name.
toml_contents	A dictionary of the parsed TOML contents of config files or strings, keyed by file path or string index.
toml_files	A list of TOML file paths resolved from the initial config paths.
validated	A boolean flag indicating successful validation.

build_config() → None

Build a combined configuration from the loaded files.

This method does pairwise merging of the loaded config data from individual files to build a single configuration dictionary, looking for duplicate definitions in the config tree.

Raises:

ConfigurationError – if duplicate config definitions occur across files.

build_schema() → None

Build a schema to validate the model configuration.

This method identifies the modules to be configured from the top-level configuration keys, setting the requested modules to be used in the configured simulation. The schemas for the requested modules are then loaded and combined using the merge_schemas() function to generate a single validation schema for model configuration.

cfg_paths: list[Path]

The configuration file paths, normalised from the cfg_paths argument.

cfg_strings: list[str]

A list of strings containing TOML content, provided by the cfg_strings argument.

collect_config_paths() → None

Collect TOML config files from provided paths.

The Config class is initialised with a list of paths to either individual TOML config files or directories containing possibly multiple config files. This method examines that list to collect all the individual TOML config files in the provided locations and then populates the toml_files attribute.

Raises:

ConfigurationError – this is raised if any of the paths: do not exist, are directories that do not contain TOML files, are not TOML files or if the resolved files contain duplicate entries.

config_errors: list[tuple[str, Any]]

Configuration errors, as a list of tuples of key path and error details.

export_config(outfile: Path) → None

Exports a validated and merged configuration as a single file.

This method will only export a configuration file if the Config instance has been successfully validated.

Parameters:

outfile – An output path for the TOML configuration file.

from_cfg_strings: bool

A boolean flag indicating whether paths or strings were used to create the instance.

load_config_toml() → None

Load the contents of resolved configuration files.

This method populates the toml_contents dictionary with the contents of the configuration files set in toml_files. That attribute is normally populated by providing a set of paths to Config and running the collect_config_paths() method, but it can also be set directly.

Raises:

ConfigurationError – Invalid TOML content in config files.

load_config_toml_string() → None

Load the contents of a config provided as a string.

This method populates the toml_contents dictionary with the contents of a provided TOML formatted string.

Raises:

ConfigurationError – Invalid TOML string.

merge_conflicts: list

A list of configuration keys duplicated across configuration files.

merged_schema: dict

The merged schema for the core and modules present in the configuration.

model_classes: dict[str, Any]

A dictionary of the model classes specified in the configuration, keyed by model name.

override_config(override_params: dict[str, Any]) → None

Override any parameters desired.

Parameters:

override_params – Extra parameter settings

resolve_config_file_paths() → None

Resolve the locations of configured file paths.

Configuration files can contain paths to other resources, such as the paths to files containing input data variables. These paths can be absolute, but may also be relative to the location of the configuration file itself. This method is used to resolve the location of files to the common root of the provided set of configuration files, typically the path where a simulation is started.

toml_contents: dict[str | Path, dict]

A dictionary of the parsed TOML contents of config files or strings, keyed by file path or string index.

toml_files: list[str | Path]

A list of TOML file paths resolved from the initial config paths.

validate_config() → None

Validate the model configuration.

This method first validates the ‘core’ configuration and applies defaults, which ensures that the modules to be used in the configured model are set. The schemas for the requested modules are then applied to each configuration section to validate the contents and apply any default values.

Raises:

ConfigurationError – if the loaded configuration is not compatible with the configuration schemas.

validated: bool

A boolean flag indicating successful validation.

virtual_ecosystem.core.config.config_merge(dest: dict, source: dict, conflicts: tuple = (), path: str = '') → tuple[dict, tuple]

Recursively merge two dictionaries detecting duplicated key definitions.

This function returns a copy of the input dest dictionary that has been extended recursively with the entries from the input source dictionary. The two input dictionaries must not share any key paths and when duplicated key paths are found, the value from the source dictionary is used and the function extends the returned conflicts tuple with the duplicated key path.

Parameters:

• dest – A dictionary to extend

• source – A dictionary of key value pairs to extend dest

• conflicts – A tuple of duplicated key paths between the two dictionaries

• path – A string giving the current key path.

Returns:

A copy of dest, extended recursively with values from source, and a tuple of duplicate key paths.