Config¶

This module provides a base class for configs that can be loaded from files, environment variables or command line arguments (via yuio.app).

Derive your config from the Config base class. Inside of its body, define config fields using type annotations, just like dataclasses:

class AppConfig(Config):
    #: Trained model to execute.
    model: pathlib.Path

    #: Input data for the model.
    data: pathlib.Path

    #: Enable or disable gpu.
    use_gpu: bool = True

Then use config’s constructors and the update() method (or the | / |= operators) to load it from various sources:

# Load config from a file.
config = AppConfig.load_from_json_file("~/.my_app_cfg.json")

# Update config with values from env.
config |= AppConfig.load_from_env()

Config base class¶

class yuio.config.Config(*args: Self | dict[str, Any], **kwargs)¶

Base class for configs.

Pass keyword args to set fields, or pass another config to copy it:

Config(config1, config2, ..., field1=value1, ...)

Upon creation, all fields that aren’t explicitly initialized and don’t have defaults are considered missing. Accessing them will raise AttributeError.

Note

Unlike dataclasses, Yuio does not provide an option to create new instances of default values upon config instantiation. This is done so that default values don’t override non-default ones when you update one config from another.

update(other: Self | dict[str, Any], /)¶

Update fields in this config with fields from another config.

This function is similar to dict.update().

Nested configs are updated recursively.

Parameters:: other – data for update.

__or__(value: Self, /) → Self¶

Merge two configs using the | operator, returning a new config.

Creates a copy of this config, updates it with value, and returns the result. Neither operand is modified:

merged = config1 | config2

Parameters:: value – config to merge from.
Returns:: a new config with fields from both operands.

__ior__(value: Self, /) → Self¶

Update this config in-place using the |= operator.

Equivalent to calling update():

config |= other
# same as
config.update(other)

Parameters:: value – config to merge from.

classmethod load_from_env(prefix: str = '') → Self¶

Load config from environment variables.

Parameters:: prefix – if given, names of all environment variables will be prefixed with this string and an underscore.
Returns:: a parsed config.
Raises:: ParsingError.

classmethod load_from_json_file( path: str | Path, /, *, ignore_unknown_fields: bool = False, ignore_missing_file: bool = False, ) → Self¶

Load config from a .json file.

Parameters:

path – path of the config file.
ignore_unknown_fields – if True, this method will ignore fields that aren’t listed in config class.
ignore_missing_file – if True, silently ignore a missing file error. This is useful when loading a config from a home directory.

Returns:

a parsed config.

Raises:

ParsingError if config parsing has failed or if config file doesn’t exist.

classmethod load_from_yaml_file( path: str | Path, /, *, ignore_unknown_fields: bool = False, ignore_missing_file: bool = False, ) → Self¶

Load config from a .yaml file.

This requires PyYaml package to be installed.

Parameters:

path – path of the config file.
ignore_unknown_fields – if True, this method will ignore fields that aren’t listed in config class.
ignore_missing_file – if True, silently ignore a missing file error. This is useful when loading a config from a home directory.

Returns:

a parsed config.

Raises:

ParsingError if config parsing has failed or if config file doesn’t exist. Can raise ImportError if PyYaml is not available.

classmethod load_from_toml_file( path: str | Path, /, *, ignore_unknown_fields: bool = False, ignore_missing_file: bool = False, ) → Self¶

Load config from a .toml file.

This requires tomllib or toml package to be installed.

Parameters:

path – path of the config file.
ignore_unknown_fields – if True, this method will ignore fields that aren’t listed in config class.
ignore_missing_file – if True, silently ignore a missing file error. This is useful when loading a config from a home directory.

Returns:

a parsed config.

Raises:

ParsingError if config parsing has failed or if config file doesn’t exist. Can raise ImportError if toml is not available.

classmethod load_from_parsed_file( parsed: dict[str, object], /, *, ignore_unknown_fields: bool = False, path: str | Path | None = None, ) → Self¶

Load config from parsed config file.

This method takes a dict with arbitrary values that you’d get from parsing type-rich configs such as yaml or json.

For example:

with open("conf.yaml") as file:
    config = Config.load_from_parsed_file(yaml.load(file))

Parameters:

parsed – data from parsed file.
ignore_unknown_fields – if True, this method will ignore fields that aren’t listed in config class.
path – path of the original file, used for error reporting.

Returns:

a parsed config.

Raises:

ParsingError.

classmethod to_json_schema( ctx: JsonSchemaContext, ) → JsonSchemaType¶

Create a JSON schema object based on this config.

The purpose of this method is to make schemas for use in IDEs, i.e. to provide autocompletion or simple error checking. The returned schema is not guaranteed to reflect all constraints added to the parser.

Parameters:: ctx – context for building a schema.
Returns:: a JSON schema that describes structure of this config.

to_json_value( *, include_defaults: bool = True, ) → JsonValue¶

Convert this config to a representation suitable for JSON serialization.

Parameters:: include_defaults – if False, default values will be skipped.
Returns:: a config converted to JSON-serializable representation.
Raises:: TypeError if any of the config fields contain values that can’t be converted to JSON by their respective parsers.

Advanced field configuration¶

By default, Config infers names for env variables and flags, appropriate parsers, and other things from field’s name, type hint, and comments.

If you need to override them, you can use yuio.app.field() and yuio.app.inline() functions (also available from yuio.config):

class AppConfig(Config):
    model: pathlib.Path | None = field(
        default=None,
        help="trained model to execute",
    )

Nesting configs¶

You can nest configs to achieve modularity:

class ExecutorConfig(Config):
    #: Number of threads to use.
    threads: int

    #: Enable or disable gpu.
    use_gpu: bool = True


class AppConfig(Config):
    #: Executor parameters.
    executor: ExecutorConfig

    #: Trained model to execute.
    model: pathlib.Path

To initialise a nested config, pass either an instance of if or a dict with its variables to the config’s constructor:

# The following lines are equivalent:
config = AppConfig(executor=ExecutorConfig(threads=16))
config = AppConfig(executor={"threads": 16})
# ...although type checkers will complain about dict =(

Parsing environment variables¶

You can load config from environment through load_from_env().

Names of environment variables are just capitalized field names. Use the yuio.app.field() function to override them:

class KillCmdConfig(Config):
    # Will be loaded from `SIGNAL`.
    signal: int

    # Will be loaded from `PROCESS_ID`.
    pid: int = field(env="PROCESS_ID")

In nested configs, environment variable names are prefixed with name of a field that contains the nested config:

class BigConfig(Config):
    # `kill_cmd.signal` will be loaded from `KILL_CMD_SIGNAL`.
    kill_cmd: KillCmdConfig

    # `kill_cmd_2.signal` will be loaded from `KILL_SIGNAL`.
    kill_cmd_2: KillCmdConfig = field(env="KILL")

    # `kill_cmd_3.signal` will be loaded from `SIGNAL`.
    kill_cmd_3: KillCmdConfig = field(env="")

You can also disable loading a field from an environment altogether:

class KillCmdConfig(Config):
    # Will not be loaded from env.
    pid: int = field(env=yuio.DISABLED)

To prefix all variable names with some string, pass the prefix parameter to the load_from_env() function:

# config.kill_cmd.field will be loaded
# from `MY_APP_KILL_CMD_SIGNAL`
config = BigConfig.load_from_env("MY_APP")

Parsing config files¶

You can load config from structured config files, such as json, yaml or toml:

class ExecutorConfig(Config):
    threads: int
    use_gpu: bool = True


class AppConfig(Config):
    executor: ExecutorConfig
    model: pathlib.Path


config = AppConfig.load_from_json_file("~/.my_app_cfg.json")

In this example, contents of the above config would be:

{
    "executor": {
        "threads": 16,
        "use_gpu": true
    },
    "model": "/path/to/model"
}

Note that, unlike with environment variables, there is no way to inline nested configs.

Merging configs¶

Configs are specially designed to be merge-able. The basic pattern is to create an empty config instance, then update() it with every config source:

config = AppConfig()
config.update(AppConfig.load_from_json_file("~/.my_app_cfg.json"))
config.update(AppConfig.load_from_env())
# ...and so on.

The update() function ignores default values, and only overrides keys that were actually configured.

If you need a more complex update behavior, you can add a merge function for a field:

class AppConfig(Config):
    plugins: list[str] = field(
        default=[],
        merge=lambda left, right: [*left, *right],
    )

Here, whenever we update() AppConfig, plugins from both instances will be concatenated.

Warning

Merge function shouldn’t mutate its arguments. It should produce a new value instead.

Warning

Merge function will not be called for default value. It’s advisable to keep the default value empty, and add the actual default to the initial empty config:

config = AppConfig(plugins=["markdown", "rst"])
config.update(...)

Collections of configs¶

When you use configs inside of other collections, the config will be parsed from JSON. This is mostly useful when loading configs from files.

class yuio.config.ConfigParser(inner: type[Cfg] | None = None, /)¶

Parser for configs that reads them as JSON strings.

This parser kicks in when you use configs as collection members, i.e. list[Config] or dict[str, Config]. On top level, the usual logic for nested configs applies.

Re-imports¶

yuio.config.field(): Alias of yuio.app.field

yuio.config.inline(): Alias of yuio.app.inline

yuio.config.bool_option(): Alias of yuio.app.bool_option

yuio.config.count_option(): Alias of yuio.app.count_option

yuio.config.parse_many_option(): Alias of yuio.app.parse_many_option

yuio.config.parse_one_option(): Alias of yuio.app.parse_one_option

yuio.config.store_const_option(): Alias of yuio.app.store_const_option

yuio.config.store_false_option(): Alias of yuio.app.store_false_option

yuio.config.store_true_option(): Alias of yuio.app.store_true_option

yuio.config.merge_dicts(): Alias of yuio.util.merge_dicts()

yuio.config.merge_dicts_opt(): Alias of yuio.util.merge_dicts_opt()

type yuio.config.HelpGroup: Alias of yuio.cli.HelpGroup.

type yuio.config.MutuallyExclusiveGroup: Alias of yuio.cli.MutuallyExclusiveGroup.

type yuio.config.OptionCtor: Alias of yuio.app.OptionCtor.

type yuio.config.OptionSettings: Alias of yuio.app.OptionSettings.

yuio.config.MISC_GROUP: Alias of yuio.cli.MISC_GROUP.

yuio.config.OPTS_GROUP: Alias of yuio.cli.OPTS_GROUP.

yuio.config.SUBCOMMANDS_GROUP: Alias of yuio.cli.SUBCOMMANDS_GROUP.