Sphinx extension¶

A Sphinx extension for documenting Yuio apps and configs.

Installation¶

Add yuio.ext.sphinx to the list of extensions in your conf.py:

extensions = [
    "yuio.ext.sphinx",
]

Yuio’s extension adds new domain cli with directives and roles to declare and cross-reference various Yuio objects.

Documented object names¶

CLI domain maintains two separate namespaces for documented objects. One is cfg namespace which corresponds to names of fields in Python; the other is cmd namespace which contains command names, flags and positional argument names as they appear on the command line.

This distinction is important. Consider the following application:

import yuio.app
import pathlib


@yuio.app.app(prog="my-app")
def main(
    quiet: bool,
    input: pathlib.Path,
): ...

In cfg namespace you would address quiet as main.quiet, while in cmd namespace you would address quiet as my-app --quiet.

Declaring commands and arguments¶

.. cli:command:: cmd-name¶

Documents a command or a subcommand. Can contain arguments and flags, but not other commands.

cmd-name is name of the command in cmd namespace; it can contain spaces to separate command from subcommands:

.. cli:command:: app-example create
                 app-example c

    Documentation.

If command name contains spaces or backslashes, they should be backslash-escaped:

.. cli:command:: app\ example

Command names can also contain quoted strings and parens; spaces within them don’t break command names:

.. cli:command:: <app example>

    Documentation.

Command’s cfg name will be automatically derived by trimming leading dashes, replacing all dashes and whitespaces with underscores, and removing all non-alphanumeric symbols:

Config name for :cli:cmd:`<app example>` would be :cli:obj:`app_example`.

cfg name can be overridden by using name option.

.. cli:argument:: <metavar>¶

.. cli:flag:: -f, --flag <metavar>¶

Documents positional arguments and flags for commands.

Flags should be separated by commas and can be followed by optional metavar; positionals are addressed by their metavar only.

Additional aliases can be added by using :flag-alias: field-list:

.. cli:flag:: -f, --flag <metavar>

    Documentation.

    :flag-alias --flag={true|false}:

Declaring configs and fields¶

.. cli:config:: name¶

Documents a config. Can contain fields and nested configs.

Config names usually correspond to names of their Python classes. You can add display name to a config to override how config entry and cross-references are displayed, though:

.. cli:config:: ConfigExample
    :display-name: .app_config.yaml

    Documentation.

If your config can be loaded from CLI arguments, add parent-command option to specify which command loads it; this will set up proper cmd namespace for flags within config fields:

.. cli:config:: ConfigExample
    :parent-command: app

    .. cli:field:: example

        Documentation.

        :flag --example:

Here, Yuio will known that --example flag belongs to the command app.

.. cli:field:: name: type = default¶

Documents a config field with optional type and default value. Should not contain nested objects.

Like with configs, field names correspond to their respective Python names, and can be adjusted using display-name option.

If field can be loaded from CLI or environment variables, you can specify this by giving :flag:, :flag-alias:, and :env: field-lists:

.. cli:field:: example

    Documentation.

    :flag --example:
    :alias --example={true|false}:
    :env APP_EXAMPLE:

Declaring environment variables¶

.. cli:envvar:: name: type = value¶

Documents an environment variable with optional type and default value:

.. cli:envvar:: ENV_VAR_EXAMPLE: boolean = false

    Documentation.

Directive parameters¶

All directives that document Yuio objects accept the standard parameters:

:no-index:¶
:no-index-entry:¶
:no-contents-entry:¶
:no-typesetting:¶: The standard Sphinx options available to all object descriptions.

:annotation: <str>¶: Adds custom annotation in front of the object.

:name: <cfg-path>¶

For commands, flags, and arguments, overrides automatically generated cfg name.

These objects primarily referenced via cmd namespace, but their names in cfg namespace are used for other purposes (for example, they’re used in HTML anchors).

It is generally a good idea to keep cfg object names in sync with names used in Python code.

:display-name: <str>¶: Overrides object name that’s shown on the pace, in table of contents, and in cross-references.

:python-name: <python-path>¶: Explicitly links a manually documented object with a Python object. This can help with linking objects in auto-generated field type signatures.

Note

This will not make an object referenceable via Python domain.

:parent-command: <cmd-path>¶

For configs, specifies cmd path to a command that loads them.

Specifying this option will ensure that config fields with flags are correctly nested within the respective command.

:enum:¶: For configs, changes “config” annotation to “enum” annotation. This can be used to document enums with cli:config.

Referencing objects¶

:cli:cmd:¶

:cli:flag:¶

:cli:arg:¶

:cli:opt:¶

:cli:cli:¶

References objects in cmd namespace:

cli:cmd references commands,
cli:flag references flags,
cli:arg references arguments,
cli:opt references flags and arguments,
cli:cli references all of the above.

Space and escaping rules from cli:command apply to all cmd references; be aware that Sphinx processes escapes to separate explicit titles from references, so you’ll have to double escape slashes:

:cli:cmd:`app\\ example` will reference command app example (notice the double escape);
:cli:arg:`my-app \<input>` will reference argument <input> in command my-app;
:cli:arg:`my-app <input>` will reference argument input and give this reference an explicit title my-app. This is because role text matches Sphinx’s syntax for explicit titles: :role-name:`title <reference>`.

Example:

Reference to :cli:cmd:`app-example create`, or to :cli:flag:`app --verbose`.

:cli:cfg:¶

:cli:field:¶

:cli:obj:¶

References objects in cfg namespace:

cli:cfg references configs,
cli:field references fields,
cli:obj references any Yuio object except for environment variables.

Example:

Reference to :cli:cfg:`ConfigExample`, or to :cli:field:`Config.sub_config.field`.

:cli:env:¶

References environment variables.

Example:

Reference to :cli:env:`YUIO_DEBUG`.

:cli:any:¶: References any Yuio object. First it tries to look up object in cmd namespace, then in cfg namespace, and finally in namespace for environment variables.

Target specification¶

Yuio extension follows the standard Sphinx rules for target specification:

you may supply an explicit title and reference target: :cli:app:`application <app>` will refer to the application app, but the link text will be “application”;
if you prefix the content with an exclamation mark (!), no reference/hyperlink will be created.
if you prefix the content with ~, the link text will only be the last component of the target. For example, :cli:field:`~AppConfig.suppress_errors` will refer to AppConfig.suppress_errors, but only display suppress_errors as the link text.

Target resolution¶

Target resolution process also follows the standard Sphinx behavior.

First, it tries searching an object at the top level, then within each object in the current path.

For example, reference :cli:field:`AppConfig.suppress_errors` that appears in documentation for object AppConfig.verbosity will try the following paths, in order:

suppress_errors,
AppConfig.suppress_errors,
AppConfig.verbosity.suppress_errors.

If object path begins with a dot (in case of cmd references, dot should be followed by space), search order is reversed. For example, :cli:field:`.suppress_errors` or :cli:cmd:`. ls` will start search from the current object.

Also, if object path begins with a dot, and no exact match is found, the target is taken as a suffix and all object paths with that suffix are searched. For example, :cli:flag:`. --help` will search for any flag named --help.

Generating documentation¶

.. cli:autoobject:: <python-path>¶

This directive takes a fully qualified path to a Python object, and recursively generates documentation for it.

A given object can be an App instance, a Config, an Enum, or a field or enumerator within.

Use dots to traverse to Python properties and app parameters, and use # to traverse to subcommands:

.. cli:autoobject:: my_module.MyConfig.user[1]_

.. cli:autoobject:: my_module.app#process.inputs[2]_

Will document field user from config my_module.MyConfig.
Will document field inputs from subcommand process of app my_module.app.

As a hint to module resolver, you can use : to separate module path from object path:

.. cli:autoobject:: my_module:MyConfig.user

:no-index:
:no-index-entry:
:no-contents-entry:
:no-typesetting:: The standard Sphinx options available to all object descriptions.

:annotation: <str>
:display-name: <str>
:name: <cfg-path>
:parent-command: <cmd-path>: These options work like their counterparts from CLI directives.

:flags:¶
:flag-prefix: <str>¶: If documenting a config, enables generation of flags, and adds the given prefix to them. Has no effect if documenting a non-config.

:env:¶
:env-prefix: <str>¶: If documenting a config, enables generation of flags, and adds the given prefix to them. Has no effect if documenting a non-config.

:subcommands:¶: If documenting a command, this option will add documentation for all its subcommands as well.

:prog: <str>¶: If documenting a command or a subcommand, this option overrides program name of the top-most App object.

:by-name:¶

:no-by-name:¶

:to-dash-case:¶

:no-to-dash-case:¶

If documenting an enum or its member, this option allows specifying how enumerators are parsed (see yuio.parse.Enum).

__yuio_by_name__ and __yuio_to_dash_case__ are respected as well.

Controlling content of generated help¶

.. if-sphinx::¶

.. if-not-sphinx::¶

This directive renders its content when generating documentation with Sphinx, but how when rendering CLI help messages. This can be useful to adjust help text in docstrings to better suit the environment.

For example, we can add a link to online documentation when user runs our program with --help:

import yuio.app

@yuio.app.app
def main():
    """
    A program that does a thing.

    .. if-not-sphinx::

        See full documentation at https://www.example.com  [1]_

    """

    ...

This paragraph will not be visible in Sphinx.

.. if-opt-doc::¶

.. if-not-opt-doc::¶

This directive renders its content when it is evaluated as part of a flag/argument help, but not when it’s part of a config help.

This becomes handy when the same config class appears in documentation twice, one time as part of a command, and another time on its own.

For example, see ContainerConfig in autodoc example below. It can be loaded from flags in the app start command, but it also appears on its own in default_container_config.

.. cut-if-not-sphinx::¶

This directive cuts help message short when it’s rendered in CLI. This way, you can show a brief help in CLI, but extended help in Sphinx:

import yuio.app

@yuio.app.app
def main():
    """
    A program that does a thing.

    .. cut-if-not-sphinx::

    This portion of docstring would not be visible in CLI, only in Sphinx.

    """

    ...

Note

If you find yourself using this directive in every docstring of a config, consider setting __yuio_short_help__=True as class attribute:

import yuio.config

class MyConfig(yuio.config.Config):
    __yuio_short_help__ = True

    socket: str | None = None
    """
    First paragraph will always be visible.

    All consequent paragraphs will only show up in Sphinx, not in CLI.

    """

Autodoc example¶

Output

app¶

$ app <options> <subcommand> ...

A lightweight container manager demonstration.

This tool showcases how to build a CLI application with the Yuio library, featuring configuration file loading, environment variables, and CLI flags.

Subcommands

list, ls: List running containers.
start: Start a container.
stop: Stop a running container.

Global options

--cfg-*¶

These options override ones given in ./app_config.json and in environment variables. Run application with -vv to see loaded config.

See config documentation for details.

Misc options

-h, --help¶: Print this message and exit.

-V, --version¶: Print program version and exit.

-v, --verbose...¶: Increase output verbosity.

--bug-report¶: Print environment data for bug report and exit.

--completions [<shell>]¶

Install or update autocompletion scripts and exit.

Supported shells: all, uninstall, bash, zsh, fish, or pwsh.

--color, --no-color¶

Enable or disable ANSI colors.

Aliases:: --color={true|false|ansi|ansi-256|ansi-true}

app list¶

app ls

$ app list <options> [--filter <regexp>] [--format {text|json}]

List running containers.

Options

--filter <regexp>¶: Filter containers by name pattern.

--format {text|json}¶

Output format (text, json).

Default:: text

app start¶

$ app start <options> [-n <str>] [-c <str>] [-i] [--mount <host-path>:<container-path>] [--env <name>=<value>] <image>

Start a container.

Starts and runs a container from the given image. The container will run the specified command or the default entry point.

Arguments

<image>¶: Base image for the container.

Options

-n, --name <str>¶: Container name.

-c, --cmd <str>¶: Override the default command.

-i, --interactive¶: Run in interactive mode.

Container configuration

--mount <host-path>:<container-path>¶: Mount points (can be used multiple times).

--env <name>=<value>¶: Environment variables (can be used multiple times).

--cpu <float>¶

Maximum CPU cores the container can use.

Default is set via config.default_container_config.cpu.

--memory <float>¶

Maximum memory in GB.

Default is set via config.default_container_config.memory.

app stop¶

$ app stop <options> [-t {<seconds>|HH:MM:SS}] <container-id>

Stop a running container.

Sends SIGTERM to the container process. If it doesn’t stop within the timeout, SIGKILL is sent.

Arguments

<container-id>¶: Container name or ID to stop.

Options

-t, --timeout {<seconds>|HH:MM:SS}¶

Timeout before force-killing.

Default:: 10.0

config app_config.json¶

Global configuration for the container manager.

Settings are loaded in this order (later overrides earlier):

default values in code,
configuration file (./app_config.json),
environment variables (APP_*),
CLI flags and arguments (--cfg-*).

registry: string = "https://docker.io"¶

Default registry URL.

Flags:: --cfg-registry <str>
Environment variables:: APP_REGISTRY

data_dir: string = "~/.containers"¶

Data directory for container storage.

Flags:: --cfg-data-dir <dir>
Environment variables:: APP_DATA_DIR

default_container_config¶

Default config used with app start.

cpu: number = 1.0¶

Default value for maximum CPU cores the container can use.

Environment variables:: APP_APP_CPU

memory: number = 0.5¶

Default value for maximum memory in GB.

Environment variables:: APP_APP_MEMORY

RST

.. cli:autoobject:: app_example.main
    :prog: app
    :subcommands:
    :no-contents-entry:

.. cli:autoobject:: app_example.GlobalConfig
    :name: config
    :display-name: app_config.json
    :parent-command: app
    :flags:
    :flag-prefix: --cfg
    :env:
    :env-prefix: APP
    :no-contents-entry:

Python

#! /usr/bin/env python

from __future__ import annotations

import datetime
import logging
import pathlib

import yuio.app
import yuio.config
import yuio.io
import yuio.parse
import yuio.ty

from typing import Annotated, Literal

_logger = logging.getLogger(__name__)


class ContainerConfig(yuio.config.Config):
    """
    Default config used with :cli:cmd:`app start`.

    """

    #: .. if-opt-doc::
    #:
    #:      Maximum CPU cores the container can use.
    #:
    #:      Default is set via :cli:field:`config.default_container_config.cpu`.
    #:
    #: .. if-not-opt-doc::
    #:
    #:      Default value for maximum CPU cores the container can use.
    cpu: yuio.ty.PosFloat = yuio.app.field(default=1.0, default_desc="")

    #: .. if-opt-doc::
    #:
    #:      Maximum memory in GB.
    #:
    #:      Default is set via :cli:field:`config.default_container_config.memory`.
    #:
    #: .. if-not-opt-doc::
    #:
    #:      Default value for maximum memory in GB.
    memory: yuio.ty.PosFloat = yuio.app.field(default=0.5, default_desc="")


class GlobalConfig(yuio.config.Config):
    """
    Global configuration for the container manager.

    Settings are loaded in this order (later overrides earlier):

    1. default values in code,
    2. configuration file (``./app_config.json``),
    3. environment variables (``APP_*``),
    4. CLI flags and arguments (``--cfg-*``).

    """

    #: Default registry URL.
    registry: str = "https://docker.io"

    #: Data directory for container storage.
    data_dir: yuio.ty.Dir = pathlib.Path("~/.containers")

    #: Default config used with :cli:cmd:`app start`.
    default_container_config: ContainerConfig = yuio.app.field(
        default=yuio.MISSING, flags=yuio.DISABLED, env="APP"
    )


# Global config instance, will be loaded in `main`.
GLOBAL_CONFIG = GlobalConfig()


@yuio.app.app(
    version="1.0.0",  # Adds `--version`.
    bug_report=True,  # Adds `--bug-report`.
    is_dev_mode=True,  # Prints warnings from Yuio.
)
def main(
    #: Global options.
    #:
    #: These options override ones given in :file:`./app_config.json`
    #: and in environment variables. Run application with
    #: :cli:flag:`-vv <-v>` to see loaded config.
    #:
    #: .. if-sphinx::
    #:
    #:      See :cli:cfg:`config documentation <config>` for details.
    #:
    #: .. if-not-sphinx::
    #:
    #:      See `config documentation`__ for details.
    #:
    #:      __ https://yuio.readthedocs.io/en/stable/main_api/sphinx_ext.html#cli-config
    #:
    config: GlobalConfig = yuio.config.field(
        flags="--cfg", usage=yuio.COLLAPSE, help_group=yuio.COLLAPSE
    ),
):
    """
    A lightweight container manager demonstration.

    This tool showcases how to build a CLI application with the Yuio library,
    featuring configuration file loading, environment variables, and CLI flags.

    """

    config_file = pathlib.Path(__file__).parent / "app_config.json"

    GLOBAL_CONFIG.update(GlobalConfig.load_from_json_file(config_file))
    GLOBAL_CONFIG.update(GlobalConfig.load_from_env("APP"))
    GLOBAL_CONFIG.update(config)

    _logger.debug("Global config loaded: %#+r", GLOBAL_CONFIG)


main.epilog = """
.. if-not-sphinx::

    Formatting
    ----------

    Prolog is formatted using ReStructuredText. Yuio supports all RST formatting except
    tables and option lists. It also doesn't have access to all RST directives and roles,
    so only a limited subset of them is available.

    Example of what we can do:


    Quotes
    ~~~~~~

        | Beautiful python
        | Explicit and simple form
        | Winding through clouds
        |
        | -- from heroku art


    Numbered lists
    ~~~~~~~~~~~~~~

    1.  First item,
    #.  second item,

    a.  or using letters,
    #.  continues.


    Code blocks
    ~~~~~~~~~~~

    .. code-block:: python

        for i in range(10):
            print(f"Hello, {i}!")


    Admonitions
    ~~~~~~~~~~~

    .. note::

        This is an admonition.


    Definition and field lists
    ~~~~~~~~~~~~~~~~~~~~~~~~~~

    term
        Definition of the term.

    :field: Contents of the field.


    Inline markup
    ~~~~~~~~~~~~~

    -   **strong emphasis**;
    -   *emphasis*;
    -   ``code``;
    -   flag: :flag:`--flag`;
    -   python roles: :class:`~yuio.app.App`;
    -   other roles: :file:`{HOME}/.config`, click :menuselection:`&File-->New &Window`,
        then :guilabel:`&Create`;
    -   footnotes [#]_ and hyperlinks__.

    __ https://example.com
    .. [#] This is a footnote.
"""


@main.subcommand(name="list", aliases=["ls"])
def ls(
    #: Filter containers by name pattern.
    filter: Annotated[str, yuio.parse.WithMeta(desc="<regexp>")] | None = None,
    #: Output format (text, json).
    format: Literal["text", "json"] = "text",
):
    """
    List running containers.

    """

    yuio.io.info("Listing containers (filter=%r, format=%r)", filter, format)


CONTAINER_CONFIG_GROUP = yuio.app.HelpGroup("Container configuration")


@main.subcommand()
def start(
    #: Base image for the container.
    image: str,
    /,
    *,
    #: Container name.
    name: str | None = yuio.config.field(default=None, flags=["-n", "--name"]),
    #: Override the default command.
    command: str | None = yuio.config.field(default=None, flags=["-c", "--cmd"]),
    #: Run in interactive mode.
    interactive: bool = yuio.config.field(default=False, flags=["-i", "--interactive"]),
    #: Mount points (can be used multiple times).
    mounts: dict[
        Annotated[
            pathlib.Path,
            yuio.parse.WithMeta(desc="<host-path>"),
        ],
        Annotated[
            yuio.ty.NonEmptyStr,
            yuio.parse.WithMeta(desc="<container-path>"),
        ],
    ] = yuio.config.field(
        default={},
        flags="--mount",
        option_ctor=yuio.config.collect_option(),
        help_group=CONTAINER_CONFIG_GROUP,
    ),
    #: Environment variables (can be used multiple times).
    env_vars: Annotated[
        dict[
            Annotated[
                str,
                yuio.parse.WithMeta(desc="<name>"),
            ],
            Annotated[
                str,
                yuio.parse.WithMeta(desc="<value>"),
            ],
        ],
        yuio.parse.Dict(pair_delimiter="="),
    ] = yuio.config.field(
        default={},
        flags="--env",
        option_ctor=yuio.config.collect_option(),
        help_group=CONTAINER_CONFIG_GROUP,
    ),
    #: Container configuration.
    container_config: ContainerConfig = yuio.config.inline(
        usage=yuio.COLLAPSE,
        help_group=CONTAINER_CONFIG_GROUP,
    ),
):
    """
    Start a container.

    Starts and runs a container from the given image.
    The container will run the specified command or the default entry point.

    """

    yuio.io.info(
        "Running container %s (name=%r, interactive=%r)", image, name, interactive
    )
    yuio.io.info("Command: %s", command or "(default)")

    full_container_config = ContainerConfig()
    full_container_config.update(GLOBAL_CONFIG.default_container_config)
    full_container_config.update(container_config)

    yuio.io.info("Container config: %#+r", full_container_config)
    yuio.io.info("Mounts: %#+r", mounts)
    yuio.io.info("Env vars: %#+r", env_vars)


@main.subcommand()
def stop(
    #: Container name or ID to stop.
    container_id: str,
    /,
    *,
    #: Timeout before force-killing.
    timeout: yuio.ty.NonNegSeconds | yuio.ty.NonNegTimeDelta = yuio.config.field(
        default=datetime.timedelta(seconds=10), flags=["-t", "--timeout"]
    ),
):
    """
    Stop a running container.

    Sends ``SIGTERM`` to the container process. If it doesn't stop within the timeout,
    ``SIGKILL`` is sent.

    """

    yuio.io.info("Stopping container %s (timeout: %r)", container_id, timeout)


if __name__ == "__main__":
    main.run()