Parse¶

Everything to do with parsing user input.

Use provided classes to construct parsers and add validation:

>>> # Parses a string that matches the given regex.
>>> ident = Regex(Str(), r'^[a-zA-Z_][a-zA-Z0-9_]*$')

>>> # Parses a non-empty list of strings.
>>> idents = LenGe(List(ident), 1)

Pass a parser to other yuio functions:

>>> yuio.io.ask('List of modules to reformat', parser=idents)

Or parse strings yourself:

>>> idents.parse('sys os enum dataclasses')
['sys', 'os', 'enum', 'dataclasses']

Build a parser from type hints:

>>> from_type_hint(list[int] | None)
Optional(List(Int))

Parser basics¶

All parsers are derived from the same base class Parser, which describes parsing API.

class yuio.parse.Parser¶

Base class for parsers.

final parse(value: str, /) → T_co¶

Parse user input, raise ParsingError on failure.

Parameters:: value – value to parse.
Returns:: a parsed and processed value.
Raises:: ParsingError.

parse_many(value: Sequence[str], /) → T_co¶

For collection parsers, parse and validate collection by parsing its items one-by-one.

Parameters:

value – collection of values to parse.

Returns:

each value parsed and assembled into the target collection.

Raises:

ParsingError. Also raises RuntimeError if trying to call this method on a parser that doesn’t supports parsing collections of objects.

Example:

>>> # Let's say we're parsing a set of ints.
>>> parser = Set(Int())

>>> # And the user enters collection items one-by-one.
>>> user_input = ['1', '2', '3']

>>> # We can parse collection from its items:
>>> parser.parse_many(user_input)
{1, 2, 3}

abstractmethod supports_parse_many() → bool¶

Return True if this parser returns a collection and so supports parse_many().

Returns:: True if parse_many() is safe to call.

final parse_config(value: object, /) → T_co¶

Parse value from a config, raise ParsingError on failure.

This method accepts python values that would result from parsing json, yaml, and similar formats.

Parameters:

value – config value to parse.

Returns:

verified and processed config value.

Raises:

ParsingError.

Example:

>>> # Let's say we're parsing a set of ints.
>>> parser = Set(Int())

>>> # And we're loading it from json.
>>> import json
>>> user_config = json.loads('[1, 2, 3]')

>>> # We can process parsed json:
>>> parser.parse_config(user_config)
{1, 2, 3}

class yuio.parse.ParsingError(

msg: LiteralString,

/,

*args: Any,

ctx: ConfigParsingContext | StrParsingContext | None = None,

fallback_msg: LiteralString | None = None,

**kwargs,

)¶

class yuio.parse.ParsingError(

msg: str,

/,

*,

ctx: ConfigParsingContext | StrParsingContext | None = None,

fallback_msg: LiteralString | None = None,

**kwargs,

)

Raised when parsing or validation fails.

Parameters:

msg –
message to format. Can be a literal string or any other colorable object.

If it’s given as a literal string, additional arguments for %-formatting may be given. Otherwise, giving additional arguments will cause a TypeError.
args – arguments for %-formatting the message.
fallback_msg –
fallback message that’s guaranteed not to include representation of the faulty value, will replace msg when parsing secret values.

Warning

This parameter must not include contents of the faulty value. It is typed as LiteralString as a deterrent; if you need string interpolation, create an instance of ParsingError and set fallback_msg directly.
ctx – current error context that will be used to set raw, pos, and other attributes.
kwargs – other keyword arguments set raw, pos, n_arg, path.

fallback_msg: Printable | ColorizedStrProtocol | ColorizedReprProtocol | RichReprProtocol | str | BaseException | None¶: This message will be used if error occurred while parsing a secret value.

Warning

This colorable must not include contents of the faulty value.

raw: str | None¶: For errors that happened when parsing a string, this attribute contains the original string.

pos: tuple[int, int] | None¶: For errors that happened when parsing a string, this attribute contains position in the original string in which this error has occurred (start and end indices).

n_arg: int | None¶: For errors that happened in parse_many(), this attribute contains index of the string in which this error has occurred.

path: list[tuple[Any, str | None]] | None¶: For errors that happened in parse_config_with_ctx(), this attribute contains path to the value in which this error has occurred.

classmethod type_mismatch(value: Any, /, *expected: type | str, **kwargs)¶

Make an error with a standard message “expected type X, got type Y”.

Parameters:

value – value of an unexpected type.
expected – expected types. Each argument can be a type or a string that describes a type.
kwargs – keyword arguments will be passed to constructor.

Example:

>>> raise ParsingError.type_mismatch(10, str)
Traceback (most recent call last):
...
yuio.parse.ParsingError: Expected str, got int: 10

Value parsers¶

class yuio.parse.Str¶: Parser for str values.

class yuio.parse.Int¶: Parser for int values.

class yuio.parse.Float¶: Parser for float values.

class yuio.parse.Bool¶: Parser for bool values, such as "yes" or "no".

class yuio.parse.Enum( enum_type: Type[E], /, *, by_name: bool | None = None, to_dash_case: bool | None = None, doc_inline: bool = False, )¶

Parser for enums, as defined in the standard enum module.

Parameters:

enum_type – enum class that will be used to parse and extract values.
by_name –
if True, the parser will use enumerator names, instead of their values, to match the input.

If not given, Yuio will search for __yuio_by_name__ attribute on the given enum class to infer value for this option.
to_dash_case –
convert enum names/values to dash case.

If not given, Yuio will search for __yuio_to_dash_case__ attribute on the given enum class to infer value for this option.
doc_inline –
inline this enum in json schema and in documentation.

Useful for small enums that don’t warrant a separate section in documentation.

If not given, Yuio will search for __yuio_doc_inline__ attribute on the given enum class to infer value for this option.

class yuio.parse.Literal(*literal_values: L)¶

Parser for literal values.

This parser accepts a set of allowed values, and parses them using semantics of Enum parser. It can be used with creating an enum for some value isn’t practical, and semantics of OneOf is limiting.

Allowed values should be strings, ints, bools, or instances of enum.Enum.

If instances of enum.Enum are passed, Literal will rely on enum’s __yuio_by_name__ and __yuio_to_dash_case__ attributes to parse these values.

class yuio.parse.Decimal¶: Parser for decimal.Decimal.

class yuio.parse.Fraction¶: Parser for fractions.Fraction.

class yuio.parse.DateTime¶: Parse a datetime in ISO (‘YYYY-MM-DD HH:MM:SS’) format.

class yuio.parse.Date¶: Parse a date in ISO (‘YYYY-MM-DD’) format.

class yuio.parse.Time¶: Parse a time in ISO (‘HH:MM:SS’) format.

class yuio.parse.TimeDelta¶: Parse a time delta.

class yuio.parse.Seconds¶: Parse a float and convert it to a time delta as a number of seconds.

class yuio.parse.Json(inner: Parser[T] | None = None, /)¶

A parser that tries to parse value as JSON.

This parser will load JSON strings into python objects. If inner parser is given, Json will validate parsing results by calling parse_config_with_ctx() on the inner parser.

Parameters:: inner – a parser used to convert and validate contents of json.

class yuio.parse.List(inner: Parser[T], /, *, delimiter: str | None = None)¶

Parser for lists.

Will split a string by the given delimiter, and parse each item using a subparser.

Parameters:

inner – inner parser that will be used to parse list items.
delimiter – delimiter that will be passed to str.split().

class yuio.parse.Set(inner: Parser[T], /, *, delimiter: str | None = None)¶

Parser for sets.

Will split a string by the given delimiter, and parse each item using a subparser.

Parameters:

inner – inner parser that will be used to parse set items.
delimiter – delimiter that will be passed to str.split().

class yuio.parse.FrozenSet(inner: Parser[T], /, *, delimiter: str | None = None)¶

Parser for frozen sets.

Will split a string by the given delimiter, and parse each item using a subparser.

Parameters:

inner – inner parser that will be used to parse set items.
delimiter – delimiter that will be passed to str.split().

class yuio.parse.Dict( key: Parser[K], value: Parser[V], /, *, delimiter: str | None = None, pair_delimiter: str = ':', )¶

Parser for dicts.

Will split a string by the given delimiter, and parse each item using a Tuple parser.

Parameters:

key – inner parser that will be used to parse dict keys.
value – inner parser that will be used to parse dict values.
delimiter – delimiter that will be passed to str.split().
pair_delimiter – delimiter that will be used to split key-value elements.

class yuio.parse.Tuple(*parsers: Parser[...], delimiter: str | None = None)¶

Parser for tuples of fixed lengths.

Parameters:

parsers – parsers for each tuple element.
delimiter – delimiter that will be passed to str.split().

class yuio.parse.Optional(inner: Parser[T], /)¶

Parser for optional values.

Allows handling Nones when parsing config. Does not change how strings are parsed, though.

Parameters:: inner – a parser used to extract and validate contents of an optional.

class yuio.parse.Union(*parsers: Parser[T])¶

Tries several parsers and returns the first successful result.

Warning

Order of parsers matters. Since parsers are tried in the same order as they’re given, make sure to put parsers that are likely to succeed at the end.

For example, this parser will always return a string because Str can’t fail:

>>> parser = Union(Str(), Int())  # Always returns a string!
>>> parser.parse("10")
'10'

To fix this, put Str at the end so that Int is tried first:

>>> parser = Union(Int(), Str())
>>> parser.parse("10")
10
>>> parser.parse("not an int")
'not an int'

class yuio.parse.Path(*, extensions: str | Collection[str] | None = None)¶

Parse a file system path, return a pathlib.Path.

Parameters:: extensions – list of allowed file extensions, including preceding dots.

class yuio.parse.NonExistentPath(*, extensions: str | Collection[str] | None = None)¶

Parse a file system path and verify that it doesn’t exist.

Parameters:: extensions – list of allowed file extensions, including preceding dots.

class yuio.parse.ExistingPath(*, extensions: str | Collection[str] | None = None)¶

Parse a file system path and verify that it exists.

Parameters:: extensions – list of allowed file extensions, including preceding dots.

class yuio.parse.File(*, extensions: str | Collection[str] | None = None)¶

Parse a file system path and verify that it points to a regular file.

Parameters:: extensions – list of allowed file extensions, including preceding dots.

class yuio.parse.Dir¶: Parse a file system path and verify that it points to a directory.

class yuio.parse.GitRepo¶

Parse a file system path and verify that it points to a git repository.

This parser just checks that the given directory has a subdirectory named .git.

class yuio.parse.Secret(inner: Parser[U], /)¶: Wraps result of the inner parser into SecretValue and ensures that yuio.io.ask() doesn’t show value as user enters it.

Validators¶

class yuio.parse.Regex( inner: Parser[str], regex: str | re.Pattern[str], /, *, group: int | str = 0, )¶

Matches the parsed string with the given regular expression.

If regex has capturing groups, parser can return contents of a group.

Parameters:

regex – regular expression for matching.
group – name or index of a capturing group that should be used to get the final parsed value.

class yuio.parse.Bound( inner: Parser[Cmp], /, *, lower: Cmp | None = None, lower_inclusive: Cmp | None = None, upper: Cmp | None = None, upper_inclusive: Cmp | None = None, )¶

Check that value is upper- or lower-bound by some constraints.

Parameters:

inner – parser whose result will be validated.
lower – set lower bound for value, so we require that value > lower. Can’t be given if lower_inclusive is also given.
lower_inclusive – set lower bound for value, so we require that value >= lower. Can’t be given if lower is also given.
upper – set upper bound for value, so we require that value < upper. Can’t be given if upper_inclusive is also given.
upper_inclusive – set upper bound for value, so we require that value <= upper. Can’t be given if upper is also given.

Example:

>>> # Int in range `0 < x <= 1`:
>>> Bound(Int(), lower=0, upper_inclusive=1)
Bound(Int, 0 < x <= 1)

class yuio.parse.Gt(inner: Parser[Cmp], bound: Cmp, /)¶

Alias for Bound.

Parameters:

inner – parser whose result will be validated.
bound – lower bound for parsed values.

class yuio.parse.Ge(inner: Parser[Cmp], bound: Cmp, /)¶

Alias for Bound.

Parameters:

inner – parser whose result will be validated.
bound – lower inclusive bound for parsed values.

class yuio.parse.Lt(inner: Parser[Cmp], bound: Cmp, /)¶

Alias for Bound.

Parameters:

inner – parser whose result will be validated.
bound – upper bound for parsed values.

class yuio.parse.Le(inner: Parser[Cmp], bound: Cmp, /)¶

Alias for Bound.

Parameters:

inner – parser whose result will be validated.
bound – upper inclusive bound for parsed values.

class yuio.parse.LenBound( inner: Parser[Sz], /, *, lower: int | None = None, lower_inclusive: int | None = None, upper: int | None = None, upper_inclusive: int | None = None, )¶

Check that length of a value is upper- or lower-bound by some constraints.

The signature is the same as of the Bound class.

Parameters:

inner – parser whose result will be validated.
lower – set lower bound for value’s length, so we require that len(value) > lower. Can’t be given if lower_inclusive is also given.
lower_inclusive – set lower bound for value’s length, so we require that len(value) >= lower. Can’t be given if lower is also given.
upper – set upper bound for value’s length, so we require that len(value) < upper. Can’t be given if upper_inclusive is also given.
upper_inclusive – set upper bound for value’s length, so we require that len(value) <= upper. Can’t be given if upper is also given.

Example:

>>> # List of up to five ints:
>>> LenBound(List(Int()), upper_inclusive=5)
LenBound(List(Int), len <= 5)

class yuio.parse.LenGt(inner: Parser[Sz], bound: int, /)¶

Alias for LenBound.

Parameters:

inner – parser whose result will be validated.
bound – lower bound for parsed values’s length.

class yuio.parse.LenGe(inner: Parser[Sz], bound: int, /)¶

Alias for LenBound.

Parameters:

inner – parser whose result will be validated.
bound – lower inclusive bound for parsed values’s length.

class yuio.parse.LenLt(inner: Parser[Sz], bound: int, /)¶

Alias for LenBound.

Parameters:

inner – parser whose result will be validated.
bound – upper bound for parsed values’s length.

class yuio.parse.LenLe(inner: Parser[Sz], bound: int, /)¶

Alias for LenBound.

Parameters:

inner – parser whose result will be validated.
bound – upper inclusive bound for parsed values’s length.

class yuio.parse.OneOf(inner: Parser[T], values: Collection[T], /)¶

Check that the parsed value is one of the given set of values.

Note

This parser is meant to validate results of other parsers; if you’re looking to parse enums or literal values, check out Enum or Literal.

Parameters:

inner – parser whose result will be validated.
values – collection of allowed values.

Example:

>>> # Accepts only strings 'A', 'B', or 'C':
>>> OneOf(Str(), ['A', 'B', 'C'])
OneOf(Str)

Auxiliary parsers¶

class yuio.parse.Map( inner: Parser[U], fn: Callable[[U], T], rev: Callable[[T | object], U] | None = None, /, )¶

A wrapper that maps result of the given parser using the given function.

Parameters:

inner – a parser whose result will be mapped.
fn – a function to convert a result.
rev –
a function used to un-map a value.

This function is used in Parser.describe_value() and Parser.to_json_value() to convert parsed value back to its original state.

Note that, since parser’s type parameter is covariant, this function is not guaranteed to receive a value of the same type that this parser produces. In this case, you should raise a TypeError.

Example:

>>> parser = yuio.parse.Map(
...     yuio.parse.Int(),
...     lambda x: 2 ** x,
...     lambda x: int(math.log2(x)),
... )
>>> parser.parse("10")
1024
>>> parser.describe_value(1024)
'10'

class yuio.parse.Apply(inner: Parser[T], fn: Callable[[T], None], /)¶

A wrapper that applies the given function to the result of a wrapped parser.

Parameters:

inner – a parser used to extract and validate a value.
fn – a function that will be called after parsing a value.

Example:

>>> # Run `Int` parser, then print its output before returning.
>>> print_output = Apply(Int(), lambda x: print(f"Value is {x}"))
>>> result = print_output.parse("10")
Value is 10
>>> result
10

class yuio.parse.Lower(inner: Parser[str], /)¶

Applies str.lower() to the result of a string parser.

Parameters:: inner – a parser whose result will be mapped.

class yuio.parse.Upper(inner: Parser[str], /)¶

Applies str.upper() to the result of a string parser.

Parameters:: inner – a parser whose result will be mapped.

class yuio.parse.CaseFold(inner: Parser[str], /)¶

Applies str.casefold() to the result of a string parser.

Parameters:: inner – a parser whose result will be mapped.

class yuio.parse.Strip(inner: Parser[str], /)¶

Applies str.strip() to the result of a string parser.

Parameters:: inner – a parser whose result will be mapped.

class yuio.parse.WithMeta( inner: Parser[T], /, *, desc: str, completer: yuio.complete.Completer | None | MISSING = MISSING, )¶

Overrides inline help messages and other meta information of a wrapped parser.

Inline help messages will show up as hints in autocompletion and widgets.

Parameters:

inner – inner parser.
desc – description override. This short string will be used in CLI, widgets, and completers to describe expected value.
completer – completer override. Pass None to disable completion.

Deriving parsers from type hints¶

There is a way to automatically derive basic parsers from type hints (used by yuio.config):

yuio.parse.from_type_hint(ty: type[T], /) → Parser[T]¶

Create parser from a type hint.

Parameters:

ty –

a type hint.

This type hint should not contain strings or forward references. Make sure they’re resolved before passing it to this function.

Returns:

a parser instance created from type hint.

Raises:

TypeError if type hint contains forward references or types that don’t have associated parsers.

Example:

>>> from_type_hint(list[int] | None)
Optional(List(Int))

Partial parsers¶

Sometimes it’s not convenient to provide a parser for a complex type when all we need is to make a small adjustment to a part of the type. For example:

class AppConfig(Config):
    max_line_width: int | str = field(
        default="default",
        parser=Union(
            Gt(Int(), 0),
            OneOf(Str(), ["default", "unlimited", "keep"]),
        ),
    )

Instead, we can use typing.Annotated to attach validating parsers directly to type hints:

from typing import Annotated


class AppConfig(Config):
    max_line_width: (
        Annotated[int, Gt(0)]
        | Annotated[str, OneOf(["default", "unlimited", "keep"])]
    ) = "default"

Notice that we didn’t specify inner parsers for Gt and OneOf. This is because their internal parsers are derived from type hint, so we only care about their settings.

Parsers created in such a way are called “partial”. You can’t use a partial parser on its own because it doesn’t have full information about the object’s type. You can only use partial parsers in type hints:

>>> partial_parser = List(delimiter=",")
>>> partial_parser.parse_with_ctx("1,2,3")
Traceback (most recent call last):
...
TypeError: List requires an inner parser
...

Other parser methods¶

Parser defines some more methods and attributes. They’re rarely used because Yuio handles everything they do itself. However, you can still use them in case you need to.

class yuio.parse.Parser

Base class for parsers.

__wrapped_parser__: Parser[object] | None = None¶: An attribute for unwrapping parsers that validate or map results of other parsers.

abstractmethod parse_with_ctx(ctx: StrParsingContext, /) → T_co¶

Actual implementation of parse(), receives parsing context instead of a raw string.

Parameters:: ctx – value to parse, wrapped into a parsing context.
Returns:: a parsed and processed value.
Raises:: ParsingError.

abstractmethod parse_many_with_ctx( ctxs: Sequence[StrParsingContext], /, ) → T_co¶

Actual implementation of parse_many(), receives parsing contexts instead of a raw strings.

Parameters:: ctxs – values to parse, wrapped into a parsing contexts.
Returns:: a parsed and processed value.
Raises:: ParsingError.

abstractmethod parse_config_with_ctx( ctx: ConfigParsingContext, /, ) → T_co¶

Actual implementation of parse_config(), receives parsing context instead of a raw value.

Parameters:: ctx – config value to parse, wrapped into a parsing contexts.
Returns:: verified and processed config value.
Raises:: ParsingError.

abstractmethod get_nargs() → Literal['+', '*'] | int¶

Generate nargs for argparse.

Returns:: nargs as defined by argparse. If supports_parse_many() returns True, value should be "+" or an integer. Otherwise, value should be 1.

abstractmethod check_type(value: object, /) → TypeGuard[T_co]¶

Check whether the parser can handle a particular value in its describe_value() and other methods.

This function is used to raise TypeErrors in function that accept unknown values. Parsers like Union rely on TypeErrors to dispatch values to correct sub-parsers.

Note

For performance reasons, this method should not inspect contents of containers, only their type (otherwise some methods turn from linear to quadratic).

This also means that validating and mapping parsers can always return True.

Parameters:: value – value that needs a type check.
Returns:: True if the value matches the type of this parser.

assert_type(value: object, /) → TypeGuard[T_co]¶

Call check_type() and raise a TypeError if it returns False.

This method always returns True or throws an error, but type checkers don’t know this. Use assert parser.assert_type(value) so that they understand that type of the value has narrowed.

Parameters:: value – value that needs a type check.
Returns:: always returns True.
Raises:: TypeError.

abstractmethod describe() → str | None¶

Return a human-readable description of an expected input.

Used to describe expected input in widgets.

Returns:: human-readable description of an expected input. Can return None for simple values that don’t need a special description.

abstractmethod describe_or_def() → str¶

Like describe(), but guaranteed to return something.

Used to describe expected input in CLI help.

Returns:: human-readable description of an expected input.

abstractmethod describe_many() → str | tuple[str, ...]¶

Return a human-readable description of a container element.

Used to describe expected input in CLI help.

Returns:: human-readable description of expected inputs. If the value is a string, then it describes an individual member of a collection. The the value is a tuple, then each of the tuple’s element describes an expected value at the corresponding position.
Raises:: RuntimeError if trying to call this method on a parser that doesn’t supports parsing collections of objects.

abstractmethod describe_value(value: object, /) → str¶

Return a human-readable description of the given value.

Used in error messages, and to describe returned input in widgets.

Note that, since parser’s type parameter is covariant, this function is not guaranteed to receive a value of the same type that this parser produces. Call assert_type() to check for this case.

Parameters:: value – value that needs a description.
Returns:: description of a value in the format that this parser would expect to see in a CLI argument or an environment variable.
Raises:: TypeError if the given value is not of type that this parser produces.

abstractmethod options() → Collection[Option[T_co]] | None¶

Return options for a Choice or a Multiselect widget.

This function can be implemented for parsers that return a fixed set of pre-defined values, like Enum or Literal. Collection and union parsers may use this data to improve their widgets. For example, the Set parser will use a Multiselect widget.

Returns:

a full list of options that will be passed to a choice widget, or None if the set of possible values is not known.

Note that returning None is not equivalent to returning an empty array: None signals other parsers that they can’t use choice widgets, while an empty array signals that there are simply no choices to add.

abstractmethod completer() → Completer | None¶

Return a completer for values of this parser.

This function is used when assembling autocompletion functions for shells, and when reading values from user via yuio.io.ask().

Returns:: a completer that will be used with CLI arguments or widgets.

abstractmethod widget( default: object | Missing, input_description: str | None, default_description: str | None, /, ) → Widget[T_co | Missing]¶

Return a widget for reading values of this parser.

This function is used when reading values from user via yuio.io.ask().

The returned widget must produce values of type T. If default is given, and the user input is empty, the widget must produce the MISSING constant (not the default constant). This is because the default value might be of any type (for example None), and validating parsers should not check it.

Validating parsers must wrap the widget they got from __wrapped_parser__ into Map or Apply in order to validate widget’s results.

Parameters:

default – default value that will be used if widget returns MISSING.
input_description – a string describing what input is expected.
default_description – a string describing default value.

Returns:

a widget that will be used to ask user for values. The widget can choose to use completer() or options(), or implement some custom logic.

abstractmethod to_json_schema( ctx: JsonSchemaContext, /, ) → JsonSchemaType¶

Create a JSON schema object based on this parser.

The purpose of this method is to make schemas for use in IDEs, i.e. to provide autocompletion or simple error checking. The returned schema is not guaranteed to reflect all constraints added to the parser. For example, OneOf and Regex parsers will not affect the generated schema.

Parameters:: ctx – context for building a schema.
Returns:: a JSON schema that describes structure of values expected by this parser.

abstractmethod to_json_value(value: object, /) → JsonValue¶

Convert given value to a representation suitable for JSON serialization.

Note that, since parser’s type parameter is covariant, this function is not guaranteed to receive a value of the same type that this parser produces. Call assert_type() to check for this case.

Returns:: a value converted to JSON-serializable representation.
Raises:: TypeError if the given value is not of type that this parser produces.

abstractmethod is_secret() → bool¶: Indicates that input functions should use secret input, i.e. getpass() or yuio.widget.SecretInput.

Building your own parser¶

Understanding parser hierarchy¶

The topmost class in the parser hierarchy is PartialParser. It provides abstract methods to deal with partial parsers. The primary parser interface, Parser, is derived from it. Below Parser, there are several abstract classes that provide boilerplate implementations for common use cases.

---
config:
    class:
        hideEmptyMembersBox: true
---
classDiagram

class PartialParser
click PartialParser href "#yuio.parse.PartialParser" "yuio.parse.PartialParser"

class Parser
click Parser href "#yuio.parse.Parser" "yuio.parse.Parser"
PartialParser <|-- Parser

class ValueParser
click ValueParser href "#yuio.parse.ValueParser" "yuio.parse.ValueParser"
Parser <|-- ValueParser

class WrappingParser
click WrappingParser href "#yuio.parse.WrappingParser" "yuio.parse.WrappingParser"
Parser <|-- WrappingParser

class MappingParser
click MappingParser href "#yuio.parse.MappingParser" "yuio.parse.MappingParser"
WrappingParser <|-- MappingParser

class Map
click Map href "#yuio.parse.Map" "yuio.parse.Map"
MappingParser <|-- Map

class Apply
click Apply href "#yuio.parse.Apply" "yuio.parse.Apply"
MappingParser <|-- Apply

class ValidatingParser
click ValidatingParser href "#yuio.parse.ValidatingParser" "yuio.parse.ValidatingParser"
Apply <|-- ValidatingParser

class CollectionParser
click CollectionParser href "#yuio.parse.CollectionParser" "yuio.parse.CollectionParser"
ValueParser <|-- CollectionParser
WrappingParser <|-- CollectionParser

The reason for separation of PartialParser and Parser is better type checking. We want to prevent users from making a mistake of providing a partial parser to a function that expect a fully initialized parser. For example, consider this code:

yuio.io.ask("Enter some names", parser=List())

This will fail because List needs an inner parser to function.

To annotate this behavior, we provide type hints for __new__ methods on each parser. When an inner parser is given, __new__ is annotated as returning an instance of Parser. When inner parser is omitted, __new__ is annotated as returning an instance of PartialParser:

from typing import TYPE_CHECKING, Any, Generic, overload

class List(..., Generic[T]):
    if TYPE_CHECKING:
        @overload
        def __new__(cls, delimiter: str | None = None) -> PartialParser:
            ...
        @overload
        def __new__(cls, inner: Parser[T], delimiter: str | None = None) -> PartialParser:
            ...
        def __new__(cls, *args, **kwargs) -> Any:
            ...

With these type hints, our example will fail to type check: yuio.io.ask() expects a Parser, but List.__new__ returns a PartialParser.

Unfortunately, this means that all parsers derived from WrappingParser must provide appropriate type hints for their __new__ method.

class yuio.parse.PartialParser¶

An interface of a partial parser.

abstractmethod wrap( parser: Parser[Any], ) → Parser[Any]¶

Apply this partial parser.

When Yuio checks type annotations, it derives a parser for the given type hint, and the applies all partial parsers to it.

For example, given this type hint:

field: Annotated[str, Map(str.lower)]

Yuio will first infer parser for string (Str), then it will pass this parser to Map.wrap.

Parameters:: parser – a parser instance that was created by inspecting type hints and previous annotations.
Returns:: a result of upgrading this parser from partial to full. This method usually returns copy of self.
Raises:: TypeError if this parser can’t be wrapped. Specifically, this method should raise a TypeError for any non-partial parser.

Parsing contexts¶

To track location of errors, parsers work with parsing context: StrParsingContext for parsing raw strings, and ConfigParsingContext for parsing configs.

When raising a ParsingError, pass context to it so that we can show error location to the user.

class yuio.parse.StrParsingContext(content: str, /, *, n_arg: int | None = None)¶

String parsing context tracks current position in the string.

Parameters:

content – content to parse.
n_arg – content index when using parse_many().

start: int¶: Start position of the value.

end: int¶: End position of the value.

content: str¶: Full content of the value that was passed to Parser.parse().

value: str¶: Part of the content that’s currently being parsed.

n_arg: int | None¶: For parse_many(), this attribute contains index of the value that is being parsed. For parse(), this is None.

split( delimiter: str | None = None, /, maxsplit: int = -1, ) → Generator[StrParsingContext, None, None]¶: Split current value by the given delimiter while keeping track of the current position.

strip( chars: str | None = None, /, ) → StrParsingContext¶: Strip current value while keeping track of the current position.

strip_if_non_space() → StrParsingContext¶: Strip current value unless it entirely consists of spaces.

class yuio.parse.ConfigParsingContext( value: object, /, *, parent: ConfigParsingContext | None = None, key: Any = None, desc: str | None = None, )¶

Config parsing context tracks path in the config, similar to JSON path.

value: object¶: Config value to be validated and parsed.

parent: ConfigParsingContext | None¶: Parent context.

key: Any¶

Key that was accessed when we’ve descended from parent context to this one.

Root context has key None.

desc: str | None¶: Additional description of the key.

descend( value: Any, key: Any, desc: str | None = None, ) → ConfigParsingContext¶

Create a new context that adds a new key to the path.

Parameters:

value – inner value that was derived from the current value by accessing it with the given key.
key –
key that we use to descend into the current value.

For example, let’s say we’re parsing a list. We iterate over it and pass its elements to a sub-parser. Before calling a sub-parser, we need to make a new context for it. In this situation, we’ll pass current element as value, and is index as key.
desc –
human-readable description for the new context. Will be colorized and %-formatted with a single named argument key.

This is useful when parsing structures that need something more complex than JSON path. For example, when parsing a key in a dictionary, it is helpful to set description to something like "key of element #%(key)r". This way, parsing errors will have a more clear message:
```
Parsing error:
  In key of element #2:
    Expected str, got int: 10
```

make_path() → list[tuple[Any, str | None]]¶

Capture current path.

Returns:: a list of tuples. First element of each tuple is a key, second is an additional description.

Base classes¶

class yuio.parse.ValueParser(ty: type[T], /, *args, **kwargs)¶

Base implementation for a parser that returns a single value.

Implements all method, except for parse_with_ctx(), parse_config_with_ctx(), to_json_schema(), and to_json_value().

Parameters:

ty – type of the produced value, used in check_type().

Example:

class MyTypeParser(ValueParser[MyType]):
    def __init__(self):
        super().__init__(MyType)

    def parse_with_ctx(self, ctx: StrParsingContext, /) -> MyType:
        return MyType(ctx.value)

    def parse_config_with_ctx(self, ctx: ConfigParsingContext, /) -> MyType:
        if not isinstance(ctx.value, str):
            raise ParsingError.type_mismatch(value, str, ctx=ctx)
        return MyType(ctx.value)

    def to_json_schema(
        self, ctx: yuio.json_schema.JsonSchemaContext, /
    ) -> yuio.json_schema.JsonSchemaType:
        return yuio.json_schema.String()

    def to_json_value(self, value: object, /) -> yuio.json_schema.JsonValue:
        assert self.assert_type(value)
        return value.data

>>> MyTypeParser().parse('pancake')
MyType(data='pancake')

class yuio.parse.WrappingParser(inner: U | None, /, *args, **kwargs)¶

A base for a parser that wraps another parser and alters its output.

This base simplifies dealing with partial parsers.

The _inner attribute is whatever internal state you need to store. When it is None, the parser is considered partial. That is, you can’t use such a parser to actually parse anything, but you can use it in a type annotation. When it is not None, the parser is considered non partial. You can use it to parse things, but you can’t use it in a type annotation.

Warning

All descendants of this class must include appropriate type hints for their __new__ method, otherwise type annotations from this base will shadow implementation’s __init__ signature.

See section on parser hierarchy for details.

Parameters:: inner – inner data or None.

_inner¶

Internal resource wrapped by this parser.

Raises:

Accessing it when the parser is in a partial state triggers an error and warns user that they didn’t provide an inner parser.

Setting a new value when the parser is not in a partial state triggers an error and warns user that they shouldn’t provide an inner parser in type annotations.

_inner_raw¶: Unchecked access to the wrapped resource.

class yuio.parse.MappingParser(inner: Parser[U] | None, /)¶

This is base abstraction for Map and Optional. Forwards all calls to the inner parser, except for parse_with_ctx(), parse_many_with_ctx(), parse_config_with_ctx(), options(), check_type(), describe_value(), widget(), and to_json_value().

Parameters:: inner – mapped parser or None.

class yuio.parse.ValidatingParser(inner: Parser[T] | None = None, /)¶

Base implementation for a parser that validates result of another parser.

This class wraps another parser and passes all method calls to it. All parsed values are additionally passed to _validate().

Parameters:

inner – a parser which output will be validated.

Example:

class IsLower(ValidatingParser[str]):
    def _validate(self, value: str, /):
        if not value.islower():
            raise ParsingError(
                "Value should be lowercase: `%r`",
                value,
                fallback_msg="Value should be lowercase",
            )

>>> IsLower(Str()).parse("Not lowercase!")
Traceback (most recent call last):
...
yuio.parse.ParsingError: Value should be lowercase: 'Not lowercase!'

__wrapped_parser__: Parser[object] | None: An attribute for unwrapping parsers that validate or map results of other parsers.

abstractmethod _validate(value: T, /)¶

Implementation of value validation.

Parameters:: value – value which needs validating.
Raises:: should raise ParsingError if validation fails.

class yuio.parse.CollectionParser(inner: Parser[T] | None, /, **kwargs)¶

A base class for implementing collection parsing. It will split a string by the given delimiter, parse each item using a subparser, and then pass the result to the given constructor.

Parameters:

inner – parser that will be used to parse collection items.
ty – type of the collection that this parser returns.
ctor – factory of instances of the collection that this parser returns. It should take an iterable of parsed items, and return a collection.
iter – a function that is used to get an iterator from a collection. This defaults to iter(), but sometimes it may be different. For example, Dict is implemented as a collection of pairs, and its iter is dict.items().
config_type – type of a collection that we expect to find when parsing a config. This will usually be a list.
config_type_iter – a function that is used to get an iterator from a config value.
delimiter – delimiter that will be passed to str.split().

The above parameters are exposed via protected attributes: self._inner, self._ty, etc.

For example, let’s implement a list parser that repeats each element twice:

from typing import Iterable, Generic


class DoubleList(CollectionParser[list[T], T], Generic[T]):
    def __init__(self, inner: Parser[T], /, *, delimiter: str | None = None):
        super().__init__(inner, ty=list, ctor=self._ctor, delimiter=delimiter)

    @staticmethod
    def _ctor(values: Iterable[T]) -> list[T]:
        return [x for value in values for x in [value, value]]

    def to_json_schema(
        self, ctx: yuio.json_schema.JsonSchemaContext, /
    ) -> yuio.json_schema.JsonSchemaType:
        return {"type": "array", "items": self._inner.to_json_schema(ctx)}

    def to_json_value(self, value: object, /) -> yuio.json_schema.JsonValue:
        assert self.assert_type(value)
        return [self._inner.to_json_value(item) for item in value[::2]]

>>> parser = DoubleList(Int())
>>> parser.parse("1 2 3")
[1, 1, 2, 2, 3, 3]
>>> parser.to_json_value([1, 1, 2, 2, 3, 3])
[1, 2, 3]

_allow_completing_duplicates: ClassVar[bool] = True¶: If set to False, autocompletion will not suggest item duplicates.

Adding type hint conversions¶

You can register a converter so that from_type_hint() can derive custom parsers from type hints:

yuio.parse.register_type_hint_conversion(cb: Cb) → Cb¶

This function takes a callback that accepts three positional arguments:

a type hint,
a type hint’s origin (as defined by typing.get_origin()),
a type hint’s args (as defined by typing.get_args()).

The callback should return a parser if it can, or None otherwise.

All registered callbacks are tried in the same order as they were registered.

If uses_delim is True, callback can use suggest_delim_for_type_hint_conversion().

This function can be used as a decorator.

Parameters:

cb – a function that should inspect a type hint and possibly return a parser.
uses_delim – indicates that callback will use suggest_delim_for_type_hint_conversion().

Example:

@register_type_hint_conversion
def my_type_conversion(ty, origin, args):
    if ty is MyType:
        return MyTypeParser()
    else:
        return None

>>> from_type_hint(MyType)
MyTypeParser

When implementing a callback, you might need to specify a delimiter for a collection parser. Use suggest_delim_for_type_hint_conversion():

yuio.parse.suggest_delim_for_type_hint_conversion() → str | None¶

Suggests a delimiter for use in type hint converters.

When creating a parser for a collection of items based on a type hint, it is important to use different delimiters for nested collections. This function can suggest such a delimiter based on the current type hint’s depth.

Raises:

RuntimeError if called from a type converter that didn’t set uses_delim to True.

Example:

@register_type_hint_conversion(uses_delim=True)
def my_collection_conversion(ty, origin, args):
    if origin is MyCollection:
        return MyCollectionParser(
            from_type_hint(args[0]),
            delimiter=suggest_delim_for_type_hint_conversion(),
        )
    else:
        return None

>>> parser = from_type_hint(MyCollection[MyCollection[str]])
>>> parser
MyCollectionParser(MyCollectionParser(Str))
>>> # First delimiter is `None`, meaning split by whitespace:
>>> parser._delimiter is None
True
>>> # Second delimiter is `","`:
>>> parser._inner._delimiter == ","
True

Re-imports¶

type yuio.parse.JsonValue: Alias of yuio.json_schema.JsonValue.

type yuio.parse.SecretString: Alias of yuio.secret.SecretString.

type yuio.parse.SecretValue: Alias of yuio.secret.SecretValue.