argparse

Hooks for command-line interface with params

class pydrobert.param.argparse.DeserializationAction(option_strings, dest, nargs=None, const='json', default=None, type=<class 'param.parameterized.Parameterized'>, choices=None, required=False, help=None, metavar=None)[source]

Action to deserialize a parameterized object from file

Given some subclass of param.parameterized.Parameterized, MyParameterized, the action can be added by calling, e.g.

>>> parser.add_argument(
...     '--param', type=MyParameterized, action=DeserializationAction)

In this example, the argument passed with the flag --param is treated as a path to a JSON file from which a serialized copy of MyParameterized is read and instantiatied.

Deserialization is performed with the param.parameterized.Parameterized.param.deserialize_parameters(). The deserialization mode and optionally the subset of parameters deserialized can be changed by passing the const keyword argument to add_argument(). const can be either a string (just the mode) or a tuple of a string (mode) and set of strings (subset).

See also

ParameterizedFileReadAction

Same intent, but using pydrobert.param custom deserialization routines.

register_serializer

To enable custom parsing modes.

class pydrobert.param.argparse.ParameterizedFileReadAction(option_strings, dest, parameterized=None, type=None, deserializer_name_dict=None, deserializer_type_dict=None, on_missing='warn', required=False, help=None, metavar=None, nargs=None, const=None)[source]

Base class for deserializing files into a Parameterized object

Subclasses of this class can be added as the ‘action’ keyword to an argparse.ArgumentParser.add_argument() call. The action will read the file path passed as an argument via command line and use the contents of the file to populate param.parameterized.Parameterized instances. The subclass deserializes the contents of the file according to the pydrobert.param.serialization.deserialize_from_filetype() function, where filetype is replaced with the subclass’ file type.

There are three ways to specify parameterized objects to populate. They are mutually exclusive:

  1. Set the keyword type with the subclass of param.parameterized.Parameterized you want to deserialize into. A new instance of that subclass will be created with the name type.__name__. The instance will be returned in the parsed namespace’s attribute whose name matches dest as well.

  2. Set the keyword parameterized with an instance of param.parameterized.Parameterized. That instance will be populated and also returned in the parsed namespace’s attribute whose name matches dest.

  3. Set the keyword parameterized as a hierarchical dictionary of param.parameterized.Parameterized instances. The leaves of the dictionary will be populated according to the “hierarchical mode” specified in the documentation of pydrobert.param.serialization.deserialize_from_dict(). The same dictionary will be returned in the parsed namespace’s attribute whose name matches dest.

Parameters:

  • option_strings (List[str]) – A list of command-line option strings which should be associated with this action.

  • dest (str) – The name of the attribute to hold the created object.

  • parameterized (Union[Parameterized, dict, None]) –

  • type (Optional[Type[Parameterized]]) –

  • deserializer_name_dict (Optional[dict]) – Use specific deserializers for parameters with specific names.

  • deserializer_type_dict (Optional[dict]) – Use specific deserializers for parameters with exactly matching types.

  • on_missing (Literal['ignore', 'warn', 'raise']) – What to do if the parameterized instance does not have a parameter listed in the config file.

  • required (bool) – True if the action must always be specified at the command line. This is only meaningful for optional command-line arguments.

  • help (Optional[str]) – The help string describing the argument.

  • metavar (Optional[str]) – The name to be used for the option’s argument with the help string. If None, the dest value will be used as the name.

  • nargs (Union[str, int, None]) – The number of command line arguments to be consumed. When more than one argument is specified, each will be deserialized in the order that they were presented on the command line. Thus, later config values will clobber earlier ones

  • const (Optional[str]) – If nargs is '?' but the flag is present on the command line, this value will be supplied as the missing argument

See also

pydrobert.param.serialization.deserialize_from_dict

For more information on how values are deserialized from the config

abstract deserialize(fp)[source]

Read the file pointer into parameterized objects

Called during the callable section of this class. Implemented by subclasses.

class pydrobert.param.argparse.ParameterizedIniPrintAction(option_strings, dest, parameterized=None, type=None, serializer_name_dict=None, serializer_type_dict=None, only=None, on_missing='raise', include_help=True, help=None, out_stream=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, help_prefix='#', one_param_section=None)[source]

Print parameters as INI and exit

Parameters:

See also

ParameterizedPrintAction

A full description of the parameters and behaviour of like actions

pydrobert.param.serialization.serialize_to_ini

A description of the serialization process and of the additional parameters

class pydrobert.param.argparse.ParameterizedIniReadAction(option_strings, dest, parameterized=None, type=None, deserializer_name_dict=None, deserializer_type_dict=None, on_missing='warn', required=False, help=None, metavar=None, nargs=None, const=None, defaults=None, comment_prefixes=('#', ';'), inline_comment_prefixes=(';',), one_param_section=None)[source]

Deserialize an INI file into a parameterized object

Parameters:

See also

ParameterizedFileReadAction

A full description of the parameters and behaviour of like actions.

pydrobert.param.serialization.deserialize_from_ini

A description of the deserialization process and of the additional parameters.

class pydrobert.param.argparse.ParameterizedJsonPrintAction(option_strings, dest, parameterized=None, type=None, serializer_name_dict=None, serializer_type_dict=None, only=None, on_missing='raise', help=None, out_stream=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, indent=2)[source]

Print parameters as JSON and exit

Parameters:

See also

ParameterizedPrintAction

A full description of the parameters and behaviour of like actions

pydrobert.param.serialization.serialize_to_json

A description of the serialization process and of the additional parameters

class pydrobert.param.argparse.ParameterizedJsonReadAction(option_strings, dest, parameterized=None, type=None, deserializer_name_dict=None, deserializer_type_dict=None, on_missing='warn', required=False, help=None, metavar=None, nargs=None, const=None)[source]

Deserialize a JSON file into a parameterized object

Parameters:

See also

ParameterizedFileReadAction

A full description of the parameters and behaviour of like actions.

pydrobert.param.serialization.deserialize_from_json

A description of the deserialization process.

class pydrobert.param.argparse.ParameterizedPrintAction(option_strings, dest, parameterized=None, type=None, serializer_name_dict=None, serializer_type_dict=None, only=None, on_missing='raise', include_help=True, help=None, out_stream=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)[source]

Base class for printing parameters to stdout and exiting

Subclasses of this class can be added as the ‘action’ keyword to an argparse.ArgumentParser.add_argument() call. Like the --help flag, after this action is called, the program will try to exit, but not before printing out parameters.

There are three ways to specify what parameters to print, analogous to how they are specified in ParameterizedFileReadAction:

  1. Set the keyword type with a subclass of param.parameterized.Parameterized. A new instance of that type will be created to be printed. Its name will be type.__name__

  2. Set the keyword parameterized with an instance of param.parameterized.Parameterized. That instance will be printed.

  3. Set the keyword parameterized as a hierarchical dictionary of param.parameterized.Parameterized instances. The leaves of the dictionary will be populated according to the “hierarchical mode” specified in the documentation of pydrobert.param.serialization.serialize_to_dict()

Note that if a ParameterizedFileReadAction has been called on the command line prior to the print that shares the same parameterized value as in 2. or 3., parameterized will be populated by that file’s contents.

Parameters:

  • option_strings (List[str]) – A list of command-line option strings which should be associated with this action.

  • dest (str) – Ignored

  • parameterized (Union[Parameterized, dict, None]) –

  • type (Optional[Type[Parameterized]]) –

  • serializer_name_dict (Optional[dict]) – Use specific serializers for parameters with specific names

  • serializer_type_dict (Optional[dict]) – Use specific serializers for parameters with exactly matching types

  • only (Optional[Collection[str]]) – If specified, only the parameters with their names in this set will be printed

  • on_missing (Literal['ignore', 'warn', 'raise']) – What to do if the parameterized instance does not have a parameter listed in only

  • include_help (bool) – Whether to print parameter help when printing parameters

  • help (Optional[str]) – The help string describing the argument

  • out_stream (TextIO) – Where to print the parameters to

abstract print_parameters()[source]

Print the parameters

Called during the callable section of this class. Should print to the attribute out_stream

class pydrobert.param.argparse.ParameterizedYamlPrintAction(option_strings, dest, parameterized=None, type=None, serializer_name_dict=None, serializer_type_dict=None, only=None, on_missing='raise', include_help=True, help=None, out_stream=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)[source]

Print parameters as YAML and exit

Parameters:

See also

ParameterizedPrintAction

A full description of the parameters and behaviour of like actions

pydrobert.param.serialization.serialize_to_yaml

A description of the serialization process and of the additional parameters

class pydrobert.param.argparse.ParameterizedYamlReadAction(option_strings, dest, parameterized=None, type=None, deserializer_name_dict=None, deserializer_type_dict=None, on_missing='warn', required=False, help=None, metavar=None, nargs=None, const=None)[source]

Deserialize a YAML file into a parameterized object

Parameters:

See also

ParameterizedFileReadAction

A full description of the parameters and behaviour of like actions

pydrobert.param.serialization.deserialize_from_yaml

A description of the deserialization process.

class pydrobert.param.argparse.SerializationAction(option_strings, dest, nargs='?', const='json', default=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, type=<class 'param.parameterized.Parameterized'>, choices=None, required=False, help=None, metavar=None)[source]

Action to serialize a parameterized object and then exit

The counterpart to DeserializationAction, adding this action as an argument to an argparse.ArgumentParser as such

>>> parser.add_argument(
...     '--print', type=MyParameterized, action=SerializationAction)

will, by default, serialize a new MyParameterized instance as JSON and print it to stdout if '--print' is parsed without an argument. Afterwards, the command will terminate. If '--print' is passed a path, the JSON will be printed there instead.

Serialization is performed with param.parameterized.Parameterized.param.serialize_parameters(). The serialization mode and optionally the subset of parameters serialized can be changed by passing the const keyword argument to add_argument(). const can be either a string (just the mode) or a tuple of a string (mode) and set of strings (subset).

The type argument can be either a subclass of param.parameterized.Parameterized or an instance of one. In the latter case, the parameter values of that instance will be serialized instead.

See also

ParameterizedFilePrintAction

Same intent, but using pydrobert.param custom serialization routines.

register_serializer

To enable custom parsing modes.

pydrobert.param.argparse.add_deserialization_group_to_parser(parser, pobj, dest, file_formats=None, subset=None, reckless=False, flag_format_str='--read-{file_format}', help_format_str='Read as a {file_format} file', title_format_str='{pobj_name} deserialization', desc_format_str='Flags to read in {pobj_name} parameters from file. Only one may be specified', required=False, register_missing=True)[source]

Add flags to parser for deserializing parameterized objects from file

A convenience function for coordinating DeserializationAction arguments over multiple file formats. The usual case might look like

>>> add_deserialization_group_to_parser(parser, MyParameterized, 'param')
>>> namespace = parser.parse_args()
>>> namespace.param  # stores the MyParameterized instance or None

Here it adds mutually exclusive flags to deserialize a MyParameterized instance using serialization protocols for the available file formats.

Parameters:

  • parser (ArgumentParser) –

  • pobj (Union[Parameterized, Type[Parameterized]]) – Either a subclass of param.parameterized.Parameterized or an instance of one. Determines the type of parameterized object to deserialize. If pobj is a type, the default value for the parameters in the namespace will be None. Otherwise (when pobj is an instance), pobj will be the default value.

  • dest (str) – The name of the attribute in namespace to store the deserialized instance under.

  • file_formats (Union[Literal['json', 'yaml'], Collection[Literal['json', 'yaml']], None]) – If specified, one or a list of file formats to add flags for. If unspecified, flags for every available file format will be added. Availability means both the correct backend is installed to parse the file and, if register_missing is False, that the corresponding mode has already been registered. The 'json' format is always available.

  • subset (Optional[Collection[str]]) – If specified, only the parameters with names in this set will be deserialized.

  • reckless (bool) – Whether to allow simplifying assumptions to make parsing easier.

  • flag_format_str (Union[str, Sequence[str]]) –

    One or more Python format strings which, after formatting, will act as the flags for the argument. The following keys are available for formatting:

    • file_format, an entry in file_formats

    • dest

    • pobj_name, either pobj.name if pobj is an instance or pobj.__name__ if pobj is a class.

  • help_format_str (Optional[str]) – A python format string which, after formatting, describes the flags. The same keys are available as those to flag_format_str.

  • title_format_str (Optional[str]) – A python format string which, after formatting, is used to name the group grp. Can only be formatted with the keys dest and pobj_name.

  • desc_format_str (Optional[str]) – A python format string which, after formatting, is used to describe the group grp. Can only be formatted with the keys dest and pobj_name.

  • required (bool) – Whether to require the user to specify one flag.

  • register_missing (bool) – Whether to register any custom modes corresponding to the file_formats (and reckless) which have yet to be registered via register_serializer(). Setting to False will restrict the dynamically chosen value of file_formats. If file_formats is manually specified, the parser will throw when it tries to deserialize using an unspecified mode.

Returns:

grp – The group containing the added arguments. The mutually-exclusive group containing them is nested in a regular argument group.

Warning

Functionality is in beta and subject to additions and modifications.

See also

add_parameterized_read_group

An analogous function using custom deserialization routines.

pydrobert.param.serialization.register_serializer

More information on the file formats and modes of serialization.

pydrobert.param.argparse.add_parameterized_print_group(parser, type=None, parameterized=None, include_yaml=None, ini_option_strings=('--print-ini',), json_option_strings=('--print-json',), yaml_option_strings=('--print-yaml',), ini_kwargs={}, json_kwargs={}, yaml_kwargs={})[source]

Add flags to print parameters as INI, JSON, or YAML

This convenience function adds a group of print actions to parser to print parameters and exit in one of INI, JSON, or YAML format.

What to print is determined by the keyword args type or parameterized.

  1. If type is specified, it will be instantiated and printed with whatever defaults it has. The instance will have the name type.__name__

  2. If parameterized is a param.parameterized.Parameterized instance, that instance will be printed

  3. If parameterized is a dictionary of param.parameterized.Parameterized instances, those instances will be serialized to dictionaries, then the dictionary of dictionaries will be printed. parameterized can contain nested dictionaries of param.parameterized.Parameterized instances, but it will be unable to be printed as an INI file, only JSON or YAML

Parameters:

  • parser (ArgumentParser) –

  • type (Optional[Type[Parameterized]]) –

  • parametrized –

  • include_yaml (Optional[bool]) – Whether to include the YAML print flags. YAML requires one of ruamel.yaml or yaml to be installed. If unset, we will include the flags if it is possible to import a YAML module.

  • ini_option_strings (Sequence[str]) – Zero or more option strings specifying that INI format should be printed. If no option strings are specified, INI printing is disabled

  • json_option_strings (Sequence[str]) – Zero or more option strings specifying that JSON format should be printed. If no option strings are specified, JSON printing is disabled

  • yaml_option_strings (Sequence[str]) – Zero or more option strings specifying that YAML format should be printed. If no option strings are specified, YAML printing is disabled

  • ini_kwargs (dict) – Additional keyword arguments to use when creating the INI flag. See ParameterizedIniPrintAction for more info

  • json_kwargs (dict) – Additional keyword arguments to use when creating the JSON flag. See ParameterizedJsonPrintAction for more info

  • yaml_kwargs (dict) – Additional keyword arguments to use when creating the YAML flag. See ParameterizedYamlPrintAction for more info

Returns:

group (obj) – The group containing the flags

Examples

>>> import param, argparse
>>> class MyParams(param.Parameterized):
...     an_int = param.Integer(1)
...     a_bool = param.Boolean(True)
>>> parser = argparse.ArgumentParser()
>>> add_parameterized_print_group(parser, type=MyParams)
>>> try:
...     parser.parse_args(['--print-ini'])
... except SystemExit:
...     pass
[MyParams]
a_bool = true
an_int = 1
>>> try:
...     # only works if ruamel.yaml/ruamel_yaml or pyyaml installed
...     parser.parse_args(['--print-yaml'])
... except SystemExit:
...     pass
a_bool: true
an_int: 1

>>> import param, argparse
>>> class A(param.Parameterized):
...     something = param.Integer(None)
...     else_ = param.List([1, 2])
>>> class B(param.Parameterized):
...     float_ = param.Number(3.14)
>>> parameterized = {'A': {'AA': A()}, 'B': B()}
>>> parser = argparse.ArgumentParser()
>>> add_parameterized_print_group(
...     parser, parameterized=parameterized, json_kwargs={'indent': None})
>>> try:
...     parser.parse_args(['--print-json'])
... except SystemExit:
...     pass
{"A": {"AA": {"else_": [1, 2], "something": null}}, "B": {"float_": 3.14}}

Notes

The returned group is technically mutally exclusive. However, since the print action ends with a sys.exit() call, mutual exclusivity will never be enforced

pydrobert.param.argparse.add_parameterized_read_group(parser, parameterized=None, type=None, include_yaml=None, ini_option_strings=('--read-ini',), json_option_strings=('--read-json',), yaml_option_strings=('--read-yaml',), dest='params', ini_kwargs={}, json_kwargs={}, yaml_kwargs={})[source]

Add flags to read configs from INI, JSON, or YAML sources

This convenience function adds a mutually exclusive group of read actions to parser to read in parameters from 3 file types: INI, JSON, or YAML.

What to read into is determined by the keyword args type or parameterized.

  1. If type is specified, it will be instantiated and populated. Its name will match type.__name__

  2. If parameterized is specified and is a param.parameterized.Parameterized instance, it will be populated directly.

  3. If parameterized is a dictionary whose leaves are param.parameterized.Parameterized, sections of the config whose keys match the keys of the dictionary will populate the corresponding param.parameterized.Parameterized instances. parameterized can nest those instances repeatedly, but only a shallow dict will be able to be parsed from an INI file

Parameters:

  • parser (ArgumentParser) –

  • type (Optional[Type[Parameterized]]) –

  • parametrized –

  • include_yaml (Optional[bool]) – Whether to include the YAML config flags. YAML requires one of ruamel.yaml or yaml to be installed. If unset, we will include the flags if it is possible to import a YAML module.

  • ini_option_strings (Sequence[str]) – Zero or more option strings specifying that the next argument is an INI file to be read. If no option strings are specified, INI reading is disabled

  • json_option_strings (Sequence[str]) – Zero or more option strings specifying that the next argument is an JSON file to be read. If no option strings are specified, JSON reading is disabled

  • yaml_option_strings (Sequence[str]) – Zero or more option strings specifying that the next argument is an YAML file to be read. If no option strings are specified, YAML reading is disabled

  • dest (str) – Under what name to store parameters in the returned namespace of parser.parse_args(...)

  • ini_kwargs (dict) – Additional keyword arguments to use when creating the INI flag. See ParameterizedIniReadAction for more info

  • json_kwargs (dict) – Additional keyword arguments to use when creating the JSON flag. See :class`ParameterizedJsonReadAction` for more info

  • yaml_kwargs (dict) – Additional keyword arguments to use when creating the YAML flag. See :class`ParameterizedYamlReadAction` for more info

Returns:

group (obj) – The mutually exclusive group containing the flags

Examples

>>> # write some configs
>>> with open('config.ini', 'w') as f:
>>>     f.write('[DEFAULT]\nfoo = a\n')
>>> with open('config.json', 'w') as f:
>>>     f.write('{"foo": "b"}\n')
>>> # make our Parameterized type
>>> import param, argparse
>>> class MyParams(param.Parameterized):
>>>     foo = param.String(None)
>>> # make our parser
>>> parser = argparse.ArgumentParser()
>>> add_parameterized_read_group(parser, type=MyParams)
>>> # parse an INI
>>> options = parser.parse_args(['--read-ini', 'config.ini'])
>>> assert options.params.foo == "a"
>>> # parse a JSON
>>> options = parser.parse_args(['--read-json', 'config.json'])
>>> assert options.params.foo == "b"

>>> # write a hierarchical config
>>> with open('config.yaml', 'w') as f:
>>>     f.write("""
... A:
...   foo: bar
...   baz: 1
... B:
...   B.1:
...     me: I may
...   B.2:
...     me: me me mee
... """)
>>> # make our Parameterized types
>>> class A(param.Parameterized):
>>>     foo = param.String(None)
>>>     bar = param.Integer(None)
>>> class B(param.Parameterized):
>>>     me = param.String(None)
>>> parameterized = {'A': A(), 'B': {'B.1': B(), 'B.2': B()}}
>>> # make our parser
>>> parser = argparse.ArgumentParser()
>>> add_parameterized_read_group(parser, parameterized=parameterized)
>>> # parse YAML (requires package ruamel.yaml/ruamel_yaml or pyyaml)
>>> parser.parse_args(['--read-yaml', 'config.yaml'])
>>> assert parameterized['A'].baz == 1
>>> assert parameterized['B']['B.2'].me == 'me me mee'
pydrobert.param.argparse.add_serialization_group_to_parser(parser, pobj, file_formats=None, subset=None, reckless=False, flag_format_str='--print-{file_format}', help_format_str='Print as a {file_format} file', title_format_str='{pobj_name} serialization', desc_format_str='Flags to print {pobj_name} parameters to file (one arg) or stdout (no args) and then exit', register_missing=True)[source]

Add flags to parser to serialize parameters to file or stdout

A convenience function for coordinating SerializationAction arguments over multiple file formats. The usual case might look like

>>> add_deserialization_group_to_parser(parser, MyParameterized())
>>> parser.parse_args()

Here it adds flags to serialize a MyParameterized instance to either a file or stdout using serialization protocols for the available file formats. If any associated flag is passed as an argument, the program will exit (and the parse_args() call will not return).

Parameters:

  • parser (ArgumentParser) –

  • pobj (Union[Parameterized, Type[Parameterized]]) – Either a subclass of param.parameterized.Parameterized or an instance of one. Determines the parameterized object to serialize. If pobj is a type, only the default values of the parameters of the class will end up serialized. If pobj is an instance, that instance’s parameters will be serialized.

  • file_formats (Union[Literal['json', 'yaml'], Collection[Literal['json', 'yaml']], None]) – If specified, one or a list of file formats to add flags for. If unspecified, flags for every available file format will be added. Availability means both the correct backend is installed to parse the file and, if register_missing is False, that the corresponding mode has already been registered. The 'json' format is always available.

  • subset (Optional[Collection[str]]) – If specified, only the parameters with names in this set will be serialized.

  • reckless (bool) – Whether to allow simplifying assumptions to make parsing easier.

  • flag_format_str (Union[str, Sequence[str]]) –

    One or more Python format strings which, after formatting, will act as the flags for the argument. The following keys are available for formatting:

    • file_format, an entry in file_formats

    • pobj_name, either pobj.name if pobj is an instance or pobj.__name__ if pobj is a class.

  • help_format_str (Optional[str]) – A python format string which, after formatting, describes the flags. The same keys are available as those to flag_format_str.

  • title_format_str (Optional[str]) – A python format string which, after formatting, is used to name the group grp. Can only be formatted with the key pobj_name.

  • desc_format_str (Optional[str]) – A python format string which, after formatting, is used to describe the group grp. Can only be formatted with the key pobj_name.

  • register_missing (bool) – Whether to register any custom modes corresponding to the file_formats (and reckless) which have yet to be registered via register_serializer(). Setting to False will restrict the dynamically chosen value of file_formats. If file_formats is manually specified, the parser will throw when it tries to serialize using an unspecified mode.

Returns:

grp – The group which the flags have beed added to. Note that only the first argument in the group can ever be parsed as the program will try to exit after printing.

Warning

Functionality is in beta and subject to additions and modifications.

See also

add_parameterized_print_group

An analogous function using custom serialization routines.

pydrobert.param.serialization.register_serializer

More information on the file formats and modes of serialization.