datalad.interface.base

High-level interface generation

class datalad.interface.base.Interface

Bases: ABC

Abstract base class for DataLad command implementations

Any DataLad command implementation must be derived from this class. The code snippet below shows a complete sketch of a Python class with such an implementation.

Importantly, no instances of command classes will created. Instead the main entry point is a static __call__() method, which must be implemented for any command. It is incorporated as a function in datalad.api, by default under the name of the file the implementation resides (e.g., command for a command.py file). Therefore the file should have a name that is a syntax-compliant function name. The default naming rule can be overwritten with an explicit alternative name (see datalad.interface.base.get_api_name()).

For commands implementing functionality that is operating on DataLad datasets, a command can be also be bound to the Dataset class as a method using the @datasetmethod decorator, under the specified name.

Any __call__() implementation should be decorated with datalad.interface.utils.eval_results(). This adds support for standard result processing, and a range of common command parameters that do not need to be manually added to the signature of __call__(). Any implementation decorated in this way should be implemented as a generator, and yield result records.

Any argument or keyword argument that appears in the signature of __call__() must have a matching item in Interface._params_. The dictionary maps argument names to datalad.support.param.Parameter specifications. The specification contain CLI argument declarations, value constraint and data type conversation specifications, documentation, and optional argparse-specific arguments for CLI parser construction.

The class decorator datalad.interface.base.build_doc() inspects an Interface implementation, and builds a standard docstring from various sources of structured information within the class (also see below). The documentation is automatically tuned differently, depending on the target API (Python vs CLI).

@build_doc
class ExampleCommand(Interface):
    """SHORT DESCRIPTION

    LONG DESCRIPTION
    ...
    """

    # COMMAND PARAMETER DEFINITIONS
    _params_ = dict(
        example=Parameter(
            args=("--example",),
            doc="""Parameter description....""",
            constraints=...),
        ...
        )
    )

    # RESULT PARAMETER OVERRIDES
    return_type= 'list'
    ...

    # USAGE EXAMPLES
    _examples_ = [
        dict(text="Example description...",
             code_py="Example Python code...",
             code_cmd="Example shell code ..."),
        ...
    ]

    @staticmethod
    @datasetmethod(name='example_command')
    @eval_results
    def __call__(example=None, ...):
        ...

        yield dict(...)

The basic implementation setup described above can be customized for individual commands in various way that alter the behavior and presentation of a specific command. The following overview uses the code comment markers in the above snippet to illustrate where in the class implementation these adjustments can be made.

(SHORT/LONG) DESCRIPTION

Interface.short_description can be defined to provide an explicit short description to be used in documentation and help output, replacing the auto-generated extract from the first line of the full description.

COMMAND PARAMETER DEFINITIONS

When a parameter specification declares Parameter(args=tuple(), ...), i.e. no arguments specified, it will be ignored by the CLI. Likewise, any Parameter specification for which is_api_arg() returns False will also be ignored by the CLI. Additionally, any such parameter will not be added to the parameter description list in the Python docstring.

RESULT PARAMETER OVERRIDES

The datalad.interface.utils.eval_results() decorator automatically add a range of additional arguments to a command, which are defined in datalad.interface.common_opts.eval_params. For any such parameter an Interface implementation can define an interface-specific default value, by declaring a class member with the respective parameter name and the desired default as its assigned value. This feature can be used to tune the default command behavior, for example, with respect to the default result rendering style, or its error behavior.

In addition to the common parameters of the Python API, an additional Interface.result_renderer_cmdline can be defined, in order to instruct the CLI to prefer the specified alternative result renderer over an Interface.result_renderer specification.

USAGE EXAMPLES

Any number of usage examples can be described in an _examples_ list class attribute. Such an example contains a description, and code examples for Python and CLI.

classmethod get_refds_path(dataset)

Return a resolved reference dataset path from a dataset argument

Deprecated since version 0.16: Use require_dataset() instead.

on_failure = 'continue'

result_filter = None

result_renderer = 'tailored'

result_xfm = None

return_type = 'list'

datalad.interface.base.alter_interface_docs_for_api(docs)

Apply modifications to interface docstrings for Python API use.

datalad.interface.base.build_doc(cls, **kwargs)

Decorator to build docstrings for datalad commands

It’s intended to decorate the class, the __call__-method of which is the actual command. It expects that __call__-method to be decorated by eval_results.

Note that values for any eval_params keys in cls._params_ are ignored. This means one class may extend another’s _params_ without worrying about filtering out eval_params.

Parameters:

cls (Interface) – DataLad command implementation

datalad.interface.base.build_example(example, api='python')

Build a code example.

Take a dict from a classes _example_ specification (list of dicts) and build a string with an api or cmd example (for use in cmd help or docstring).

Parameters:

api ({'python', 'cmdline'}) – If ‘python’, build Python example for docstring. If ‘cmdline’, build cmd example.

Returns:

ex – Concatenated examples for the given class.

Return type:

str

datalad.interface.base.dedent_docstring(text)

Remove uniform indentation from a multiline docstring

datalad.interface.base.eval_results(wrapped)

Decorator for return value evaluation of datalad commands.

Note, this decorator is only compatible with commands that return status dict sequences!

Two basic modes of operation are supported: 1) “generator mode” that yields individual results, and 2) “list mode” that returns a sequence of results. The behavior can be selected via the kwarg return_type. Default is “list mode”.

This decorator implements common functionality for result rendering/output, error detection/handling, and logging.

Result rendering/output configured via the result_renderer keyword argument of each decorated command. Supported modes are: ‘generic’ (a generic renderer producing one line per result with key info like action, status, path, and an optional message); ‘json’ (a complete JSON line serialization of the full result record), ‘json_pp’ (like ‘json’, but pretty-printed spanning multiple lines), ‘tailored’ custom output formatting provided by each command class (if any), or ‘disabled’ for no result rendering.

Error detection works by inspecting the status item of all result dictionaries. Any occurrence of a status other than ‘ok’ or ‘notneeded’ will cause an IncompleteResultsError exception to be raised that carries the failed actions’ status dictionaries in its failed attribute.

Status messages will be logged automatically, by default the following association of result status and log channel will be used: ‘ok’ (debug), ‘notneeded’ (debug), ‘impossible’ (warning), ‘error’ (error). Logger instances included in the results are used to capture the origin of a status report.

Parameters:

func (function) – __call__ method of a subclass of Interface, i.e. a datalad command definition

datalad.interface.base.get_allargs_as_kwargs(call, args, kwargs)

Generate a kwargs dict from a call signature and *args, **kwargs

Basically resolving the argnames for all positional arguments, and resolving the defaults for all kwargs that are not given in a kwargs dict

datalad.interface.base.get_api_name(intfspec)

Given an interface specification return an API name for it

datalad.interface.base.get_cmd_doc(interface)

Return the documentation for the command defined by interface.

Parameters:

interface (subclass of Interface) –

datalad.interface.base.get_cmd_summaries(descriptions, groups, width=79)

Return summaries for the commands in groups.

Parameters:

• descriptions (dict) – A map of group names to summaries.

• groups (list of tuples) – A list of groups and commands in the form described by get_interface_groups.

• width (int, optional) – The maximum width of each line in the summary text.

Returns:

• A list with a formatted entry for each command. The first command of each

• group is preceded by an entry describing the group.

datalad.interface.base.get_interface_groups(include_plugins=False)

Return a list of command groups.

Return type:

A list of tuples with the form (GROUP_NAME, GROUP_DESCRIPTION, COMMANDS).

datalad.interface.base.is_api_arg(arg)

Return True if argument is our API argument or self or used for internal purposes

datalad.interface.base.load_interface(spec)

Load and return the class for spec.

Parameters:

spec (tuple) – For a standard interface, the first item is the datalad source module and the second object name for the interface.

Return type:

The interface class or, if importing the module fails, None.

datalad.interface.base.update_docstring_with_examples(cls_doc, ex)

Update a commands docstring with examples.

Take _examples_ of a command, build the Python examples, and append them to the docstring.

Parameters:

• cls_doc (str) – docstring

• ex (list) – list of dicts with examples

datalad.interface.base.update_docstring_with_parameters(func, params, prefix=None, suffix=None, add_args=None)

Generate a useful docstring from a parameter spec

Amends any existing docstring of a callable with a textual description of its parameters. The Parameter spec needs to match the number and names of the callables arguments.