datalad.runner.protocol

Base class of a protocol to be used with the DataLad runner

class datalad.runner.protocol.GeneratorMixIn

Bases: object

Protocol mix in that will instruct runner.run to return a generator

When this class is in the parent of a protocol given to runner.run (and some other functions/methods) the run-method will return a Generator, which yields whatever the protocol callbacks send to the Generator, via the send_result-method of this class.

This allows to use runner.run() in constructs like:

for result in runner.run(…):

# do something, for example write to stdin of the subprocess

send_result(result)

class datalad.runner.protocol.WitlessProtocol(done_future=None, encoding=None)

Bases: object

Subprocess communication protocol base class for run_async_cmd

This class implements basic subprocess output handling. Derived classes like StdOutCapture should be used for subprocess communication that need to capture and return output. In particular, the pipe_data_received() method can be overwritten to implement “online” processing of process output.

This class defines a default return value setup that causes run_async_cmd() to return a 2-tuple with the subprocess’s exit code and a list with bytestrings of all captured output streams.

Parameters:

• done_future (Optional[Any])

• encoding (Optional[str])

connection_lost(exc)

Called when the connection is lost or closed.

The argument is an exception object or None (the latter meaning a regular EOF is received or the connection was aborted or closed).

Parameters:

exc (Optional[BaseException])

Return type:

None

connection_made(process)

Parameters:

process (Popen)

Return type:

None

pipe_connection_lost(fd, exc)

Called when a file descriptor associated with the child process is closed.

fd is the int file descriptor that was closed.

Parameters:

• fd (int)

• exc (Optional[BaseException])

Return type:

None

pipe_data_received(fd, data)

Parameters:

• fd (int)

• data (bytes)

Return type:

None

proc_err = False

proc_out = False

process_exited()

Return type:

None

timeout(fd)

Called if the timeout parameter to WitlessRunner.run() is not None and a process file descriptor could not be read (stdout or stderr) or not be written (stdin) within the specified time in seconds, or if waiting for a subprocess to exit takes longer than the specified time.

stdin timeouts are only caught when the type of the stdin- parameter to WitlessRunner.run() is either a Queue, a str, or bytes. Stdout or stderr timeouts are only caught of proc_out and proc_err are True in the protocol class. Process wait timeouts are always caught if timeout is not None. In this case the fd-argument will be None.

fd:

The file descriptor that timed out or None if no progress was made at all, i.e. no stdin element was enqueued and no output was read from either stdout or stderr.

Return type:

bool

Returns:

If the callback returns True, the file descriptor (if any was given) will be closed and no longer monitored. If the return values is anything else than True, the file-descriptor will be monitored further and additional timeouts might occur indefinitely. If None was given, i.e. a process runtime-timeout was detected, and True is returned, the process will be terminated.

Parameters:

fd (Optional[int])