datalad.cmd

Class the starts a subprocess and keeps it around to communicate with it via stdin. For each instruction send over stdin, a response is read and returned. The response structure is determined by “output_proc”

class datalad.cmd.BatchedCommand(cmd, path=None, output_proc=None, timeout=None, exception_on_timeout=False)

Bases: SafeDelCloseMixin

Container for a running subprocess. Supports communication with the subprocess via stdin and stdout.

Parameters:

• cmd (Union[str, Tuple, List])

• path (Optional[str])

• output_proc (Optional[Callable])

• timeout (Optional[float])

• exception_on_timeout (bool)

classmethod clean_inactive()

close(return_stderr=False)

Close communication and wait for process to terminate. If the “timeout” parameter to the constructor was not None, and if the configuration setting “datalad.runtime.stalled-external” is set to “abandon”, the method will return latest after “timeout” seconds. If the subprocess did not exit within this time, the attribute “wait_timed_out” will be set to “True”.

Parameters:

return_stderr (bool) – if set to “True”, the call will return all collected stderr content as string. In addition, if return_stderr is True and the log level is 5 or lower, and the configuration setting “datalad.log.outputs” evaluates to “True”, the content of stderr will be logged.

Returns:

stderr output if return_stderr is True, None otherwise

Return type:

str, optional

get_one_line()

Get a single stdout line from the generator.

If timeout was specified, and exception_on_timeout is False, and if a timeout occurs, return None. Otherwise, return the string that was read from the generator.

Return type:

Optional[str]

get_requested_error_output(return_stderr)

Parameters:

return_stderr (bool)

get_timeout_exception(fd)

Get a process timeout exception if timeout exceptions should be generated for a process that continues longer than timeout seconds after self.close() was initiated.

Parameters:

fd (Optional[int])

Return type:

Optional[TimeoutExpired]

proc1(single_command)

Simulate the old interface. This method is used only once in AnnexRepo.get_metadata()

Parameters:

single_command (str)

process_request(request)

Parameters:

request (Union[Tuple, str])

Return type:

Any | None

process_running()

Return type:

bool

exception datalad.cmd.BatchedCommandError(cmd='', last_processed_request='', msg='', code=None, stdout='', stderr='', cwd=None, **kwargs)

Bases: CommandError

class datalad.cmd.BatchedCommandProtocol(batched_command, done_future=None, encoding=None, output_proc=None)

Bases: GeneratorMixIn, StdOutErrCapture

Parameters:

• batched_command (BatchedCommand)

• done_future (Optional[Any])

• encoding (Optional[str])

• output_proc (Optional[Callable])

pipe_connection_lost(fd, exc)

Called when a file descriptor associated with the child process is closed.

fd is the int file descriptor that was closed.

Parameters:

• fd (int)

• exc (Optional[BaseException])

pipe_data_received(fd, data)

Parameters:

• fd (int)

• data (bytes)

timeout(fd)

Called if the timeout parameter to WitlessRunner.run() is not None and a process file descriptor could not be read (stdout or stderr) or not be written (stdin) within the specified time in seconds, or if waiting for a subprocess to exit takes longer than the specified time.

stdin timeouts are only caught when the type of the stdin- parameter to WitlessRunner.run() is either a Queue, a str, or bytes. Stdout or stderr timeouts are only caught of proc_out and proc_err are True in the protocol class. Process wait timeouts are always caught if timeout is not None. In this case the fd-argument will be None.

fd:

The file descriptor that timed out or None if no progress was made at all, i.e. no stdin element was enqueued and no output was read from either stdout or stderr.

Return type:

bool

Returns:

If the callback returns True, the file descriptor (if any was given) will be closed and no longer monitored. If the return values is anything else than True, the file-descriptor will be monitored further and additional timeouts might occur indefinitely. If None was given, i.e. a process runtime-timeout was detected, and True is returned, the process will be terminated.

Parameters:

fd (Optional[int])

class datalad.cmd.ReadlineEmulator(batched_command)

Bases: object

This class implements readline() on the basis of an instance of BatchedCommand. Its purpose is to emulate stdout’s for output_procs, This allows us to provide a BatchedCommand API that is identical to the old version, but with an implementation that is based on the threaded runner.

Parameters:

batched_command (BatchedCommand)

readline()

Read from the stdout provider until we have a line or None (which indicates some error).

class datalad.cmd.SafeDelCloseMixin

Bases: object

A helper class to use where __del__ would call .close() which might fail if “too late in GC game”

datalad.cmd.readline_rstripped(stdout)