datalad.customremotes.base¶
Base classes to custom git-annex remotes (e.g. extraction from archives)
-
class
datalad.customremotes.base.
AnnexCustomRemote
(path=None, cost=None, fin=None, fout=None)[source]¶ Bases:
object
Base class to provide custom special remotes for git-annex
Implements git-annex special custom remotes protocol described at http://git-annex.branchable.com/design/external_special_remote_protocol/
-
AVAILABILITY
= 'LOCAL'¶
-
COST
= 100¶
-
CUSTOM_REMOTE_NAME
= None¶
-
SUPPORTED_SCHEMES
= ()¶
-
get_DIRHASH
(key, full=False)[source]¶ Gets a two level hash associated with a Key.
Parameters: - full (bool, optional) – If True, would spit out full DIRHASH path, i.e. with a KEY/ directory
- like "abc/def". This is always the same for any given Key, so (Something) –
- be used for eg, creating hash directory structures to store Keys in. (can) –
-
get_contentlocation
(key, absolute=False, verify_exists=True)[source]¶ Return (relative to top or absolute) path to the file containing the key
This is a wrapper around AnnexRepo.get_contentlocation which provides caching of the result (we are asking the location for the same archive key often)
-
read
(req=None, n=1)[source]¶ Read a message from git-annex
Parameters: - req (string, optional) – Expected request - first msg of the response
- n (int) – Number of response elements after first msg
-
req_CHECKPRESENT
(key)[source]¶ - CHECKPRESENT-SUCCESS Key
- Indicates that a key has been positively verified to be present in the remote.
- CHECKPRESENT-FAILURE Key
- Indicates that a key has been positively verified to not be present in the remote.
- CHECKPRESENT-UNKNOWN Key ErrorMsg
- Indicates that it is not currently possible to verify if the key is present in the remote. (Perhaps the remote cannot be contacted.)
-
req_CHECKURL
(url)[source]¶ The remote replies with one of CHECKURL-FAILURE, CHECKURL-CONTENTS, or CHECKURL-MULTI.
- CHECKURL-CONTENTS Size|UNKNOWN Filename
- Indicates that the requested url has been verified to exist. The Size is the size in bytes, or use “UNKNOWN” if the size could not be determined. The Filename can be empty (in which case a default is used), or can specify a filename that is suggested to be used for this url.
- CHECKURL-MULTI Url Size|UNKNOWN Filename …
- Indicates that the requested url has been verified to exist, and contains multiple files, which can each be accessed using their own url. Note that since a list is returned, neither the Url nor the Filename can contain spaces.
- CHECKURL-FAILURE
- Indicates that the requested url could not be accessed.
-
req_INITREMOTE
(*args)[source]¶ Initialize this remote. Provides high level abstraction.
Specific implementation should go to _initialize
-
req_PREPARE
(*args)[source]¶ Prepare “to deliver”. Provides high level abstraction
Specific implementation should go to _prepare
-
req_REMOVE
(key)[source]¶ - REMOVE-SUCCESS Key
- Indicates the key has been removed from the remote. May be returned if the remote didn’t have the key at the point removal was requested.
- REMOVE-FAILURE Key ErrorMsg
- Indicates that the key was unable to be removed from the remote.
-
req_WHEREIS
(key)[source]¶ Added in 5.20150812-17-g6bc46e3
provide any information about ways to access the content of a key stored in it, such as eg, public urls. This will be displayed to the user by eg, git annex whereis. The remote replies with WHEREIS-SUCCESS or WHEREIS-FAILURE. Note that users expect git annex whereis to run fast, without eg, network access. This is not needed when SETURIPRESENT is used, since such uris are automatically displayed by git annex whereis.
- WHEREIS-SUCCESS String
- Indicates a location of a key. Typically an url, the string can be anything that it makes sense to display to the user about content stored in the special remote.
- WHEREIS-FAILURE
- Indicates that no location is known for a key.
-
send
(*args)[source]¶ Send a message to git-annex
Parameters: *args (list of strings) – arguments to be joined by a space and passed to git-annex
-
-
class
datalad.customremotes.base.
AnnexExchangeProtocol
(repopath, custom_remote_name=None)[source]¶ Bases:
datalad.support.protocol.ProtocolInterface
A little helper to protocol interactions of custom remote with annex
-
HEADER
= '#!/bin/bash\n\nset -e\n\n# Gets a VALUE response and stores it in $RET\nreport () {\n echo "$@" >&2\n}\n\nrecv () {\n read resp\n #resp=${resp%\\n}\n target="$@"\n if [ "$resp" != "$target" ]; then\n report "! exp $target"\n report " got $resp"\n else\n report "+ got $resp"\n fi\n}\n\nsend () {\n echo "$@"\n report "sent $@"\n}\n\n'¶
-
add_section
(cmd, exception)[source]¶ Adds a section to the protocol.
This is an alternative to the use of start_section() and end_section(). In opposition to start_section, this one can be called anytime.
Parameters:
-
do_execute_callables
¶
-
do_execute_ext_commands
¶
-
end_section
(id_, exception)[source]¶ Ends the section id.
To call after the command call to be recorded. This ends the section defined by id as returned by start_section().
Parameters:
-
records_callables
¶
-
records_ext_commands
¶
-
start_section
(cmd)[source]¶ Starts a new section of the protocol.
To call before the command call to be recorded. To be used with a corresponding call of end_section().
Parameters: cmd (list) – The actual command and its options/arguments as a list Returns: An id of the started section to be used as argument of the corresponding call of end_section(). Return type: int
-