datalad.customremotes.base

Base classes to custom git-annex remotes (e.g. extraction from archives)

class datalad.customremotes.base.AnnexCustomRemote(path=None, cost=None, fin=None, fout=None)

Bases: object

Base class to provide custom special remotes for git-annex

Implements git-annex special custom remotes protocol described at http://git-annex.branchable.com/design/external_special_remote_protocol/

AVAILABILITY = 'LOCAL'

COST = 100

CUSTOM_REMOTE_NAME = None

SUPPORTED_SCHEMES = ()

debug(msg)

error(msg, annex_err='ERROR')

gen_URLS(key)

Yield URL(s) associated with a Key.

get_DIRHASH(key, full=False)

Gets a two level hash associated with a Key.

Parameters:

• full (bool, optional) – If True, would spit out full DIRHASH path, i.e. with a KEY/ directory
• like "abc/def". This is always the same for any given Key, so (Something) –
• be used for eg, creating hash directory structures to store Keys in. (can) –

get_URLS(key)

Gets URL(s) associated with a Key.

Use a generator gen_URLS where possible. This one should be deprecated in 0.15.

get_contentlocation(key, absolute=False, verify_exists=True)

Return (relative to top or absolute) path to the file containing the key

This is a wrapper around AnnexRepo.get_contentlocation which provides caching of the result (we are asking the location for the same archive key often)

heavydebug(msg, *args, **kwargs)

info(msg)

main()

Interface to the command line tool

progress(bytes)

read(req=None, n=1)

Read a message from git-annex

Parameters:

• req (string, optional) – Expected request - first msg of the response
• n (int) – Number of response elements after first msg

req_CHECKPRESENT(key)

CHECKPRESENT-SUCCESS Key

Indicates that a key has been positively verified to be present in the remote.

CHECKPRESENT-FAILURE Key

Indicates that a key has been positively verified to not be present in the remote.

CHECKPRESENT-UNKNOWN Key ErrorMsg

Indicates that it is not currently possible to verify if the key is present in the remote. (Perhaps the remote cannot be contacted.)

req_CHECKURL(url)

The remote replies with one of CHECKURL-FAILURE, CHECKURL-CONTENTS, or CHECKURL-MULTI.

CHECKURL-CONTENTS Size|UNKNOWN Filename

Indicates that the requested url has been verified to exist. The Size is the size in bytes, or use “UNKNOWN” if the size could not be determined. The Filename can be empty (in which case a default is used), or can specify a filename that is suggested to be used for this url.

CHECKURL-MULTI Url Size|UNKNOWN Filename …

Indicates that the requested url has been verified to exist, and contains multiple files, which can each be accessed using their own url. Note that since a list is returned, neither the Url nor the Filename can contain spaces.

CHECKURL-FAILURE

Indicates that the requested url could not be accessed.

req_CLAIMURL(url)

req_EXPORTSUPPORTED()

req_GETAVAILABILITY()

req_GETCOST()

req_INITREMOTE(*args)

Initialize this remote. Provides high level abstraction.

Specific implementation should go to _initialize

req_PREPARE(*args)

Prepare “to deliver”. Provides high level abstraction

Specific implementation should go to _prepare

req_REMOVE(key)

REMOVE-SUCCESS Key

Indicates the key has been removed from the remote. May be returned if the remote didn’t have the key at the point removal was requested.

REMOVE-FAILURE Key ErrorMsg

Indicates that the key was unable to be removed from the remote.

req_TRANSFER(cmd, key, file)

req_WHEREIS(key)

Added in 5.20150812-17-g6bc46e3

provide any information about ways to access the content of a key stored in it, such as eg, public urls. This will be displayed to the user by eg, git annex whereis. The remote replies with WHEREIS-SUCCESS or WHEREIS-FAILURE. Note that users expect git annex whereis to run fast, without eg, network access. This is not needed when SETURIPRESENT is used, since such uris are automatically displayed by git annex whereis.

WHEREIS-SUCCESS String

Indicates a location of a key. Typically an url, the string can be anything that it makes sense to display to the user about content stored in the special remote.

WHEREIS-FAILURE

Indicates that no location is known for a key.

send(*args)

Send a message to git-annex

Parameters:*args (list of strings) – arguments to be joined by a space and passed to git-annex

send_unsupported(msg=None)

Send UNSUPPORTED-REQUEST to annex and log optional message in our log

stop(msg=None)

exception datalad.customremotes.base.AnnexRemoteQuit

Bases: Exception

datalad.customremotes.base.ensure_datalad_remote(repo, remote=None, encryption=None, autoenable=False)

Initialize and enable datalad special remote if it isn’t already.

Parameters:

• repo (AnnexRepo) –
• remote (str, optional) – Special remote name. This should be one of the values in datalad.consts.DATALAD_SPECIAL_REMOTES_UUIDS and defaults to datalad.consts.DATALAD_SPECIAL_REMOTE.
• autoenable (encryption,) – Passed to init_datalad_remote.

datalad.customremotes.base.generate_uuids()

Generate UUIDs for our remotes. Even though quick, for consistency pre-generated and recorded in consts.py

datalad.customremotes.base.get_function_nargs(f)

datalad.customremotes.base.init_datalad_remote(repo, remote, encryption=None, autoenable=False, opts=[])

Initialize datalad special remote