datalad.customremotes.base

Base classes to custom git-annex remotes (e.g. extraction from archives)

class datalad.customremotes.base.AnnexCustomRemote(annex)[source]

Bases: SpecialRemote

AVAILABILITY = 'LOCAL'
COST = 100
gen_URLS(key)[source]

Yield URL(s) associated with a key, and keep stats on protocols.

getavailability()[source]

Asks the remote if it is locally or globally available. (Ie stored in the cloud vs on a local disk.)

Returns

Allowed values are “global” or “local”.

Return type

str

getcost()[source]

Requests the remote to return a use cost. Higher costs are more expensive.

cheapRemoteCost = 100 nearlyCheapRemoteCost = 110 semiExpensiveRemoteCost = 175 expensiveRemoteCost = 200 veryExpensiveRemoteCost = 1000 (taken from Config/Cost.hs)

Returns

Indicates the cost of the remote.

Return type

int

initremote()[source]

Gets called when git annex initremote or git annex enableremote are run. This is where any one-time setup tasks can be done, for example creating the remote folder. Note: This may be run repeatedly over time, as a remote is initialized in different repositories, or as the configuration of a remote is changed. So any one-time setup tasks should be done idempotently.

Raises

RemoteError – If the remote could not be initialized.

prepare()[source]

Tells the remote that it’s time to prepare itself to be used. Gets called whenever git annex is about to access any of the below methods, so it shouldn’t be too expensive. Otherwise it will slow down operations like git annex whereis or git annex info.

Internet connection can be established here, though it’s recommended to defer this until it’s actually needed.

Raises

RemoteError – If the remote could not be prepared.

remove(key)[source]

Requests the remote to remove a key’s contents.

Parameters

key (str) –

Raises

RemoteError – If the key couldn’t be deleted from the remote.

send_progress(progress)[source]

Indicates the current progress of the transfer (in bytes).

May be repeated any number of times during the transfer process.

Too frequent updates are wasteful but bear in mind that this is used both to display a progress meter for the user, and for annex.stalldetection. So, sending an update on each 1% of the file may not be frequent enough, as it could appear to be a stall when transferring a large file.

Parameters

progress (int) – The current progress of the transfer in bytes.

transfer_store(key, local_file)[source]

Store the file in local_file to a unique location derived from key.

It’s important that, while a Key is being stored, checkpresent(key) not indicate it’s present until all the data has been transferred. While the transfer is running, the remote can repeatedly call annex.progress(size) to indicate the number of bytes already stored. This will influence the progress shown to the user.

Parameters
  • key (str) – The Key to be stored in the remote. In most cases, this is going to be the remote file name. It should be at least be unambigiously derived from it.

  • local_file (str) – Path to the file to upload. Note that in some cases, local_file may contain whitespace. Note that local_file should not influence the filename used on the remote.

Raises

RemoteError – If the file could not be stored to the remote.

datalad.customremotes.base.ensure_datalad_remote(repo, remote=None, encryption=None, autoenable=False)[source]

Initialize and enable datalad special remote if it isn’t already.

Parameters
  • repo (AnnexRepo) –

  • remote (str, optional) – Special remote name. This should be one of the values in datalad.consts.DATALAD_SPECIAL_REMOTES_UUIDS and defaults to datalad.consts.DATALAD_SPECIAL_REMOTE.

  • encryption (optional) – Passed to init_datalad_remote.

  • autoenable (optional) – Passed to init_datalad_remote.

datalad.customremotes.base.generate_uuids()[source]

Generate UUIDs for our remotes. Even though quick, for consistency pre-generated and recorded in consts.py

datalad.customremotes.base.init_datalad_remote(repo, remote, encryption=None, autoenable=False, opts=[])[source]

Initialize datalad special remote