datalad.api.download_url(urls, dataset=None, path=None, overwrite=False, archive=False, save=True, message=None)

Download content

It allows for a uniform download interface to various supported URL schemes, re-using or asking for authentication details maintained by datalad.


$ datalad download-url s3://bucket/file2.dat

  • urls (non-empty sequence of str) – URL(s) to be downloaded.
  • dataset (Dataset or None, optional) – specify the dataset to add files to. If no dataset is given, an attempt is made to identify the dataset based on the current working directory and/or the path given. Use save=False to prevent adding files to the dataset. [Default: None]
  • path (str or None, optional) – path (filename or directory path) where to store downloaded file(s). In case of multiple URLs provided, must point to a directory. Otherwise current directory is used. [Default: None]
  • overwrite (bool, optional) – flag to overwrite it if target file exists. [Default: False]
  • archive (bool, optional) – pass the downloaded files to add_archive_content(…, delete=True). [Default: False]
  • save (bool, optional) – by default all modifications to a dataset are immediately saved. Giving this option will disable this behavior. [Default: True]
  • message (str or None, optional) – a description of the state or the changes made to a dataset. [Default: None]
  • on_failure ({'ignore', 'continue', 'stop'}, optional) – behavior to perform on failure: ‘ignore’ any failure is reported, but does not cause an exception; ‘continue’ if any failure occurs an exception will be raised at the end, but processing other actions will continue for as long as possible; ‘stop’: processing will stop on first failure and an exception is raised. A failure is any result with status ‘impossible’ or ‘error’. Raised exception is an IncompleteResultsError that carries the result dictionaries of the failures in its failed attribute. [Default: ‘continue’]
  • proc_post – Like proc_pre, but procedures are executed after the main command has finished. [Default: None]
  • proc_pre – DataLad procedure to run prior to the main command. The argument a list of lists with procedure names and optional arguments. Procedures are called in the order their are given in this list. It is important to provide the respective target dataset to run a procedure on as the dataset argument of the main command. [Default: None]
  • result_filter (callable or None, optional) – if given, each to-be-returned status dictionary is passed to this callable, and is only returned if the callable’s return value does not evaluate to False or a ValueError exception is raised. If the given callable supports **kwargs it will additionally be passed the keyword arguments of the original API call. [Default: None]
  • result_renderer ({'default', 'json', 'json_pp', 'tailored'} or None, optional) – format of return value rendering on stdout. [Default: None]
  • result_xfm ({'paths', 'relpaths', 'datasets', 'successdatasets-or-none', 'metadata'} or callable or None, optional) – if given, each to-be-returned result status dictionary is passed to this callable, and its return value becomes the result instead. This is different from result_filter, as it can perform arbitrary transformation of the result value. This is mostly useful for top- level command invocations that need to provide the results in a particular format. Instead of a callable, a label for a pre-crafted result transformation can be given. [Default: None]
  • return_type ({'generator', 'list', 'item-or-list'}, optional) – return value behavior switch. If ‘item-or-list’ a single value is returned instead of a one-item return value list, or a list in case of multiple return values. None is return in case of an empty list. [Default: ‘list’]