datalad meta-add

Synopsis

datalad meta-add [-h] [-d DATASET] [-o] [-u] [-i] [--json-lines] [-b] [--version] METADATA [ADDITIONAL_VALUES]

Description

Add metadata to a dataset.

This command reads metadata from a source and adds this metadata to a dataset. The source can either be a file, or standard input. The metadata format is a string with the JSON-serialized dictionary, or list of dictionaries (or individual dictionaries in JSON-Lines format) that describes the metadata.

In case of an API-call metadata can also be provided in a python dictionary or a list of dictionaries.

If metadata is read from a source, additional parameter can overwrite or amend information that is provided by the source.

The ADDITIONAL_VALUES arguments can be pre-fixed by ‘@’, in which case the pre-fixed argument is interpreted as a file-name and the argument value is read from the file.

The metadata key “dataset-id” must be identical to the ID of the dataset that receives the metadata, unless -i or –allow-id-mismatch is provided.

Examples

Add metadata stored in the file “metadata-123.json” to the dataset in the current directory.:

% datalad meta-add metadata-123.json

Add metadata stored in the JSON lines file “metadata-entries.jsonl” to the dataset in the current directory.:

% datalad meta-add --json-lines metadata-123.json

Add metadata stored in the file “metadata-123.json” to the dataset “/home/user/dataset_0”:

% datalad meta-add -d /home/user/dataset_0 metadata-123.json

Add metadata stored in the file “metadata-123.json” to the dataset in the current directory and overwrite the “dataset_id” value provided in “metadata-123.json”:

% datalad meta-add -d /home/user/dataset_0 -i metadata-123.json '{"dataset_id": "00010203-1011-2021-3031-404142434445"}'

Add metadata read from standard input to the dataset in the current directory:

% datalad meta-add -

Add metadata stored in the file “metadata-123.json” to the dataset in the current directory and overwrite the valuesfrom “metadata-123.json” with the values stored in “extra-info.json”:

% datalad meta-add metadata-123.json @extra-info.json

Options

METADATA

path of the file that contains the metadata that should be added to the metadata store (metadata records must be provided as a JSON-serialized metadata dictionary). The file may contain a single metadata-record or a JSON-array with multiple metadata-records. If –json-lines is given, the file may also contain one dictionary per line. If the path is “-”, the metadata file is read from standard input. The dictionary must contain the following keys: ‘type’ ‘extractor_name’ ‘extractor_version’ ‘extraction_parameter’ ‘extraction_time’ ‘agent_name’ ‘agent_email’ ‘dataset_id’ ‘dataset_version’ ‘extracted_metadata’ If the metadata is associated with a file, the following key indicates the file path: ‘path’ If the metadata should refer to a sub-dataset element (that means if an “aggregated” record should be stored (see meta-aggregate for more info), the following key indicates the path of the sub-dataset from the root of the “containing dataset”: ‘dataset_path’ The containing dataset, aka. the “root dataset”, is the dataset version, specified by dataset-id and version that contains the given sub-dataset version at the path ‘dataset_path’. If the containing dataset, aka. the “root dataset”, is known, it can be specified by providing the following keys: ‘root_dataset_id’ ‘root_dataset_version’ If the version of the root dataset that contains the given subdataset at the given path is not known, the “root_dataset_*”-keys can be omitted. Such a situation might arise, if the sub-dataset metadata was extracted in an old version of the sub- dataset, and the relation of this old version to the root-dataset is not known (we assume that the root-dataset would be the dataset to which the metadata is added.) In this case the metadata is added with an “anonymous” root dataset, but with the given sub-dataset-path. (This makes sense, if the sub-dataset at the given path contains a version that is defined in the metadata, i.e. in dataset_version. The metadata will then be added to all versions that meet this condition.). Constraints: value must be a string

ADDITIONAL_VALUES

A string that contains a JSON serialized dictionary of key value-pairs. These key values-pairs are used in addition to the key value pairs in the metadata dictionary to describe the metadata that should be added. If an additional key is already present in the metadata, an error is raised, unless -o, –allow- override is provided. In this case, the additional values will override the value in metadata and a warning is issued. NB! If multiple records are provided in METADATA, the additional values will be applied to all of them. Constraints: value must be a string or value must be NONE

-h, –help, –help-np

show this help message. –help-np forcefully disables the use of a pager for displaying the help message

-d DATASET, –dataset DATASET

“dataset to which metadata should be added. If not provided, the dataset is assumed to be given by the current directory. Constraints: Value must be a Dataset or a valid identifier of a Dataset (e.g. a path) or value must be NONE

-o, –allow-override

Allow the additional values to override values given in metadata.

-u, –allow-unknown

Allow unknown keys. By default, unknown keys generate an errors. If this switch is True, unknown keys will only be reported. For processing unknown keys will be ignored.

-i, –allow-id-mismatch

Allow insertion of metadata, even if the “dataset-id” in the metadata source does not match the ID of the target dataset.

–json-lines

Interpret metadata input as JSON lines. i.e. expect one metadata record per line. This is the format used by commands like “datalad meta-dump”.

-b, –batch-mode

Enable batch mode. In batch mode metadata-records are read from stdin, one record per line, and a result is written to stdout, one result per line. Batch mode can be exited by sending an empty line that just consists of a newline. Meta-add will return an empty line that just consists of a newline to confirm the exit request. When this flag is given, the metadata file name should be set to “-” (minus).

–version

show the module and its version which provides the command

Authors

datalad is developed by The DataLad Team and Contributors <team@datalad.org>.