diff --git a/docs/source/diagrams/high_level/multimodal_1a.drawio b/docs/source/diagrams/high_level/multimodal_1a.drawio new file mode 100644 index 0000000..d846fe1 --- /dev/null +++ b/docs/source/diagrams/high_level/multimodal_1a.drawio @@ -0,0 +1,37 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/source/diagrams/high_level/multimodal_1a.drawio.png b/docs/source/diagrams/high_level/multimodal_1a.drawio.png new file mode 100644 index 0000000..1bcb001 Binary files /dev/null and b/docs/source/diagrams/high_level/multimodal_1a.drawio.png differ diff --git a/docs/source/diagrams/high_level/multimodal_1b.drawio b/docs/source/diagrams/high_level/multimodal_1b.drawio new file mode 100644 index 0000000..d614434 --- /dev/null +++ b/docs/source/diagrams/high_level/multimodal_1b.drawio @@ -0,0 +1,106 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/source/diagrams/high_level/multimodal_1b.drawio.png b/docs/source/diagrams/high_level/multimodal_1b.drawio.png new file mode 100644 index 0000000..aff7447 Binary files /dev/null and b/docs/source/diagrams/high_level/multimodal_1b.drawio.png differ diff --git a/docs/source/diagrams/high_level/multimodal_2a.drawio b/docs/source/diagrams/high_level/multimodal_2a.drawio new file mode 100644 index 0000000..1b72f8f --- /dev/null +++ b/docs/source/diagrams/high_level/multimodal_2a.drawio @@ -0,0 +1,55 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/source/diagrams/high_level/multimodal_2a.drawio.png b/docs/source/diagrams/high_level/multimodal_2a.drawio.png new file mode 100644 index 0000000..f768353 Binary files /dev/null and b/docs/source/diagrams/high_level/multimodal_2a.drawio.png differ diff --git a/docs/source/diagrams/high_level/multimodal_2b.drawio b/docs/source/diagrams/high_level/multimodal_2b.drawio new file mode 100644 index 0000000..f1a5847 --- /dev/null +++ b/docs/source/diagrams/high_level/multimodal_2b.drawio @@ -0,0 +1,133 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/source/diagrams/high_level/multimodal_2b.drawio.png b/docs/source/diagrams/high_level/multimodal_2b.drawio.png new file mode 100644 index 0000000..ba05027 Binary files /dev/null and b/docs/source/diagrams/high_level/multimodal_2b.drawio.png differ diff --git a/docs/source/diagrams/index.md b/docs/source/diagrams/index.md index d609c5e..a37e066 100644 --- a/docs/source/diagrams/index.md +++ b/docs/source/diagrams/index.md @@ -24,6 +24,7 @@ components encompass how data are generated and move through the system: lab_management data_acquisition +multimodal_acquisition data_staging data_storage_processing dynamic_foraging diff --git a/docs/source/diagrams/multimodal_acquisition.md b/docs/source/diagrams/multimodal_acquisition.md new file mode 100644 index 0000000..c1e1880 --- /dev/null +++ b/docs/source/diagrams/multimodal_acquisition.md @@ -0,0 +1,33 @@ +# Multimodal Data Acquisition + +A single data acquisition session can use multiple modalities (e.g. behavior, fip, pophys) that record data simulatneously. In this case, there are several options for outputting metadata such that it adheres to the [aind-data-schema](https://aind-data-schema.readthedocs.io/en/latest/index.html) structure. The below diagrams use the `acquisition.json` file as an example, however the same patterns can be applied to `instrument.json` as well. + +## 1. Using AIND Data Schema + +### a. Single output file + +The most straightforward scenario requires all modalities to write to the same JSON metadata file. The output should be formatted according to `aind-data-schema`. + +![Multimodal acquisition 1a](high_level/multimodal_1a.drawio.png) + +### b. Per-modality output files + +Alternatively, each modality can create its own JSON file that aligns with `aind-data-schema`. The most important consideration when combining metadata from multiple modalities is ensuring that unique fields match across files. For example, the `start_time` of the session should match across all modalities used in that session. More details on the merging process can be found on the [Upload Data page](../acquire_upload/upload_data.md#merge-rules). + +![Multimodal acquisition 1b](high_level/multimodal_1b.drawio.png) + +## 2. Custom Schemas + +The data transfer service also supports the ability to use a custom schema output for each modality. This follows an `extractor/mapper` pattern where users can write code and define the model/contract that will extract data using the [aind-metadata-extractor](https://github.com/AllenNeuralDynamics/aind-metadata-extractor). The contract is then used by [aind-metadata-mapper](https://github.com/AllenNeuralDynamics/aind-metadata-mapper) where the extracted metadata are mapped into `aind-data-schema` compliant metadata. If you are interested in implementing this pattern for your modality, see the instructions on the [Acquire Data page](../acquire_upload/acquire_data.md#acquisition). + +### a. Single custom output file + +Similar to 1a, this scenario has all modalities writing to the same file, except this file outputs in a custom schema format. The result is passed to the mapper where a predefined data contract is used to convert the information to `aind-data-schema`. + +![Multimodal acquisition 2a](high_level/multimodal_2a.drawio.png) + +### b. Per-modality custom and standard output files + +The `extractor/mapper` pattern can also be implemented on the per-modality level. For example, we currently maintain a custom mapping for `fip`, defined in [aind_metadata_extractor.models](https://github.com/AllenNeuralDynamics/aind-metadata-extractor/blob/main/src/aind_metadata_extractor/models/fip.json). As described in 1b, each modality writes to its own JSON file. The modalities that have a data model will be passed through a mapper before all files are merged together. + +![Multimodal acquisition 2b](high_level/multimodal_2b.drawio.png) \ No newline at end of file