Skip to content

Commit

Permalink
Merge pull request #536 from SabrinaZander/elab2ARCtool
Browse files Browse the repository at this point in the history
elab2ARC tool
  • Loading branch information
Brilator authored Dec 10, 2024
2 parents b0589fc + 74a943f commit 7351776
Show file tree
Hide file tree
Showing 11 changed files with 124 additions and 0 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added src/assets/images/elab2arc/elabftw-id.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
title: Project Access Token
authors:
- sabrina-zander
lastUpdated: 2024-11-28
---

import { Steps } from '@astrojs/starlight/components';

This guide shows you how to generate a Project Access Token. Project Access Token are similar to [Personal Access Token](/nfdi4plants.knowledgebase/datahub/navigation-settings/datahub-generate-pat), but Project Access Tokens are scoped to a project, so you cannot use them to access resources from other projects.

<Steps>

1. Sign in to the [DataHUB](https://git.nfdi4plants.org/)
2. Open your Project (ARC)
3. Navigate to 'Settings' (1), then 'Access Tokens' (2): ![Alt text](@images/arc-commander/arc-commander-access-2.png)
3. Choose a name for your token (1), set an expiration date (optional) (2), assign a role (3), configure permissions (4), and create your token (5):
![Alt text](@images/arc-commander/arc-commander-access-3.png)
4. Once created, you can copy the token:
![Alt text](@images/arc-commander/arc-commander-access-4.png)

</Steps>

:::note
Make sure you save your token, because it is only displayed once.
:::
2 changes: 2 additions & 0 deletions src/content/docs/fundamentals/electronic-lab-notebooks.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ ELNs are designed to help overcome the limitations of classical pen-and-paper la

Protocols generated in ELNs integrate well with the [Annotated Research Context (ARC)](/nfdi4plants.knowledgebase/core-concepts/arc). Protocols and the metadata they contain have a dedicated place and schema within the ARC. Each study and assay subfolder in the ARC comes with a folder called `protocols`. This folder collects all protocols associated with a particular set of data files in the `dataset` folder. In addition, the `isa.study.xlsx` and `isa.assay.xlsx` workbooks associated with each study or assay, help to communicate the protocol metadata in a structured and machine-readable way.

To simplify the integration of protocols and metadata from ELNs into ARC, DataPLANT has delevoped the tool [elab2ARC](/nfdi4plants.knowledgebase/resources/elab2arc/), which automates the conversion of [eLabFTW](https://www.elabftw.net/) experiments into ARC format.

Since ARCs are designed as FAIR digital objects which combine the data files with the metadata into one research data package, they facilitate the storage, shareing, and reuse of the research data as a whole.

![Arcify Protocols](@images/fundamentals/protocols2arc.drawio.png)
96 changes: 96 additions & 0 deletions src/content/docs/resources/elab2ARC.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
---
title: elab2ARC
lastUpdated: 2024-11-28
description: elab2ARC is a tool for converting eLabFTW experiment data into ARC format.
authors:
- xiaoran-zhou
- sabrina-zander
sidebar:
badge:
text: new
variant: tip
---

import { Steps } from '@astrojs/starlight/components';

[elab2ARC](http://nfdi4plants.org/elab2arc) is a tool for converting [eLabFTW](https://www.elabftw.net/) experiments into [Annotated Research Context (ARC)](/nfdi4plants.knowledgebase/core-concepts/arc/) format. It allows researchers to easily fetch experiment data from eLabFTW instances and update or push these files to DataHUB in ARC format. The application streamlines the management of data from lab experiments to repositories, improving efficiency in scientific data handling.

## Why is elab2ARC needed?

Many researchers rely on [electronic lab notebooks (ELNs)](/nfdi4plants.knowledgebase/fundamentals/electronic-lab-notebooks/) to document their experiments due to their flexibility in handling free-form text and supporting data files up to 100 MB. However, when metadata standardization becomes necessary or when raw data files exceed the file size limit, other research data standards, such as Annotated Research Context (ARC), may be required.

ARC offers robust metadata annotation tools and customizable templates that help standardize the unstructured content typically found in ELNs. Additionally, ARC supports large files up to 10 GB, making it an excellent complement to ELNs like eLabFTW, which are not designed to handle raw data storage or larger file sizes.

Manually transferring metadata from ELNs to ARC can be both error-prone and time-consuming. To address this challenge, elab2ARC provides an efficient solution by automating the extraction, transformation, and loading (ETL) of data from ELN records into ARC-compliant structures. This tool streamlines the process, reduces errors, and ensures consistency, enhancing research data management workflows.

## Features of elab2ARC

**elab2ARC** simplifies converting eLabFTW experiment data into Annotated Research Context (ARC) format. Key features include:

- **Direct Data Retrieval**: Extracts experiment data from eLabFTW using experiment IDs and API keys.
- **ARC Conversion**: Transforms data into ARC format, organizing protocols, datasets, and metadata.
- **DataHub Integration**: Automates ARC file updates and pushes them to a DataHub repository.
- **Multi-Instance Support**: Manages data from multiple eLabFTW instances seamlessly.

These features streamline workflows, ensuring efficient and standardized data management.

## Setting up elab2ARC


### Create an Project Access Token in DataHUB

import DataHubPat from '../datahub/navigation-settings/datahub-generate-project-access-token.mdx'

<DataHubPat />

### Create an eLabFTW API key

<Steps>
1. Login to your elabFTW instance
2. Navigate to 'User Settings'
![](@images/elab2arc/elabftw-user-settings.png)
3. Navigate to 'API KEYS' Tab. Enter a name and a **Read Only** permissions
![](@images/elab2arc/elabftw-user-api-key.png)
4. Save the key securely, as it will only be displayed once. Treat it like a password, as it grants access to eLabFTW.
![](@images/elab2arc/elabftw-user-api-key-2.png)

</Steps>

### Modify eLabFTW experiment


<Steps>
1. Select the eLabFTW experiment, which you want to convert and go to edit mode
2. Add an extra field to your eLabFTW experiment (bottom)
![](@images/elab2arc/elabftw-add-extra-field.png)
3. Use field type **URL** and as a name **datahub_url** and enter the **URL of your ARC** as the default value
![](@images/elab2arc/elabftw-extra-field-2.png)
4. Save the entry

</Steps>

## How to use elab2ARC

<Steps>
1. Select your elabFTW Instance or add your instance [1]
2. Paste eLabFTW API key [2]
3. Paste DataHUB Project Access Token [3]
![](@images/elab2arc/elab2arc-tool-usage.png)

4. Enter eLabFTW Experiment ID [4] (use comma to seperate multiple IDs) which can be found in the experiment settings. Make sure that you have added the DataHUB URL as an extra field before.
![](@images/elab2arc/elabftw-id.png)
5. Click **One Click Submission** [5] to start the converting.

</Steps>

## Conversion

![](@images/elab2arc/elab2arc-conversion.png)

The elab2ARC tool will automatically convert your elabFTW experiments into ARC format
- create a new assay folder with eLabFTW experiment name as assay name (green)
- create the assay folder structure (dataset/protocols/isa.assay)
- convert experiment main text into a .md file and store it in the protocol folder (purple)
- add all attachments of the eLabFTW experiment into the dataset folder (yellow)
- enter name/email/affiliation of the eLabFTW experiment metadata into the isa.assay sheet

0 comments on commit 7351776

Please sign in to comment.