Skip to content

Commit

Permalink
Merge pull request #12 from eliteportal/dev
Browse files Browse the repository at this point in the history
Added biospecimen human dependencies to assays
  • Loading branch information
nlee-sage authored Nov 14, 2023
2 parents 1a044e4 + 7c8bbc4 commit 8f63cd8
Show file tree
Hide file tree
Showing 7 changed files with 985 additions and 11 deletions.
75 changes: 75 additions & 0 deletions .github/workflows/create-template-config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# --------------------------------------------------------------------------------------------------
# GitHub Action to create a DCA template config json file for a data model
#
# This action creates a json file named with the `file` input argument using
# the data model supplied to the `data_model` argument. It will validate the
# json file against DCA's template config schema. Finally, it will create a PR
# in the repo for the new file.
#
# Copy this into your data model repo .github/workflow directory.
# Your repo settings must have Actions enabled and must allow GitHub Actions to
# create and approve pull requests.
#
# By default, this action runs by workflow dispatch. But it can be configured
# to run on other triggers. Consult the github doc below for more information.
# https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflow_dispatch
#
# The resulting file with contain one entry for each attribute in the data model that `dependsOn`
# "Component". To include a subset of these attributes, use either `include_schemas` or
# `exclude_schemas` in the call to `datacurator::write_dca_template_config()`
#
# --------------------------------------------------------------------------------------------------

name: DCA Template Config File
on:
workflow_dispatch:
inputs:
data_model:
description: URL to a jsonld data model file
required: true
file:
description: Directory to save the template config
required: true
include_schemas:
description: Space-separated string of schemas to include in output. Must be empty if using exclude_schemas.
required: false
exclude_schemas:
description: Space-separated string of schemas to exclude from output. Must be empty if using include_schemas.
required: false

jobs:
create-template-config:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Checkout DCA Config Repo for Schema
uses: actions/checkout@v4
with:
repository: 'Sage-Bionetworks/data_curator_config'
ref: 'dev'
path: 'data_curator_config'

- name: Create config file
uses: sage-bionetworks/dca-template-config-action@main
env:
data_model: ${{ inputs.data_model }}
file: ${{ inputs.file }}
include_schemas: ${{ inputs.include_schemas }}
exclude_schemas: ${{ inputs.exclude_schemas }}

- name: Validate Config File
uses: docker://orrosenblatt/validate-json-action:latest
env:
INPUT_SCHEMA: 'data_curator_config/schemas/dca_template_config.schema.json'
INPUT_JSONS: ${{ inputs.file }}

- name: Open PR
uses: peter-evans/create-pull-request@v5
with:
title: Update DCA Template Config File
body: Recreate the json file that populates the DCA template dropdown menu.
delete-branch: true
branch-suffix: timestamp
add-paths: |
${{ inputs.file }}
16 changes: 8 additions & 8 deletions EL.data.model.csv
Original file line number Diff line number Diff line change
Expand Up @@ -256,14 +256,14 @@ Biospecimen human,Template used for contributing metadata to the ELITE portal,,"
Biospecimen nonHuman,Template used for contributing metadata to the ELITE portal,,"individualID,specimenID,specimenIdSource,dataGenerationSite,specimenType,assay,specimenAge,parentSpecimenID,parentSpecimenType,laboratoryProcedure,samplingAge,organ,tissue,tissueVolume,tissueWeight,specimenSource,nucleicAcidSource,cellType,fastingState,isPostMortem,Component",,True,Template,Individual nonHuman,,,Template,,,
Individual Human,Template used for contributing metadata to the ELITE portal,,"individualID,cohort,studyCode,fieldCenterCode,visitCode,countryCode,consentGroupID,speciesGroup,sex,race,ethnicity,ethnicGroupCode,age,diagnosisStatus,diagnosis,Component",,True,Template,,,,Template,,,
Individual nonHuman,Template used for contributing metadata to the ELITE portal,,"individualID,cohort,studyCode,taxon,speciesGroup,speciesName,commonName,sex,lifeStage,speciesAge,captivityStatus,captivityDuration,Component",,True,Template,,,,Template,,,
Metabolomics Human,Template used for contributing metadata to the ELITE portal,,"Filename,specimenID,sampleType,specifySampleType,specifyMeasurementTechnique,technologyPlatformVersion,specifyPlatformVersion,platformLocation,specifyPlatformLocation,msTarget,msAnalyzerTypeMS1,specifyMSAnalyzerTypeMS1,msAnalyzerTypeMS2,specifyMSAnalyzerTypeMS2,msAssayTechnique,specifyMSAssayTechnique,msAnalyteType,specifyMSAnalyteType,msInstrumentModel,specifyMSInstrumentModel,sampleIntroduction,specifySampleIntroduction,extractionMethod,acquisitionMode,acquisitionSoftware,acquistionSoftwareVersion,samplePrepProtocol,databaseName,specifyDatabaseName,databaseSource,specifyDatabaseSource,databaseWeblink,spectrometerFrequency,hasIonizationSource,ionProperty,vacuumPressure,vacuumPressureUnit,specifyVacuumPressueUnits,lensVoltages,lensVoltagesUnit,specifyLensVoltageUnits,gasFlowTemperature,gasFlowTemperatureUnit,hasAssayControl,controlType,specifyControlType,experiementalBatchSize,experiementalBatchSizeUnit,specifyExperimentalBatchSizeUnits,batchSize,batchSizeUnit,specifyBatchSizeUnits,batchID,batchLabel,sampleBatchID,sampleBatchSize,sampleBatchSizeUnit,specifySampleBatchSizeUnits,acquisitionBatchID,acquisitionBatchSize,acquisitionBatchSizeUnit,specifyAcquisitionBatchSizeUnits,processingBatchID,processingBatchSize,processingBatchSizeUnit,specifyProcessingBatchSizeUnit,Component",,True,Template,,,,Template,,,
Microbiome,Template used for contributing metadata to the ELITE portal,,"Filename,specimenID,sampleType,specifySampleType,specifyMeasurementTechnique,technologyPlatformVersion,specifyPlatformVersion,platformLocation,specifyPlatformLocation,extractionMethod,specifyExtractionMethod,libraryBatchID,sequencingBatchID,sequencingBatchSize,sequencingBatchSizeUnit,specifySequencingBatchSizeUnit,dnaBatchID,dnaBatchSize,dnaBatchSizeUnit,specifyDNABatchSizeUnit,libraryPrep,specifyLibraryPrep,libraryPreparationMethod,specifyLibraryPreparationMethod,libraryVersion,specifyLibraryVersion,isStranded,readStrandOrigin,readLength,readLengthUnits,specifyReadLengthUnits,runType,totalReads,Component",,True,Template,,,,Template,,,
RNAseq,Template used for contributing metadata to the ELITE portal,,"Filename,specimenID,sampleType,specifySampleType,specifyMeasurementTechnique,technologyPlatformVersion,specifyPlatformVersion,platformLocation,specifyPlatformLocation,referenceTranscriptID,repositoryName,specifyRepositoryName,transcriptType,specifyTranscriptType,resultUnit,valueReported,libraryPreparationMethod,specifyLibraryPreparationMethod,libraryVersion,specifyLibraryVersion,isStranded,readStrandOrigin,readLength,readLengthUnits,specifyReadLengthUnits,runType,totalReads,Component",,True,Template,,,,Template,,,
Whole Genome Sequencing,Template used for contributing metadata to the ELITE portal,,"Filename,specimenID,sampleType,specifySampleType,specifyMeasurementTechnique,technologyPlatformVersion,specifyPlatformVersion,platformLocation,specifyPlatformLocation,referenceTranscriptID,repositoryName,specifyRepositoryName,transcriptType,specifyTranscriptType,resultUnit,valueReported,rnaBatchID,libraryBatchID,sequencingBatchID,libraryPrep,specifyLibraryPrep,libraryPreparationMethod,specifyLibraryPreparationMethod,libraryVersion,specifyLibraryVersion,isStranded,readStrandOrigin,readLength,readLengthUnits,specifyReadLengthUnits,runType,totalReads,Component",,True,Template,,,,Template,,,
bsSeq,bisulfite-seq WGBS methylseq methylomics,,"Filename,specimenID,sampleType,specifySampleType,specifyMeasurementTechnique,technologyPlatformVersion,specifyPlatformVersion,platformLocation,specifyPlatformLocation,referenceTranscriptID,dnaBatchID,dnaBatchID,dnaBatchSize,dnaBatchSizeUnit,specifyDNABatchSizeUnit,libraryBatchID,libraryPrep,specifyLibraryPrep,libraryPreparationMethod,specifyLibraryPreparationMethod,libraryVersion,specifyLibraryVersion,sequencingBatchID,sequencingBatchSize,sequencingBatchSizeUnit,specifySequencingBatchSizeUnit,libraryPreparationMethod,specifyLibraryPreparationMethod,libraryVersion,specifyLibraryVersion,isStranded,readStrandOrigin,readLength,readLengthUnits,specifyReadLengthUnits,runType,totalReads,directionalBSseqLibrary,gDNAconc,lambdaDNAconc,pcrCycles,meanCoverage,conversionRatio,conversionRatioUnits,specifyConversionRatioUnit,enrichmentMethod,specifyEnrichmentMethod,Component",,True,Template,,,,Template,,,
genotyping,Template used for contributing metadata to the ELITE portal,,"Filename,specimenID,sampleType,specifySampleType,useReagent,reagentID(s),reagentName,reagentManufacturer,reagentCatalogNumber,reagentLotNumber,reagentWeblink,reagentContact,useTreatment,treatmentID(s),treatmentName,treatmentAmountValue,treatmentAmountUnit,treatmentDurationValue,treatmentDurationUnit,treatmentTemperatureValue,treatmentTemperatureUnit,specifyMeasurementTechnique,Component",,True,Template,,,,Template,,,
proteomics,Template used for contributing metadata to the ELITE portal,,"Filename,specimenID,sampleType,specifySampleType,specifyMeasurementTechnique,technologyPlatformVersion,specifyPlatformVersion,platformLocation,specifyPlatformLocation,msTarget,msAnalyzerTypeMS1,specifyMSAnalyzerTypeMS1,msAnalyzerTypeMS2,specifyMSAnalyzerTypeMS2,msAssayTechnique,specifyMSAssayTechnique,msAnalyteType,specifyMSAnalyteType,protoemicsAssayType,specifyProteomicsAssayType,msInstrumentModel,specifyMSInstrumentModel,digestionMethod,specifyDigestionMethod,fractionIdentifier,acquisitionMode,acquisitionSoftware,acquistionSoftwareVersion,samplePrepProtocol,databaseName,specifyDatabaseName,databaseSource,specifyDatabaseSource,databaseWeblink,spectrometerFrequency,modificationParameters,specifyOtherModificationParameters,cleavageAgents,specifyOtherCleavageAgent,fragmentMassTolerance,precursorMassTolerance,dataFile,Component",,True,Template,,,,Template,,,
scRNAseq,Template used for contributing metadata to the ELITE portal,,"Filename,specimenID,sampleType,specifySampleType,specifyMeasurementTechnique,technologyPlatformVersion,specifyPlatformVersion,platformLocation,specifyPlatformLocation,referenceTranscriptID,repositoryName,specifyRepositoryName,transcriptType,specifyTranscriptType,resultUnit,valueReported,rnaBatchID,rnaBatchSize,rnaBatchSizeUnit,specifyRNABatchSizeUnit,libraryBatchID,sequencingBatchID,sequencingBatchSize,sequencingBatchSizeUnit,specifySequencingBatchSizeUnit,libraryPrep,specifyLibraryPrep,libraryPreparationMethod,specifyLibraryPreparationMethod,libraryVersion,specifyLibraryVersion,isStranded,readStrandOrigin,readLength,readLengthUnits,specifyReadLengthUnits,runType,totalReads,Component",,True,Template,,,,Template,,,
Metabolomics Human,Template used for contributing metadata to the ELITE portal,,"Filename,specimenID,sampleType,specifySampleType,specifyMeasurementTechnique,technologyPlatformVersion,specifyPlatformVersion,platformLocation,specifyPlatformLocation,msTarget,msAnalyzerTypeMS1,specifyMSAnalyzerTypeMS1,msAnalyzerTypeMS2,specifyMSAnalyzerTypeMS2,msAssayTechnique,specifyMSAssayTechnique,msAnalyteType,specifyMSAnalyteType,msInstrumentModel,specifyMSInstrumentModel,sampleIntroduction,specifySampleIntroduction,extractionMethod,acquisitionMode,acquisitionSoftware,acquistionSoftwareVersion,samplePrepProtocol,databaseName,specifyDatabaseName,databaseSource,specifyDatabaseSource,databaseWeblink,spectrometerFrequency,hasIonizationSource,ionProperty,vacuumPressure,vacuumPressureUnit,specifyVacuumPressueUnits,lensVoltages,lensVoltagesUnit,specifyLensVoltageUnits,gasFlowTemperature,gasFlowTemperatureUnit,hasAssayControl,controlType,specifyControlType,experiementalBatchSize,experiementalBatchSizeUnit,specifyExperimentalBatchSizeUnits,batchSize,batchSizeUnit,specifyBatchSizeUnits,batchID,batchLabel,sampleBatchID,sampleBatchSize,sampleBatchSizeUnit,specifySampleBatchSizeUnits,acquisitionBatchID,acquisitionBatchSize,acquisitionBatchSizeUnit,specifyAcquisitionBatchSizeUnits,processingBatchID,processingBatchSize,processingBatchSizeUnit,specifyProcessingBatchSizeUnit,Component",,True,Template,Biospecimen human,,,Template,,,
Microbiome,Template used for contributing metadata to the ELITE portal,,"Filename,specimenID,sampleType,specifySampleType,specifyMeasurementTechnique,technologyPlatformVersion,specifyPlatformVersion,platformLocation,specifyPlatformLocation,extractionMethod,specifyExtractionMethod,libraryBatchID,sequencingBatchID,sequencingBatchSize,sequencingBatchSizeUnit,specifySequencingBatchSizeUnit,dnaBatchID,dnaBatchSize,dnaBatchSizeUnit,specifyDNABatchSizeUnit,libraryPrep,specifyLibraryPrep,libraryPreparationMethod,specifyLibraryPreparationMethod,libraryVersion,specifyLibraryVersion,isStranded,readStrandOrigin,readLength,readLengthUnits,specifyReadLengthUnits,runType,totalReads,Component",,True,Template,Biospecimen human,,,Template,,,
RNAseq,Template used for contributing metadata to the ELITE portal,,"Filename,specimenID,sampleType,specifySampleType,specifyMeasurementTechnique,technologyPlatformVersion,specifyPlatformVersion,platformLocation,specifyPlatformLocation,referenceTranscriptID,repositoryName,specifyRepositoryName,transcriptType,specifyTranscriptType,resultUnit,valueReported,libraryPreparationMethod,specifyLibraryPreparationMethod,libraryVersion,specifyLibraryVersion,isStranded,readStrandOrigin,readLength,readLengthUnits,specifyReadLengthUnits,runType,totalReads,Component",,True,Template,Biospecimen human,,,Template,,,
Whole Genome Sequencing,Template used for contributing metadata to the ELITE portal,,"Filename,specimenID,sampleType,specifySampleType,specifyMeasurementTechnique,technologyPlatformVersion,specifyPlatformVersion,platformLocation,specifyPlatformLocation,referenceTranscriptID,repositoryName,specifyRepositoryName,transcriptType,specifyTranscriptType,resultUnit,valueReported,rnaBatchID,libraryBatchID,sequencingBatchID,libraryPrep,specifyLibraryPrep,libraryPreparationMethod,specifyLibraryPreparationMethod,libraryVersion,specifyLibraryVersion,isStranded,readStrandOrigin,readLength,readLengthUnits,specifyReadLengthUnits,runType,totalReads,Component",,True,Template,Biospecimen human,,,Template,,,
bsSeq,bisulfite-seq WGBS methylseq methylomics,,"Filename,specimenID,sampleType,specifySampleType,specifyMeasurementTechnique,technologyPlatformVersion,specifyPlatformVersion,platformLocation,specifyPlatformLocation,referenceTranscriptID,dnaBatchID,dnaBatchID,dnaBatchSize,dnaBatchSizeUnit,specifyDNABatchSizeUnit,libraryBatchID,libraryPrep,specifyLibraryPrep,libraryPreparationMethod,specifyLibraryPreparationMethod,libraryVersion,specifyLibraryVersion,sequencingBatchID,sequencingBatchSize,sequencingBatchSizeUnit,specifySequencingBatchSizeUnit,libraryPreparationMethod,specifyLibraryPreparationMethod,libraryVersion,specifyLibraryVersion,isStranded,readStrandOrigin,readLength,readLengthUnits,specifyReadLengthUnits,runType,totalReads,directionalBSseqLibrary,gDNAconc,lambdaDNAconc,pcrCycles,meanCoverage,conversionRatio,conversionRatioUnits,specifyConversionRatioUnit,enrichmentMethod,specifyEnrichmentMethod,Component",,True,Template,Biospecimen human,,,Template,,,
genotyping,Template used for contributing metadata to the ELITE portal,,"Filename,specimenID,sampleType,specifySampleType,useReagent,reagentID(s),reagentName,reagentManufacturer,reagentCatalogNumber,reagentLotNumber,reagentWeblink,reagentContact,useTreatment,treatmentID(s),treatmentName,treatmentAmountValue,treatmentAmountUnit,treatmentDurationValue,treatmentDurationUnit,treatmentTemperatureValue,treatmentTemperatureUnit,specifyMeasurementTechnique,Component",,True,Template,Biospecimen human,,,Template,,,
proteomics,Template used for contributing metadata to the ELITE portal,,"Filename,specimenID,sampleType,specifySampleType,specifyMeasurementTechnique,technologyPlatformVersion,specifyPlatformVersion,platformLocation,specifyPlatformLocation,msTarget,msAnalyzerTypeMS1,specifyMSAnalyzerTypeMS1,msAnalyzerTypeMS2,specifyMSAnalyzerTypeMS2,msAssayTechnique,specifyMSAssayTechnique,msAnalyteType,specifyMSAnalyteType,protoemicsAssayType,specifyProteomicsAssayType,msInstrumentModel,specifyMSInstrumentModel,digestionMethod,specifyDigestionMethod,fractionIdentifier,acquisitionMode,acquisitionSoftware,acquistionSoftwareVersion,samplePrepProtocol,databaseName,specifyDatabaseName,databaseSource,specifyDatabaseSource,databaseWeblink,spectrometerFrequency,modificationParameters,specifyOtherModificationParameters,cleavageAgents,specifyOtherCleavageAgent,fragmentMassTolerance,precursorMassTolerance,dataFile,Component",,True,Template,Biospecimen human,,,Template,,,
scRNAseq,Template used for contributing metadata to the ELITE portal,,"Filename,specimenID,sampleType,specifySampleType,specifyMeasurementTechnique,technologyPlatformVersion,specifyPlatformVersion,platformLocation,specifyPlatformLocation,referenceTranscriptID,repositoryName,specifyRepositoryName,transcriptType,specifyTranscriptType,resultUnit,valueReported,rnaBatchID,rnaBatchSize,rnaBatchSizeUnit,specifyRNABatchSizeUnit,libraryBatchID,sequencingBatchID,sequencingBatchSize,sequencingBatchSizeUnit,specifySequencingBatchSizeUnit,libraryPrep,specifyLibraryPrep,libraryPreparationMethod,specifyLibraryPreparationMethod,libraryVersion,specifyLibraryVersion,isStranded,readStrandOrigin,readLength,readLengthUnits,specifyReadLengthUnits,runType,totalReads,Component",,True,Template,Biospecimen human,,,Template,,,
AU/ml,,,,,False,ValidValue,,,,Unit,NUMERIC,,
DK units/ml,,,,,False,ValidValue,,,,Unit,NUMERIC,,
Days,,,,,False,ValidValue,,,,Unit,NUMERIC,,
Expand Down
17 changes: 15 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,21 +26,34 @@ After cloning the repository, run the following command:

Run `scripts/convert_csv_to_json.py`

### Create Data Model Visualization Tree

[Schematic API](https://schematic.api.sagebionetworks.org/v1/ui/)
[Visualization Repository](https://github.com/Sage-Bionetworks/schema_visualization)

# Updates

## Oct 20th, 2023

Attribute renaming to harmonize with current backend and ADKP

- consortium -> program
- studyCode -> project
- study -> studyKey

New Terms

- 'studyFocs'
- valid values are 'Longevity' currently.
- valid values are 'Longevity' currently.

Attribute Changes:

Attribute Changes:
- Remove valid values from speciesName

File Annotation Updates

- change "species" to "speciesGroup"

## To Do

- [ ] Organize directories/files
12 changes: 12 additions & 0 deletions _data/manifest_generation_results.txt
Original file line number Diff line number Diff line change
Expand Up @@ -57,3 +57,15 @@ BsSeq : /Users/nlee/Documents/Projects/ELITE/ELITE-data-models/EL.xlsx
Genotyping : /Users/nlee/Documents/Projects/ELITE/ELITE-data-models/EL.xlsx
Proteomics : /Users/nlee/Documents/Projects/ELITE/ELITE-data-models/EL.xlsx
ScRNAseq : /Users/nlee/Documents/Projects/ELITE/ELITE-data-models/EL.xlsx
Biospecimenhuman : /Users/nlee/Documents/Projects/ELITE/ELITE-data-models/EL.xlsx
BiospecimennonHuman : /Users/nlee/Documents/Projects/ELITE/ELITE-data-models/EL.xlsx
IndividualHuman : /Users/nlee/Documents/Projects/ELITE/ELITE-data-models/EL.xlsx
IndividualnonHuman : /Users/nlee/Documents/Projects/ELITE/ELITE-data-models/EL.xlsx
MetabolomicsHuman : /Users/nlee/Documents/Projects/ELITE/ELITE-data-models/EL.xlsx
Microbiome : /Users/nlee/Documents/Projects/ELITE/ELITE-data-models/EL.xlsx
RNAseq : /Users/nlee/Documents/Projects/ELITE/ELITE-data-models/EL.xlsx
WholeGenomeSequencing : /Users/nlee/Documents/Projects/ELITE/ELITE-data-models/EL.xlsx
BsSeq : /Users/nlee/Documents/Projects/ELITE/ELITE-data-models/EL.xlsx
Genotyping : /Users/nlee/Documents/Projects/ELITE/ELITE-data-models/EL.xlsx
Proteomics : /Users/nlee/Documents/Projects/ELITE/ELITE-data-models/EL.xlsx
ScRNAseq : /Users/nlee/Documents/Projects/ELITE/ELITE-data-models/EL.xlsx
2 changes: 2 additions & 0 deletions _scripts/get-dca-config-workflow.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#! bin/bash
wget -O ../.github/workflows/create-template-config.yml https://raw.githubusercontent.com/Sage-Bionetworks/data-models/2f863dd7d767c612c8419e6ad83877b0b2fa15f2/.github/workflows/create-template-config.yml
3 changes: 3 additions & 0 deletions dev/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Development

This folder is used as the directory to hold all code used to create the data models.
Loading

0 comments on commit 8f63cd8

Please sign in to comment.