Merge pull request #46 from eliteportal/update-non-human-templates

I updated individual non-human and biospecimen attributes for LC metabolomics data contributions.

README.md

@@ -48,6 +48,8 @@ EL Metadata Dictionary is a [Jekyll](https://jekyllrb.com/) site utilizing [Just
 
 Interim process to update the metadata dictionary site after changes have been made to the data model:
 
+:note: Note: do this in a SEPARATE PR after changes to the data model are merged to main. The scripts to do this reference the data model at the url in `processess/.env`, which is the main branch of this repo. It's not the most elegant right now but keeping the data model updates and the dictionary site updates as separate steps will make rolling back errors easier while we shore up this process.
+
 1. Make a new branch. Run `poetry install` and then `poetry shell` on the command line to install dependencies and open a virtual environment.
 
 2. From the main data-models directory, run `python proccesses/data_manager.py`. This should update some files within `_data/`

modules/ManifestColumn/captivityStatus.csv

@@ -1,2 +1,2 @@
 Attribute,Description,Valid Values,DependsOn,DependsOn Component,Required,Parent,Validation Rules,Properties,Source,module,multivalue,columnType,Ontology,Notes,UsedIn
-captivityStatus,"The status of the individual with regard to captivity.note- Wild, captive, and stranded values are applicable, especially for marine mammals. Depending on life stage terminology for individual species, other values are possible. Please let the data curation team know.","Captive,Stranded,Wild",,,True,ManifestColumn,,ManifestColumn,,ManifestColumn,False,STRING,"https://www.animalaudiograms.org/audiogram_metadata_scheme#:~:text=of%20the%20animal.-,captivity%20status,-The%20status%20of", ,Individual_non_Human
+captivityStatus,"The status of the individual with regard to captivity.note- Wild, captive, and stranded values are applicable, especially for marine mammals. Depending on life stage terminology for individual species, other values are possible. Please let the data curation team know.","Captive,Stranded,Wild, Not applicable, Not collected, Not specified, Other, Unknown",,,True,ManifestColumn,,ManifestColumn,,ManifestColumn,False,STRING,"https://www.animalaudiograms.org/audiogram_metadata_scheme#:~:text=of%20the%20animal.-,captivity%20status,-The%20status%20of", ,Individual_non_Human

modules/ManifestColumn/lifeStage.csv

@@ -1,2 +1,2 @@
 Attribute,Description,Valid Values,DependsOn,DependsOn Component,Required,Parent,Validation Rules,Properties,Source,module,multivalue,columnType,Ontology,Notes,UsedIn
-lifeStage,The life stage of the individual.note- Other values are possible depending on life stage terminology for individual species. Please let the data curation team know.,"Adult,Juvenile,Post-Juvenile",,,True,ManifestColumn,,ManifestColumn,,ManifestColumn,False,STRING,"Biological Collections Ontology, https://www.animalaudiograms.org/audiogram_metadata_scheme#:~:text=range%3A%20female%3B%20male-,life%20stage,-The%20life%20stage, https://www.ebi.ac.uk/ols/ontologies/pato", ,Individual_non_Human
+lifeStage,The life stage of the individual.note- Other values are possible depending on life stage terminology for individual species. Please let the data curation team know.,"Adult,Juvenile,Post-Juvenile, Not applicable, Not collected, Not specified, Other, Unknown",,,True,ManifestColumn,,ManifestColumn,,ManifestColumn,False,STRING,"Biological Collections Ontology, https://www.animalaudiograms.org/audiogram_metadata_scheme#:~:text=range%3A%20female%3B%20male-,life%20stage,-The%20life%20stage, https://www.ebi.ac.uk/ols/ontologies/pato", ,Individual_non_Human

modules/biospecimen/cellType.csv

@@ -1,2 +1,2 @@
 Attribute,Description,Valid Values,DependsOn,DependsOn Component,Required,Parent,Validation Rules,Properties,Source,module,multivalue,columnType,Ontology,Notes,UsedIn
-cellType,Indicate the cell type.,"A549, arachnoid, astrocytes, B-lymphocytes, CD138+, CD8+ T-Cells, CNON, dopaminergic neurons, Embryonic stem cells, epithelial, epithelial-like, fibroblast, GABAergic neurons, glia, GLUtamatergic neurons, immune cell, iPSC, iPSC-derived astrocytes, iPSC-derived glia, iPSC-derived neuron, iPSC-derived neuronal progenitor cell, iPSC-derived telencephalic organoids, lymphoblast, lymphoblastoid cell line, macrophages, meningioma, microglia, monocytes, monocyte-derived microglia, NCX NES, NeuN-, NeuN+, neural progenitor cell, neuron, oligodendrocyte, peripheral blood mononuclear cell, polygonal, round, schwann, Schwann cell precusor, schwannoma, SH-SY5Y",,,True,ManifestColumn,,ManifestColumn,sage.annotations-experimentalData.cellType-0.0.9,biospecimen,False,STRING,"https://bioportal.bioontology.org/ontologies/CCF/?p=classes&conceptid=http%3A%2F%2Fpurl.org%2Fccf%2Fcell_type,https://www.ebi.ac.uk/ols/ontologies/ccf/terms?iri=http%3A%2F%2Fpurl.org%2Fccf%2Fcell_type", ,"Biospecimen_human,Biospecimen_non_Human"
+cellType,Indicate the cell type.,"A549, arachnoid, astrocytes, B-lymphocytes, CD138+, CD8+ T-Cells, CNON, dopaminergic neurons, Embryonic stem cells, epithelial, epithelial-like, fibroblast, GABAergic neurons, glia, GLUtamatergic neurons, immune cell, iPSC, iPSC-derived astrocytes, iPSC-derived glia, iPSC-derived neuron, iPSC-derived neuronal progenitor cell, iPSC-derived telencephalic organoids, lymphoblast, lymphoblastoid cell line, macrophages, meningioma, microglia, monocytes, monocyte-derived microglia, NCX NES, NeuN-, NeuN+, neural progenitor cell, neuron, oligodendrocyte, peripheral blood mononuclear cell, polygonal, round, schwann, Schwann cell precusor, schwannoma, SH-SY5Y, Not collected, Not specified, Not applicable, Other, Unknown",,,True,ManifestColumn,,ManifestColumn,sage.annotations-experimentalData.cellType-0.0.9,biospecimen,False,STRING,"https://bioportal.bioontology.org/ontologies/CCF/?p=classes&conceptid=http%3A%2F%2Fpurl.org%2Fccf%2Fcell_type,https://www.ebi.ac.uk/ols/ontologies/ccf/terms?iri=http%3A%2F%2Fpurl.org%2Fccf%2Fcell_type", ,"Biospecimen_human,Biospecimen_non_Human"

modules/biospecimen/organ.csv

@@ -1,2 +1,2 @@
 Attribute,Description,Valid Values,DependsOn,DependsOn Component,Required,Parent,Validation Rules,Properties,Source,module,multivalue,columnType,Ontology,Notes,UsedIn
-organ,Indicate the organ the specimen is from,"blood, bone marrow, brain, breast, Bursa Of Fabricius, cerebrospinal fluid, colon, kidney, large intestine, liver, lung, lymph node, mammary gland, nerves, nose, ovary, pancreas, prostate, skin, spleen",,,True,ManifestColumn,,ManifestColumn,sage.annotations-experimentalData.organ-0.0.4,biospecimen,False,STRING,http://purl.obolibrary.org/obo/NCIT_C13018, ,"Biospecimen_human,Biospecimen_non_Human"
+organ,Indicate the organ the specimen is from,"blood, bone marrow, brain, breast, Bursa Of Fabricius, cerebrospinal fluid, colon, kidney, large intestine, liver, lung, lymph node, mammary gland, nerves, nose, ovary, pancreas, prostate, skin, spleen, Not collected, Not specified, Not applicable, Other, Unknown",,,True,ManifestColumn,,ManifestColumn,sage.annotations-experimentalData.organ-0.0.4,biospecimen,False,STRING,http://purl.obolibrary.org/obo/NCIT_C13018, ,"Biospecimen_human,Biospecimen_non_Human"

modules/experimentalData/assay.csv

@@ -1,2 +1,2 @@
 Attribute,Description,Valid Values,DependsOn,DependsOn Component,Required,Parent,Validation Rules,Properties,Source,module,multivalue,columnType,Ontology,Notes,UsedIn
-assay,The analysis or technology used to generate the data in this file,"10x multiome, 16SrRNAseq, active avoidance learning behavior, anxiety-related behavior, ATACSeq, atomicForceMicroscopy, autoradiography, Baker Lipidomics, Biocrates Bile Acids, Biocrates p180, Biocrates Q500, bisulfiteSeq, Blood Chemistry Measurement, brightfieldMicroscopy, cellViabilityAssay, ChIPSeq, CITESeq, contextual conditioning behavior, CUT&Tag, DIA, DNA optical mapping, electrochemiluminescence, elevated plus maze test, elevated T maze apparatus method, ELISA, errBisulfiteSeq, exomeSeq, FIA-MSMS, FitBark, frailty assessment, Genotyping, HI-C, HiChIPseq, high content screen, HPLC, HPLC-MSMS, Immunocytochemistry, immunofluorescence, immunohistochemistry, in vivo bioluminescence, ISOSeq, jumpingLibrary, kinesthetic behavior, label free mass spectrometry, Laser Speckle Imaging, LC-MS, LC-MSMS, LC-SRM, Leiden Oxylipins, lentiMPRA, LFP, liquid chromatography-electrochemical detection, lncrnaSeq, locomotor activation behavior, long-read rnaSeq, LTP, MDMS-SL, memory behavior, Metabolon, methylationArray, MIB/MS, microRNAcounts, mirnaArray, mirnaSeq, MRI, mRNAcounts, MudPIT, m6A-rnaSeq, nextGenerationTargetedSequencing, Nightingale NMR, NOMe-Seq, novelty response behavior, open field test, oxBS-Seq, pharmacodynamics, pharmacokinetics, photograph, polymeraseChainReaction, Positron Emission Tomography, proximity extension assay, questionnaire, Rader Lipidomics, Real Time PCR, Ribo-Seq, rotarod performance test, rnaArray, rnaSeq, RPPA, sandwich ELISA, Sanger sequencing, scale, scATACSeq, scCGIseq, scirnaSeq, scrnaSeq, scwholeGenomeSeq, SiMoA, snpArray, snATACSeq, snrnaSeq, spontaneous alternation, STARRSeq, TMT quantitation, tractionForceMicroscopy, UPLC-MSMS, UPLC-ESI-QTOF-MS, UC Davis GCTOF, UCSD Untargeted Metabolomics, Vernier Caliper, von Frey test, westernBlot, wheel running, whole-cell patch clamp, wholeGenomeSeq, Wishart Catecholamines, Wishart High Value Metabolites, Zeno Electronic Walkway",,,True,ManifestColumn,,ManifestColumn,sage.annotations-experimentalData.assay-0.0.26,experimentalData,False,STRING,Sage Bionetworks, ,Biospecimen_non_Human
+assay,The analysis or technology used to generate the data in this file,"10x multiome, 16SrRNAseq, active avoidance learning behavior, anxiety-related behavior, ATACSeq, atomicForceMicroscopy, autoradiography, Baker Lipidomics, Biocrates Bile Acids, Biocrates p180, Biocrates Q500, bisulfiteSeq, Blood Chemistry Measurement, brightfieldMicroscopy, cellViabilityAssay, ChIPSeq, CITESeq, contextual conditioning behavior, CUT&Tag, DIA, DNA optical mapping, electrochemiluminescence, elevated plus maze test, elevated T maze apparatus method, ELISA, errBisulfiteSeq, exomeSeq, FIA-MSMS, FitBark, frailty assessment, Genotyping, HI-C, HiChIPseq, high content screen, HPLC, HPLC-MSMS, Immunocytochemistry, immunofluorescence, immunohistochemistry, in vivo bioluminescence, ISOSeq, jumpingLibrary, kinesthetic behavior, label free mass spectrometry, Laser Speckle Imaging, LC-MS, LC-MSMS, LC-SRM, Leiden Oxylipins, lentiMPRA, LFP, liquid chromatography-electrochemical detection, lncrnaSeq, locomotor activation behavior, long-read rnaSeq, LTP, MDMS-SL, memory behavior, Metabolon, methylationArray, MIB/MS, microRNAcounts, mirnaArray, mirnaSeq, MRI, mRNAcounts, MudPIT, m6A-rnaSeq, nextGenerationTargetedSequencing, Nightingale NMR, NOMe-Seq, novelty response behavior, open field test, oxBS-Seq, pharmacodynamics, pharmacokinetics, photograph, polymeraseChainReaction, Positron Emission Tomography, proximity extension assay, questionnaire, Rader Lipidomics, Real Time PCR, Ribo-Seq, rotarod performance test, rnaArray, rnaSeq, RPPA, sandwich ELISA, Sanger sequencing, scale, scATACSeq, scCGIseq, scirnaSeq, scrnaSeq, scwholeGenomeSeq, SiMoA, snpArray, snATACSeq, snrnaSeq, spontaneous alternation, STARRSeq, TMT quantitation, tractionForceMicroscopy, UPLC-MSMS, UPLC-ESI-QTOF-MS, UC Davis GCTOF, UCSD Untargeted Metabolomics, Vernier Caliper, von Frey test, westernBlot, wheel running, whole-cell patch clamp, wholeGenomeSeq, Wishart Catecholamines, Wishart High Value Metabolites, Zeno Electronic Walkway, Not collected, Not specified, Not applicable, Other, Unknown",,,True,ManifestColumn,,ManifestColumn,sage.annotations-experimentalData.assay-0.0.26,experimentalData,False,STRING,Sage Bionetworks, ,Biospecimen_non_Human

modules/individual/cohort.csv

@@ -1,2 +1,2 @@
 Attribute,Description,Valid Values,DependsOn,DependsOn Component,Required,Parent,Validation Rules,Properties,Source,module,multivalue,columnType,Ontology,Notes,UsedIn
-cohort,Name of the cohort the individual belongs to,"ABC-DS, ACT, ADNI, Banner, BEB-Miller, BLSA, CHDWB, CLINCOR, DiCAD, EHBS, Emory ADRC, Framingham, HBTRC, HPGP, HUP, LBP, MARS, MAYO, MC, MCJ, MCR, MSBB, NYBB, Pitt ADRC, RADC, ROSMAP, SMRI, UK Biobank, UPBB, UPenn, UW ADRC,ABC-DS,ACT,ADNI,Banner,BEB-Miller,Biggs Institute Brain Bank,BLSA,CHDWB,CLINCOR,Columbia ADRC,DiCAD,EFIGA,EHBS,Emory ADRC,FBS,Framingham,HBCC,HBTRC,HPGP,HUP,LBP,MARS,Mayo Clinic,MC,MCJ,MCR,MSBB,NYBB,Pitt ADRC,RADC,ROSMAP,SMRI,UFL,UK Biobank,UPBB,UPenn,UW ADRC,WHICAP",,,True,ManifestColumn,,ManifestColumn,"http://purl.obolibrary.org/obo/NCIT_C61512,http://purl.obolibrary.org/obo/STATO_0000203",individual,True,STRING,Sage Bionetworks, ,"Individual_Human,Individual_non_Human,Phenotype_human"
+cohort,Name of the cohort the individual belongs to,"ABC-DS, ACT, ADNI, Banner, BEB-Miller, BLSA, CHDWB, CLINCOR, DiCAD, EHBS, Emory ADRC, Framingham, HBTRC, HPGP, HUP, LBP, MARS, MAYO, MC, MCJ, MCR, MSBB, NYBB, Pitt ADRC, RADC, ROSMAP, SMRI, UK Biobank, UPBB, UPenn, UW ADRC,ABC-DS,ACT,ADNI,Banner,BEB-Miller,Biggs Institute Brain Bank,BLSA,CHDWB,CLINCOR,Columbia ADRC,DiCAD,EFIGA,EHBS,Emory ADRC,FBS,Framingham,HBCC,HBTRC,HPGP,HUP,LBP,MARS,Mayo Clinic,MC,MCJ,MCR,MSBB,NYBB,Pitt ADRC,RADC,ROSMAP,SMRI,UFL,UK Biobank,UPBB,UPenn,UW ADRC,WHICAP",,,False,ManifestColumn,,ManifestColumn,"http://purl.obolibrary.org/obo/NCIT_C61512,http://purl.obolibrary.org/obo/STATO_0000203",individual,True,STRING,Sage Bionetworks, ,"Individual_Human,Individual_non_Human,Phenotype_human"

modules/template/templates.csv

@@ -4,7 +4,7 @@ biospecimen_non_human_template,Template for Biospecimen_non_Human,,"Filename,ind
 assay_bsSeq_template,Template for bsSeq,,"Filename,specimenID,sampleType,measurementTechnique,technologyPlatformVersion,platformLocation,referenceTranscriptID,dnaBatchID,dnaBatchSize,dnaBatchSizeUnit,libraryBatchID,libraryPrep,libraryPreparationMethod,libraryVersion,sequencingBatchID,sequencingBatchSize,sequencingBatchSizeUnit,isStranded,readStrandOrigin,readLength,readLengthUnits,runType,totalReads,directionalBSseqLibrary,gDNAconc,lambdaDNAconc,pcrCycles,meanCoverage,conversionRatio,conversionRatioUnits,enrichmentMethod,Component",,False,Component,,template,Sage Bionetworks,template,,,Sage Bionetworks,,
 genotyping_human_template,Template for Genotyping_Human,,"Filename,specimenID,stage,numberIndividuals,sampleType,useReagent,reagentIDs,reagentName,reagentManufacturer,reagentCatalogNumber,reagentLotNumber,reagentWeblink,reagentContact,useTreatment,treatmentIDs,treatmentName,treatmentAmountValue,treatmentAmountUnit,treatmentDurationValue,treatmentDurationUnit,treatmentTemperatureValue,treatmentTemperatureUnit,measurementTechnique,genotypeTechnology,arrayManufacturer,arrayInformation,imputation,imputationPanel,imputationSoftware,variantCount,statisticalModel,adjustedCovariates,reportedTrait,backgroundTrait,useReadMeFile,readmeFile,useSummaryFile,summaryFile,summaryStatisticsAssembly,Component",,False,Component,,template,Sage Bionetworks,template,,,Sage Bionetworks,,
 individual_human_template,Template for Individual_Human,,"Filename,individualID,cohort,studyCode,fieldCenterCode,visitCode,countryCode,consentGroupID,speciesGroup,sex,race,ethnicity,ethnicGroupCode,age,diagnosisStatus,diagnosis,Component",,False,Component,,template,Sage Bionetworks,template,,,Sage Bionetworks,,
-individual_non_human_template,Template for Individual_non_Human,,"Filename,individualID,cohort,studyCode,taxon,speciesGroup,speciesName,commonName,sex,lifeStage,speciesAge,captivityStatus,captivityDuration,Component",,False,Component,,template,Sage Bionetworks,template,,,Sage Bionetworks,,
+individual_non_human_template,Template for Individual_non_Human,,"Filename,individualID,studyCode,taxon,speciesGroup,speciesName,commonName,sex,lifeStage,speciesAge,captivityStatus,captivityDuration,Component",,False,Component,,template,Sage Bionetworks,template,,,Sage Bionetworks,,
 assay_metabolomics_template,Template for Metabolomics,,"Filename,specimenID,sampleType,measurementTechnique,technologyPlatformVersion,platformLocation,instrumentModel,samplePrepProtocol,dataFile,databaseSource,databaseWeblink,databaseName,msTarget,msAssayTechnique,msAnalyteType,sampleIntroduction,extractionMethod,acquisitionMode,acquisitionSoftware,acquistionSoftwareVersion,spectrometerFrequency,hasIonizationSource,ionProperty,vacuumPressure,vacuumPressureUnit,lensVoltages,lensVoltagesUnit,gasFlowTemperature,gasFlowTemperatureUnit,hasAssayControl,controlType,experimentalBatchSize,experimentalBatchSizeUnit,batchSize,batchSizeUnit,batchID,batchLabel,sampleBatchID,sampleBatchSize,sampleBatchSizeUnit,acquisitionBatchID,acquisitionBatchSize,acquisitionBatchSizeUnit,processingBatchID,processingBatchSize,processingBatchSizeUnit,Component",,False,Component,,template,Sage Bionetworks,template,,,Sage Bionetworks,,
 assay_metagenomics_template,Template for Metagenomics,,"Filename,specimenID,sampleType,experimentType,measurementTechnique,technologyPlatformVersion,platformLocation,instrumentModel,samplePrepProtocol,dataFile,databaseSource,databaseWeblink,databaseName,databaseVersion,databaseLibrary,libraryPrep,libSize,mid,assemblyName,nucleAcidExt,sourceMatID,libReadsSeqd,numberContig,libLayout,assemblyQual,libVector,adapters,assemblySoftware,experimentalFactor,associatedResource,sop,targetGene,seqMethod,seqPlatform,DNAextraction,Component",,False,Component,,template,Sage Bionetworks,template,,,Sage Bionetworks,,
 assay_microbiome_template,Template for Microbiome,,"Filename,specimenID,sampleType,measurementTechnique,technologyPlatformVersion,platformLocation,extractionMethod,libraryBatchID,sequencingBatchID,sequencingBatchSize,sequencingBatchSizeUnit,dnaBatchID,dnaBatchSize,dnaBatchSizeUnit,libraryPrep,libraryPreparationMethod,libraryVersion,isStranded,readStrandOrigin,readLength,readLengthUnits,runType,totalReads,Component",,False,Component,,template,Sage Bionetworks,template,,,Sage Bionetworks,,