Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add metadata attributes to gen3manifest #741

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Add metadata attributes to gen3manifest #741

wants to merge 1 commit into from

Conversation

adamjtaylor
Copy link
Contributor

This PR adds the same metadata fields as the SB-CGC manifest to enable joining with other information.

@vercel Vercel
Copy link

vercel bot commented Jan 7, 2025 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
htan-portal-nextjs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jan 7, 2025 0:51am

@adamjtaylor adamjtaylor linked an issue Jan 7, 2025 that may be closed by this pull request
Add metadata attributes to gen3 manifest #740
Open
@adamjtaylor
Copy link
Contributor Author

Tested in the preview and this seems to be working. Manifest contains useful extra keys and the gen3 command still works as expected

(gen3) ataylor@ajt-mbp Downloads % cat gen3_extra_keys.json
[
  {
    "object_id": "dg.4DFC/9c6f52d6-c9b2-11ed-b0f4-130f4a9208be",
    "name": "HTAPP-264-SMP-662_WES_normal.bam",
    "atlas_name": "HTAN HTAPP",
    "assay_name": "Bulk DNA",
    "level": "Level 2",
    "data_file_id": "HTA1_264_000150101",
    "parent_biospecimen_id": "HTA1_264_0001501",
    "parent_data_file_id": "Not Applicable - L1 file not provided"
  }
]%                                                                                                 (gen3) ataylor@ajt-mbp Downloads % gen3 \
    --endpoint=nci-crdc.datacommons.io \
    drs-pull \
    manifest gen3_extra_keys.json .
[2025-01-07 12:56:21,988][WARNING] Unable to process WTS response. Likely no WTS service running on this commons. Certain commands might fail.
Resolving objects: 100%|█████████████████████████████████████████████| 1/1 [00:00<00:00,  2.54it/s]
HTAPP-264-SMP-662_WES_normal.bam             :   0%|                                   | 4.60M/10.8^C

@adamjtaylor adamjtaylor requested a review from inodb January 7, 2025 12:57
inodb
inodb reviewed Jan 7, 2025
View reviewed changes
packages/data-portal-explore/src/components/FileTable.tsx
Comment on lines +101 to +108
name: f.viewers?.cds?.name,
atlas_name: f.atlas_name,
assay_name: f.assayName,
level: f.level,
// make sure to replace all possible commas since we are generating a CSV file
data_file_id: f.DataFileID?.replace(/,/g, ' ').trim(),
parent_biospecimen_id: f.ParentBiospecimenID?.replace(/,/g, ' ').trim(),
parent_data_file_id: f.ParentDataFileID?.replace(/,/g, ' ').trim(),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be better to use some JSON to CSV conversion package (@onursumer @alisman do u have a rec?)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://www.npmjs.com/package/json-2-csv looks promising, but I guess we can refactor this later since we're already doing something similar in the generateCdsManifestFile function.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@onursumer onursumer onursumer left review comments

@inodb inodb inodb left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add metadata attributes to gen3 manifest
3 participants
@adamjtaylor @inodb @onursumer