-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
In 1101 base workflow #21
Changes from all commits
21414da
769d6da
ead9f88
430f3d2
dc07a15
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,10 @@ | ||
class InvalidDSpaceMetadataError(Exception): | ||
pass | ||
|
||
|
||
class InvalidSQSMessageError(Exception): | ||
pass | ||
|
||
|
||
class ItemMetadatMissingRequiredFieldError(Exception): | ||
pass |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
import json | ||
import logging | ||
from dataclasses import dataclass | ||
from typing import Any | ||
|
||
from dsc.utilities.aws.s3 import S3Client | ||
|
||
logger = logging.getLogger(__name__) | ||
|
||
|
||
@dataclass | ||
class ItemSubmission: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do you think There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I did have a There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Side note: If we find that |
||
"""A class to store the required values for a DSpace submission.""" | ||
|
||
dspace_metadata: dict[str, Any] | ||
bitstream_uris: list[str] | ||
metadata_s3_key: str | ||
metadata_uri: str = "" | ||
Comment on lines
+17
to
+18
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Minor, but maybe worth considering renaming There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Updating to |
||
|
||
def upload_dspace_metadata(self, bucket: str) -> None: | ||
"""Upload DSpace metadata to S3 using the specified bucket and keyname. | ||
|
||
Args: | ||
bucket: The S3 bucket for uploading the item metadata file. | ||
""" | ||
s3_client = S3Client() | ||
s3_client.put_file(json.dumps(self.dspace_metadata), bucket, self.metadata_s3_key) | ||
metadata_uri = f"s3://{bucket}/{self.metadata_s3_key}" | ||
logger.info(f"Metadata uploaded to S3: {metadata_uri}") | ||
self.metadata_uri = metadata_uri |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
"""dsc.workflows. | ||
|
||
All primary functions used by CLI are importable from here. | ||
""" | ||
|
||
from dsc.workflows.base import BaseWorkflow | ||
|
||
__all__ = [ | ||
"BaseWorkflow", | ||
] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Taking some of the other file reorganizing and renaming into consideration -- and thanks BTW, it's feeling good to navigate around! -- I could envision this file called something more high level like
items.py
. If other classes ever made sense to add specific to items, it would be a natural place for it.I don't have evidence or even a pointable philosophy to support it, but when the classname mirrors the filename, it feels a bit off. While we may not need more "item" type classes, it's tight coupling between file and class names.
Totally optional.