Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lastModified field related to time artifact pulled from Proxy. #27

Open
verticalambition opened this issue Apr 3, 2024 · 5 comments · May be fixed by #28
Open

lastModified field related to time artifact pulled from Proxy. #27

verticalambition opened this issue Apr 3, 2024 · 5 comments · May be fixed by #28

Comments

@verticalambition
Copy link

verticalambition commented Apr 3, 2024

  • What are you trying to do?
    Trying to copy select artifacts from Nexus, using Terraform Format repo, based on last modified time of the artifact, which could be equivalent to when the artifact was pulled from proxy.

  • What feature or behavior is this required for?
    We have a situation where we have a Nexus repo in an offline environment. We want to be able to pull down artifacts that have been modified (pulled) within the last 48 hours in our primary environment and transfer them through offline process to our other system.

For general repositories in Nexus, when we query through the API:
{nexusUrl}/service/rest/v1/components?repository={repoName}"

we get a response (truncated but for example)

      "repository": "terraform",
      "format": "raw",
      "group": "/registry.terraform.io/hashicorp/kubernetes",
      "name": "registry.terraform.io/hashicorp/kubernetes/2.26.0.json",
      "version": null,
      "assets": [
        {
          "downloadUrl": "https://nexus/terraform/registry.terraform.io/hashicorp/kubernetes/2.26.0.json",
          "path": "registry.terraform.io/hashicorp/kubernetes/2.26.0.json",
          "id": "dGVycmFmb3JtOjgwNDgyZjkyY2VmMDI5ODFiOGFlM2I2ZTVhMmZkZDNm",
          "repository": "terraform",
          "format": "raw",
          "checksum": {
            "sha1": "0b27edd55714bc401718441171c4fecab1ee4431",
            "md5": "4d2a81bed96d1acea4a2f8a0ad117a0b"
          },
          "contentType": "application/json",
          "lastModified": "2024-03-12T19:07:26.770+00:00",
          "lastDownloaded": "2024-03-15T07:01:16.770+00:00",
          "uploader": "user",
          "uploaderIp": "10.42.62.192",
          "fileSize": 203,
          "blobCreated": "2024-03-12T19:07:26.770+00:00"
        }

When we query the Terraform formatted repository the same way, we receive a response like

{
      "id": "dGVycmFmb3JtLXJlZ2lzdHJ5OmMwYWMyYWI2YzVlOTNhNGFjMWZhYTQyNjJjNGYzNjNi",
      "repository": "terraform-registry",
      "format": "terraform",
      "group": null,
      "name": "/v1/providers/registry.terraform.io/hashicorp/aws/5.19.0.json",
      "version": null,
      "assets": []
    }

It doesn't appear to log a modified time along with other metrics like downloadUrl, etc. We would like to have a field of lastModified that is related to when the image was pulled from the proxy repo.

If this seems like a reasonable idea, wondering where in the code we would work on trying to implement this functionality or if anyone has ideas, etc.

  • How could we solve this issue? (Not knowing is okay!)
    Have the lastModified be the time in which the artifact was pulled from the proxy and able to be access through API/curl
  • Anything else?
    Thank you.
    cc @DarthHater @bhamail
@davejab
Copy link
Owner

davejab commented Apr 3, 2024

@verticalambition been a while since i have looked at this code base, I'll need to familiarise myself with it again and see how this could be done. If you are feeling brave you could have a look yourself, definitely start in here:

https://github.com/davejab/nexus-repository-terraform/blob/main/nexus-repository-terraform/src/main/java/org/sonatype/nexus/plugins/terraform/orient/internal/TerraformProxyFacetImpl.java

Also worth checking source for other plugins that are doing what you are looking for and seeing what it is that is missing.

@verticalambition
Copy link
Author

verticalambition commented Apr 3, 2024 via email

@davejab
Copy link
Owner

davejab commented Apr 7, 2024

Have experimented a little with this, I can achieve it by creating a component linked asset with asset = tx.createAsset(bucket, component);, as is conveniently included under a @todo here:

// @todo
// if (assetKind.equals(AssetKind.PROVIDER_ARCHIVE)) {
// if (asset == null) {
// asset = tx.createAsset(bucket, component);
// asset.name(assetPath);
// asset.formatAttributes().set(P_ASSET_KIND, assetKind.name());
// }
// } else {
if (asset == null) {
asset = tx.createAsset(bucket, getRepository().getFormat());
asset.name(assetPath);
for (String key : matcherState.getTokens().keySet()) {
asset.formatAttributes().set(key, matcherState.getTokens().get(key));
}
}
// }

This does have some unintended side affects, which I have yet to fix

@verticalambition
Copy link
Author

Awesome, you go. Kind of a noob question for you but going to ask it anyway. I had cloned the project and tried to mess with it myself. I swear I'm a SW Eng with 7 years of experience but its been about 5 since I've coded a project with Java. I tried opening it with VSCode and Eclipse and I could not for the life of me get the project to not complain and smear my editor with red. Dumb stuff I think related to my compiler settings or whatnot. I'll be honest I only spent about 3 hours on it before work called me to a different task.

May have something to do with us using a flavor of Java 17 and was trying to point it to a JDK 8 I downloaded since this project requires it.

Do you by chance have any advice on what editor you use or configuration so this project is happy in an editor? I wanted it good so I could access the Sonatype libraries and see parameters.

Regardless, thanks for giving it a go. I appreciate the effort.

@davejab
Copy link
Owner

davejab commented Apr 7, 2024

I have had the most success using IntelliJ for Java projects

@davejab davejab linked a pull request Apr 8, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants