Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Npm/config #653

Merged
merged 65 commits into from
Feb 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
7fb4a50
the go way to 'publish' a NPM package
hilmarf Jan 24, 2024
7d32ed1
npmjs further drafting
hilmarf Jan 29, 2024
6d862c8
init npmjs
hilmarf Jan 30, 2024
b5b2873
BlobHandler, Identity, RegistrationHandler for npmjs
hilmarf Jan 30, 2024
1b7cd72
publish
hilmarf Jan 31, 2024
bcfec0e
config test
hilmarf Jan 31, 2024
7fc42b2
some cleanup and test
hilmarf Jan 31, 2024
16f766c
fix some lint issues
hilmarf Jan 31, 2024
921c337
Merge branch 'main' into main
hilmarf Jan 31, 2024
b93d2d5
use Logger instead
hilmarf Feb 5, 2024
14908b5
//nolint:gosec
hilmarf Feb 5, 2024
b82ec71
use errors.Is(...
hilmarf Feb 5, 2024
ad34a99
nolint:gosec
hilmarf Feb 5, 2024
fc34e07
clean Package struct
hilmarf Feb 5, 2024
a4f84d1
Update publish.go
hilmarf Feb 5, 2024
8177ccc
gofumpt
hilmarf Feb 6, 2024
fe552bb
Merge remote-tracking branch 'hilmarf/main'
hilmarf Feb 6, 2024
581d6d6
don't re-upload same artifacts
hilmarf Feb 6, 2024
19055b5
return error
hilmarf Feb 6, 2024
9eaf836
logging
hilmarf Feb 7, 2024
16167c2
npmjs -> NPM
hilmarf Feb 7, 2024
2d846cb
description
hilmarf Feb 7, 2024
5342d8d
description
hilmarf Feb 7, 2024
c3dadad
rename package
hilmarf Feb 7, 2024
4b9c7db
unused parameters
hilmarf Feb 7, 2024
aca0895
doc has to end with .
hilmarf Feb 7, 2024
c151abb
check if URL is configured
hilmarf Feb 7, 2024
b41d870
REALM
hilmarf Feb 7, 2024
a224c36
npmjs.com
hilmarf Feb 7, 2024
3faf576
Merge branch 'open-component-model:main' into main
hilmarf Feb 9, 2024
74b19d6
Merge branch 'main' of github.com:open-component-model/ocm
hilmarf Feb 13, 2024
97503f8
draft
hilmarf Feb 13, 2024
b86eab1
revert
hilmarf Feb 14, 2024
a104900
identity
hilmarf Feb 14, 2024
61b4e90
gone
hilmarf Feb 14, 2024
35481c2
npm credential repository
hilmarf Feb 14, 2024
8c511a2
Merge branch 'main' of github.com:open-component-model/ocm into npm/c…
hilmarf Feb 14, 2024
d1ec79e
fix lint and test issues
hilmarf Feb 14, 2024
4626c2f
obsolete... useless leftover from copy&paste
hilmarf Feb 16, 2024
3982246
obsolete... useless leftover from copy&paste
hilmarf Feb 16, 2024
a804a5f
check for existing token and reuse it, instead of login
hilmarf Feb 16, 2024
4c8e2f9
moved logging REALM
hilmarf Feb 16, 2024
9eacd28
npm.CONSUMER_TYPE
hilmarf Feb 16, 2024
23ecab9
init
hilmarf Feb 16, 2024
dcfb880
Merge branch 'open-component-model:main' into main
hilmarf Feb 16, 2024
0702046
fix import cycle in test
hilmarf Feb 16, 2024
1a5644a
Merge branch 'main' of github.com:open-component-model/ocm into npm/c…
hilmarf Feb 16, 2024
85a98b1
Merge branch 'main' of github.com:hilmarf/ocm into npm/config
hilmarf Feb 16, 2024
a9477ab
Merge branch 'main' into npm/config
hilmarf Feb 16, 2024
b6058b0
fix cred names + harmonize propagation handling
mandelsoft Feb 16, 2024
e6935fb
Merge branch 'npm/config' of github.com:open-component-model/ocm into…
hilmarf Feb 19, 2024
8ff9037
Merge branch 'main' into npm/config
hilmarf Feb 19, 2024
300ccaf
improve description handling
mandelsoft Feb 16, 2024
3f468d1
generate cli docu (includes npm upload)
mandelsoft Feb 17, 2024
ac88604
default config location + handle default config
mandelsoft Feb 17, 2024
688831c
Merge branch 'npm/config' of github.com:open-component-model/ocm into…
hilmarf Feb 19, 2024
9f6eb3b
fix '/' desaster
hilmarf Feb 19, 2024
f7c262e
more tests for RepositorySpec
hilmarf Feb 19, 2024
803a49b
replace os.Getenv("HOME") with os.UserHomeDir()
hilmarf Feb 20, 2024
82ad97f
use utils.ResolvePath() in readNpmConfigFile()
hilmarf Feb 20, 2024
d74999c
Update configure.go
hilmarf Feb 20, 2024
afd8455
Merge branch 'main' into npm/config
hilmarf Feb 20, 2024
8565e58
Merge branch 'main' into npm/config
hilmarf Feb 22, 2024
83658b2
ignore err
hilmarf Feb 22, 2024
90f0425
ignore err
hilmarf Feb 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion docs/reference/ocm.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,9 +50,12 @@ location a default configuration is composed according to known type specific
configuration files.

The following configuration sources are used:
- The docker configuration file at <code>~/.docker/config.jaon</code> is
- The docker configuration file at <code>~/.docker/config.json</code> is
read to feed in the configured credentials for OCI registries.

- The npm configuration file at <code>~/.npmrc</code> is
read to feed in the configured credentials for NPM registries.



With the option <code>--cred</code> it is possible to specify arbitrary credentials
Expand Down
28 changes: 28 additions & 0 deletions docs/reference/ocm_credential-handling.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,19 @@ The following credential consumer types are used/supported:
- <code>certificateAuthority</code>: the certificate authority certificate used to verify certificates


- <code>Registry.npmjs.com</code>: NPM repository

It matches the <code>Registry.npmjs.com</code> consumer type and additionally acts like
the <code>hostpath</code> type.

Credential consumers of the consumer type Registry.npmjs.com evaluate the following credential properties:

- <code>username</code>: the basic auth user name
- <code>password</code>: the basic auth password
- <code>email</code>: NPM registry, require an email address
- <code>token</code>: the token attribute. May exist after login at any npm registry. Check your .npmrc file!


- <code>S3</code>: S3 credential matcher

This matcher is a hostpath matcher.
Expand Down Expand Up @@ -306,6 +319,21 @@ behaviours are described in the following list:
is read.


- Credential provider <code>NPMConfig</code>

This repository type can be used to access credentials stored in a file
following the NPM npmrc format (~/.npmrc). It take into account the
credentials helper section, also. If enabled, the described
credentials will be automatically assigned to appropriate consumer ids.

The following versions are supported:
- Version <code>v1</code>

The repository specification supports the following fields:
- <code>npmrcFile</code>: *string*: the file path to a NPM npmrc file
- <code>propagateConsumerIdentity</code>: *bool*(optional): enable consumer id propagation



### SEE ALSO

Expand Down
13 changes: 13 additions & 0 deletions docs/reference/ocm_get_credentials.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,19 @@ Matchers exist for the following usage contexts or consumer types:
- <code>certificateAuthority</code>: the certificate authority certificate used to verify certificates


- <code>Registry.npmjs.com</code>: NPM repository

It matches the <code>Registry.npmjs.com</code> consumer type and additionally acts like
the <code>hostpath</code> type.

Credential consumers of the consumer type Registry.npmjs.com evaluate the following credential properties:

- <code>username</code>: the basic auth user name
- <code>password</code>: the basic auth password
- <code>email</code>: NPM registry, require an email address
- <code>token</code>: the token attribute. May exist after login at any npm registry. Check your .npmrc file!


- <code>S3</code>: S3 credential matcher

This matcher is a hostpath matcher.
Expand Down
1 change: 1 addition & 0 deletions docs/reference/ocm_logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ The following *tags* are used by the command line tool:

The following *realms* are used by the command line tool:
- <code>ocm</code>: general realm used for the ocm go library.
- <code>ocm/NPM</code>: NPM registry
- <code>ocm/accessmethod/ociartifact</code>: access method ociArtifact
- <code>ocm/compdesc</code>: component descriptor handling
- <code>ocm/config</code>: configuration management
Expand Down
17 changes: 13 additions & 4 deletions docs/reference/ocm_ocm-uploadhandlers.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,6 @@ resource blob), it is possible to pass a target configuration controlling the
exact behaviour of the handler for selected artifacts.

The following handler names are possible:
- <code>plugin</code>: [downloaders provided by plugins]

sub namespace of the form <code>&lt;plugin name>/&lt;handler></code>

- <code>ocm/ociArtifacts</code>: downloading OCI artifacts

The <code>ociArtifacts</code> downloader is able to download OCI artifacts
Expand All @@ -64,6 +60,19 @@ The following handler names are possible:
Alternatively, a single string value can be given representing an OCI repository
reference.

- <code>plugin</code>: [downloaders provided by plugins]

sub namespace of the form <code>&lt;plugin name>/&lt;handler></code>

- <code>ocm/npmPackage</code>: uploading npm artifacts

The <code>ocm/npmPackage</code> uploader is able to upload npm artifacts
as artifact archive according to the npm package spec.
If registered the default mime type is: application/x-tgz

It accepts a plain string for the URL or a config with the following field:
'url': the URL of the npm repository.



See [ocm ocm-uploadhandlers](ocm_ocm-uploadhandlers.md) for further details on using
Expand Down
17 changes: 13 additions & 4 deletions docs/reference/ocm_transfer_commontransportarchive.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,10 +110,6 @@ are configured for the operation. It has the following format
</center>

The uploader name may be a path expression with the following possibilities:
- <code>plugin</code>: [downloaders provided by plugins]

sub namespace of the form <code>&lt;plugin name>/&lt;handler></code>

- <code>ocm/ociArtifacts</code>: downloading OCI artifacts

The <code>ociArtifacts</code> downloader is able to download OCI artifacts
Expand All @@ -138,6 +134,19 @@ The uploader name may be a path expression with the following possibilities:
Alternatively, a single string value can be given representing an OCI repository
reference.

- <code>plugin</code>: [downloaders provided by plugins]

sub namespace of the form <code>&lt;plugin name>/&lt;handler></code>

- <code>ocm/npmPackage</code>: uploading npm artifacts

The <code>ocm/npmPackage</code> uploader is able to upload npm artifacts
as artifact archive according to the npm package spec.
If registered the default mime type is: application/x-tgz

It accepts a plain string for the URL or a config with the following field:
'url': the URL of the npm repository.



See [ocm ocm-uploadhandlers](ocm_ocm-uploadhandlers.md) for further details on using
Expand Down
17 changes: 13 additions & 4 deletions docs/reference/ocm_transfer_componentversions.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,10 +167,6 @@ are configured for the operation. It has the following format
</center>

The uploader name may be a path expression with the following possibilities:
- <code>plugin</code>: [downloaders provided by plugins]

sub namespace of the form <code>&lt;plugin name>/&lt;handler></code>

- <code>ocm/ociArtifacts</code>: downloading OCI artifacts

The <code>ociArtifacts</code> downloader is able to download OCI artifacts
Expand All @@ -195,6 +191,19 @@ The uploader name may be a path expression with the following possibilities:
Alternatively, a single string value can be given representing an OCI repository
reference.

- <code>plugin</code>: [downloaders provided by plugins]

sub namespace of the form <code>&lt;plugin name>/&lt;handler></code>

- <code>ocm/npmPackage</code>: uploading npm artifacts

The <code>ocm/npmPackage</code> uploader is able to upload npm artifacts
as artifact archive according to the npm package spec.
If registered the default mime type is: application/x-tgz

It accepts a plain string for the URL or a config with the following field:
'url': the URL of the npm repository.



See [ocm ocm-uploadhandlers](ocm_ocm-uploadhandlers.md) for further details on using
Expand Down
6 changes: 1 addition & 5 deletions pkg/contexts/config/configutils/configure.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
// SPDX-FileCopyrightText: 2022 SAP SE or an SAP affiliate company and Open Component Model contributors.
//
// SPDX-License-Identifier: Apache-2.0

package configutils

import (
Expand All @@ -27,7 +23,7 @@ func Configure(path string) error {
func ConfigureContext(ctxp config.ContextProvider, path string) error {
ctx := config.FromProvider(ctxp)

h := os.Getenv("HOME")
h, _ := os.UserHomeDir()
if path == "" {
if h != "" {
cfg := h + "/.ocmconfig"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package npm
package identity

import (
"path"
Expand All @@ -9,13 +9,32 @@ import (
"github.com/open-component-model/ocm/pkg/contexts/credentials/cpi"
"github.com/open-component-model/ocm/pkg/contexts/credentials/identity/hostpath"
"github.com/open-component-model/ocm/pkg/listformat"
"github.com/open-component-model/ocm/pkg/logging"
)

const (
// CONSUMER_TYPE is the npm repository type.
CONSUMER_TYPE = "Registry.npmjs.com"

// ATTR_USERNAME is the username attribute. Required for login at any npm registry.
ATTR_USERNAME = cpi.ATTR_USERNAME
// ATTR_PASSWORD is the password attribute. Required for login at any npm registry.
ATTR_PASSWORD = cpi.ATTR_PASSWORD
// ATTR_EMAIL is the email attribute. Required for login at any npm registry.
ATTR_EMAIL = cpi.ATTR_EMAIL
// ATTR_TOKEN is the token attribute. May exist after login at any npm registry.
ATTR_TOKEN = cpi.ATTR_TOKEN
)

// Logging Realm.
var REALM = logging.DefineSubRealm("NPM registry", "NPM")

func init() {
attrs := listformat.FormatListElements("", listformat.StringElementDescriptionList{
ATTR_USERNAME, "the basic auth user name",
ATTR_PASSWORD, "the basic auth password",
ATTR_EMAIL, "NPM registry, require an email address",
ATTR_TOKEN, "the token attribute. May exist after login at any npm registry. Check your .npmrc file!",
})

cpi.RegisterStandardIdentity(CONSUMER_TYPE, hostpath.IdentityMatcher(CONSUMER_TYPE), `NPM repository
Expand Down
20 changes: 11 additions & 9 deletions pkg/contexts/credentials/repositories/dockerconfig/type.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ import (
"fmt"

"github.com/open-component-model/ocm/pkg/contexts/credentials/cpi"
"github.com/open-component-model/ocm/pkg/generics"
"github.com/open-component-model/ocm/pkg/runtime"
"github.com/open-component-model/ocm/pkg/utils"
)

const (
Expand All @@ -27,19 +29,19 @@ type RepositorySpec struct {
runtime.ObjectVersionedType `json:",inline"`
DockerConfigFile string `json:"dockerConfigFile,omitempty"`
DockerConfig json.RawMessage `json:"dockerConfig,omitempty"`
PropgateConsumerIdentity bool `json:"propagateConsumerIdentity,omitempty"`
PropgateConsumerIdentity *bool `json:"propagateConsumerIdentity,omitempty"`
}

func (s RepositorySpec) WithConsumerPropagation(propagate bool) *RepositorySpec {
s.PropgateConsumerIdentity = propagate
s.PropgateConsumerIdentity = &propagate
return &s
}

// NewRepositorySpec creates a new memory RepositorySpec.
func NewRepositorySpec(path string, prop ...bool) *RepositorySpec {
p := false
for _, e := range prop {
p = p || e
var p *bool
if len(prop) > 0 {
p = generics.Pointer(utils.Optional(prop...))
}
if path == "" {
path = "~/.docker/config.json"
Expand All @@ -52,9 +54,9 @@ func NewRepositorySpec(path string, prop ...bool) *RepositorySpec {
}

func NewRepositorySpecForConfig(data []byte, prop ...bool) *RepositorySpec {
p := false
for _, e := range prop {
p = p || e
var p *bool
if len(prop) > 0 {
p = generics.Pointer(utils.Optional(prop...))
}
return &RepositorySpec{
ObjectVersionedType: runtime.NewVersionedTypedObject(Type),
Expand All @@ -73,5 +75,5 @@ func (a *RepositorySpec) Repository(ctx cpi.Context, creds cpi.Credentials) (cpi
if !ok {
return nil, fmt.Errorf("failed to assert type %T to Repositories", r)
}
return repos.GetRepository(ctx, a.DockerConfigFile, a.DockerConfig, a.PropgateConsumerIdentity)
return repos.GetRepository(ctx, a.DockerConfigFile, a.DockerConfig, utils.AsBool(a.PropgateConsumerIdentity, true))
}
10 changes: 6 additions & 4 deletions pkg/contexts/credentials/repositories/gardenerconfig/type.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,9 @@ import (
"github.com/open-component-model/ocm/pkg/contexts/credentials/internal"
gardenercfgcpi "github.com/open-component-model/ocm/pkg/contexts/credentials/repositories/gardenerconfig/cpi"
"github.com/open-component-model/ocm/pkg/contexts/credentials/repositories/gardenerconfig/identity"
"github.com/open-component-model/ocm/pkg/generics"
"github.com/open-component-model/ocm/pkg/runtime"
"github.com/open-component-model/ocm/pkg/utils"
)

const (
Expand All @@ -30,19 +32,19 @@ type RepositorySpec struct {
URL string `json:"url"`
ConfigType gardenercfgcpi.ConfigType `json:"configType"`
Cipher Cipher `json:"cipher"`
PropagateConsumerIdentity bool `json:"propagateConsumerIdentity"`
PropagateConsumerIdentity *bool `json:"propagateConsumerIdentity,omitempty"`
}

var _ cpi.ConsumerIdentityProvider = (*RepositorySpec)(nil)

// NewRepositorySpec creates a new memory RepositorySpec.
func NewRepositorySpec(url string, configType gardenercfgcpi.ConfigType, cipher Cipher, propagateConsumerIdentity bool) *RepositorySpec {
func NewRepositorySpec(url string, configType gardenercfgcpi.ConfigType, cipher Cipher, propagateConsumerIdentity ...bool) *RepositorySpec {
return &RepositorySpec{
ObjectVersionedType: runtime.NewVersionedTypedObject(Type),
URL: url,
ConfigType: configType,
Cipher: cipher,
PropagateConsumerIdentity: propagateConsumerIdentity,
PropagateConsumerIdentity: generics.Pointer(utils.OptionalDefaultedBool(true, propagateConsumerIdentity...)),
}
}

Expand All @@ -62,7 +64,7 @@ func (a *RepositorySpec) Repository(ctx cpi.Context, creds cpi.Credentials) (cpi
return nil, fmt.Errorf("unable to get key from context: %w", err)
}

return repos.GetRepository(ctx, a.URL, a.ConfigType, a.Cipher, key, a.PropagateConsumerIdentity)
return repos.GetRepository(ctx, a.URL, a.ConfigType, a.Cipher, key, utils.AsBool(a.PropagateConsumerIdentity, true))
}

func (a *RepositorySpec) GetConsumerId(uctx ...internal.UsageContext) internal.ConsumerIdentity {
Expand Down
5 changes: 1 addition & 4 deletions pkg/contexts/credentials/repositories/init.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
// SPDX-FileCopyrightText: 2022 SAP SE or an SAP affiliate company and Open Component Model contributors.
//
// SPDX-License-Identifier: Apache-2.0

package repositories

import (
Expand All @@ -11,5 +7,6 @@ import (
_ "github.com/open-component-model/ocm/pkg/contexts/credentials/repositories/gardenerconfig"
_ "github.com/open-component-model/ocm/pkg/contexts/credentials/repositories/memory"
_ "github.com/open-component-model/ocm/pkg/contexts/credentials/repositories/memory/config"
_ "github.com/open-component-model/ocm/pkg/contexts/credentials/repositories/npm"
_ "github.com/open-component-model/ocm/pkg/contexts/credentials/repositories/vault"
)
18 changes: 18 additions & 0 deletions pkg/contexts/credentials/repositories/npm/a_usage.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
package npm

import (
"github.com/open-component-model/ocm/pkg/listformat"
)

var usage = `
This repository type can be used to access credentials stored in a file
following the NPM npmrc format (~/.npmrc). It take into account the
credentials helper section, also. If enabled, the described
credentials will be automatically assigned to appropriate consumer ids.
`

var format = `The repository specification supports the following fields:
` + listformat.FormatListElements("", listformat.StringElementDescriptionList{
"npmrcFile", "*string*: the file path to a NPM npmrc file",
"propagateConsumerIdentity", "*bool*(optional): enable consumer id propagation",
})
33 changes: 33 additions & 0 deletions pkg/contexts/credentials/repositories/npm/cache.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
package npm

import (
"github.com/open-component-model/ocm/pkg/contexts/credentials/cpi"
"github.com/open-component-model/ocm/pkg/contexts/datacontext"
)

type Cache struct {
repos map[string]*Repository
}

func createCache(_ datacontext.Context) interface{} {
return &Cache{
repos: map[string]*Repository{},
}
}

func (r *Cache) GetRepository(ctx cpi.Context, name string, prop bool) (*Repository, error) {
var (
err error = nil
repo *Repository
)
if name != "" {
repo = r.repos[name]
}
if repo == nil {
repo, err = NewRepository(ctx, name, prop)
if err == nil {
r.repos[name] = repo
}
}
return repo, err
}
Loading
Loading