Sync All Secrets to K8s

[manedurphy]

Sync All Secrets to K8s

Addresses #529

Purpose

• The purpose for this feature is to give users an easy way to sync all of the secrets listed in the parameters of the SecretProviderClass to K8s.
• This means that the user no longer has to manually define a SecretObject for each secret that they want to sync. Instead, they just have to declare that all secrets that are mounted to the filesystem of the pod should be synced to K8s, as well as the K8s secret type that should be used for each secret listed in the parameters.

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: e2e-provider-sync-all
spec:
  provider: e2e-provider
  parameters:
    objects: |
      array:
        - |
          objectName: foo
          objectVersion: v1
        - |
          objectName: fookey
          objectVersion: v1
  syncOptions:
    type: Opaque
    syncAll: true

Implementation

Type Definitions

• The SyncOptions type has been defined and added as a field for the SecretProviderClass.
• This means that there will be an update to the Helm chart.

Type Definitions

type SyncOptions struct {
	// syncs all secrets listed in the parameters field of SecretProviderClass
	SyncAll bool `json:"syncAll,omitempty"`
	// type of K8s secret object
	Type string `json:"type,omitempty"`
}

// SecretProviderClassSpec defines the desired state of SecretProviderClass
type SecretProviderClassSpec struct {
	// Configuration for provider name
	Provider Provider `json:"provider,omitempty"`
	// Configuration for specific provider
	Parameters    map[string]string `json:"parameters,omitempty"`
	SecretObjects []*SecretObject   `json:"secretObjects,omitempty"`
	SyncOptions   SyncOptions       `json:"syncOptions,omitempty"`
}

Patcher

• Since Patcher loops through the SecretObjects that are defined in the SecretProviderClass, we need to build them up when syncOptions.syncAll is set to true since the user is not expected to manually create secrets objects in the configuration.

Patcher

if spc.Spec.SyncOptions.SyncAll {
    files, err := fileutil.GetMountedFiles(spcPodStatus.Status.TargetPath)
    if err != nil {
        return fmt.Errorf("failed to get mounted files for pod %s/%s: %v", namespace, pod.Name, err)
    } else {
        if len(spc.Spec.SecretObjects) == 0 {
            spc.Spec.SecretObjects = spcutil.BuildSecretObjects(files, secretutil.GetSecretType(strings.TrimSpace(spc.Spec.SyncOptions.Type)))
        } else {
            spc.Spec.SecretObjects = append(spc.Spec.SecretObjects, spcutil.BuildSecretObjects(files, secretutil.GetSecretType(strings.TrimSpace(spc.Spec.SyncOptions.Type)))...)
        }
    }
}

for _, secret := range spc.Spec.SecretObjects {
    key := types.NamespacedName{Name: secret.SecretName, Namespace: namespace}
    val, exists := secretOwnerMap[key]
    if exists {
        secretOwnerMap[key] = append(val, ownerRefs...)
    } else {
        secretOwnerMap[key] = ownerRefs
    }
}

Reconcile

• Like shown above, Reconcile also iterates through secrets objects, so we must build them when syncOptions.syncAll is set to true.
• This same strategy is also seen in pkg/rotation/reconciler.go.

Reconcile

files, err := fileutil.GetMountedFiles(spcPodStatus.Status.TargetPath)
if err != nil {
    r.generateEvent(pod, corev1.EventTypeWarning, secretCreationFailedReason, fmt.Sprintf("failed to get mounted files, err: %+v", err))
    klog.ErrorS(err, "failed to get mounted files", "spc", klog.KObj(spc), "pod", klog.KObj(pod), "spcps", klog.KObj(spcPodStatus))
    return ctrl.Result{RequeueAfter: 10 * time.Second}, err
}

if spc.Spec.SyncOptions.SyncAll {
    if len(spc.Spec.SecretObjects) == 0 {
        spc.Spec.SecretObjects = spcutil.BuildSecretObjects(files, secretutil.GetSecretType(strings.TrimSpace(spc.Spec.SyncOptions.Type)))
    } else {
        spc.Spec.SecretObjects = append(spc.Spec.SecretObjects, spcutil.BuildSecretObjects(files, secretutil.GetSecretType(strings.TrimSpace(spc.Spec.SyncOptions.Type)))...)
    }
}

BuildSecretObjects

• The BuildSecretObjects function in pkg/util/spcutil/secret_object_builder.go takes the files and K8s secret type as parameters and builds the secret objects on the SecretProviderClass.

BuildSecretObjects

func BuildSecretObjects(files map[string]string, secretType corev1.SecretType) []*secretsstorev1.SecretObject {
	var (
		secretObject  *secretsstorev1.SecretObject
		secretObjects []*secretsstorev1.SecretObject
	)

	secretObjects = make([]*secretsstorev1.SecretObject, 0)
	for key := range files {

		switch secretType {
		case corev1.SecretTypeOpaque:
			secretObject = createOpaqueSecretDataObject(key)
		case corev1.SecretTypeTLS:
			secretObject = createTLSSecretDataObject(key)
		case corev1.SecretTypeDockerConfigJson:
			secretObject = createDockerConfigJsonSecretDataObject(key)
		case corev1.SecretTypeBasicAuth:
			secretObject = createBasicAuthSecretDataObject(key)
		case corev1.SecretTypeSSHAuth:
			secretObject = createSSHSecretDataObject(key)
		}

		secretObjects = append(secretObjects, secretObject)
	}

	return secretObjects
}

Usage

• All of the details, along with examples, have been added to docs/book/src/topics/sync-all-secrets-to-k8s.md.
• The examples utilize the Vault provider, but this feature is meant to be provider-agnostic and should work with any of them.

Additional Notes

• As always I am happy to receive any and all feedback @aramase @tam7t @ritazh
• I've added a couple of tests within e2e-provider.bats as well.

[k8s-ci-robot]

Hi @manedurphy. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: manedurphy
To complete the pull request process, please assign tam7t after the PR has been reviewed.
You can assign the PR to them by writing /assign @tam7t in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[fubar]

I think what many people (including myself) who are using AWS Secrets Manager are looking for is the ability to automatically populate key/value pairs in a Kubernetes Secret with the key/value pairs of an AWS Secrets Manager secret. Currently, the AWS Secrets & Configuration Provider (ASCP) for the Secrets Store CSI Driver requires us to explicitly list all key/value pairs in a SecretProviderClass resource, and it needs to be done twice: On the input side, the AWS Secrets Manager secret is made available as a JSON object which includes all key/value pairs in the secret, and the desired keys need to be specified with a JSON path reference (jmesPath); On the output side, those selected values then need to be set as entries in the Kubernetes Secret. The desired solution here is to have it automatically create one entry in a Kubernetes Secret for each entry of the JSON object, using the same keys. Does this PR help with that? Would it require any changes on the ASCP side?

[manedurphy]

It has been a while since I've had a look at this, so I'd have to read through it to refresh my memory of all of the changes that were made, but this was meant to be provider-agnostic. Regardless of which provider that the csi-driver gets its secrets from, the SyncAll configuration option is meant to sync each secret as its own Kubernetes secret. Additionally, for opaque secrets, the user can specify that they wish to sync all key-value pairs listed in this objects field of the SecretProviderClass into a single Kubernetes secret. I am going to spend some time look back at the changes I made an confirm that no changes need to be made on the provider's side. I will also attend this week's meeting to see if we can make progress on this.

[manedurphy]

Having gone through the motions of running the driver with the AWS provider, it seems that the AWS provider does not send files back to the driver as the other providers do, see here.

In order for the SyncAll feature to be provider-agnostic, the driver should be solely responsible for mounting secrets, so the AWS provider would need to make this change to take advantage of this feature.

[fubar]

Appreciate the efforts @manedurphy. Aside from the AWS provider not adhering to the standard with respect to sending back files as you mentioned, a key point with AWS secrets is that their value is provided as a JSON object. If AWS were to adhere to the standard, would the changes in this PR enable converting an untyped JSON object into a single Kubernetes secret with all entries of the JSON object added as individual key/value pairs in the Kubernetes secret?

[manedurphy]

I think I understand your question now, but please correct me if need be. When a user decides to sync all of the secrets listed in the provider's configuration, it is done so by referring to the name of the file that is mounted, and the value stored in that file will be the value of the secret that is synced to K8s. Having had another chance to play with the AWS provider, what I've noticed is that if we use jmesPath in the provider configuration, that a file for each key is written in addition to the full JSON object. In the example below, we can expect 3 files to be mounted:

1. db-creds: the full JSON object
2. username: the value assigned to the username key in secrets manager
3. password: the value assigned to the password key in secrets manager

SecretProviderClassAWS

apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
  name: aws-secrets
spec:
  provider: aws
  parameters:
    objects: |
      - objectName: "db-creds"
        objectType: "secretsmanager"
        jmesPath:
        - path: username
          objectAlias: username
        - path: password
          objectAlias: password
  syncOptions:
    syncAll: true
    type: Opaque

Since we have 3 files mounted, we can expect 3 secrets to be synced to K8s named db-creds, username, and password with their respective values being the same as specified above. So for AWS, there would just be an additional secret with the full JSON object as a value.

Should the AWS provider start adhering to the standard of sending files back to the driver, when jmesPath is configured in the SPC it can send one file per key-value pair that is specified, each of which will subsequently be synced as their own K8s secret. Otherwise, the provider will send a single file back to the driver which contains the full JSON object, which will subsequently be synced as a single K8s secret.

[fubar]

Thanks for taking a closer look at the AWS provider and for providing an example. Your observations of the files that are mounted and their contents are correct. As for your last paragraph and to make sure I properly explained the desired outcome: The desired solution from my perspective is that jmesPath does not need to be specified. Instead, based on your example:

apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
  name: aws-secrets
spec:
  provider: aws
  parameters:
    objects: |
      - objectName: "db-creds"
        objectType: "secretsmanager"
  syncOptions:
    syncAll: true
    type: Opaque

This would create a single Kubernetes secret, containing username and password as individual values. Ie. either the driver or the AWS provider would JSON-decode the AWS secret, and allow for all contained key/value pairs to be set in a single Kubernetes secret, without having to list those key/value pairs anywhere.

In light of this added context, do you think this is possible? If so, do the required changes to make it work need to be made only in the AWS provider, or would this PR need additional work as well?

[manedurphy]

Got it, thank you for clarifying @fubar. So, with the changes that are currently in place, that functionality is not available. However, it would be totally possible. My only concern would be implementing features in the driver that are provider-specific when the feature is meant to be provider-agnostic. However, I feel that I can implement this functionality in a way that can be used by any provider.

• Add a boolean json field to syncOptions in the SecretProviderClass.
• When true, the driver will expect that all of the secrets that are listed in the SPC are in JSON format, and handles the error accordingly if any of them are not.
• When the driver successfully JSON-decodes each secret defined in the SPC, it can then build a secret object for each mounted secret, and add an entry to the secret object's data field for each key-value pair within the SPC following a similar approach to what is described in the PR description above (recently updated).
• This would mean that the user should expect to have multiple SPCs defined if they are intending to have a mixture of JSON and non-JSON secrets.
• For both of the SPC definitions below, this would result in a single K8s secret with 2 key-value pairs

{"username": "my-db-user","password": "my-db-password"}

SecretProviderClassAWS

apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
  name: aws-secrets
spec:
  provider: aws
  parameters:
    objects: |
      - objectName: "db-creds"
        objectType: "secretsmanager"
  syncOptions:
    type: Opaque
    syncAll: true
    json: true

SecretProviderClassVault

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: vault-opaque
spec:
  provider: vault
  parameters:
    roleName: "csi"
    vaultAddress: "http://vault.vault:8200"
    objects: |
      - secretPath: "secret/data/db-creds"
        objectName: "database/creds"
        secretKey: "creds"
  syncOptions:
    type: Opaque
    syncAll: true
    json: true

What are your thoughts on this approach?

[fubar]

That sounds excellent! I am not familiar enough with the various secrets providers currently out there to know if support for JSON-encoded values should be the responsibility of a provider or the driver. However, it is not hard to imagine that other (and future) providers would use JSON under the hood - even aside from that, having the option of JSON support on the driver level would enable us to deliberately use JSON-encoded values with any provider if we find it beneficial for a given use case. Thus I think it would be a valuable addition to the driver.

Having to create separate SPCs for JSON-encoded secrets and all others would be a bit cumbersome. What do you think of using a list to indicate which secrets are JSON, and only attempt to decode those? E.g.:

apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
  name: aws-secrets
spec:
  provider: aws
  parameters:
    objects: |
      - objectName: "db-creds"
        objectType: "secretsmanager"
  syncOptions:
    type: Opaque
    syncAll: true
    decodeJsonSecrets:
      - secretName: "db-creds"

I recognize that adding a list is somewhat going against the original motivation for this PR, but I feel that this feature is different enough that a simple list of secrets would be the easiest to use. And it's optional, after all.

Taking this one step further, one could imagine that other formats aside from JSON could be supported in the future. A specification like this would make it more extensible:

  syncOptions:
    type: Opaque
    syncAll: true
    decodeSecrets:
      - secretName: "db-creds"
        format: json

[manedurphy]

I like the idea of supporting multiple formats in the driver. I would assume that most would opt in for JSON, but leaving it open for other possibilities gives the driver more flexibility than the boolean value that I presented before. A couple of things comes to mind to bounce off of your idea for a decodeSecrets list.

1. Format field hierarchy

• We can have a top-level format field within the syncOptions object that tells the driver to expect all mounted secrets to adhere to that format, and use the decodeSecrets list to specify which secrets (if any) will use an alternative format.

• In the example below for Vault, we would expect 2 secrets, username and password, to be individual secrets in K8s with 1 key-value pair each; we would also expect a third secret, db-creds, to be a single secret in K8s with 2 key-value pairs.

• In the example below for AWS, we would expect 2 secrets, db-creds and jwt-auth, to be individual secrets in K8s with their respective number of key-value pairs because they inherit the json format from the top-level in syncOptions.

DB-Creds Secret

{
  "username": "my-db-user",
  "password": "my-db-password"
}

SecretProviderClassVault

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: vault-spc
spec:
  provider: vault
  parameters:
    roleName: "csi"
    vaultAddress: "http://vault.vault:8200"
    objects: |
      - secretPath: "secret/data/database"
        objectName: "username"
        secretKey: "username"
      - secretPath: "secret/data/database"
        objectName: "password"
        secretKey: "password"
      - secretPath: "secret/data/database"
        objectName: "db-creds"
  syncOptions:
    type: Opaque
    syncAll: true
    format: default
    decodeSecrets:
      - secretName: "db-creds"
        format: json

SecretProviderClassAWS

apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
  name: aws-spc
spec:
  provider: aws
  parameters:
    objects: |
      - objectName: "db-creds"
        objectType: "secretsmanager"
      - objectName: "jwt-auth"
        objectType: "secretsmanager"
  syncOptions:
    type: Opaque
    syncAll: true
    format: json

2. Secret format auto-detection

• We can set the value of the format field to auto.
• The value auto means that the driver should not expect the secret to follow any specific format, but that it should try to acquire a secret's key-value pairs for all supported formats before resorting to the driver's default behavior.
• In the POC program below, we have 3 secrets. One of which is in JSON format, another in YAML, and the last one plain text. The readSecret function first attempts to JSON-unmarshal the secret to a map and return on success. On failure, it moves on to the next supported format, YAML, following the same procedure. When all supported formats fail, the function resorts to returning the plain text contents of the file as a string.
• This allows the user to trust that the driver will handle a set of mounted secrets accordingly, as well as the ability to strictly configure the format should they prefer/need to do that instead.

Mounted Secrets

• secrets/db-creds-json

{
    "username": "db-user-json",
    "password": "db-password-json"
}

• secrets/db-creds-yaml

username: db-user-yaml
password: db-password-yaml

• secrets/default

default-secret

POC Program

package main

import (
	"encoding/json"
	"flag"
	"fmt"
	"os"

	"github.com/hashicorp/go-hclog"
	"sigs.k8s.io/yaml"
)

const (
	jsonSecret   = "secrets/db-creds-json"
	yamlSecret   = "secrets/db-creds-yaml"
	normalSecret = "secrets/default"
)

var (
	logger hclog.Logger
	debug  bool
)

func init() {
	logger = hclog.Default()
	flag.BoolVar(&debug, "debug", false, "set logger level to debug")
	flag.Parse()

	if debug {
		logger.SetLevel(hclog.Debug)
	}
}

func main() {
	var (
		secret interface{}
		err    error
	)
	if secret, err = readSecret(jsonSecret); err != nil {
		panic(err)
	}
	logger.Info("secret received", "secret", secret.(map[string]interface{}))

	if secret, err = readSecret(yamlSecret); err != nil {
		panic(err)
	}
	logger.Info("secret received", "secret", secret.(map[string]interface{}))

	if secret, err = readSecret(normalSecret); err != nil {
		panic(err)
	}
	logger.Info("secret received", "secret", secret.(string))
}

func readSecret(path string) (interface{}, error) {
	var (
		fileContent []byte
		secretsMap  map[string]interface{}
		err         error
	)

	if fileContent, err = os.ReadFile(path); err != nil {
		return nil, fmt.Errorf("could not read content: %v", err)
	}
	if err = json.Unmarshal(fileContent, &secretsMap); err == nil {
		return secretsMap, nil
	}
	logger.Debug("could not read JSON data", "error", err)
	if err = yaml.Unmarshal(fileContent, &secretsMap); err == nil {
		return secretsMap, nil
	}
	logger.Debug("could not read YAML data", "error", err)

	return string(fileContent), nil
}

POC Logs

2022-03-22T21:24:44.667-0700 [INFO]  secret received: secret="map[password:db-password-json username:db-user-json]"
2022-03-22T21:24:44.667-0700 [DEBUG] could not read JSON data: error="invalid character 'u' looking for beginning of value"
2022-03-22T21:24:44.667-0700 [INFO]  secret received: secret="map[password:db-password-yaml username:db-user-yaml]"
2022-03-22T21:24:44.667-0700 [DEBUG] could not read JSON data: error="invalid character 'd' looking for beginning of value"
2022-03-22T21:24:44.667-0700 [DEBUG] could not read YAML data: error="error unmarshaling JSON: while decoding JSON: json: cannot unmarshal string into Go value of type map[string]interface {}"
2022-03-22T21:24:44.667-0700 [INFO]  secret received: secret=default-secret

3. Other possibilies for syncOptions

Note: This last idea is unrelated to the current discussion. Just something I thought of recently and wanted to make note of here for future reference and to keep the brainstorming alive.

• A removeOnPodDeletion boolean field in syncOptions.
• When set to false, a secret that was synced to K8s will remain after a pod's deletion.
• I'm imagining a scenario where a user may want to launch a short-lived pod that is deleted once the mounted secrets have been synced to K8s.

Example

syncOptions:
  type: Opaque
  syncAll: true
  format: auto
  removeOnPodDeletion: false

Another thing that I am thinking about is how to inform each provider which format should be used. AWS mounts the secret as a JSON object by default, so that would be supported immediately. Vault, however, parses through the .data.data map that is nested in the response body as mentioned here. When a value for secretKey is not specified in the SecretProviderClass parameters, the provider returns the entire JSON response it receives from Vault's API, shown below.

Vault Response

{
  "request_id": "1e9243e4-5d32-69c6-c3f0-7131eafd4c00",
  "lease_id": "",
  "lease_duration": 0,
  "renewable": false,
  "data": {
    "data": {
      "password": "db-password",
      "username": "db-username"
    },
    "metadata": {
      "created_time": "2022-03-23T02:26:26.993678358Z",
      "custom_metadata": null,
      "deletion_time": "",
      "destroyed": false,
      "version": 1
    }
  },
  "warnings": null
}

This makes me think that for the Vault provider, in order to take advantage of the driver's supported formats, it would need the parameters in the SecretProviderClass to specify the format. In the case of JSON, it would also need to only JSON-marshal the contents that are nested in .data.data rather than the entire response body. Maybe @tomhjp will have some more insight on how to handle such a case.

SecretProviderClassVault

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: vault-spc
spec:
  provider: vault
  parameters:
    roleName: "csi"
    vaultAddress: "http://vault.vault:8200"
    objects: |
      - secretPath: "secret/data/database"
        objectName: "database/db-creds"
        format: json
  syncOptions:
    syncAll: true
    type: Opaque
    format: json

Apologies for the verboseness of this response. I am enjoying the exchange of these ideas!

[fubar]

First off, thanks for putting together those details, this is going in a great direction!

1. Specifying an optional default format that applies to all secrets is a good idea, and making it a direct property of syncOptions makes sense. I'd suggest making the naming between format and decodeSecrets more consistent; they both refer to the same thing but the names don't imply that. How about reducing decodeSecrets to just secrets; as for format: default, if I understand the driver correctly, it expects secret values to be in plaintext at the moment, so the default format would be plaintext. This would give us:

   syncOptions:
     type: Opaque
     syncAll: true
     format: plaintext
     secrets:
       - secretName: "db-creds"
         format: json

   The secrets list would thereby be generic enough to be re-used for future use cases that are not related to the secret's formatting.

2. I like the idea of automatic format detection. I imagine that attempting to decode using the wrong format fails fast, hence there would be no performance penalty worth worrying about.

3. I'm not sure I follow the use case here, but when I started to set up the integration with AWS I expected the secrets to be present without having a pod that references them, so I see the general point. In my case it was not related to the deletion of a pod but rather to its existence in the first place. Also, you would need to be able to remove a secret that is never going to be referenced by a pod again, and I imagine the driver would need to do this after unsetting this config value again.

As for Vault and its nested JSON object, I would have expected providers to return only the value of a secret to the driver, and not a construct that includes metadata, but I can see why someone would want it all. If we consider this a Vault "format", and assuming it's unlikely to change, it could be made an option of the format parameter:

syncOptions:
  type: Opaque
  syncAll: true
  format: vault

Alternatively, a JSON path could work. Though a bit clunky, this would make it extensible for any other JSON object:

syncOptions:
  type: Opaque
  syncAll: true
  secrets:
    - secretName: "db-creds"
      format: json
      jsonPath: "$.data.data"

Thanks for being open to my suggestions! This has the potential to be a significant improvement to the whole integration of external secrets providers.

[manedurphy]

So I really like the jsonPath field. It takes away the burden of handling such a case from a specific provider, and is available for use by all current providers and future providers. I also agree with changing the name from decodeSecrets to secrets for the same reasons that you have specified.

I think what I would like to do is spend some time this weekend to work on some POCs and cut a new branch where I can implement these ideas. I will try to break down the strategies the same as before, so that the changes that need to be made in the code are clear. I will return to this discussion with any insights, obstacles, and successes. Trying my best to balance time with work as it has recently been busy, but I'm excited to see these ideas through. Really appreciate your inputs, @fubar!

[fubar]

Awesome, looking forward to the results!

[manedurphy]

Alright, I was able to implement these changes and have several examples to demonstrate the new functionality. I will still need to write unit tests for the new functions I've added as well as update the documentation, but I want to make sure that this is going in the right direction before I address those areas.

These examples were done with the Vault provider, but the functionality is provider-agnostic. These are the secrets that are stored in Vault.

# Opaque
kubectl exec -n vault vault-0 -- vault kv put secret/database username="db-username" password="db-password"
kubectl exec -n vault vault-0 -- vault kv put secret/auth client_id="my-client-id" client_secret="my-client-secret"

# Basic
kubectl exec -n vault vault-0 -- vault kv put secret/basic-plaintext credentials="basic-username,basic-password"
kubectl exec -n vault vault-0 -- vault kv put secret/basic-json username="basic-username" password="basic-password"

# TLS
kubectl exec -n vault vault-0 -- vault kv put secret/tls-json tls.key="$TLS_KEY" tls.crt="$TLS_CERT" 
kubectl exec -n vault vault-0 -- vault kv put secret/tls-plaintext data="$CERT"

tls-plaintext content

# CERT

-----BEGIN CERTIFICATE-----
MIIDOTCCAiGgAwIBAgIJAP0J5Z7N0Y5fMA0GCSqGSIb3DQEBCwUAMDMxFzAVBgNV
BAMMDmRlbW8uYXp1cmUuY29tMRgwFgYDVQQKDA9ha3MtaW5ncmVzcy10bHMwHhcN
MjAwNDE1MDQyMzQ2WhcNMjEwNDE1MDQyMzQ2WjAzMRcwFQYDVQQDDA5kZW1vLmF6
dXJlLmNvbTEYMBYGA1UECgwPYWtzLWluZ3Jlc3MtdGxzMIIBIjANBgkqhkiG9w0B
AQEFAAOCAQ8AMIIBCgKCAQEAyS3Zky3n8JlLBxPLzgUpKZYxvzRadeWLmWVbK9by
o08S0Ss8Jao7Ay1wHtnLbn52rzCX6IX1sAe1TAT755Gk7JtLMkshtj6F8BNeelEy
E1gsBE5ntY5vyLTm/jZUIKz2Z9TLnqvQTmp6gJ68BKJ1NobnsHiAcKc6hI7kmY9C
oshmAi5qiKYBgzv/thji0093vtVSa9iwHhQp+AEIMhkvM5ZZkiU5eE6MT9SBEcVW
KmWF28UsB04daYwS2MKJ5l6d4n0LUdAG0FBt1lCoT9rwUDj9l3Mqmi953gw26LUr
NrYnM/8N2jl7Cuyw5alIWaUDrt5i+pu8wdWfzVk+fO7x8QIDAQABo1AwTjAdBgNV
HQ4EFgQUwFBbR014McETdrGGklpEQcl71Q0wHwYDVR0jBBgwFoAUwFBbR014McET
drGGklpEQcl71Q0wDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEATgTy
gg1Q6ISSekiBCe12dqUTMFQh9GKpfYWKRbMtjOjpc7Mdwkdmm3Fu6l3RfEFT28Ij
fy97LMYv8W7beemDFqdmneb2w2ww0ZAFJg+GqIJZ9s/JadiFBDNU7CmJMhA225Qz
XC8ovejiePslnL4QJWlhVG93ZlBJ6SDkRgfcoIW2x4IBE6wv7jmRF4lOvb3z1ddP
iPQqhbEEbwMpXmWv7/2RnjAHdjdGaWRMC5+CaI+lqHyj6ir1c+e6u1QUY54qjmgM
koN/frqYab5Ek3kauj1iqW7rPkrFCqT2evh0YRqb1bFsCLJrRNxnOZ5wKXV/OYQa
QX5t0wFGCZ0KlbXDiw==
-----END CERTIFICATE-----
-----BEGIN PRIVATE KEY-----
MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDJLdmTLefwmUsH
E8vOBSkpljG/NFp15YuZZVsr1vKjTxLRKzwlqjsDLXAe2ctufnavMJfohfWwB7VM
BPvnkaTsm0sySyG2PoXwE156UTITWCwETme1jm/ItOb+NlQgrPZn1Mueq9BOanqA
nrwEonU2hueweIBwpzqEjuSZj0KiyGYCLmqIpgGDO/+2GOLTT3e+1VJr2LAeFCn4
AQgyGS8zllmSJTl4ToxP1IERxVYqZYXbxSwHTh1pjBLYwonmXp3ifQtR0AbQUG3W
UKhP2vBQOP2XcyqaL3neDDbotSs2ticz/w3aOXsK7LDlqUhZpQOu3mL6m7zB1Z/N
WT587vHxAgMBAAECggEAJb0qIYftCJ9ZCbzW8JDbRefc8SdbCN7Er0PqNHEgFy6Q
MxjPMambZF8ztzXYCaRDk12kQYRPsHPhuJ7+ulQCAjinhIm/izZzXbPkd0GgCSzz
JOOoZNCRe68j3fBHG9IWbyfmAp/sdalXzaT5VE09e7sW323bekaEnbVIgN30/CAS
gI77YdaIhG+PT/pSCOc11MTkBJp+VhT1tEtlRAR78b1RXbGi1oUHRee7C3Ia8IKQ
3L5dPxR9RsYsR2O66908kEi8ZcuIjcbIuRPDXYHY+5Nwm3mXuZlkyjyfxJXsIA8i
qBrQrSpHGgAn1TVlLDSCKPLbkRzBRRvAW0zL/cDTuQKBgQDq/9Yxx9QivAuUxxdE
u0VO5CzzZYFWhDxAXS3/wYyo1YnoPtUz/lGCvMWp0k2aaa0+KTXv2fRCUGSujHW7
Jfo4kuMPkauAhoXx9QJAcjoK0nNbYEaqoJyMoRID+Qb9XHkj+lmBTmMVgALCT9DI
HekHj/M3b7CknbfWv1sOZ/vpQwKBgQDbKEuP/DWQa9DC5nn5phHD/LWZLG/cMR4X
TmwM/cbfRxM/6W0+/KLAodz4amGRzVlW6ax4k26BSE8Zt/SiyA1DQRTeFloduoqW
iWF4dMeItxw2am+xLREwtoN3FgsJHu2z/O/0aaBAOMLUXIPIyiE4L6OnEPifE/pb
AM8EbM5auwKBgGhdABIRjbtzSa1kEYhbprcXjIL3lE4I4f0vpIsNuNsOInW62dKC
Yk6uaRY3KHGn9uFBSgvf/qMost310R8xCYPwb9htN/4XQAspZTubvv0pY0O0aQ3D
0GJ/8dFD2f/Q/pekyfUsC8Lzm8YRzkXhSqkqG7iF6Kviw08iolyuf2ijAoGBANaA
pRzDvWWisUziKsa3zbGnGdNXVBEPniUvo8A/b7RAK84lWcEJov6qLs6RyPfdJrFT
u3S00LcHICzLCU1+QsTt4U/STtfEKjtXMailnFrq5lk4aiPfOXEVYq1fTOPbesrt
Katu6uOQ6tjRyEbx1/vXXPV7Peztr9/8daMeIAdbAoGBAOYRJ1CzMYQKjWF32Uas
7hhQxyH1QI4nV56Dryq7l/UWun2pfwNLZFqOHD3qm05aznzNKvk9aHAsOPFfUUXO
7sp0Ge5FLMSw1uMNnutcVcMz37KAY2fOoE2xoLM4DU/H2NqDjeGCsOsU1ReRS1vB
J+42JGwBdLV99ruYKVKOWPh4
-----END PRIVATE KEY-----

tls-json content

# TLS_KEY

-----BEGIN PRIVATE KEY-----
MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDJLdmTLefwmUsH
E8vOBSkpljG/NFp15YuZZVsr1vKjTxLRKzwlqjsDLXAe2ctufnavMJfohfWwB7VM
BPvnkaTsm0sySyG2PoXwE156UTITWCwETme1jm/ItOb+NlQgrPZn1Mueq9BOanqA
nrwEonU2hueweIBwpzqEjuSZj0KiyGYCLmqIpgGDO/+2GOLTT3e+1VJr2LAeFCn4
AQgyGS8zllmSJTl4ToxP1IERxVYqZYXbxSwHTh1pjBLYwonmXp3ifQtR0AbQUG3W
UKhP2vBQOP2XcyqaL3neDDbotSs2ticz/w3aOXsK7LDlqUhZpQOu3mL6m7zB1Z/N
WT587vHxAgMBAAECggEAJb0qIYftCJ9ZCbzW8JDbRefc8SdbCN7Er0PqNHEgFy6Q
MxjPMambZF8ztzXYCaRDk12kQYRPsHPhuJ7+ulQCAjinhIm/izZzXbPkd0GgCSzz
JOOoZNCRe68j3fBHG9IWbyfmAp/sdalXzaT5VE09e7sW323bekaEnbVIgN30/CAS
gI77YdaIhG+PT/pSCOc11MTkBJp+VhT1tEtlRAR78b1RXbGi1oUHRee7C3Ia8IKQ
3L5dPxR9RsYsR2O66908kEi8ZcuIjcbIuRPDXYHY+5Nwm3mXuZlkyjyfxJXsIA8i
qBrQrSpHGgAn1TVlLDSCKPLbkRzBRRvAW0zL/cDTuQKBgQDq/9Yxx9QivAuUxxdE
u0VO5CzzZYFWhDxAXS3/wYyo1YnoPtUz/lGCvMWp0k2aaa0+KTXv2fRCUGSujHW7
Jfo4kuMPkauAhoXx9QJAcjoK0nNbYEaqoJyMoRID+Qb9XHkj+lmBTmMVgALCT9DI
HekHj/M3b7CknbfWv1sOZ/vpQwKBgQDbKEuP/DWQa9DC5nn5phHD/LWZLG/cMR4X
TmwM/cbfRxM/6W0+/KLAodz4amGRzVlW6ax4k26BSE8Zt/SiyA1DQRTeFloduoqW
iWF4dMeItxw2am+xLREwtoN3FgsJHu2z/O/0aaBAOMLUXIPIyiE4L6OnEPifE/pb
AM8EbM5auwKBgGhdABIRjbtzSa1kEYhbprcXjIL3lE4I4f0vpIsNuNsOInW62dKC
Yk6uaRY3KHGn9uFBSgvf/qMost310R8xCYPwb9htN/4XQAspZTubvv0pY0O0aQ3D
0GJ/8dFD2f/Q/pekyfUsC8Lzm8YRzkXhSqkqG7iF6Kviw08iolyuf2ijAoGBANaA
pRzDvWWisUziKsa3zbGnGdNXVBEPniUvo8A/b7RAK84lWcEJov6qLs6RyPfdJrFT
u3S00LcHICzLCU1+QsTt4U/STtfEKjtXMailnFrq5lk4aiPfOXEVYq1fTOPbesrt
Katu6uOQ6tjRyEbx1/vXXPV7Peztr9/8daMeIAdbAoGBAOYRJ1CzMYQKjWF32Uas
7hhQxyH1QI4nV56Dryq7l/UWun2pfwNLZFqOHD3qm05aznzNKvk9aHAsOPFfUUXO
7sp0Ge5FLMSw1uMNnutcVcMz37KAY2fOoE2xoLM4DU/H2NqDjeGCsOsU1ReRS1vB
J+42JGwBdLV99ruYKVKOWPh4
-----END PRIVATE KEY-----

# TLS_CERT

-----BEGIN CERTIFICATE-----
MIIDOTCCAiGgAwIBAgIJAP0J5Z7N0Y5fMA0GCSqGSIb3DQEBCwUAMDMxFzAVBgNV
BAMMDmRlbW8uYXp1cmUuY29tMRgwFgYDVQQKDA9ha3MtaW5ncmVzcy10bHMwHhcN
MjAwNDE1MDQyMzQ2WhcNMjEwNDE1MDQyMzQ2WjAzMRcwFQYDVQQDDA5kZW1vLmF6
dXJlLmNvbTEYMBYGA1UECgwPYWtzLWluZ3Jlc3MtdGxzMIIBIjANBgkqhkiG9w0B
AQEFAAOCAQ8AMIIBCgKCAQEAyS3Zky3n8JlLBxPLzgUpKZYxvzRadeWLmWVbK9by
o08S0Ss8Jao7Ay1wHtnLbn52rzCX6IX1sAe1TAT755Gk7JtLMkshtj6F8BNeelEy
E1gsBE5ntY5vyLTm/jZUIKz2Z9TLnqvQTmp6gJ68BKJ1NobnsHiAcKc6hI7kmY9C
oshmAi5qiKYBgzv/thji0093vtVSa9iwHhQp+AEIMhkvM5ZZkiU5eE6MT9SBEcVW
KmWF28UsB04daYwS2MKJ5l6d4n0LUdAG0FBt1lCoT9rwUDj9l3Mqmi953gw26LUr
NrYnM/8N2jl7Cuyw5alIWaUDrt5i+pu8wdWfzVk+fO7x8QIDAQABo1AwTjAdBgNV
HQ4EFgQUwFBbR014McETdrGGklpEQcl71Q0wHwYDVR0jBBgwFoAUwFBbR014McET
drGGklpEQcl71Q0wDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEATgTy
gg1Q6ISSekiBCe12dqUTMFQh9GKpfYWKRbMtjOjpc7Mdwkdmm3Fu6l3RfEFT28Ij
fy97LMYv8W7beemDFqdmneb2w2ww0ZAFJg+GqIJZ9s/JadiFBDNU7CmJMhA225Qz
XC8ovejiePslnL4QJWlhVG93ZlBJ6SDkRgfcoIW2x4IBE6wv7jmRF4lOvb3z1ddP
iPQqhbEEbwMpXmWv7/2RnjAHdjdGaWRMC5+CaI+lqHyj6ir1c+e6u1QUY54qjmgM
koN/frqYab5Ek3kauj1iqW7rPkrFCqT2evh0YRqb1bFsCLJrRNxnOZ5wKXV/OYQa
QX5t0wFGCZ0KlbXDiw==
-----END CERTIFICATE-----

Opaque

Example 1

• For the following SecretProviderClass, we can expect 4 secrets to be synced to K8s.
• The top-level value for format is set to plaintext, which every secret will inherit unless specified in the secrets list with a different format (e.g. json).
• We can see 1 secret is specified in the secrets list with a different format, json. We can also see that a value for jsonPath is specified in that object, $.data.data.
• This means that each key-value pair in the nested $.data.data object will be set in a single K8s secrets named db-creds.
• We can also see, at the top-level, a value for jsonPath is set as $.bad.path. Since no other secrets are configured for a json format, this value does not affect anything. We will explore changing this value in the next example, but for now it does nothing.
• The 3 secrets that will use a plaintext format will be named database-username, database-password, and jwt-auth.
• The driver names the secrets automatically when syncAll is set to true, and alters the value of the objectName to be hyphen-separated.
• For the Vault provider, when a secretKey is not specified in the object parameters, the full JSON object, including metadata, is sent to the driver to be mounted. This means that the jwt-auth secret will be synced as the full JSON object base64 encoded to K8s since it has a plaintext format for this example. We will change it to json in the next example.

Details

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: vault-spc
spec:
  provider: vault
  parameters:
    roleName: "csi"
    vaultAddress: "http://vault.vault:8200"
    objects: |
      - secretPath: "secret/data/database"
        objectName: "database/username"
        secretKey: "username"
      - secretPath: "secret/data/database"
        objectName: "database/password"
        secretKey: "password"
      - secretPath: "secret/data/database"
        objectName: "db-creds"
      - secretPath: "secret/data/auth"
        objectName: "jwt-auth"
  syncOptions:
    syncAll: true
    type: Opaque
    format: plaintext
    jsonPath: "$.bad.path"
    secrets:
      - secretName: db-creds
        format: json
        jsonPath: "$.data.data"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: busybox-deployment-opaque
  labels:
    app: busybox
spec:
  replicas: 1
  selector:
    matchLabels:
      app: busybox
  template:
    metadata:
      labels:
        app: busybox
    spec:
      terminationGracePeriodSeconds: 0
      containers:
        - image: k8s.gcr.io/e2e-test-images/busybox:1.29
          name: busybox
          imagePullPolicy: IfNotPresent
          command:
            - "/bin/sleep"
            - "10000"
          env:
            - name: DB_USER
              valueFrom:
                secretKeyRef:
                  name: database-username
                  key: username
            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: database-password
                  key: password
          volumeMounts:
            - name: secrets-store-inline
              mountPath: "/mnt/secrets-store"
              readOnly: true
          resources:
            requests:
              memory: "64Mi"
              cpu: "250m"
            limits:
              memory: "128Mi"
              cpu: "500m"
      volumes:
        - name: secrets-store-inline
          csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
              secretProviderClass: "vault-spc"

kubectl get secrets

# Output
database-password     Opaque                                1      2s
database-username     Opaque                                1      2s
db-creds              Opaque                                2      2s
default-token-nv2v8   kubernetes.io/service-account-token   3      7h24m
jwt-auth              Opaque                                1      2s

kubectl get secret database-username -o yaml                                       

# Output
apiVersion: v1
data:
  username: ZGItdXNlcm5hbWU=
kind: Secret
metadata:
  creationTimestamp: "2022-03-28T00:57:55Z"
  labels:
    secrets-store.csi.k8s.io/managed: "true"
  name: database-username
  namespace: default
  ownerReferences:
  - apiVersion: apps/v1
    kind: ReplicaSet
    name: busybox-deployment-opaque-c4bf48669
    uid: 22ee0f18-6494-430c-9f36-084f8b43a51f
  resourceVersion: "48583"
  uid: 63eca3cf-4cdd-45fb-8f90-9aed2962d699
type: Opaque

kubectl get secret database-password -o yaml

# Output
apiVersion: v1
data:
  password: ZGItcGFzc3dvcmQ=
kind: Secret
metadata:
  creationTimestamp: "2022-03-28T00:57:55Z"
  labels:
    secrets-store.csi.k8s.io/managed: "true"
  name: database-password
  namespace: default
  ownerReferences:
  - apiVersion: apps/v1
    kind: ReplicaSet
    name: busybox-deployment-opaque-c4bf48669
    uid: 22ee0f18-6494-430c-9f36-084f8b43a51f
  resourceVersion: "48582"
  uid: 8d078727-f611-48bd-a9d4-821577de9372
type: Opaque

kubectl get secrets db-creds -o yaml             

# Output
apiVersion: v1
data:
  password: ZGItcGFzc3dvcmQ=
  username: ZGItdXNlcm5hbWU=
kind: Secret
metadata:
  creationTimestamp: "2022-03-28T00:57:55Z"
  labels:
    secrets-store.csi.k8s.io/managed: "true"
  name: db-creds
  namespace: default
  ownerReferences:
  - apiVersion: apps/v1
    kind: ReplicaSet
    name: busybox-deployment-opaque-c4bf48669
    uid: 22ee0f18-6494-430c-9f36-084f8b43a51f
  resourceVersion: "48580"
  uid: a8941489-54bc-4ee7-b2e6-655131b422c1
type: Opaque

kubectl get secrets jwt-auth -o yaml

# Output
apiVersion: v1
data:
  jwt-auth: eyJyZXF1ZXN0X2lkIjoiNmM0NDI3MjEtMzM2Yy1lOTA2LTY2ZTQtMGNlYTliNTU1N2U0IiwibGVhc2VfaWQiOiIiLCJsZWFzZV9kdXJhdGlvbiI6MCwicmVuZXdhYmxlIjpmYWxzZSwiZGF0YSI6eyJkYXRhIjp7ImNsaWVudF9pZCI6Im15LWNsaWVudC1pZCIsImNsaWVudF9zZWNyZXQiOiJteS1jbGllbnQtc2VjcmV0In0sIm1ldGFkYXRhIjp7ImNyZWF0ZWRfdGltZSI6IjIwMjItMDMtMjdUMjA6MzM6MzQuODI0NjYyNzk5WiIsImN1c3RvbV9tZXRhZGF0YSI6bnVsbCwiZGVsZXRpb25fdGltZSI6IiIsImRlc3Ryb3llZCI6ZmFsc2UsInZlcnNpb24iOjF9fSwid2FybmluZ3MiOm51bGx9
kind: Secret
metadata:
  creationTimestamp: "2022-03-28T00:57:55Z"
  labels:
    secrets-store.csi.k8s.io/managed: "true"
  name: jwt-auth
  namespace: default
  ownerReferences:
  - apiVersion: apps/v1
    kind: ReplicaSet
    name: busybox-deployment-opaque-c4bf48669
    uid: 22ee0f18-6494-430c-9f36-084f8b43a51f
  resourceVersion: "50439"
  uid: de8fba16-4625-4550-9b11-c1119fca8892
type: Opaque

Example 2

• For this example, we are now listing the jwt-auth secret in the secrets list. Since we are not specifying a value for jsonPath, so it will inherit the value $.bad.path.
• Since this path does not exist on that object, the secret will fail to sync.

Details

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: vault-spc
spec:
  provider: vault
  parameters:
    roleName: "csi"
    vaultAddress: "http://vault.vault:8200"
    objects: |
      - secretPath: "secret/data/database"
        objectName: "database/username"
        secretKey: "username"
      - secretPath: "secret/data/database"
        objectName: "database/password"
        secretKey: "password"
      - secretPath: "secret/data/database"
        objectName: "db-creds"
      - secretPath: "secret/data/auth"
        objectName: "jwt-auth"
  syncOptions:
    syncAll: true
    type: Opaque
    format: plaintext
    jsonPath: "$.bad.path"
    secrets:
      - secretName: db-creds
        format: json
        jsonPath: "$.data.data"
      - secretName: jwt-auth
        format: json

kubectl get secrets

# Output
NAME                  TYPE                                  DATA   AGE
database-password     Opaque                                1      3s
database-username     Opaque                                1      3s
db-creds              Opaque                                2      3s
default-token-nv2v8   kubernetes.io/service-account-token   3      7h50m

# Driver logs
E0328 01:24:10.663014       1 reconciler.go:493] "failed to get data in spc for secret" err="invalid json path $.bad.path" spc="default/vault-spc" secret="default/jwt-auth" controller="rotation"

Example 3

• In this example, we will remove the top-level value for jsonPath.
• This means that the same jwt-auth secret will sync all of the fields, including metadata, in that JSON object.
• Each top-level key in the JSON object will be set in the synced K8s secret, and each value will be base64 encoded (including nested objects)

Details

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: vault-spc
spec:
  provider: vault
  parameters:
    roleName: "csi"
    vaultAddress: "http://vault.vault:8200"
    objects: |
      - secretPath: "secret/data/database"
        objectName: "database/username"
        secretKey: "username"
      - secretPath: "secret/data/database"
        objectName: "database/password"
        secretKey: "password"
      - secretPath: "secret/data/database"
        objectName: "db-creds"
      - secretPath: "secret/data/auth"
        objectName: "jwt-auth"
  syncOptions:
    syncAll: true
    type: Opaque
    format: plaintext
    secrets:
      - secretName: db-creds
        format: json
        jsonPath: "$.data.data"
      - secretName: jwt-auth
        format: json

{
  "request_id": "d912c8b6-7f44-cee3-847c-9276f0239399",
  "lease_id": "",
  "lease_duration": 0,
  "renewable": false,
  "data": {
    "data": {
      "client_id": "my-client-id",
      "client_secret": "my-client-secret"
    },
    "metadata": {
      "created_time": "2022-03-27T20:33:34.824662799Z",
      "custom_metadata": null,
      "deletion_time": "",
      "destroyed": false,
      "version": 1
    }
  },
  "warnings": null
}

kubectl get secrets jwt-auth -o yaml

# Output
apiVersion: v1
data:
  data: eyJkYXRhIjp7ImNsaWVudF9pZCI6Im15LWNsaWVudC1pZCIsImNsaWVudF9zZWNyZXQiOiJteS1jbGllbnQtc2VjcmV0In0sIm1ldGFkYXRhIjp7ImNyZWF0ZWRfdGltZSI6IjIwMjItMDMtMjdUMjA6MzM6MzQuODI0NjYyNzk5WiIsImN1c3RvbV9tZXRhZGF0YSI6bnVsbCwiZGVsZXRpb25fdGltZSI6IiIsImRlc3Ryb3llZCI6ZmFsc2UsInZlcnNpb24iOjF9fQ==
  lease_duration: MA==
  lease_id: ""
  renewable: ZmFsc2U=
  request_id: ZmUwYWQ3ZWYtNjc0OS1iMWIxLTQ0NjItNDdiNDlmYWZhMWRl
  warnings: bnVsbA==
kind: Secret
metadata:
  creationTimestamp: "2022-03-28T01:30:04Z"
  labels:
    secrets-store.csi.k8s.io/managed: "true"
  name: jwt-auth
  namespace: default
  ownerReferences:
  - apiVersion: apps/v1
    kind: ReplicaSet
    name: busybox-deployment-opaque-c4bf48669
    uid: c85f1469-590d-461d-835b-4fdeb5802c36
  resourceVersion: "52040"
  uid: 0b9bc95c-cced-4b96-8cec-13f5e9c682ee
type: Opaque

Basic

Example

• For this example, we can expect 2 secrets of type kubernetes.io/basic-auth to be synced to K8s.
• In Vault, the basic-json secret has two fields, username and password; and the basic-plaintext secret has one field, credentials.
• When mounting the secret as a JSON object, we can take advantage of the fact that the driver can read the keys that are needed to create a secret of this type in K8s within the mounted JSON object.
• When mounting the secret as plaintext, we need to mount the secrets in a way that allows the driver to extract the username and password values respectively. I have chose to mount the secret as a comma-separated string, basic-username,basic-password. This is similar to how the driver currenlty handles separating values for a key and certificate for TLS secrets.
• I think with this implementation, that most users would opt in for the json format since they can store separate values for username and password in their respective secret stores rather than one key-value pair that contains both values as a comma-separated string, but I think this approach allows us to accomodate all supported formats for this secret type.

Details

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: vault-spc
spec:
  provider: vault
  parameters:
    roleName: "csi"
    vaultAddress: "http://vault.vault:8200"
    objects: |
      - secretPath: "secret/data/basic-json"
        objectName: "basic-json"
      - secretPath: "secret/data/basic-plaintext"
        objectName: "basic-plaintext"
        secretKey: "credentials"
  syncOptions:
    syncAll: true
    type: kubernetes.io/basic-auth
    format: json
    jsonPath: "$.data.data"
    secrets:
      - secretName: basic-plaintext
        format: plaintext

kubectl get secrets

# Output
basic-json            kubernetes.io/basic-auth              2      11s
basic-plaintext       kubernetes.io/basic-auth              2      11s
default-token-nv2v8   kubernetes.io/service-account-token   3      8h

kubectl get secrets basic-json -o yaml

# Output
apiVersion: v1
data:
  password: YmFzaWMtcGFzc3dvcmQ=
  username: YmFzaWMtdXNlcm5hbWU=
kind: Secret
metadata:
  creationTimestamp: "2022-03-28T01:59:11Z"
  labels:
    secrets-store.csi.k8s.io/managed: "true"
  name: basic-json
  namespace: default
  ownerReferences:
  - apiVersion: apps/v1
    kind: ReplicaSet
    name: busybox-deployment-basic-54b4b754f4
    uid: 5f297eee-541d-43a7-9ecc-5d26ff4e406e
  resourceVersion: "55103"
  uid: d5feb59a-61cf-4dbe-a092-73fb97200d6f
type: kubernetes.io/basic-auth

kubectl get secrets basic-plaintext -o yaml

# Output
apiVersion: v1
data:
  password: YmFzaWMtcGFzc3dvcmQ=
  username: YmFzaWMtdXNlcm5hbWU=
kind: Secret
metadata:
  creationTimestamp: "2022-03-28T01:59:11Z"
  labels:
    secrets-store.csi.k8s.io/managed: "true"
  name: basic-plaintext
  namespace: default
  ownerReferences:
  - apiVersion: apps/v1
    kind: ReplicaSet
    name: busybox-deployment-basic-54b4b754f4
    uid: 5f297eee-541d-43a7-9ecc-5d26ff4e406e
  resourceVersion: "55104"
  uid: db403427-ab1d-4b82-b001-d4c7d8d9ca9b
type: kubernetes.io/basic-auth

TLS

Example

• The driver currently separates the private key and certificate values from a file where these two values are concatenated; this is handled when using the plaintext format.
• When using the json format, we can extract the individual values for tls.key and tls.crt from the JSON object that is mounted.

Details

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: vault-tls
spec:
  provider: vault
  parameters:
    roleName: "csi"
    vaultAddress: "http://vault.vault:8200"
    objects: |
      - secretPath: "secret/data/tls-json"
        objectName: "tls-json"
      - secretPath: "secret/data/tls-plaintext"
        objectName: "tls-plaintext"
        secretKey: "data"
  syncOptions:
    syncAll: true
    type: kubernetes.io/tls
    format: plaintext
    jsonPath: "$.data.data"
    secrets:
      - secretName: tls-json
        format: json

kubectl get secrets    

# Output
NAME                  TYPE                                  DATA   AGE
default-token-nv2v8   kubernetes.io/service-account-token   3      8h
tls-json              kubernetes.io/tls                     2      3s
tls-plaintext         kubernetes.io/tls                     2      3s

kubectl get secrets tls-plaintext -o yaml

# Output
apiVersion: v1
data:
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURPVENDQWlHZ0F3SUJBZ0lKQVAwSjVaN04wWTVmTUEwR0NTcUdTSWIzRFFFQkN3VUFNRE14RnpBVkJnTlYKQkFNTURtUmxiVzh1WVhwMWNtVXVZMjl0TVJnd0ZnWURWUVFLREE5aGEzTXRhVzVuY21WemN5MTBiSE13SGhjTgpNakF3TkRFMU1EUXlNelEyV2hjTk1qRXdOREUxTURReU16UTJXakF6TVJjd0ZRWURWUVFEREE1a1pXMXZMbUY2CmRYSmxMbU52YlRFWU1CWUdBMVVFQ2d3UFlXdHpMV2x1WjNKbGMzTXRkR3h6TUlJQklqQU5CZ2txaGtpRzl3MEIKQVFFRkFBT0NBUThBTUlJQkNnS0NBUUVBeVMzWmt5M244SmxMQnhQTHpnVXBLWll4dnpSYWRlV0xtV1ZiSzlieQpvMDhTMFNzOEphbzdBeTF3SHRuTGJuNTJyekNYNklYMXNBZTFUQVQ3NTVHazdKdExNa3NodGo2RjhCTmVlbEV5CkUxZ3NCRTVudFk1dnlMVG0valpVSUt6Mlo5VExucXZRVG1wNmdKNjhCS0oxTm9ibnNIaUFjS2M2aEk3a21ZOUMKb3NobUFpNXFpS1lCZ3p2L3RoamkwMDkzdnRWU2E5aXdIaFFwK0FFSU1oa3ZNNVpaa2lVNWVFNk1UOVNCRWNWVwpLbVdGMjhVc0IwNGRhWXdTMk1LSjVsNmQ0bjBMVWRBRzBGQnQxbENvVDlyd1VEajlsM01xbWk5NTNndzI2TFVyCk5yWW5NLzhOMmpsN0N1eXc1YWxJV2FVRHJ0NWkrcHU4d2RXZnpWaytmTzd4OFFJREFRQUJvMUF3VGpBZEJnTlYKSFE0RUZnUVV3RkJiUjAxNE1jRVRkckdHa2xwRVFjbDcxUTB3SHdZRFZSMGpCQmd3Rm9BVXdGQmJSMDE0TWNFVApkckdHa2xwRVFjbDcxUTB3REFZRFZSMFRCQVV3QXdFQi96QU5CZ2txaGtpRzl3MEJBUXNGQUFPQ0FRRUFUZ1R5CmdnMVE2SVNTZWtpQkNlMTJkcVVUTUZRaDlHS3BmWVdLUmJNdGpPanBjN01kd2tkbW0zRnU2bDNSZkVGVDI4SWoKZnk5N0xNWXY4VzdiZWVtREZxZG1uZWIydzJ3dzBaQUZKZytHcUlKWjlzL0phZGlGQkROVTdDbUpNaEEyMjVRegpYQzhvdmVqaWVQc2xuTDRRSldsaFZHOTNabEJKNlNEa1JnZmNvSVcyeDRJQkU2d3Y3am1SRjRsT3ZiM3oxZGRQCmlQUXFoYkVFYndNcFhtV3Y3LzJSbmpBSGRqZEdhV1JNQzUrQ2FJK2xxSHlqNmlyMWMrZTZ1MVFVWTU0cWptZ00Ka29OL2ZycVlhYjVFazNrYXVqMWlxVzdyUGtyRkNxVDJldmgwWVJxYjFiRnNDTEpyUk54bk9aNXdLWFYvT1lRYQpRWDV0MHdGR0NaMEtsYlhEaXc9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
  tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBeVMzWmt5M244SmxMQnhQTHpnVXBLWll4dnpSYWRlV0xtV1ZiSzlieW8wOFMwU3M4CkphbzdBeTF3SHRuTGJuNTJyekNYNklYMXNBZTFUQVQ3NTVHazdKdExNa3NodGo2RjhCTmVlbEV5RTFnc0JFNW4KdFk1dnlMVG0valpVSUt6Mlo5VExucXZRVG1wNmdKNjhCS0oxTm9ibnNIaUFjS2M2aEk3a21ZOUNvc2htQWk1cQppS1lCZ3p2L3RoamkwMDkzdnRWU2E5aXdIaFFwK0FFSU1oa3ZNNVpaa2lVNWVFNk1UOVNCRWNWV0ttV0YyOFVzCkIwNGRhWXdTMk1LSjVsNmQ0bjBMVWRBRzBGQnQxbENvVDlyd1VEajlsM01xbWk5NTNndzI2TFVyTnJZbk0vOE4KMmpsN0N1eXc1YWxJV2FVRHJ0NWkrcHU4d2RXZnpWaytmTzd4OFFJREFRQUJBb0lCQUNXOUtpR0g3UWlmV1FtOAoxdkNRMjBYbjNQRW5Xd2pleEs5RDZqUnhJQmN1a0RNWXp6R3BtMlJmTTdjMTJBbWtRNU5kcEVHRVQ3Qno0YmllCi9ycFVBZ0k0cDRTSnY0czJjMTJ6NUhkQm9Ba3M4eVRqcUdUUWtYdXZJOTN3Unh2U0ZtOG41Z0tmN0hXcFY4MmsKK1ZSTlBYdTdGdDl0MjNwR2hKMjFTSURkOVB3Z0VvQ08rMkhXaUlSdmowLzZVZ2puTmRURTVBU2FmbFlVOWJSTApaVVFFZS9HOVVWMnhvdGFGQjBYbnV3dHlHdkNDa055K1hUOFVmVWJHTEVkanV1dmRQSkJJdkdYTGlJM0d5TGtUCncxMkIyUHVUY0p0NWw3bVpaTW84bjhTVjdDQVBJcWdhMEswcVJ4b0FKOVUxWlN3MGdpankyNUVjd1VVYndGdE0KeS8zQTA3a0NnWUVBNnYvV01jZlVJcndMbE1jWFJMdEZUdVFzODJXQlZvUThRRjB0LzhHTXFOV0o2RDdWTS81UgpncnpGcWRKTm1tbXRQaWsxNzluMFFsQmtyb3gxdXlYNk9KTGpENUdyZ0lhRjhmVUNRSEk2Q3RKelcyQkdxcUNjCmpLRVNBL2tHL1Z4NUkvcFpnVTVqRllBQ3drL1F5QjNwQjQvek4yK3dwSjIzMXI5YkRtZjc2VU1DZ1lFQTJ5aEwKai93MWtHdlF3dVo1K2FZUncveTFtU3h2M0RFZUYwNXNEUDNHMzBjVFArbHRQdnlpd0tIYytHcGhrYzFaVnVtcwplSk51Z1VoUEdiZjBvc2dOUTBFVTNoWmFIYnFLbG9saGVIVEhpTGNjTm1wdnNTMFJNTGFEZHhZTENSN3RzL3p2CjlHbWdRRGpDMUZ5RHlNb2hPQytqcHhENG54UDZXd0RQQkd6T1dyc0NnWUJvWFFBU0VZMjdjMG10WkJHSVc2YTMKRjR5Qzk1Uk9DT0g5TDZTTERiamJEaUoxdXRuU2dtSk9ybWtXTnloeHAvYmhRVW9MMy82aktMTGQ5ZEVmTVFtRAo4Ry9ZYlRmK0YwQUxLV1U3bTc3OUtXTkR0R2tOdzlCaWYvSFJROW4vMFA2WHBNbjFMQXZDODV2R0VjNUY0VXFwCktodTRoZWlyNHNOUElxSmNybjlvb3dLQmdRRFdnS1VjdzcxbG9yRk00aXJHdDgyeHB4blRWMVFSRDU0bEw2UEEKUDIrMFFDdk9KVm5CQ2FMK3FpN09rY2ozM1NheFU3dDB0TkMzQnlBc3l3bE5ma0xFN2VGUDBrN1h4Q283VnpHbwpwWnhhNnVaWk9Hb2ozemx4RldLdFgwemoyM3JLN1NtcmJ1cmprT3JZMGNoRzhkZjcxMXoxZXozczdhL2YvSFdqCkhpQUhXd0tCZ1FEbUVTZFFzekdFQ28xaGQ5bEdyTzRZVU1jaDlVQ09KMWVlZzY4cXU1ZjFGcnA5cVg4RFMyUmEKamh3OTZwdE9XczU4elNyNVBXaHdMRGp4WDFGRnp1N0tkQm51UlN6RXNOYmpEWjdyWEZYRE05K3lnR05uenFCTgpzYUN6T0ExUHg5amFnNDNoZ3JEckZOVVhrVXRid1NmdU5pUnNBWFMxZmZhN21DbFNqbGo0ZUE9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=
kind: Secret
metadata:
  creationTimestamp: "2022-03-28T02:15:37Z"
  labels:
    secrets-store.csi.k8s.io/managed: "true"
  name: tls-plaintext
  namespace: default
  ownerReferences:
  - apiVersion: apps/v1
    kind: ReplicaSet
    name: busybox-deployment-tls-656dfb6d85
    uid: d1a66939-cd8e-40b4-8487-ad5aed859880
  resourceVersion: "56816"
  uid: 17b05390-cf84-4d09-b54a-00781f06064f
type: kubernetes.io/tls

kubectl get secrets tls-json -o yaml     
apiVersion: v1
data:
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURPVENDQWlHZ0F3SUJBZ0lKQVAwSjVaN04wWTVmTUEwR0NTcUdTSWIzRFFFQkN3VUFNRE14RnpBVkJnTlYKQkFNTURtUmxiVzh1WVhwMWNtVXVZMjl0TVJnd0ZnWURWUVFLREE5aGEzTXRhVzVuY21WemN5MTBiSE13SGhjTgpNakF3TkRFMU1EUXlNelEyV2hjTk1qRXdOREUxTURReU16UTJXakF6TVJjd0ZRWURWUVFEREE1a1pXMXZMbUY2CmRYSmxMbU52YlRFWU1CWUdBMVVFQ2d3UFlXdHpMV2x1WjNKbGMzTXRkR3h6TUlJQklqQU5CZ2txaGtpRzl3MEIKQVFFRkFBT0NBUThBTUlJQkNnS0NBUUVBeVMzWmt5M244SmxMQnhQTHpnVXBLWll4dnpSYWRlV0xtV1ZiSzlieQpvMDhTMFNzOEphbzdBeTF3SHRuTGJuNTJyekNYNklYMXNBZTFUQVQ3NTVHazdKdExNa3NodGo2RjhCTmVlbEV5CkUxZ3NCRTVudFk1dnlMVG0valpVSUt6Mlo5VExucXZRVG1wNmdKNjhCS0oxTm9ibnNIaUFjS2M2aEk3a21ZOUMKb3NobUFpNXFpS1lCZ3p2L3RoamkwMDkzdnRWU2E5aXdIaFFwK0FFSU1oa3ZNNVpaa2lVNWVFNk1UOVNCRWNWVwpLbVdGMjhVc0IwNGRhWXdTMk1LSjVsNmQ0bjBMVWRBRzBGQnQxbENvVDlyd1VEajlsM01xbWk5NTNndzI2TFVyCk5yWW5NLzhOMmpsN0N1eXc1YWxJV2FVRHJ0NWkrcHU4d2RXZnpWaytmTzd4OFFJREFRQUJvMUF3VGpBZEJnTlYKSFE0RUZnUVV3RkJiUjAxNE1jRVRkckdHa2xwRVFjbDcxUTB3SHdZRFZSMGpCQmd3Rm9BVXdGQmJSMDE0TWNFVApkckdHa2xwRVFjbDcxUTB3REFZRFZSMFRCQVV3QXdFQi96QU5CZ2txaGtpRzl3MEJBUXNGQUFPQ0FRRUFUZ1R5CmdnMVE2SVNTZWtpQkNlMTJkcVVUTUZRaDlHS3BmWVdLUmJNdGpPanBjN01kd2tkbW0zRnU2bDNSZkVGVDI4SWoKZnk5N0xNWXY4VzdiZWVtREZxZG1uZWIydzJ3dzBaQUZKZytHcUlKWjlzL0phZGlGQkROVTdDbUpNaEEyMjVRegpYQzhvdmVqaWVQc2xuTDRRSldsaFZHOTNabEJKNlNEa1JnZmNvSVcyeDRJQkU2d3Y3am1SRjRsT3ZiM3oxZGRQCmlQUXFoYkVFYndNcFhtV3Y3LzJSbmpBSGRqZEdhV1JNQzUrQ2FJK2xxSHlqNmlyMWMrZTZ1MVFVWTU0cWptZ00Ka29OL2ZycVlhYjVFazNrYXVqMWlxVzdyUGtyRkNxVDJldmgwWVJxYjFiRnNDTEpyUk54bk9aNXdLWFYvT1lRYQpRWDV0MHdGR0NaMEtsYlhEaXc9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0t
  tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUV2Z0lCQURBTkJna3Foa2lHOXcwQkFRRUZBQVNDQktnd2dnU2tBZ0VBQW9JQkFRREpMZG1UTGVmd21Vc0gKRTh2T0JTa3BsakcvTkZwMTVZdVpaVnNyMXZLalR4TFJLendscWpzRExYQWUyY3R1Zm5hdk1KZm9oZld3QjdWTQpCUHZua2FUc20wc3lTeUcyUG9Yd0UxNTZVVElUV0N3RVRtZTFqbS9JdE9iK05sUWdyUFpuMU11ZXE5Qk9hbnFBCm5yd0VvblUyaHVld2VJQndwenFFanVTWmowS2l5R1lDTG1xSXBnR0RPLysyR09MVFQzZSsxVkpyMkxBZUZDbjQKQVFneUdTOHpsbG1TSlRsNFRveFAxSUVSeFZZcVpZWGJ4U3dIVGgxcGpCTFl3b25tWHAzaWZRdFIwQWJRVUczVwpVS2hQMnZCUU9QMlhjeXFhTDNuZUREYm90U3MydGljei93M2FPWHNLN0xEbHFVaFpwUU91M21MNm03ekIxWi9OCldUNTg3dkh4QWdNQkFBRUNnZ0VBSmIwcUlZZnRDSjlaQ2J6VzhKRGJSZWZjOFNkYkNON0VyMFBxTkhFZ0Z5NlEKTXhqUE1hbWJaRjh6dHpYWUNhUkRrMTJrUVlSUHNIUGh1SjcrdWxRQ0FqaW5oSW0vaXpaelhiUGtkMEdnQ1N6egpKT09vWk5DUmU2OGozZkJIRzlJV2J5Zm1BcC9zZGFsWHphVDVWRTA5ZTdzVzMyM2Jla2FFbmJWSWdOMzAvQ0FTCmdJNzdZZGFJaEcrUFQvcFNDT2MxMU1Ua0JKcCtWaFQxdEV0bFJBUjc4YjFSWGJHaTFvVUhSZWU3QzNJYThJS1EKM0w1ZFB4UjlSc1lzUjJPNjY5MDhrRWk4WmN1SWpjYkl1UlBEWFlIWSs1TndtM21YdVpsa3lqeWZ4SlhzSUE4aQpxQnJRclNwSEdnQW4xVFZsTERTQ0tQTGJrUnpCUlJ2QVcwekwvY0RUdVFLQmdRRHEvOVl4eDlRaXZBdVV4eGRFCnUwVk81Q3p6WllGV2hEeEFYUzMvd1l5bzFZbm9QdFV6L2xHQ3ZNV3AwazJhYWEwK0tUWHYyZlJDVUdTdWpIVzcKSmZvNGt1TVBrYXVBaG9YeDlRSkFjam9LMG5OYllFYXFvSnlNb1JJRCtRYjlYSGtqK2xtQlRtTVZnQUxDVDlESQpIZWtIai9NM2I3Q2tuYmZXdjFzT1ovdnBRd0tCZ1FEYktFdVAvRFdRYTlEQzVubjVwaEhEL0xXWkxHL2NNUjRYClRtd00vY2JmUnhNLzZXMCsvS0xBb2R6NGFtR1J6VmxXNmF4NGsyNkJTRThadC9TaXlBMURRUlRlRmxvZHVvcVcKaVdGNGRNZUl0eHcyYW0reExSRXd0b04zRmdzSkh1MnovTy8wYWFCQU9NTFVYSVBJeWlFNEw2T25FUGlmRS9wYgpBTThFYk01YXV3S0JnR2hkQUJJUmpidHpTYTFrRVloYnByY1hqSUwzbEU0STRmMHZwSXNOdU5zT0luVzYyZEtDCllrNnVhUlkzS0hHbjl1RkJTZ3ZmL3FNb3N0MzEwUjh4Q1lQd2I5aHROLzRYUUFzcFpUdWJ2djBwWTBPMGFRM0QKMEdKLzhkRkQyZi9RL3Bla3lmVXNDOEx6bThZUnprWGhTcWtxRzdpRjZLdml3MDhpb2x5dWYyaWpBb0dCQU5hQQpwUnpEdldXaXNVemlLc2EzemJHbkdkTlhWQkVQbmlVdm84QS9iN1JBSzg0bFdjRUpvdjZxTHM2UnlQZmRKckZUCnUzUzAwTGNISUN6TENVMStRc1R0NFUvU1R0ZkVLanRYTWFpbG5GcnE1bGs0YWlQZk9YRVZZcTFmVE9QYmVzcnQKS2F0dTZ1T1E2dGpSeUVieDEvdlhYUFY3UGV6dHI5LzhkYU1lSUFkYkFvR0JBT1lSSjFDek1ZUUtqV0YzMlVhcwo3aGhReHlIMVFJNG5WNTZEcnlxN2wvVVd1bjJwZndOTFpGcU9IRDNxbTA1YXpuek5Ldms5YUhBc09QRmZVVVhPCjdzcDBHZTVGTE1TdzF1TU5udXRjVmNNejM3S0FZMmZPb0UyeG9MTTREVS9IMk5xRGplR0NzT3NVMVJlUlMxdkIKSis0MkpHd0JkTFY5OXJ1WUtWS09XUGg0Ci0tLS0tRU5EIFBSSVZBVEUgS0VZLS0tLS0=
kind: Secret
metadata:
  creationTimestamp: "2022-03-28T02:15:37Z"
  labels:
    secrets-store.csi.k8s.io/managed: "true"
  name: tls-json
  namespace: default
  ownerReferences:
  - apiVersion: apps/v1
    kind: ReplicaSet
    name: busybox-deployment-tls-656dfb6d85
    uid: d1a66939-cd8e-40b4-8487-ad5aed859880
  resourceVersion: "56815"
  uid: 76b63d13-f1fd-41fd-9fc5-ca83a524c53d
type: kubernetes.io/tls

Code

• I'm going to try to go over the code changes in more details sometime this week, but this snippet is probably the most important highlight -- GetSecretData in pkg/util/secretutil/secret.go.
• It takes in the format and jsonPath as parameters, and contains conditional logic for handling json and plaintext formats (plaintext being the default case).

Snippet

func GetSecretData(secretObjData []*secretsstorev1.SecretObjectData, secretType corev1.SecretType, files map[string]string, format, jsonPath string) (map[string][]byte, error) {
	datamap := make(map[string][]byte)
	for _, data := range secretObjData {
		objectName := strings.TrimSpace(data.ObjectName)
		dataKey := strings.TrimSpace(data.Key)

		if len(objectName) == 0 {
			return datamap, fmt.Errorf("object name in secretObjects.data is empty")
		}
		if len(dataKey) == 0 {
			return datamap, fmt.Errorf("key in secretObjects.data is empty")
		}
		file, ok := files[objectName]
		if !ok {
			return datamap, fmt.Errorf("file matching objectName %s not found in the pod", objectName)
		}
		content, err := os.ReadFile(file)
		if err != nil {
			return datamap, fmt.Errorf("failed to read file %s, err: %w", objectName, err)
		}

		// TODO (manedurphy) Take auto-detection into consideration
		switch format {
		case formatJSON:
			var (
				jsonContent map[string]interface{}
				valBytes    []byte
				err         error
			)
			if err = json.Unmarshal(content, &jsonContent); err == nil {
				if jsonPath != "" {
					var (
						jsonPathSplit []string
						valid         bool
					)
					jsonPathSplit = strings.Split(jsonPath, ".")[1:]
					for _, path := range jsonPathSplit {
						if jsonContent, valid = jsonContent[path].(map[string]interface{}); !valid {
							return datamap, fmt.Errorf("invalid json path %s", jsonPath)
						}
					}
				}
				for key, val := range jsonContent {
					switch val := val.(type) {
					case string:
						valBytes = []byte(val)
					default:
						if valBytes, err = json.Marshal(val); err != nil {
							return datamap, fmt.Errorf("failed to marshal value %v, err: %w", val, err)
						}
					}
					datamap[key] = valBytes
				}
				continue
			}
			return datamap, fmt.Errorf("failed to unmarshal JSON file contents %s, err: %w", file, err)
		default:
			datamap[dataKey] = content
			if secretType == corev1.SecretTypeTLS {
				c, err := GetCertPart(content, dataKey)
				if err != nil {
					return datamap, fmt.Errorf("failed to get cert data from file %s, err: %w", file, err)
				}
				datamap[dataKey] = c
			}
			if secretType == corev1.SecretTypeBasicAuth {
				username, password := getBasicAuthCredentials(content)
				delete(datamap, dataKey)

				datamap[basicAuthUsername] = []byte(username)
				datamap[basicAuthPassword] = []byte(password)
			}
		}
	}
	return datamap, nil
}

Additional Notes

• I will be attending this week's meeting to demo this functionality for everyone.
• Please let me know if I need to clarify anything, I wrote this all in a somewhat tired state so I am not confident in its clarity.
• As always, any and all feedback is welcome and appreciated.

[fubar]

That all looks excellent to me, nice work! To confirm, specifying format: plaintext at the top level is equivalent to omitting it because plaintext is the default, correct?

I'd be curious to hear the feedback from the meeting if you're able to share it.

[manedurphy]

That is correct, omitting the top-level plaintext would achieve the same effect as including it. Happy to attend tomorrow's meeting and give a quick demo. I am also curious to hear everyone's thoughts.

[manedurphy]

commenting to keep this alive. the last several weeks have been busy but i will be returning to this soon.

[fubar]

What feedback did you get on this in the meeting?

[manedurphy]

@fubar that moving forward with the jsonpath would be the best first step, and building on top of that with the syncAll option after further discussion. The jsonpath would be the focus for the next PR I raise, and this one will subsequently be closed.

[fubar]

@manedurphy nice, one step at a time.