Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SyncAll feature #655

Closed
wants to merge 9 commits into from
Closed

SyncAll feature #655

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
acba337
driver can now build the secretObjects.data list after reading secret…
manedurphy Jun 24, 2021
5530055
more descriptive names SyncAll tests in vault.bats
manedurphy Jul 23, 2021
32faac6
only syncing all for type opaque
manedurphy Jul 27, 2021
f4b6267
SyncAll now handleling kubernetes.io/basic-auth secrets
manedurphy Jul 29, 2021
2233fbc
secrets synced with syncAll/syncOptions.syncAll are now delete on pod…
manedurphy Aug 15, 2021
d8a6fb0
driver will now check if secretObjects is an empty list before perfor…
manedurphy Aug 21, 2021
8888b56
cleaner import statements
manedurphy Aug 22, 2021
ab38ab3
secret_object_data_test unit tests added
manedurphy Aug 22, 2021
c863328
removed crd from charts/templates
manedurphy Aug 22, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions apis/v1alpha1/secretproviderclass_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,16 @@ type SecretObject struct {
// annotations of k8s secret object
Annotations map[string]string `json:"annotations,omitempty"`
Data []*SecretObjectData `json:"data,omitempty"`
// SyncAll can sync all secrets defined in the parameters field of SecretProviderClass
SyncAll bool `json:"syncAll,omitempty"`
}

// SyncOptions defines the desired state of synced K8s secret objects and replaces SecretObject
type SyncOptions struct {
// syncs all secrets listed in the parameters field of SecretProviderClass
SyncAll bool `json:"syncAll,omitempty"`
// type of K8s secret object
Type string `json:"type,omitempty"`
}

// SecretProviderClassSpec defines the desired state of SecretProviderClass
Expand All @@ -60,6 +70,7 @@ type SecretProviderClassSpec struct {
// Configuration for specific provider
Parameters map[string]string `json:"parameters,omitempty"`
SecretObjects []*SecretObject `json:"secretObjects,omitempty"`
SyncOptions SyncOptions `json:"syncOptions,omitempty"`
}

// ByPodStatus defines the state of SecretProviderClass as seen by
Expand Down
13 changes: 13 additions & 0 deletions charts/secrets-store-csi-driver/crds/secrets-store.csi.x-k8s.io_secretproviderclasses.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,16 @@ spec:
provider:
description: Configuration for provider name
type: string
syncOptions:
description: SyncOptions defines the secret type when syncing all secrets listed in the parameters field
properties:
syncAll:
description: SyncAll as true will sync all secrets defined in the parameters field to K8s
type: boolean
type:
description: type of K8s secret object
type: string
type: object
secretObjects:
items:
description: SecretObject defines the desired state of synced K8s
Expand All @@ -55,6 +65,9 @@ spec:
type: string
description: annotations of k8s secret object
type: object
syncAll:
description: SyncAll can sync all secrets defined in the parameters field of SecretProviderClass
type: boolean
data:
items:
description: SecretObjectData defines the desired state of
Expand Down
13 changes: 13 additions & 0 deletions config/crd/bases/secrets-store.csi.x-k8s.io_secretproviderclasses.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,16 @@ spec:
provider:
description: Configuration for provider name
type: string
syncOptions:
description: SyncOptions defines the secret type when syncing all secrets listed in the parameters field
properties:
syncAll:
description: SyncAll as true will sync all secrets defined in the parameters field to K8s
type: boolean
type:
description: type of K8s secret object
type: string
type: object
secretObjects:
items:
description: SecretObject defines the desired state of synced K8s
Expand All @@ -55,6 +65,9 @@ spec:
type: string
description: annotations of k8s secret object
type: object
syncAll:
description: SyncAll can sync all secrets defined in the parameters field of SecretProviderClass
type: boolean
data:
items:
description: SecretObjectData defines the desired state of
Expand Down
72 changes: 51 additions & 21 deletions controllers/secretproviderclasspodstatus_controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,34 +23,29 @@ import (
"sync"
"time"

"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/types"
"k8s.io/apimachinery/pkg/util/wait"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/record"
"k8s.io/klog/v2"
"sigs.k8s.io/controller-runtime/pkg/client/apiutil"
"sigs.k8s.io/controller-runtime/pkg/event"
"sigs.k8s.io/controller-runtime/pkg/predicate"

"sigs.k8s.io/controller-runtime/pkg/manager"

"k8s.io/client-go/tools/record"
"k8s.io/klog/v2"

"sigs.k8s.io/controller-runtime/pkg/predicate"
"sigs.k8s.io/secrets-store-csi-driver/apis/v1alpha1"
"sigs.k8s.io/secrets-store-csi-driver/pkg/client/clientset/versioned/scheme"
"sigs.k8s.io/secrets-store-csi-driver/pkg/util/fileutil"
"sigs.k8s.io/secrets-store-csi-driver/pkg/util/k8sutil"
"sigs.k8s.io/secrets-store-csi-driver/pkg/util/secretutil"

ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/secrets-store-csi-driver/pkg/util/spcutil"

corev1 "k8s.io/api/core/v1"
v1 "k8s.io/api/core/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
apiruntime "k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/types"
"k8s.io/apimachinery/pkg/util/wait"
clientcorev1 "k8s.io/client-go/kubernetes/typed/core/v1"
ctrl "sigs.k8s.io/controller-runtime"
client "sigs.k8s.io/controller-runtime/pkg/client"
)

const (
Expand All @@ -65,7 +60,7 @@ const (
type SecretProviderClassPodStatusReconciler struct {
client.Client
mutex *sync.Mutex
scheme *apiruntime.Scheme
scheme *runtime.Scheme
nodeID string
reader client.Reader
writer client.Writer
Expand Down Expand Up @@ -121,7 +116,7 @@ func (r *SecretProviderClassPodStatusReconciler) Patcher(ctx context.Context) er
}

spcPodStatuses := spcPodStatusList.Items
for i := range spcPodStatuses {
for i, spcPodStatus := range spcPodStatuses {
spcName := spcPodStatuses[i].Status.SecretProviderClassName
spc := &v1alpha1.SecretProviderClass{}
namespace := spcPodStatuses[i].Namespace
Expand All @@ -135,11 +130,12 @@ func (r *SecretProviderClassPodStatusReconciler) Patcher(ctx context.Context) er
spcMap[namespace+"/"+spcName] = *spc
}
// get the pod and check if the pod has a owner reference
pod := &v1.Pod{}
pod := &corev1.Pod{}
err = r.reader.Get(ctx, client.ObjectKey{Namespace: namespace, Name: spcPodStatuses[i].Status.PodName}, pod)
if err != nil {
return fmt.Errorf("failed to fetch pod during patching, err: %+v", err)
}

var ownerRefs []metav1.OwnerReference
for _, ownerRef := range pod.GetOwnerReferences() {
ownerRefs = append(ownerRefs, metav1.OwnerReference{
Expand All @@ -149,6 +145,7 @@ func (r *SecretProviderClassPodStatusReconciler) Patcher(ctx context.Context) er
Name: ownerRef.Name,
})
}

// If a pod has no owner references, then it's a static pod and
// doesn't belong to a replicaset. In this case, use the spcps as
// owner reference just like we do it today
Expand All @@ -167,6 +164,19 @@ func (r *SecretProviderClassPodStatusReconciler) Patcher(ctx context.Context) er
ownerRefs = append(ownerRefs, ref)
}

if spc.Spec.SyncOptions.SyncAll {
files, err := fileutil.GetMountedFiles(spcPodStatus.Status.TargetPath)
if err != nil {
return fmt.Errorf("failed to get mounted files for pod %s/%s: %v", namespace, pod.Name, err)
} else {
if len(spc.Spec.SecretObjects) == 0 {
spc.Spec.SecretObjects = spcutil.BuildSecretObjects(files, secretutil.GetSecretType(strings.TrimSpace(spc.Spec.SyncOptions.Type)))
} else {
spc.Spec.SecretObjects = append(spc.Spec.SecretObjects, spcutil.BuildSecretObjects(files, secretutil.GetSecretType(strings.TrimSpace(spc.Spec.SyncOptions.Type)))...)
}
}
}

for _, secret := range spc.Spec.SecretObjects {
key := types.NamespacedName{Name: secret.SecretName, Namespace: namespace}
val, exists := secretOwnerMap[key]
Expand Down Expand Up @@ -238,7 +248,7 @@ func (r *SecretProviderClassPodStatusReconciler) Reconcile(ctx context.Context,

// Obtain the full pod metadata. An object reference is needed for sending
// events and the UID is helpful for validating the SPCPS TargetPath.
pod := &v1.Pod{}
pod := &corev1.Pod{}
if err := r.reader.Get(ctx, client.ObjectKey{Namespace: req.Namespace, Name: spcPodStatus.Status.PodName}, pod); err != nil {
klog.ErrorS(err, "failed to get pod", "pod", klog.ObjectRef{Namespace: req.Namespace, Name: spcPodStatus.Status.PodName})
if apierrors.IsNotFound(err) {
Expand All @@ -249,7 +259,7 @@ func (r *SecretProviderClassPodStatusReconciler) Reconcile(ctx context.Context,
// skip reconcile if the pod is being terminated
// or the pod is in succeeded state (for jobs that complete aren't gc yet)
// or the pod is in a failed state (all containers get terminated)
if !pod.GetDeletionTimestamp().IsZero() || pod.Status.Phase == v1.PodSucceeded || pod.Status.Phase == v1.PodFailed {
if !pod.GetDeletionTimestamp().IsZero() || pod.Status.Phase == corev1.PodSucceeded || pod.Status.Phase == corev1.PodFailed {
klog.V(5).InfoS("pod is being terminated, skipping reconcile", "pod", klog.KObj(pod))
return ctrl.Result{}, nil
}
Expand All @@ -264,7 +274,7 @@ func (r *SecretProviderClassPodStatusReconciler) Reconcile(ctx context.Context,
return ctrl.Result{}, err
}

if len(spc.Spec.SecretObjects) == 0 {
if len(spc.Spec.SecretObjects) == 0 && !spc.Spec.SyncOptions.SyncAll {
klog.InfoS("no secret objects defined for spc, nothing to reconcile", "spc", klog.KObj(spc), "spcps", klog.KObj(spcPodStatus))
return ctrl.Result{}, nil
}
Expand All @@ -289,6 +299,25 @@ func (r *SecretProviderClassPodStatusReconciler) Reconcile(ctx context.Context,
klog.ErrorS(err, "failed to get mounted files", "spc", klog.KObj(spc), "pod", klog.KObj(pod), "spcps", klog.KObj(spcPodStatus))
return ctrl.Result{RequeueAfter: 10 * time.Second}, err
}

if spc.Spec.SyncOptions.SyncAll {
if len(spc.Spec.SecretObjects) == 0 {
spc.Spec.SecretObjects = spcutil.BuildSecretObjects(files, secretutil.GetSecretType(strings.TrimSpace(spc.Spec.SyncOptions.Type)))
} else {
spc.Spec.SecretObjects = append(spc.Spec.SecretObjects, spcutil.BuildSecretObjects(files, secretutil.GetSecretType(strings.TrimSpace(spc.Spec.SyncOptions.Type)))...)
}
}

for _, secretObj := range spc.Spec.SecretObjects {
if secretObj.SyncAll {
if secretutil.GetSecretType(strings.TrimSpace(secretObj.Type)) != corev1.SecretTypeOpaque {
return ctrl.Result{}, fmt.Errorf("secret provider class %s/%s cannot use secretObjects[*].syncAll for non-opaque secrets", spc.Namespace, spc.Name)
}

spcutil.BuildSecretObjectData(files, secretObj)
}
}

errs := make([]error, 0)
for _, secretObj := range spc.Spec.SecretObjects {
secretName := strings.TrimSpace(secretObj.SecretName)
Expand Down Expand Up @@ -428,6 +457,7 @@ func (r *SecretProviderClassPodStatusReconciler) createK8sSecret(ctx context.Con
}

err := r.writer.Create(ctx, secret)

if err == nil {
klog.InfoS("successfully created Kubernetes secret", "secret", klog.ObjectRef{Namespace: namespace, Name: name})
return nil
Expand Down Expand Up @@ -482,7 +512,7 @@ func (r *SecretProviderClassPodStatusReconciler) patchSecretWithOwnerRef(ctx con

// secretExists checks if the secret with name and namespace already exists
func (r *SecretProviderClassPodStatusReconciler) secretExists(ctx context.Context, name, namespace string) (bool, error) {
o := &v1.Secret{}
o := &corev1.Secret{}
secretKey := types.NamespacedName{
Namespace: namespace,
Name: name,
Expand Down
13 changes: 13 additions & 0 deletions deploy/secrets-store.csi.x-k8s.io_secretproviderclasses.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,16 @@ spec:
provider:
description: Configuration for provider name
type: string
syncOptions:
description: SyncOptions defines the secret type when syncing all secrets listed in the parameters field
properties:
syncAll:
description: SyncAll as true will sync all secrets defined in the parameters field to K8s
type: boolean
type:
description: type of K8s secret object
type: string
type: object
secretObjects:
items:
description: SecretObject defines the desired state of synced K8s
Expand All @@ -55,6 +65,9 @@ spec:
type: string
description: annotations of k8s secret object
type: object
syncAll:
description: SyncAll can sync all secrets defined in the parameters field of SecretProviderClass
type: boolean
data:
items:
description: SecretObjectData defines the desired state of
Expand Down
13 changes: 13 additions & 0 deletions ...harts/secrets-store-csi-driver/crds/secrets-store.csi.x-k8s.io_secretproviderclasses.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,16 @@ spec:
provider:
description: Configuration for provider name
type: string
syncOptions:
description: SyncOptions defines the secret type when syncing all secrets listed in the parameters field
properties:
syncAll:
description: SyncAll as true will sync all secrets defined in the parameters field to K8s
type: boolean
type:
description: type of K8s secret object
type: string
type: object
secretObjects:
items:
description: SecretObject defines the desired state of synced K8s
Expand All @@ -55,6 +65,9 @@ spec:
type: string
description: annotations of k8s secret object
type: object
syncAll:
description: SyncAll can sync all secrets defined in the parameters field of SecretProviderClass
type: boolean
data:
items:
description: SecretObjectData defines the desired state of
Expand Down
13 changes: 13 additions & 0 deletions manifest_staging/deploy/secrets-store.csi.x-k8s.io_secretproviderclasses.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,16 @@ spec:
provider:
description: Configuration for provider name
type: string
syncOptions:
description: SyncOptions defines the secret type when syncing all secrets listed in the parameters field
properties:
syncAll:
description: SyncAll as true will sync all secrets defined in the parameters field to K8s
type: boolean
type:
description: type of K8s secret object
type: string
type: object
secretObjects:
items:
description: SecretObject defines the desired state of synced K8s
Expand All @@ -55,6 +65,9 @@ spec:
type: string
description: annotations of k8s secret object
type: object
syncAll:
description: SyncAll can sync all secrets defined in the parameters field of SecretProviderClass
type: boolean
data:
items:
description: SecretObjectData defines the desired state of
Expand Down
Loading