Remove support for legacy user-provided service instance credentials

[danail-branekov]

In 0.12.0 we introduced support for objects in user-provided services credentials (#2900). When we have been working on that, we wanted to avoid workloads restart on korifi upgrade in order to ensure smooth upgrade path, and came up with some legacy support code.

With the upcoming release the smooth upgrade is not going to be possible as workloads have to restart, see

• Document the bump to gateway API 1.1.0 as a breaking change #3410
• Set CF_INSTANCE_INDEX env variable with downward API #3483

As there is no way around that, let's make use of it and get rid of the legacy secret format support:

• korifi/controllers/controllers/services/instances/upsi/controller.go

  Lines 151 to 183 in f9b24a3

  	log := logr.FromContextOrDiscard(ctx)
  	log.Info("migrating legacy secret", "legacy-secret-name", credentialsSecret.Name)
  	migratedSecret := &corev1.Secret{
  	ObjectMeta: metav1.ObjectMeta{
  	Name: cfServiceInstance.Name + "-migrated",
  	Namespace: cfServiceInstance.Namespace,
  	},
  	}
  	_, err := controllerutil.CreateOrPatch(ctx, r.k8sClient, migratedSecret, func() error {
  	migratedSecret.Type = corev1.SecretTypeOpaque
  	data := map[string]any{}
  	for k, v := range credentialsSecret.Data {
  	data[k] = string(v)
  	}
  	dataBytes, err := json.Marshal(data)
  	if err != nil {
  	log.Error(err, "failed to marshal legacy credentials secret data")
  	return err
  	}
  	migratedSecret.Data = map[string][]byte{
  	tools.CredentialsSecretKey: dataBytes,
  	}
  	return controllerutil.SetOwnerReference(cfServiceInstance, migratedSecret, r.scheme)
  	})
  	if err != nil {
  	log.Error(err, "failed to create migrated credentials secret")
  	return nil, err
  	}
  	return migratedSecret, nil

• korifi/controllers/controllers/services/bindings/upsi/controller.go

  Lines 90 to 106 in f9b24a3

  	if isLegacyServiceBinding(cfServiceBinding, cfServiceInstance) {
  	bindingSecret := &corev1.Secret{
  	ObjectMeta: metav1.ObjectMeta{
  	Name: cfServiceBinding.Status.Binding.Name,
  	Namespace: cfServiceBinding.Namespace,
  	},
  	}
  	// For legacy sevice bindings we want to keep the binding secret
  	// unchanged in order to avoid unexpected app restarts. See ADR 16 for more details.
  	err := r.k8sClient.Get(ctx, client.ObjectKeyFromObject(bindingSecret), bindingSecret)
  	if err != nil {
  	return err
  	}
  	return nil
  	}

[danail-branekov]

Turns out that maybe this would not that trivial :(

Imagine the following scenario:

• UPSI created with 0.11.0. The credentials secret referenced by the UPSI is in the old format

• Korifi bumped to 0.12.0

  • As a result of the bump, the CFServiceInstance.Spec.SecretName resource still references the initial secret (in the old format)
  • However, as a result of the migration, its CFServiceInstance.Status.Credentials.Name is set to the migrated secret with the -migrated suffix). this migrated secret is in the new format
  • NOTE that the service instance spec is not changed at this point

• Now Korifi is bumped to the release candidate

  • The upsi controller reconciles the CFServiceInstance. It now no longer has the knowledge that this might be a pre 0.12.0 upsi, therefore jumps straight into validating the secret referenced by CFServiceInstance.Spec.SecretName. As that secret is in the old format (see above), validation fails

I have not tested yet, but I am pretty sure we would see the same error if we jump from 0.11.0 to the release candidate