one IAMServiceAccountKey could create multiple service account keys

[lazy-slackoth]

Checklist

• I did not find a related open issue.
• I did not find a solution in the troubleshooting guide: (https://cloud.google.com/config-connector/docs/troubleshooting)
• If this issue is time-sensitive, I have submitted a corresponding issue with GCP support.

Bug Description

How it looks like for me:

I have a pipeline where I create IAMServiceAccount a bit later ServiceAccount/IAMPartialPolicy/IAMServiceAccountKey.

Sometimes I have the following situation IAMServiceAccountKey has Ready + UpToDate status, but if you open CRD description it doesn't have any status(private_key,public_key) at the same time you can find a new service account key.
Then it seems like it stays in this state until new reconcile and only after it create new SA key and CRD will be updated.
Looks like old service account key was not tracked(?).
When I destroy my env with all CRDs mentioned above I still have this key. Quite rare I had a situation where I had 2 keys after create/destroy process.

I played with creation intervals between IAMServiceAccount and IAMServiceAccountKey, but didn't find any connection.

Due to limitation of 10 keys per one service account we have to constantly manually delete these "untracked" keys.

Additional Diagnostic Information

Status IAMServiceAccountKey example until new reconcile,but new SA key could be found in GCP:

status:
  conditions:
    - lastTransitionTime: '2024-12-09T14:06:44Z'
      message: Update in progress
      reason: Updating
      status: 'False'
      type: Ready

Nothing special in logs:

Kubernetes Cluster Version

v1.30.4

Config Connector Version

1.105.0

Config Connector Mode

cluster mode

Log Output

No response

Steps to reproduce the issue

In my case It's enough to recreate resource from yaml snippet multiple times(sometimes 2, sometimes 6) to get "untrackable" service account key/s.

YAML snippets

apiVersion: iam.cnrm.cloud.google.com/v1beta1
kind: IAMServiceAccount
metadata:
  annotations:
    cnrm.cloud.google.com/deletion-policy: abandon
    cnrm.cloud.google.com/management-conflict-prevention-policy: none
    cnrm.cloud.google.com/project-id: x
  name: sa-project-qa
  namespace: project-qa1
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: project
  namespace: project-qa1
  annotations:
    iam.gke.io/gcp-service-account: sa-project-qa@x.iam.gserviceaccount.com
---
apiVersion: iam.cnrm.cloud.google.com/v1beta1
kind: IAMPartialPolicy
metadata:
  name: sa-project-qa-project-binding
  namespace: project-qa1
spec:
  bindings:
    - members:
        - member: serviceAccount:x.svc.id.goog[project-qa1/project]
      role: roles/iam.workloadIdentityUser
  resourceRef:
    apiVersion: iam.cnrm.cloud.google.com/v1beta1
    kind: IAMServiceAccount
    name: sa-project-qa
---
apiVersion: iam.cnrm.cloud.google.com/v1beta1
kind: IAMServiceAccountKey
metadata:
  name: sa-project-qa-project-qa1-key
  namespace: project-qa1
spec:
  keyAlgorithm: KEY_ALG_RSA_2048
  privateKeyType: TYPE_GOOGLE_CREDENTIALS_FILE
  publicKeyType: TYPE_X509_PEM_FILE
  serviceAccountRef:
    name: sa-project-qa