Add support for cloud workload identity

[ThorstenHans]

Many cloud providers offer seamless integration with their identity story, which is often referred to as "Workload Identity". Although SpinKube should provide a cloud-agnostic implementation, I'll layout the user story and explain the concept using Azure services and Azure Workload Identity.

User Story

As an Spin App developer I want to use cloud services provided by Azure without having to specify access keys, passwords or tokens manually. Instead, I want to assign an Azure Identity to my Spin App and configure RBAC in Azure to allow that particular Azure Identity interacting with chosen Azure Services.

Azure Workload Identity

By deploying Azure Workload Identity to an Azure Kubernetes Service (AKS) cluster, and configuring the cluster itself accordingly, workloads could obtain tokens ( see concept) for interacting with Azure Services.

From an app developers point of view, two things must be done, to make a workload use Azure Workload Identity:

• the correct serviceAccountName must be specified on the Pod
• the label azure.workload.identity/use: "true" must be specified on the Pod

Spin Host vs Spin App

In the context of Spin, there are two layers in which users would want to use Azure Workload Identity in order to streamline and harden integrations and interactions with Azure Services:

• Spin Host level:
  • Using Azure Key Vault as Application Variable Provider
  • Using Azure CosmosDB or Azure Cache for Redis as key-value store
• Spin App level:
  • Accessing the data plane of various Azure Services e.g. Azure Storage Account, ...

[rajatjindal]

one more use case I have in mind is to use RBAC/ServiceAccount provided by Kubernetes in our spin apps to be able to talk to Kubernetes api server.

[devigned]

During my time working on adding auth support for Azure Workload Identity in Spin, I've been thinking about how best to provide users a great experience via the operator. I believe that a user should be able to kubectl apply their workload and have the Azure (or other cloud provider) resources created on their behalf rather than needing to provision Azure resources out of the normal Kubernetes flow to enable their workload. For example, with a Workload Identity enabled AKS cluster, a workload needs the following resource to grant KV access to CosmosDB, a User Assigned Identity, Federated Identity Credential, and a CosmosDB Role Assignment (https://github.com/devigned/spin-workload-id/blob/e35152bbaf22c43bf422ca0e522f9f05a9796cfd/infra/main.tf#L54-L137). The User Assigned Identity is then used to link the K8s ServiceAccount via the User Assigned Identity "Client ID".

There are projects that exist to provision cloud resources in K8s (Azure Service Operator, Crossplane, etc.). However, we need to weight the cost / benefit of depending upon another operator to handle the provisioning of these cloud resources.

In my opinion, we should orchestrate the provisioning of all of these identity resources for each application. Each application should have its own identity and constrained set of rights. Users should not see passwords or worry about secrets. It should just work. Thoughts?