Demo of Tanzu platform and SpringAI

This repository contains artifacts necessary to build and run generative AI applications using Spring Boot and Tanzu Platform. The instructions below cover setup for both Cloud Foundry (cf) and Kubernetes (k8s) environments.

Architecture

Prerequisites

• Ensure you have the latest version of the Tanzu CLI installed.
• Access to a Route53 domain and necessary AWS permissions.
• Configured egress settings (closed by default) to connect to external services.

Installation

Cloud Foundry Runtime

Preperations

Update the following in demo.sh according to your TPCF configurations

PGVECTOR_SERVICE_NAME="vector-db"
PGVECTOR_PLAN_NAME="on-demand-postgres-db"
PGVECTOR_EXTERNAL_PORT=1025 # Need TCP Router on the TPCF foundation enabled, and Service Gateways on the Postgres tile enabled.  Choose an available port 

GENAI_CHAT_SERVICE_NAME="genai-chat" 
GENAI_CHAT_PLAN_NAME="meta-llama/Meta-Llama-3-8B-Instruct" # plan must have chat capabilty

GENAI_EMBEDDINGS_SERVICE_NAME="genai-embed" 
GENAI_EMBEDDINGS_PLAN_NAME="nomic-embed-text" # plan must have Embeddings capabilty

Build

mvn clean package -DskipTests

Deployment

Run the demo script to create all services and push the spring-metal application

cf login -u admin -p YOUR_CF_ADMIN_PASSWORD
cf target -o YOUR_ORG -s YOUR_SPACE

./demo.sh cf

Notes:

• if your Cloud Foundry Runtime srrvices are hosted on a private network, you will need to create or update your postgres service with the TCP Router and Service instance gateway. Documentation
• The contents of your Kubernetes service secret can be viewed through the service key.

Kubernetes Runtime

Preperations

• Ensure the CF runtime services are installed and your CF CLI is targeted to the org/space you used above.
• Ensure you're logged into the tanzu platform and your kubernetes context is set to your space
• Template the Kubernetes services and bindings

tanzu login
tanzu context use <my-context>
tanzu project use <my-project>
tanzu space use <my-space>
export KUBECONFIG=~/.config/tanzu/kube/config 
./demo.sh prepare-k8s

Deployment - all in one

./demo.sh k8s

Deployment - step by step

Build

Follow these commands to build your application:

tanzu build config --containerapp-registry [YOUR CONTAINER REGISTRY] 
tanzu build -o build-output

Deploy

Follow these commands to deploy your application from the build-output folder:

tanzu deploy --from-build build-output

Bind

Create and bind the pre-provisioned services :

Create secrets to external Postgres (with pgvector) and GenAI control apis running on TPCF and bind them as pre-provisioned services

tanzu context use <my-context>
kubectl apply -f .tanzu/services

Cleanup

./demo.sh cleanup

Troubleshooting

Issue: Problem with external service binding.

• Solution: Ensure that all credentials and connection details in .tanzu/config/services are correct and updated.

Issue: Application deployment fails.

• Solution: Check the build output for errors and verify the Tanzu configuration settings.

Browse your application through the app ingress link provided in the Space UI after deployment.

Contributing

Contributions to this project are welcome. Please ensure to follow the existing coding style and add unit tests for any new or changed functionality.