Update dependencies; design; tool management

.cursorrules

@@ -0,0 +1,160 @@
+# Agent OS Platform Design
+
+## System Overview
+The Agent OS Platform is a FastAPI-based system for running AI agents with dynamic tool creation and secure execution capabilities.
+
+## Core Architecture
+
+### Components
+1. **Master Container (FastAPI)**
+   - Handles user interactions
+   - Manages agent lifecycles
+   - Coordinates tool execution
+   - Runs on Python 3.13 (slim image)
+
+2. **E2B Integration**
+   - On-demand sandbox creation for tool execution
+   - One sandbox per tool execution
+   - Short-lived containers (destroyed after tool completion)
+   - Maximum execution time: 10 minutes (OpenAI limit)
+   - Firecracker microVM-based isolation
+   - Supports any Linux-compatible framework
+   - Handles package installation and I/O
+
+3. **Redis Message Bus**
+   - Inter-component communication
+   - State management
+   - Event distribution
+
+4. **Tool Registry**
+   - Firestore-based tool storage
+   - Tools stored as standalone Python code
+   - Each tool includes all dependencies
+   - No caching or hot-reloading
+
+### Development Environment
+1. **Local Development**
+   - Docker Compose based setup
+   - E2B for sandbox isolation
+   - Redis for message passing
+   - Firestore emulator for storage
+
+2. **Production Environment**
+   - Similar to local but with real Firestore
+   - E2B for production sandboxes
+   - Potential future K8s migration
+
+## Security Model
+
+### Tool Execution
+1. **E2B Sandbox**
+   - Created on-demand for each tool execution
+   - Destroyed immediately after tool completion
+   - No session pooling or reuse
+   - Clean environment for each execution
+   - Resource limits enforcement
+   - Network isolation
+   - Secure file system access
+
+2. **Tool Validation**
+   - Static code analysis
+   - Security checks
+   - Dependency scanning
+   - Automated testing
+
+### Access Control
+1. **Authentication**
+   - Firebase Authentication
+   - JWT validation
+   - Role-based access
+
+2. **Authorization**
+   - Tool execution permissions
+   - Admin capabilities
+   - User isolation
+
+## System Self-Improvement
+
+### AI-Driven Updates
+1. **Code Generation**
+   - AI creates/modifies platform code
+   - Generates pull requests
+   - Must pass all tests
+   - Manual review required
+
+2. **Tool Creation**
+   - AI generates new tools
+   - Automated validation
+   - Security scanning
+
+### Quality Assurance
+1. **Testing**
+   - Very high coverage requirement
+   - Integration tests
+   - Security tests
+   - Performance benchmarks
+
+2. **Monitoring**
+   - Execution metrics
+   - Error tracking
+   - Resource usage
+   - User analytics
+
+## Implementation Notes
+
+### Current State
+- Single container deployment
+- File-based tool storage
+- Basic security validation
+- Manual tool approval
+
+### Migration Path
+1. **Phase 1: E2B Integration**
+   - Add E2B sandbox support
+   - Implement on-demand container creation
+   - Add container cleanup after tool execution
+   - Implement security checks
+   - Add execution timeouts (10-minute limit)
+
+2. **Phase 2: Message Bus**
+   - Redis integration
+   - State management
+   - Event system
+
+3. **Phase 3: Tool Registry**
+   - Move to pure Firestore storage
+   - Remove file-based storage
+   - Implement standalone tool format
+
+### Design Principles
+1. **Simplicity**
+   - Minimal abstractions
+   - Clear responsibilities
+   - Direct communication
+
+2. **Security**
+   - Isolated execution
+   - Validated tools
+   - Access control
+
+3. **Maintainability**
+   - Clear documentation
+   - High test coverage
+   - Automated improvements
+
+## Maintenance Instructions
+
+This file (.cursorrules) serves as the source of truth for system design.
+When making significant changes:
+
+1. Update this file first
+2. Ensure consistency with implementation
+3. Remove or update other docs if needed
+4. Keep this file in sync with:
+   - README.md (high-level overview)
+   - docker-compose.yml (deployment)
+   - backend/services/* (implementation)
+
+---
+Last Updated: [Current Date]
+Update this file when making architectural changes.

.dockerignore

@@ -0,0 +1,49 @@
+# Version control
+.git
+.gitignore
+
+# Dependencies
+node_modules
+frontend/node_modules
+**/__pycache__
+*.pyc
+*.pyo
+*.pyd
+.Python
+env
+pip-log.txt
+pip-delete-this-directory.txt
+.tox
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.log
+
+# Poetry
+.venv
+dist
+poetry.toml
+poetry/
+
+# Environment variables
+.env
+.env.*
+!.env.example
+
+# Development
+*.md
+LICENSE
+tests/
+*.test.js
+*.spec.js
+.vscode
+.idea
+
+# Build artifacts
+frontend/public
+frontend/.cache
+dist
+*.egg-info/

.env.docker.testing

@@ -0,0 +1 @@
+FIREBASE_CONFIG={"apiKey":"...","authDomain":"...","projectId":"...","storageBucket":"...","messagingSenderId":"...","appId":"...","measurementId":"..."}

.env.testing

@@ -6,4 +6,5 @@
 GOOGLE_CREDENTIALS={"type":"service_account",...}
 ENCRYPTION_KEY=
 MAILCHIMP_API_KEY=
-MAILCHIMP_LIST_ID=
+MAILCHIMP_LIST_ID=
+REDIS_URL=redis://redis:6379/1

.github/workflows/test.yml

@@ -12,7 +12,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: [3.11]
+        python-version: [3.13]
       fail-fast: false
 
     steps:
@@ -26,6 +26,6 @@ jobs:
       - run: poetry install --with dev
       - run: poetry run pytest --cov=backend --cov-report=xml
       - name: Upload coverage reports to Codecov
-        uses: codecov/codecov-action@v4
+        uses: codecov/codecov-action@v5
         env:
           CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

.gitignore

@@ -8,7 +8,6 @@ __pycache__/
 
 # Distribution / packaging
 .Python
-build/
 develop-eggs/
 dist/
 downloads/
@@ -164,7 +163,11 @@ cython_debug/
 settings.json
 
 .env
+.env.docker
 
 # UI folder
 backend/ui/*
 backend/data/agency_data/*
+
+
+.DS_Store

.pre-commit-config.yaml

@@ -1,6 +1,6 @@
 repos:
   - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v4.5.0
+    rev: v5.0.0
     hooks:
       - id: trailing-whitespace
       - id: end-of-file-fixer
@@ -10,14 +10,14 @@ repos:
         language_version: python3
 
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.3.4
+    rev: v0.9.1
     hooks:
       - id: ruff
         args: [--fix]
       - id: ruff-format
 
   - repo: https://github.com/pre-commit/mirrors-mypy
-    rev: v1.9.0
+    rev: v1.14.1
     hooks:
       - id: mypy
         args:

Dockerfile

@@ -0,0 +1,75 @@
+FROM node:20.10.0 AS frontend-builder
+
+# Set working directory
+WORKDIR /app
+
+# Install rsync
+RUN apt-get update && apt-get install -y rsync && rm -rf /var/lib/apt/lists/*
+
+# Copy all package files
+COPY build/ build/
+
+# Copy frontend source
+COPY frontend/ frontend/
+
+# Create required directories
+RUN mkdir -p frontend/src/firebase frontend/static backend/ui
+
+# These environment variables are required for the build
+# They should be passed during docker build using --build-arg
+ARG FIREBASE_CONFIG
+ARG CHATBOT_WIDGET
+
+ENV FIREBASE_CONFIG=${FIREBASE_CONFIG}
+ENV CHATBOT_WIDGET=${CHATBOT_WIDGET}
+
+# Run the pre-build script from the correct location
+RUN node build/pre-build.js
+
+# Install and build frontend with error handling
+RUN cd frontend && \
+    npm install -g gatsby-cli && \
+    yarn install && \
+    yarn build && \
+    cp -r public/* ../backend/ui/
+
+# Backend build stage
+FROM python:3.13.1-slim
+
+# Set working directory
+WORKDIR /app
+
+# Install system dependencies and poetry
+RUN apt-get update && apt-get install -y \
+    build-essential \
+    curl \
+    && rm -rf /var/lib/apt/lists/* \
+    && curl -sSL https://install.python-poetry.org | python3 -
+
+# Add Poetry to PATH
+ENV PATH="/root/.local/bin:$PATH"
+
+# Copy poetry files
+COPY pyproject.toml poetry.lock ./
+
+# Configure poetry to not create a virtual environment
+RUN poetry config virtualenvs.create false
+
+# Install dependencies
+RUN poetry install --no-dev --no-interaction --no-ansi
+
+# Copy backend code
+COPY backend/ backend/
+
+# Copy built frontend from previous stage
+COPY --from=frontend-builder /app/backend/ui backend/ui
+
+# Set environment variables
+ENV PORT=8000
+ENV PYTHONPATH=/app
+
+# Expose port
+EXPOSE 8000
+
+# Command to run the application
+CMD ["gunicorn", "-w", "4", "-k", "uvicorn.workers.UvicornWorker", "--timeout", "120", "--bind", "0.0.0.0:8000", "backend.main:app"]