Skip to main content

CivStart Architecture Overview

System Architecture​

CivStart uses a distributed architecture with components deployed across different platforms for optimal performance and cost-efficiency.

┌─────────────────────────────────────────────────────────────┐
│                    Production Architecture                   │
└─────────────────────────────────────────────────────────────┘

┌──────────────────┐       ┌──────────────────┐
│   Frontend       │       │   Admin Panel    │
│   (Next.js 15)   │       │   (Next.js 15)   │
│                  │       │                  │
│   Vercel         │       │   Vercel         │
│   Auto-deploy    │       │   Auto-deploy    │
└────────┬─────────┘       └────────┬─────────┘
         │                          │
         │ HTTPS API calls          │
         └──────────┬───────────────┘
                    │
                    ▼
         ┌──────────────────────┐
         │   Backend API        │
         │   (NestJS)           │
         │                      │
         │   Production Server  │
         │   PM2 Process        │
         │   Port: 3800         │
         └──────────┬───────────┘
                    │
         ┌──────────┴───────────┐
         │                      │
         ▼                      ▼
┌─────────────────┐    ┌─────────────────┐
│   PostgreSQL    │    │   External APIs │
│   Database      │    │   - Clerk Auth  │
│   Port: 5432    │    │   - Gemini AI   │
└─────────────────┘    │   - Pinecone    │
                       │   - Airtable    │
                       └─────────────────┘

Component Breakdown​

1. Frontend (apps/frontend/)​

Technology: Next.js 15 with App Router Deployment: Vercel (automatic on push to main) Port: N/A (Vercel handles routing) URL: https://your-domain.com

Key Features:

  • Server-side rendering (SSR) for SEO
  • Static generation for performance
  • TanStack Query for API state management
  • Clerk authentication integration
  • Tailwind CSS + ShadCN UI components

Deployment Process:

  1. Push to main branch
  2. Vercel automatically builds and deploys
  3. Preview deployments on pull requests
  4. Environment variables set in Vercel dashboard

Environment Variables (Set in Vercel):

NEXT_PUBLIC_API_URL=https://api.your-domain.com
NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=pk_...
CLERK_SECRET_KEY=sk_...

2. Admin Panel (apps/admin/)​

Technology: Next.js 15 Deployment: Vercel (automatic on push to main) Port: N/A (Vercel handles routing) URL: https://admin.your-domain.com

Key Features:

  • Real-time WebSocket connection to backend
  • Signal approval workflows
  • Connection management interface
  • Admin-only authentication via Clerk

Deployment Process: Same as frontend - automatic Vercel deployment

Environment Variables (Set in Vercel):

NEXT_PUBLIC_API_URL=https://api.your-domain.com
NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=pk_...
CLERK_SECRET_KEY=sk_...

3. Backend API (apps/backend/)​

Technology: NestJS with Express Deployment: Production server via PM2 Port: 3800 URL: https://api.your-domain.com (or via reverse proxy)

Key Features:

  • RESTful API with 50+ endpoints
  • WebSocket gateway for real-time updates
  • AI/ML signal processing
  • Airtable bidirectional sync
  • Vector search with Pinecone
  • Email notifications

Deployment Process:

  1. Push to main branch
  2. GitHub Actions triggers deployment
  3. SSH to production server
  4. Run deploy-production.sh script
  5. PM2 restarts backend process
  6. Health checks verify deployment

PM2 Configuration:

// ecosystem.config.js
{
  name: 'civstart-backend',
  script: 'apps/backend/dist/main.js',
  instances: 1,
  autorestart: true,
  max_memory_restart: '1G',
  env: {
    NODE_ENV: 'production',
    PORT: 3800,
  }
}

Environment Variables (Set on server):

DATABASE_URL=postgresql://...
CLERK_SECRET_KEY=sk_...
GOOGLE_GEMINI_API_KEY=AIza...
PINECONE_API_KEY=pcsk_...
AIRTABLE_API_KEY=pat...
REDIS_URL=redis://...
PORT=3800
NODE_ENV=production

4. Database (packages/database/)​

Technology: PostgreSQL with Prisma ORM Deployment: Production server or managed service Port: 5432

Key Features:

  • 45+ database models
  • pgvector extension for ML embeddings
  • Complex relationships and junction tables
  • Prisma migrations for schema management

Schema Stats:

  • Schema size: 35,451 lines
  • Models: 45+
  • Relationships: Complex many-to-many
  • Extensions: UUID, pgvector, pg_trgm

Connection Pattern:

// Shared Prisma client
import { prisma } from "@repo/database";

Deployment Workflows​

Automated Deployment (GitHub Actions)​

Trigger: Push to main branch

Workflow:

  1. CI/CD Pipeline (.github/workflows/ci.yml):

    Quality Checks → Tests → Build → Docker → Security Scan

  2. Backend Deployment:

    SSH to server → Pull latest → Install deps → Migrate DB →
    Build backend → Restart PM2 → Health checks

  3. Frontend/Admin Deployment:

    Vercel detects push → Build Next.js → Deploy → Live

Manual Deployment​

Backend Only:

# SSH to server
ssh user@server

# Run deployment script
cd /path/to/civstart
./scripts/deploy-production.sh

Frontend/Admin: No manual deployment needed - Vercel handles automatically

Development vs Production​

ComponentDevelopmentProduction
Frontendpnpm dev:frontend (port 3801)Vercel
Adminpnpm dev:admin (port 3802)Vercel
Backendpnpm dev:backend (port 3800)PM2 on server
DatabaseDocker or local PostgreSQLProduction PostgreSQL
RedisDocker (optional)Production Redis (optional)

Data Flow​

Typical Request Flow​

User Browser → Frontend (Vercel)
              ↓ API Request
              Backend (PM2)
              ↓ Query
              PostgreSQL
              ↓ Response
              Backend (PM2)
              ↓ JSON
              Frontend (Vercel)
              ↓ Render
User Browser ← Frontend (Vercel)

Real-time Updates Flow​

Admin Panel (Vercel) ←──WebSocket──→ Backend (PM2)
                                      ↓ Database Change
                                      PostgreSQL
                                      ↓ Event
                                      Backend (PM2)
                                      ↓ WebSocket Broadcast
Admin Panel (Vercel) ←──Update───────┘

External Service Integrations​

Clerk Authentication​

  • Used by: Frontend, Admin, Backend
  • Purpose: User authentication and authorization
  • Integration: JWT token validation on backend
  • Webhooks: User creation/update events

Google Gemini AI​

  • Used by: Backend only
  • Purpose:
    • Signal generation from transcripts
    • Text embeddings for RAG
    • AI-powered matching
  • Models: gemini-2.5-flash, gemini-embedding-001

Pinecone Vector Database​

  • Used by: Backend only
  • Purpose: Vector similarity search for startup matching
  • Index: civstart-signals (3072 dimensions)

Airtable​

  • Used by: Backend only
  • Purpose: Bidirectional data sync
  • Integration: REST API + Webhooks
  • Sync Frequency: On-demand + webhook-triggered

Scaling Considerations​

Current Limitations​

ComponentConstraintSolution
BackendSingle PM2 instanceAdd load balancer + multiple instances
DatabaseSingle PostgreSQLAdd read replicas
WebSocketSingle server connectionUse Redis pub/sub for multi-server
AI ProcessingSequential batch jobsAdd job queue (Bull/BullMQ)

  1. Phase 1 (Current):

    • Single backend on PM2
    • Vercel auto-scaling for frontend
    • Suitable for 100-1000 users

  2. Phase 2 (10K+ users):

    • Add load balancer (nginx/HAProxy)
    • 2-3 backend instances with PM2 cluster mode
    • PostgreSQL read replicas
    • Redis for session/cache

  3. Phase 3 (100K+ users):

    • Migrate to container orchestration (Kubernetes)
    • Add CDN for static assets
    • Database sharding if needed
    • Consider Cloudflare Workers for read-heavy endpoints

Monitoring & Observability​

Current Setup​

Backend:

  • PM2 process monitoring
  • PM2 logs (pm2 logs civstart-backend)
  • Health check endpoint: /health

Frontend/Admin:

  • Vercel deployment logs
  • Vercel Analytics (optional)

  1. Application Performance Monitoring (APM):

    • New Relic, Datadog, or Sentry
    • Track API response times
    • Monitor error rates

  2. Log Aggregation:

    • Centralized logging (LogDNA, Papertrail)
    • Structured JSON logs

  3. Uptime Monitoring:

    • UptimeRobot or Pingdom
    • Alert on downtime

  4. Database Monitoring:

    • Query performance tracking
    • Connection pool monitoring

Security​

Current Measures​

  • ✓ Clerk authentication for all routes
  • ✓ JWT token validation on backend
  • ✓ Helmet middleware for HTTP headers
  • ✓ CORS configuration
  • ✓ Environment variables for secrets
  • ✓ GitHub Secrets for deployment
  • Rate limiting (express-rate-limit)
  • API key rotation schedule
  • Database encryption at rest
  • SSL/TLS for all connections
  • Regular security audits
  • Dependency vulnerability scanning

Cost Estimate​

ServiceUsageEstimated Cost
Vercel2 projects (Frontend + Admin)$20/mo (Pro) or $0 (Hobby)
Production ServerVPS with PM2$50-100/mo
PostgreSQLSelf-hosted or managed$0-50/mo
ClerkAuthentication$25/mo (Pro) or $0 (Free tier)
Gemini AIAPI callsVariable (~$10-50/mo)
PineconeVector storage$70/mo (Starter)
AirtableAPI calls$0-20/mo
Total$175-365/mo

Optimization Opportunities​

  1. Use Vercel Hobby Plan if traffic permits: Save $20/mo
  2. Migrate to Railway.io for backend: ~$20/mo (save $30-80/mo)
  3. Use Supabase for PostgreSQL: $25/mo (predictable cost)
  4. Add caching layer: Reduce AI API calls

Migration Paths​

Pros:

  • ✓ Working and stable
  • ✓ Cost-effective
  • ✓ Easy to maintain

Cons:

  • ✗ Manual server management
  • ✗ Single point of failure

Option 2: Migrate Backend to Railway/Fly.io​

Pros:

  • ✓ Auto-scaling
  • ✓ Better DDoS protection
  • ✓ Managed infrastructure
  • ✓ No code changes needed

Cons:

  • ✗ Slightly higher cost (~$20-30/mo)

Timeline: 1-2 weeks

Option 3: Add Cloudflare Workers (Hybrid)​

Pros:

  • ✓ 70% faster read requests
  • ✓ Global CDN
  • ✓ DDoS protection

Cons:

  • ✗ Code changes required
  • ✗ Added complexity

Timeline: 4-6 weeks

See Cloudflare Workers Analysis below for details.

Cloudflare Workers Migration​

TL;DR: Not recommended for full migration due to:

  • Long-running AI/batch processes (incompatible with 15-min limit)
  • WebSocket real-time features (limited Durable Objects support)
  • Complex database operations (connection pooling constraints)

Recommended: Hybrid approach - Workers for read caching only

See detailed analysis in previous conversation for complete breakdown.

Questions?​

For architecture questions or deployment issues, refer to:

Last Updated: January 2025 Architecture Version: 1.0