deploying-on-gcp

What this skill does

# GCP Patterns

Build applications and infrastructure using Google Cloud Platform services with appropriate service selection, architecture patterns, and best practices.

## Purpose

This skill provides decision frameworks and implementation patterns for Google Cloud Platform (GCP) services across compute, storage, databases, data analytics, machine learning, networking, and security. It guides service selection based on workload requirements and demonstrates production-ready patterns using Terraform, Python SDKs, and gcloud CLI.

## When to Use

Use this skill when:

- Selecting GCP compute services (Cloud Run, GKE, Cloud Functions, Compute Engine, App Engine)
- Choosing storage or database services (Cloud Storage, Cloud SQL, Spanner, Firestore, Bigtable, BigQuery)
- Designing data analytics pipelines (BigQuery, Pub/Sub, Dataflow, Dataproc, Composer)
- Implementing ML workflows (Vertex AI, AutoML, pre-trained APIs)
- Architecting network infrastructure (VPC, Load Balancing, CDN, Cloud Armor)
- Setting up IAM, security, and cost optimization
- Migrating from AWS or Azure to GCP
- Building multi-cloud or GCP-first architectures

## Core Concepts

### GCP Service Categories

**Compute Options:**
- **Cloud Run:** Serverless containers for stateless HTTP services (auto-scale to zero)
- **GKE (Google Kubernetes Engine):** Managed Kubernetes for complex orchestration
- **Cloud Functions:** Event-driven functions for simple processing
- **Compute Engine:** Virtual machines for full OS control
- **App Engine:** Platform-as-a-Service for web applications

**Storage & Databases:**
- **Cloud Storage:** Object storage with Standard/Nearline/Coldline/Archive tiers
- **Cloud SQL:** Managed PostgreSQL/MySQL/SQL Server (up to 96TB)
- **Cloud Spanner:** Global distributed SQL with 99.999% SLA
- **Firestore:** NoSQL document database with real-time sync
- **Bigtable:** Wide-column NoSQL for time-series and IoT (petabyte scale)
- **AlloyDB:** PostgreSQL-compatible with 4x performance improvement

**Data & Analytics:**
- **BigQuery:** Serverless data warehouse (petabyte-scale SQL analytics)
- **Pub/Sub:** Global messaging and event streaming
- **Dataflow:** Apache Beam for stream and batch processing
- **Dataproc:** Managed Spark and Hadoop clusters
- **Cloud Composer:** Managed Apache Airflow for workflows

**AI/ML Services:**
- **Vertex AI:** Unified ML platform (training, deployment, monitoring)
- **AutoML:** No-code ML for standard tasks
- **Pre-trained APIs:** Vision, Natural Language, Speech, Translation
- **TPUs:** Tensor Processing Units for large model training

### Decision Framework: Compute Service Selection

```
Need to run code in GCP?
├─ HTTP service?
│  ├─ YES → Stateless?
│  │  ├─ YES → Cloud Run (auto-scale to zero)
│  │  └─ NO → Need Kubernetes? → GKE | Compute Engine
│  └─ NO (Event-driven)
│     ├─ Simple function? → Cloud Functions
│     └─ Complex orchestration? → GKE | Cloud Run Jobs
```

**Selection Guide:**
- **First choice:** Cloud Run (unless state or Kubernetes required)
- **Need Kubernetes:** GKE Autopilot (managed) or Standard (full control)
- **Simple events:** Cloud Functions (60-min max execution)
- **Full control:** Compute Engine (VMs with custom configuration)

### Decision Framework: Database Selection

```
Choose database type:
├─ Relational (SQL)
│  ├─ Multi-region required? → Cloud Spanner
│  ├─ PostgreSQL + high performance? → AlloyDB
│  └─ Standard RDBMS → Cloud SQL (PostgreSQL/MySQL/SQL Server)
│
├─ Document (NoSQL)
│  ├─ Mobile/web with offline sync? → Firestore
│  └─ Flexible schema, no offline? → MongoDB Atlas (Marketplace)
│
├─ Key-Value
│  ├─ Time-series or IoT data? → Bigtable
│  └─ Caching layer? → Memorystore (Redis/Memcached)
│
└─ Analytics
   └─ Petabyte-scale SQL analytics → BigQuery
```

### Decision Framework: Storage Selection

```
Storage type needed?
├─ Objects/Files
│  ├─ Frequent access → Cloud Storage (Standard)
│  ├─ Monthly access → Cloud Storage (Nearline)
│  ├─ Quarterly access → Cloud Storage (Coldline)
│  └─ Yearly access → Cloud Storage (Archive)
│
├─ Block storage → Persistent Disk (SSD/Standard/Extreme)
└─ Shared filesystem → Filestore (NFS)
```

### GCP vs AWS vs Azure Service Mapping

| Category | GCP | AWS | Azure |
|----------|-----|-----|-------|
| **Serverless Containers** | Cloud Run | Fargate | Container Instances |
| **Kubernetes** | GKE | EKS | AKS |
| **Functions** | Cloud Functions | Lambda | Functions |
| **VMs** | Compute Engine | EC2 | Virtual Machines |
| **Object Storage** | Cloud Storage | S3 | Blob Storage |
| **SQL Database** | Cloud SQL | RDS | SQL Database |
| **NoSQL Document** | Firestore | DynamoDB | Cosmos DB |
| **Data Warehouse** | BigQuery | Redshift | Synapse |
| **Messaging** | Pub/Sub | SNS/SQS | Service Bus |
| **ML Platform** | Vertex AI | SageMaker | Machine Learning |

## Architecture Patterns

### Pattern 1: Serverless Web Application

**Use Case:** Stateless HTTP API with database and caching

**Architecture:**
```
Internet → Cloud Load Balancer → Cloud Run → Cloud SQL (PostgreSQL)
                                            → Memorystore (Redis)
                                            → Cloud Storage
```

**Key Services:**
- Cloud Run for API service (auto-scaling containers)
- Cloud SQL for transactional data
- Memorystore for caching
- Cloud Storage for file uploads

For detailed Terraform configuration, see `references/compute-services.md`.

### Pattern 2: Data Analytics Platform

**Use Case:** Real-time event processing and analytics

**Architecture:**
```
Data Sources → Pub/Sub → Dataflow → BigQuery → Looker/Tableau
                          ↓
                     Cloud Storage (staging)
```

**Key Services:**
- Pub/Sub for event ingestion (at-least-once delivery)
- Dataflow for stream processing (Apache Beam)
- BigQuery for analytics (partitioned tables, clustering)
- Cloud Storage for staging and backups

For BigQuery optimization patterns, see `references/data-analytics.md`.

### Pattern 3: ML Pipeline

**Use Case:** End-to-end machine learning workflow

**Architecture:**
```
Training Data (GCS) → Vertex AI Training → Model Registry → Vertex AI Endpoints
                                                              ↓
                                                         Predictions
```

**Key Services:**
- Vertex AI Workbench for notebook development
- Vertex AI Training for custom models (GPU/TPU support)
- Vertex AI Endpoints for model serving (auto-scaling)
- Vertex AI Pipelines for orchestration (Kubeflow)

For ML implementation examples, see `references/ml-ai-services.md`.

### Pattern 4: GKE Microservices Platform

**Use Case:** Complex orchestration with multiple services

**Architecture:**
```
Internet → Cloud Load Balancer → GKE Cluster
                                   ├─ Ingress Controller
                                   ├─ Service Mesh (optional)
                                   ├─ Microservice A
                                   ├─ Microservice B
                                   └─ Microservice C
```

**Key Features:**
- GKE Autopilot (fully managed nodes) or Standard (custom configuration)
- Workload Identity for secure GCP service access
- Private cluster with Private Google Access
- Config Connector for managing GCP resources via Kubernetes

For GKE setup and best practices, see `references/compute-services.md`.

## Best Practices

### Cost Optimization

**Compute:**
- Use Committed Use Discounts for predictable workloads (57% off)
- Use Spot VMs for fault-tolerant workloads (60-91% off)
- Cloud Run scales to zero when idle (no charges)
- GKE Autopilot charges only for pod resources, not nodes

**Storage:**
- Use appropriate Cloud Storage classes (Standard/Nearline/Coldline/Archive)
- Enable Object Lifecycle Management to transition cold data
- Archive backups with Coldline or Archive (99% cheaper than Standard)

**Data:**
- BigQuery: Use partitioned and clustered tables
- Query only needed columns (avoid `SELECT *`)
-