implementing-cloud-dlp-for-data-protection

What this skill does

# Implementing Cloud DLP for Data Protection

## When to Use

- When compliance frameworks (GDPR, HIPAA, PCI DSS) require automated sensitive data discovery and protection
- When building data governance programs that classify and label data across cloud storage
- When implementing data loss prevention controls for cloud-based data pipelines
- When auditing cloud environments for unprotected sensitive data (PII, PHI, financial data)
- When integrating DLP scanning into CI/CD pipelines to prevent sensitive data from reaching production

**Do not use** for endpoint DLP (use Microsoft Purview or Symantec DLP agents), for email DLP (use Microsoft 365 DLP or Google Workspace DLP), or for network-level data exfiltration prevention (use VPC endpoint policies and network firewalls).

## Prerequisites

- Amazon Macie enabled with appropriate S3 bucket permissions
- Google Cloud DLP API enabled (`gcloud services enable dlp.googleapis.com`)
- Azure Information Protection or Microsoft Purview configured
- IAM permissions for DLP service administration and data access
- Knowledge of data sensitivity categories relevant to the organization (PII, PHI, PCI, proprietary)

## Workflow

### Step 1: Deploy Amazon Macie for S3 Data Discovery

Enable Macie and configure automated sensitive data discovery jobs for S3 buckets.

```bash
# Enable Amazon Macie
aws macie2 enable-macie

# List all S3 buckets Macie can scan
aws macie2 describe-buckets \
  --query 'buckets[*].[bucketName,classifiableSizeInBytes,unclassifiableObjectCount.total]' \
  --output table

# Create a classification job for specific buckets
aws macie2 create-classification-job \
  --job-type SCHEDULED \
  --name "weekly-pii-scan" \
  --schedule-frequency-details '{"weekly":{"dayOfWeek":"MONDAY"}}' \
  --s3-job-definition '{
    "bucketDefinitions": [{
      "accountId": "ACCOUNT_ID",
      "buckets": ["customer-data-bucket", "analytics-data-lake", "backup-bucket"]
    }],
    "scoping": {
      "includes": {
        "and": [{
          "simpleScopeTerm": {
            "key": "OBJECT_EXTENSION",
            "values": ["csv", "json", "parquet", "txt", "xlsx"],
            "comparator": "EQ"
          }
        }]
      }
    }
  }' \
  --managed-data-identifier-ids '["SSN","CREDIT_CARD_NUMBER","EMAIL_ADDRESS","AWS_CREDENTIALS","PHONE_NUMBER"]'

# Create custom data identifier for internal employee IDs
aws macie2 create-custom-data-identifier \
  --name "EmployeeID" \
  --regex "EMP-[0-9]{6}" \
  --description "Internal employee ID format"

# Check job status and results
aws macie2 list-classification-jobs \
  --query 'items[*].[name,jobStatus,statistics.approximateNumberOfObjectsToProcess]' \
  --output table
```

### Step 2: Configure Google Cloud DLP API for Data Inspection

Use Google Cloud DLP to inspect and de-identify sensitive data across GCP resources.

```bash
# Inspect a Cloud Storage bucket for sensitive data
gcloud dlp inspect-content \
  --content-type=TEXT_PLAIN \
  --min-likelihood=LIKELY \
  --info-types=PHONE_NUMBER,EMAIL_ADDRESS,CREDIT_CARD_NUMBER,US_SOCIAL_SECURITY_NUMBER \
  --storage-type=CLOUD_STORAGE \
  --gcs-uri="gs://sensitive-data-bucket/data/*.csv"

# Create an inspection job for BigQuery
cat > dlp-job.json << 'EOF'
{
  "inspectJob": {
    "storageConfig": {
      "bigQueryOptions": {
        "tableReference": {
          "projectId": "PROJECT_ID",
          "datasetId": "customer_data",
          "tableId": "transactions"
        },
        "sampleMethod": "RANDOM_START",
        "rowsLimit": 10000
      }
    },
    "inspectConfig": {
      "infoTypes": [
        {"name": "CREDIT_CARD_NUMBER"},
        {"name": "US_SOCIAL_SECURITY_NUMBER"},
        {"name": "EMAIL_ADDRESS"},
        {"name": "PHONE_NUMBER"},
        {"name": "PERSON_NAME"}
      ],
      "minLikelihood": "LIKELY",
      "limits": {"maxFindingsPerRequest": 1000}
    },
    "actions": [{
      "saveFindings": {
        "outputConfig": {
          "table": {
            "projectId": "PROJECT_ID",
            "datasetId": "dlp_results",
            "tableId": "findings"
          }
        }
      }
    }]
  }
}
EOF

gcloud dlp jobs create --project=PROJECT_ID --body-from-file=dlp-job.json
```

### Step 3: Implement Data De-identification with Cloud DLP

Configure de-identification transforms to mask, tokenize, or redact sensitive data.

```python
# deidentify_pipeline.py - De-identify sensitive data using Google Cloud DLP
from google.cloud import dlp_v2

def deidentify_data(project_id, text):
    """De-identify PII in text using Cloud DLP."""
    client = dlp_v2.DlpServiceClient()

    inspect_config = {
        "info_types": [
            {"name": "EMAIL_ADDRESS"},
            {"name": "PHONE_NUMBER"},
            {"name": "CREDIT_CARD_NUMBER"},
            {"name": "US_SOCIAL_SECURITY_NUMBER"},
        ],
        "min_likelihood": dlp_v2.Likelihood.LIKELY,
    }

    deidentify_config = {
        "info_type_transformations": {
            "transformations": [
                {
                    "info_types": [{"name": "EMAIL_ADDRESS"}],
                    "primitive_transformation": {
                        "character_mask_config": {
                            "masking_character": "*",
                            "number_to_mask": 0,
                            "characters_to_ignore": [
                                {"common_characters_to_ignore": "PUNCTUATION"}
                            ],
                        }
                    },
                },
                {
                    "info_types": [{"name": "CREDIT_CARD_NUMBER"}],
                    "primitive_transformation": {
                        "crypto_replace_ffx_fpe_config": {
                            "crypto_key": {
                                "kms_wrapped": {
                                    "wrapped_key": "WRAPPED_KEY_BASE64",
                                    "crypto_key_name": "projects/PROJECT/locations/global/keyRings/dlp/cryptoKeys/tokenization",
                                }
                            },
                            "common_alphabet": "NUMERIC",
                        }
                    },
                },
                {
                    "info_types": [{"name": "US_SOCIAL_SECURITY_NUMBER"}],
                    "primitive_transformation": {
                        "redact_config": {}
                    },
                },
            ]
        }
    }

    item = {"value": text}
    parent = f"projects/{project_id}/locations/global"

    response = client.deidentify_content(
        request={
            "parent": parent,
            "deidentify_config": deidentify_config,
            "inspect_config": inspect_config,
            "item": item,
        }
    )
    return response.item.value
```

### Step 4: Configure Azure Information Protection

Set up sensitivity labels and DLP policies in Microsoft Purview for Azure resources.

```powershell
# Connect to Microsoft Purview compliance
Connect-IPPSSession

# Create sensitivity labels
New-Label -DisplayName "Confidential - PII" \
  -Name "Confidential-PII" \
  -Tooltip "Contains personally identifiable information" \
  -ContentType "File, Email"

New-Label -DisplayName "Highly Confidential - Financial" \
  -Name "HighlyConfidential-Financial" \
  -Tooltip "Contains financial data subject to PCI DSS" \
  -ContentType "File, Email"

# Create auto-labeling policy for Azure Storage
New-AutoSensitivityLabelPolicy -Name "Auto-Label-PII" \
  -ExchangeLocation All \
  -SharePointLocation All \
  -OneDriveLocation All \
  -Mode Enable

New-AutoSensitivityLabelRule -Policy "Auto-Label-PII" \
  -Name "Detect-SSN" \
  -ContentContainsSensitiveInformation @{
    Name = "U.S. Social Security Number (SSN)";
    MinCount = 1;
    MinConfidence = 85
  } \
  -ApplySensitivityLabel "Confidential-PII"
```

```bash
# Azure: Configure DLP policy for Storage accounts
az security assessment create \
  --name "storage-sensitive-data" \
  --assessed-re