alibabacloud-dataworks-infra-manage

What this skill does

# DataWorks Infrastructure Management

Unified management of **Data Sources**, **Compute Resources**, and **Resource Groups** in Alibaba Cloud DataWorks workspaces, supporting create and query operations.

## Architecture

```
DataWorks
├── Workspaces ─── Query and search workspaces
│   ├── Data Sources ─── 50 types: MySQL, Hologres, MaxCompute, ...
│   └── Compute Resources ─── Hologres, MaxCompute, Flink, Spark
└── Resource Groups ─── Serverless resource group management (cross-workspace)

Dependencies:
  Workspace ◀── Data Sources, Compute Resources (must belong to a workspace)
  Workspace ◀── Resource Groups (associated via binding; one resource group can bind to multiple workspaces)
  Connectivity Test ──depends on──▶ Resource Group (must be bound to the workspace of the data source)
  Standard Mode ──requires──▶ Dev (Development) + Prod (Production) dual data sources and compute resources
```

---

## Global Rules

### Prerequisites

1. **Aliyun CLI >= 3.3.1**: `aliyun version` (Installation guide: [references/cli-installation-guide.md](references/cli-installation-guide.md))
2. **First-time use**: `aliyun configure set --auto-plugin-install true`
3. **jq** (required for resource group operations): `which jq`
4. **Credential status**: `aliyun configure list`, verify valid credentials exist
5. **DataWorks edition**: Basic edition or above required

> **Security Rules**: **DO NOT** read/print/echo AK/SK values, **DO NOT** let users input AK/SK directly, **ONLY** use `aliyun configure list` to check credential status.

### Command Formatting

- **User-Agent (mandatory)**: All `aliyun` CLI commands **must** include the `--user-agent AlibabaCloud-Agent-Skills` parameter to identify the source.
- **Single-line commands**: When executing Bash commands, **must** construct as a **single-line string**; do not use `\` for line breaks.
- **jq step-by-step execution**: First execute the `aliyun` command to get JSON, then format with `jq` (to avoid multi-line security prompts).
- **Endpoint mandatory**: When specifying the `--region` parameter, you **must** also add `--endpoint dataworks.<REGION_ID>.aliyuncs.com`. Not needed when `--region` is not specified.

### Parameter Confirmation

> Before executing any command, all user-customizable parameters **must** be confirmed by the user. Do not assume or use default values.
> **Exception**: When the user has **explicitly specified** parameter values in the conversation, use them directly without re-confirmation.

**Resource group related parameters (mandatory user selection)**: VPC, VSwitch, Resource Group ID (for binding/connectivity testing) — involve networking and billing, **DO NOT auto-select**; must display a list for the user to explicitly choose. Confirm even if there is only one option.

### ⚠️ Write API Execution Gate — MUST Check Before Every Write Operation

> **MANDATORY**: Before calling **any** Write API (Create / Update / Delete / Bind / Unbind / Associate / Dissociate / Test), you **MUST** perform the following checks in order:
>
> 1. **Scan the entire SKILL.md** for a Security Restriction or Disabled Operations notice that mentions the target API or module.
> 2. **If a restriction exists**: **BLOCK the operation immediately**. Do NOT call the API. Respond to the user with:
>    - What operation is blocked and why
>    - The recommended alternative (e.g., use the DataWorks console, contact administrator)
> 3. **If no restriction exists**: Proceed normally with parameter confirmation and execution.
>
> **This check is NOT optional.** It applies to every single write operation without exception. Never skip this step.
>
> **Quick Reference — Blocked APIs in this skill**:
> | Module | Blocked APIs | Reason |
> |--------|-------------|--------|
> | Data Sources (Module 1) | `UpdateDataSource`, `DeleteDataSource` | Prevent accidental data loss, credential exposure, disruption of running tasks |
> | Compute Resources (Module 2) | `UpdateComputeResource`, `DeleteComputeResource` | Prevent disruption of running development and scheduling tasks |
>
> **Allowed Write APIs**: `CreateDataSource`, `CreateComputeResource`, `CreateResourceGroup`, `AssociateProjectToResourceGroup`, `DissociateProjectFromResourceGroup`, `TestDataSourceConnectivity`

### RAM Permissions

All operations require `dataworks:<APIAction>` permissions. Creating resource groups additionally requires `AliyunBSSOrderAccess` and `vpc:DescribeVpcs`, `vpc:DescribeVSwitches`.
> Full permission matrix: [references/ram-policies.md](references/ram-policies.md)

---

## Quick Start: New Workspace Infrastructure Initialization

When the user is **unsure about specific operations** or has vague requirements, guide them through the following process:

1. **Environment check** — Check CLI and credentials per Prerequisites
2. **Confirm workspace** — Use `ListProjects` to locate the workspace, `GetProject` to confirm the mode (Simple/Standard)
3. **Create compute resources** — Guide engine type selection; the system will **automatically create corresponding data sources**. Standard Mode requires Dev+Prod pairs. Only pure storage-type data sources (MySQL, Kafka, etc.) need separate data source creation
4. **Create/bind resource groups** — Query existing resource groups → let user select → bind. Guide creation when no resource groups are available
5. **Test connectivity** — Test with bound resource groups; when all pass, inform "Infrastructure configuration complete"

> After each step, proactively suggest the next action.

---

## Next Step Guidance

After each write operation is completed and verified, **proactively suggest** follow-up actions:

| Completed Operation | Recommended Next Step |
|-----------|-----------|
| Create compute resource | Standard Mode: "Create the corresponding Dev resource?"; "Test connectivity?" |
| Create data source separately | "Test connectivity?"; Standard Mode: "Create Dev/Prod environment data sources?" |
| Create resource group | "Bind to a workspace?" |
| Bind resource group | "Test data source connectivity?" |
| Connectivity test passed | "Infrastructure is ready." |
| Connectivity test failed | Analyze the error cause, guide the fix |
| Unbind resource group | "Bind to another workspace?" |

---

## Trigger Rules

**Trigger scenarios**: Data source create/query, compute resource create/query, resource group management, infrastructure initialization, colloquial aliases (DW database connection failure, configure holo/mc resources, create rg)

**Not triggered**: Data development tasks, scheduling configuration, MaxCompute table management, data integration tasks, ECS/RDS/OSS, workspace member management, data quality/lineage/preview. Standalone workspace queries are handled by the `alibabacloud-dataworks-workspace-manage` skill.

## Interaction Flow

All operations follow: **Identify module → Environment check → Collect parameters → Execute command → Verify result → Guide next step**

Common aliases: DW=DataWorks, holo=Hologres, mc/MC/odps=MaxCompute, pg=PostgreSQL, rg=Resource Group, ds=Data Source, RDS=InstanceMode MySQL/PG/SQLServer, ADB=AnalyticDB

Naming suggestions: Data source `{type}_{business}_{purpose}`, Compute resource `{type}_{business}`, Resource group `dw_{purpose}_rg_{env}`

---

# Module 0: Workspace Query

> If the `alibabacloud-dataworks-workspace-manage` skill is available, prefer using it for workspace queries. The following is only a fallback.

```bash
aliyun dataworks-public ListProjects --user-agent AlibabaCloud-Agent-Skills --Status Available --PageSize 100
```

When searching by name, first get the full list then filter `.PagingInfo.Projects[]` by Name/DisplayName using `jq`.

---

# Module 1: Data Source Management

Supports **50** data source types. See [references/data-sources/README.md](references/data-sources/README.md) for details.

> **When do you need to create a data source separately?** Creating a compute resource (Module 2) will **automatically create the corresponding data sourc