netapp-volumes

What this skill does

# Domino NetApp Volumes Skill

## Description
This skill helps users work with Domino Volumes for NetApp ONTAP — enterprise-grade storage that mounts NetApp ONTAP filesystems directly into Domino workloads, enabling multi-terabyte scale with near-instant snapshotting.

## Activation
Activate this skill when users want to:
- Create or manage NetApp Volumes in Domino
- Work with NetApp volume snapshots and versioning
- Share large-scale data across projects or teams
- Access existing NetApp ONTAP storage from Domino workspaces, jobs, or apps
- Use the NetApp Volumes REST API programmatically
- Understand mount paths for NetApp volumes
- Choose between NetApp Volumes and Domino Datasets

## What is a Domino NetApp Volume?

A Domino NetApp Volume is:
- **Enterprise-grade storage**: Backed by NetApp ONTAP via Kubernetes PVCs with the `netapp-storage` storage class
- **Multi-terabyte scale**: No practical upper limit — suitable for very large datasets
- **Near-instant snapshots**: ~3 seconds even for 100+ GB volumes using redirect-on-write (no extra storage consumed at snapshot time)
- **Versioned with commit messages**: Snapshots support human-readable tags and commit messages, unlike Datasets
- **Shareable**: Attach a single volume to multiple projects and teams
- **Persistent**: Data persists across executions

### NetApp Volumes vs. Domino Datasets

| Feature | NetApp Volumes | Domino Datasets |
|---------|---------------|-----------------|
| Storage backend | External NetApp ONTAP | Domino-managed NFS/EFS |
| Data scale | Multi-terabyte and beyond | Up to ~1 TB |
| Snapshot speed | ~3 seconds (any size) | Scales with data size |
| Snapshot storage cost | No extra space (redirect-on-write) | Duplicates physical data |
| Commit messages on snapshots | Supported | Not supported |
| Create new volume from snapshot | Read-only clones | Can create editable dataset |
| Cross-project sharing | Yes | Yes |
| Admin prerequisite | Admin must register filesystems | None |
| REST API | Full dedicated API | Via Domino Python SDK |

**Use NetApp Volumes when:**
- Data exceeds ~1 TB
- Near-instant snapshotting is required
- Your organization already has NetApp ONTAP infrastructure
- You need snapshot commit messages for audit trails
- High-performance shared storage across many teams

**Use Domino Datasets when:**
- Data is Domino-managed with no external infrastructure dependency
- You need to create editable new versions from snapshots
- Data is under ~1 TB

---

## Prerequisites

An admin must register at least one NetApp filesystem (Kubernetes PVC with the `netapp-storage` label) before users can create volumes. Contact your Domino administrator if no filesystems are available.

---

## Creating a NetApp Volume

### From the Domino Home Page
1. Navigate to **Data > NetApp Volumes** in the toolbar
2. Click **Add NetApp Volume > Create Volume**
3. Fill in the form:
   - **Name**: Letters, numbers, underscores, hyphens only
   - **Description**: Brief overview of the data
   - **Data Plane**: Select from available options
   - **NetApp Filesystem**: Choose a registered filesystem
   - **Capacity**: Set maximum storage allocation
4. Click **Next** to configure permissions
5. Assign users with **Reader / Editor / Owner** roles
6. Click **Finish**
7. Manually add the volume to projects afterward

### From within a Project (recommended — auto-associates)
1. Open a project → **Data > NetApp Volumes** (left panel)
2. Click **Add NetApp Volume > Create Volume**
3. Fill in the same form fields as above
4. The volume is automatically associated with the current project on creation

### Via REST API
```python
import os
import requests

# Auth token from in-cluster token service
token = requests.get("http://localhost:8899/access-token").text.strip()
headers = {
    "Authorization": f"Bearer {token}",
    "Content-Type": "application/json"
}

api_url = os.environ["DOMINO_API_HOST"]
remotefs_url = os.environ["DOMINO_REMOTE_FILE_SYSTEM_HOSTPORT"]

# Look up your user ID (username is available in the DOMINO_USER_NAME env var)
user_resp = requests.get(
    f"{api_url}/v4/users?userName={os.environ['DOMINO_USER_NAME']}",
    headers=headers
).json()
user_id = user_resp[0]["id"]

# Create a volume (capacity is in bytes; grants is required)
response = requests.post(
    f"{remotefs_url}/remotefs/v1/volumes",
    headers=headers,
    json={
        "name": "large-training-data",
        "description": "Multi-TB training dataset for vision models",
        "filesystemId": "<filesystem-id>",
        "capacity": 5_000_000_000_000,  # 5 TB in bytes
        "grants": [
            {"targetId": user_id, "targetRole": "VolumeOwner"}
        ]
    }
)
volume = response.json()
print(f"Created volume ID: {volume['id']}")
```

---

## Adding a Volume to a Project

### Via Domino UI
1. Go to project → **Data > NetApp Volumes**
2. Click **Add NetApp Volume > Add Existing Volume**
3. Select the volume from the list
4. Configure access level for the project

### Via REST API
```python
# Attach a volume to a project
requests.post(
    f"{remotefs_url}/remotefs/v1/rpc/attach-volume-to-project",
    headers=headers,
    json={
        "volumeId": "<volume-id>",
        "projectId": "<project-id>"
    }
)
```

---

## Mount Paths

Mount paths depend on your **project type**. Check which exists in your execution to determine your project type.

### Git-Based Projects

| Volume Type | Mount Path |
|-------------|-----------|
| Live volume | `/mnt/netapp-volumes/<volume-name>/` |
| Snapshot by number | `/mnt/netapp-volumes/snapshots/<volume-name>/<snapshot-number>/` |
| Snapshot by tag | `/mnt/netapp-volumes/snapshot-tags/<volume-name>/<tag-name>/` |

### DFS (Domino File System) Projects

| Volume Type | Mount Path |
|-------------|-----------|
| Live volume | `/domino/netapp-volumes/<volume-name>/` |
| Snapshot by number | `/domino/netapp-volumes/snapshots/<volume-name>/<snapshot-number>/` |
| Snapshot by tag | `/domino/netapp-volumes/snapshot-tags/<volume-name>/<tag-name>/` |

### Snapshot Access Behavior

There are two ways to access snapshots, with an important behavioral difference:

- **`/snapshots/<volume-name>/<number>/`** — Accessed by snapshot number. When a new snapshot is taken while a workspace is running, the new numbered directory appears **immediately** in that workspace without a restart.
- **`/snapshot-tags/<volume-name>/<tag-name>/`** — Accessed by tag name. Each snapshot has at most one active tag path — a symlink to its numbered snapshot directory. If you apply multiple tags to the same snapshot, only the most recently applied tag creates a path; earlier tags for that snapshot are not accessible by path. Tag paths for new snapshots are also **not** visible in a running workspace — you must **restart the workspace** for them to appear.

Use the numbered path when you need to access a fresh snapshot from within a live workspace. Use the tagged path for stable, named references in reproducible runs.

> **Important:** Each snapshot exposes only one tag path at a time — the most recently applied tag. Older tags on the same snapshot do not have accessible paths.

> **Important:** Renaming a volume changes its mount path. Update any hardcoded paths in your code after renaming.

### Identify Your Project Type

```python
import os

if os.path.exists("/domino/netapp-volumes"):
    print("DFS Project")
    netapp_root = "/domino/netapp-volumes"
elif os.path.exists("/mnt/netapp-volumes"):
    print("Git-Based Project")
    netapp_root = "/mnt/netapp-volumes"
```

### Permissions

- **Owners/Editors**: Read-write access to the live volume
- **Readers**: Read-only access

### Example: Reading Data

```python
import pandas as pd

# Git-Based Project
df = pd.read_parquet("/mnt/netapp-volumes/large-training-data/features.parquet")

# DFS Project
df = pd.read_parquet("/domino/netapp-volumes/large-training-data/features.parquet")

# Read from a specific snapshot tag
df = pd.read_parquet("/mnt/netapp-v