genlayernode

What this skill does

# GenLayer Validator Setup

Interactive wizard to set up a GenLayer validator node from scratch on a Linux server.

## MANDATORY: Show Process Overview at Start

**CRITICAL REQUIREMENT**: At the very beginning of ANY setup or upgrade operation, you MUST display a complete overview of ALL steps that will be performed. This is NON-NEGOTIABLE.

### Before Starting ANY Installation or Upgrade:

1. **Display the full process overview** showing all steps that will be executed
2. **Ask user to confirm** they want to proceed with the installation/upgrade
3. **Before EACH step**, briefly explain what that step will do before executing it

### Required Start Message Format:

```
## GenLayer Validator Node Setup

I'll guide you through the complete validator node installation. Here's what we'll do:

**Step 1: Determine Server Location**
- Identify where the validator will run (local/GCP/AWS/SSH)
- Configure access method for commands

**Step 2: Verify Prerequisites**
- Check system architecture (must be x86_64)
- Verify RAM, CPU, and disk space
- Check Node.js, Docker, Python installations

**Step 3: New or Existing Validator**
- New: Run staking wizard (requires 42,000+ GEN)
- Existing: Provide validator wallet address

**Step 4: Download & Extract Node Software**
- Download GenLayer node tarball from official storage
- Extract to /opt/genlayer-node/${VERSION}/
- Set up directory structure and symlinks
- Run GenVM setup to download dependencies

**Step 5: Configure Environment (.env)**
- Create .env from example template
- Configure RPC and WebSocket URLs
- Set LLM provider (you'll add API key manually)

**Step 6: Configure Node (config.yaml)**
- Set validator wallet address
- Configure operator address
- Set network endpoints and ports

**Step 7: Set Up Operator Key**
- Import keystore from staking wizard, OR
- Copy from previous installation, OR
- Generate new operator key

**Step 8: Start WebDriver Container**
- Launch WebDriver via Docker Compose
- Wait for health check to pass

**Step 9: Run Doctor Check**
- Verify GenVM binaries are installed
- Verify WebDriver connectivity

**Step 10: Choose Deployment Method**
- Systemd service (recommended)
- Docker Compose
- Manual (screen/tmux)

**Step 11: Verify Node Running**
- Check health endpoint
- Verify sync status

**Estimated time: 20-45 minutes**

Ready to begin?
```

### Before Each Step:

Always show a brief description of what will happen:

```
## Step 4: Download & Extract Node Software

This step will:
1. Download the GenLayer node v0.4.4 tarball (~XX MB)
2. Create directory /opt/genlayer-node/v0.4.4/
3. Extract binary, configs, and GenVM files
4. Set up symlinks for easy access
5. Run GenVM setup to download dependencies (~2 min)

Proceeding...
```

**NEVER skip showing what a step will do before executing it.**

---

## What This Skill Will Do

This skill guides you through the complete validator node installation process. When you invoke this skill, it will:

1. **Determine your setup environment** - Local machine, remote server (SSH), or cloud VM (GCP/AWS/Azure)
2. **Verify prerequisites** - Check that your server meets minimum requirements (architecture, RAM, software dependencies)
3. **Guide wallet setup** - Help create a new validator wallet or use an existing one
4. **Download node software** - Fetch the specified version (or latest) from official storage
5. **Generate configuration** - Create config.yaml and .env files with your specific settings
6. **Import/generate operator keys** - Set up the validator's operator account
7. **Configure LLM provider** - Set up API keys for intelligent contract execution
8. **Start the node** - Launch as systemd service, Docker Compose, or manual process
9. **Verify installation** - Check health and sync status
10. **Optional: Enable monitoring** - Set up telemetry push to central monitoring

**Total time estimate**: 20-45 minutes depending on your experience level and setup method.

## CRITICAL: Update Procedure Warning

**If you are UPDATING an existing node, read this first:**

### Zero-Downtime Update Procedure
**NEVER stop your old node before preparing the new version!**

Traditional update process (WRONG - causes 3-4 min downtime):
```
Stop node -> Download -> Extract -> GenVM setup (2 min) -> Start
```

**Correct procedure (10-15 sec downtime):**
```
Download -> Extract -> GenVM setup WHILE OLD NODE RUNS
THEN: Stop old -> Switch symlinks -> Start new
```

**Impact of wrong procedure:**
- 3-4 minutes downtime vs 10-15 seconds
- Missed 12-24 validation opportunities
- Lost rewards during downtime
- Validator needs to re-prime
- Potential slashing penalties

**See `update-procedure.md` for detailed zero-downtime update steps.**

**For fresh installations, see `install-procedure.md` for the step-by-step installation guide.**

### Database Storage Structure
For patch versions (v0.4.x), the database MUST be shared, not copied:

**Correct structure:**
```
/opt/genlayer-node/v0.4/data/node/genlayer.db    <- Shared DB
/opt/genlayer-node/v0.4.3/data/node/genlayer.db  -> symlink to shared
/opt/genlayer-node/v0.4.4/data/node/genlayer.db  -> symlink to shared
```

### LLM Strategy and Provider Configuration

Two LLM strategies are available:
- **default** — Random provider selection from enabled backends
- **greybox** — Deterministic ordered fallback via OpenRouter (requires `OPENROUTERKEY`)

```bash
# 1. Apply release LLM config (has all backends)
cp third_party/genvm/config/genvm-modules-llm-release.yaml \
   third_party/genvm/config/genvm-module-llm.yaml

# 2. For greybox strategy, switch lua script:
sed -i 's/genvm-llm-default\.lua/genvm-llm-greybox.lua/' \
  third_party/genvm/config/genvm-module-llm.yaml

# 3. Enable your provider (replace <provider> with name from mapping):
# Provider mapping (env var -> config name):
#   HEURISTKEY -> heurist
#   COMPUT3KEY -> comput3
#   IOINTELLIGENCEKEY -> ionet
#   LIBERTAI_API_KEY -> libertai
#   ANTHROPICKEY -> anthropic
#   GEMINIKEY -> google
#   OPENROUTERKEY -> openrouter
#   MORPHEUS_API_KEY -> morpheus
sed -i '/^  <provider>:/,/^  [a-z]/ s/enabled: false/enabled: true/' \
  third_party/genvm/config/genvm-module-llm.yaml
```

**Symptom if not enabled:** Node fails to start with "module_failed_to_start" error.

### Updating Greybox on a Running Node
To update the Lua script or LLM YAML without a full redeploy, see
`common-procedures.md` -> "Update Greybox Config on Running Node".
Key points:
- Update files in all GenVM instance config directories
- Restart LLM module per GenVM manager: `curl -X POST http://127.0.0.1:<port>/module/stop` then `/module/start`
- No atomic restart — each instance restarts independently

### Validator Triage: RPC Outages, Temporary Bans, and Provider Errors

When a validator reports `504 Gateway Timeout`, `get chain ID`, `validator is banned`, or `NO_PROVIDER_FOR_PROMPT`, triage before changing configuration:

1. **Classify the symptom.**
   - `dependencies: get chain ID: 504 Gateway Timeout` usually means the configured rollup RPC is unavailable or overloaded. Confirm with `eth_chainId`/`eth_blockNumber` against the configured HTTP RPC and compare with other operators before editing local node files.
   - `/health` with `temporary ban until epoch ...: validator is banned` means the node may be healthy but consensus has temporarily banned the validator. Fix the root cause first, then request unban with the validator wallet address.
   - `NO_PROVIDER_FOR_PROMPT` means GenVM could not find an enabled LLM backend that satisfies the prompt capabilities. Check enabled providers, API keys, JSON/image capability support, and restart the affected GenVM LLM module after config changes.
2. **Do not rotate endpoints blindly.** Shared Bradbury/Asimov RPC incidents can affect multiple operators at once; switching URLs without an official replacement can make recovery harder.
3. **Collect safe diagnostics only.** Share sanitized health output, node/genvm versions, network, and validator address w