code-analyzer

What this skill does

# Code Analyzer

Analyzes the cloned GitHub repository to understand the computational methodology, then generates structured notes and drafts the Methods section.

## Critical Principle: No Silent Assumptions

**NEVER assume what the code does without user confirmation.**

Code can be complex and context-dependent. When uncertain about methodology:

### When to Pause and Ask

1. **Unclear analysis purpose**
   - What is this script/notebook trying to accomplish?
   - Is this the main analysis or a side experiment?

2. **Ambiguous parameters**
   - Hardcoded values without explanation (what does `threshold = 0.7` mean?)
   - Configuration options that affect results

3. **Multiple analysis paths**
   - Which branch/version of the code was used for final results?
   - Are there deprecated scripts that shouldn't be documented?

4. **Statistical method uncertainty**
   - Is this the correct interpretation of the test being performed?
   - What assumptions were made (normality, independence, etc.)?

5. **Missing context**
   - What does variable `X` represent in domain terms?
   - Why was this specific approach chosen over alternatives?

### How to Ask

```
I have questions about the code before writing the Methods section:

**File**: analysis.ipynb, Cell 15

**Question**: I see `model = RandomForestClassifier(n_estimators=100, max_depth=5)`.
- Were these hyperparameters tuned, or are they defaults?
- If tuned, what was the tuning method (grid search, random search)?
- Should I report these specific values in the Methods?

**Why this matters**: Reviewers often ask about hyperparameter selection.
```

### Document All Clarifications

Log clarifications in `notes/code-analysis.md`:

```markdown
## Clarifications Received

| File | Question | User Response |
|------|----------|---------------|
| analysis.ipynb | Hyperparameter tuning? | Grid search with 5-fold CV |
| preprocess.py | Why z-score normalization? | Standard for this imaging modality |
```

## Prerequisites

- `scope.md` must exist
- `notes/ethics-summary.md` may exist (provides approved procedures and endpoints for cross-reference)
- `notes/ethics-scope-comparison.md` may exist (clarifies what was actually implemented vs. approved)
- `code/` directory with cloned repository (from context-ingestion)
- If code/ doesn't exist, this step produces limited output based on scope.md alone

## Workflow

```
[Read scope.md and notes/ethics-summary.md for context]
     │
     ▼
[Scan Repository Structure]
     │
     ▼
[Identify Key Files] ─── Notebooks, scripts, configs
     │
     ▼
[CHECKPOINT: Clarify Code Purpose] ─── Ask about unclear scripts/params
     │
     ▼
[Analyze Code Flow] ─── Data loading → Processing → Analysis → Output
     │
     ▼
[CHECKPOINT: Verify Methodology] ─── Confirm interpretation with user
     │
     ▼
[ETHICS CROSS-REFERENCE] ─── Compare code procedures vs. approved procedures
     │
     ▼
[Extract Methodology] ─── Generate notes/code-analysis.md
     │
     ▼
[STATISTICAL REVIEW] ─── Validate statistical methods
     │                     └── agents/statistical-reviewer.md
     ▼
[Draft Methods] ─── drafts/methods.md (with statistical sign-off)
```

## Step 1: Scan Repository Structure

```bash
# Get repository overview
find code/ -type f -name "*.py" -o -name "*.ipynb" -o -name "*.R" | head -30

# Check for dependency files
ls code/requirements.txt code/environment.yml code/setup.py 2>/dev/null

# Check for README
cat code/README.md 2>/dev/null | head -50
```

Identify:
- Primary language (Python, R, MATLAB, etc.)
- Project structure (flat, src/, notebooks/, etc.)
- Entry points (main scripts, notebooks)
- Configuration files

## Step 2: Identify Key Files

Prioritize analysis of:

### Jupyter Notebooks (.ipynb)
Most important - usually contain the full analysis workflow.

```bash
ls code/*.ipynb code/**/*.ipynb 2>/dev/null
```

### Main Analysis Scripts
```bash
ls code/main.py code/analysis.py code/run*.py 2>/dev/null
```

### Data Processing
```bash
ls code/*preprocess* code/*clean* code/*load* 2>/dev/null
```

### Model/Statistical Files
```bash
ls code/*model* code/*train* code/*stat* 2>/dev/null
```

## Step 3: Analyze Code Flow

For each key file, trace the methodology:

### 3a. Data Loading

Look for patterns:
```python
# Python patterns
pd.read_csv(...)
pd.read_excel(...)
nibabel.load(...)      # Neuroimaging
pydicom.dcmread(...)   # DICOM
SimpleITK.ReadImage(...)
```

Extract:
- Data sources (file types, databases)
- Data formats
- Initial data shape/size

### 3b. Preprocessing

Look for patterns:
```python
# Cleaning
df.dropna(...)
df.fillna(...)

# Transformation
StandardScaler()
normalize(...)
resample(...)

# Feature engineering
df['new_col'] = ...
```

Extract:
- Missing data handling
- Normalization/standardization
- Feature creation
- Exclusion criteria applied in code

### 3c. Statistical Analysis

Look for patterns:
```python
# Hypothesis tests
scipy.stats.ttest_ind(...)
scipy.stats.mannwhitneyu(...)
scipy.stats.pearsonr(...)
scipy.stats.spearmanr(...)

# Regression
statsmodels.api.OLS(...)
statsmodels.api.Logit(...)

# Multiple comparison correction
statsmodels.stats.multitest.multipletests(...)
```

Extract:
- Statistical tests used
- Significance thresholds
- Multiple comparison corrections

### 3d. Machine Learning (if applicable)

Look for patterns:
```python
# Splitting
train_test_split(..., test_size=0.2, random_state=42)
cross_val_score(...)
StratifiedKFold(...)

# Models
RandomForestClassifier(...)
LogisticRegression(...)
XGBClassifier(...)

# Evaluation
accuracy_score(...)
roc_auc_score(...)
confusion_matrix(...)
```

Extract:
- Model type and parameters
- Train/test split ratios
- Cross-validation strategy
- Performance metrics

### 3e. Dependencies and Versions

```bash
cat code/requirements.txt 2>/dev/null
```

Or extract from imports:
```python
import pandas as pd
print(pd.__version__)
```

## Step 4: Generate Code Analysis Notes

Create `notes/code-analysis.md`:

```markdown
# Code Analysis

**Repository**: [GitHub URL]
**Analyzed**: [timestamp]
**Primary Language**: Python [version]

## Repository Structure

```
code/
├── analysis.ipynb      # Main analysis
├── preprocessing.py    # Data cleaning
├── models.py          # ML models
└── requirements.txt   # Dependencies
```

## Data Pipeline

### 1. Data Loading
- **Source**: CSV files from [source]
- **Format**: Tabular data with [n] columns
- **Initial Size**: [n] rows

### 2. Preprocessing
- Missing data: [handling approach]
- Normalization: [method]
- Exclusions: [criteria]

### 3. Analysis Approach

#### Statistical Tests
| Test | Purpose | Parameters |
|------|---------|------------|
| Independent t-test | Group comparison | α = 0.05 |
| Pearson correlation | Association | |
| Mann-Whitney U | Non-parametric comparison | |

#### Machine Learning (if applicable)
- **Model**: [type]
- **Split**: [ratio] train/test
- **Cross-validation**: [k]-fold
- **Hyperparameters**: [key params]
- **Metrics**: [accuracy, AUC, etc.]

### 4. Output Generation
- Figures saved to: [location]
- Results saved to: [location]

## Dependencies

| Package | Version | Purpose |
|---------|---------|---------|
| pandas | 2.0.3 | Data manipulation |
| scikit-learn | 1.3.0 | ML models |
| scipy | 1.11.1 | Statistical tests |
| matplotlib | 3.7.2 | Visualization |

## Key Code Snippets

### Statistical Test Implementation
```python
[relevant code snippet]
```

### Model Training
```python
[relevant code snippet]
```

## Notes for Methods Section

- [Key methodological detail 1]
- [Key methodological detail 2]
- [Any unusual or noteworthy approaches]
```

## Step 4b: Ethics Cross-Reference (If Ethics Docs Exist)

**Skip this step if `notes/ethics-summary.md` does not exist.**

Compare the procedures identified in code with the approved procedures to ensure consistency and identify any discrepancies.

### Cross-Reference Table

Create a comparison in `notes/code-analysis.md`:

```markdown
## Ethics Procedure Cro