project-sharing

What this skill does

# Project Sharing and Output Preparation

Expert guidance for preparing project outputs for sharing with collaborators, reviewers, or repositories. Creates organized packages at different sharing levels while preserving your working directory.

**Supporting files in this directory:**
- [notebook-streamlining.md](notebook-streamlining.md) - Streamlining notebooks for sharing and the abridge option
- [quality-assurance.md](quality-assurance.md) - QA procedures, best practices, checklists, and dependency management
- [common-scenarios.md](common-scenarios.md) - Sharing scenarios (collaborators, manuscripts, archival, repositories) and example scripts
- [cleanup-and-deprecation.md](cleanup-and-deprecation.md) - Correcting cleanup mistakes and deprecating redundant notebooks

## When to Use This Skill

- Sharing analysis results with collaborators
- Preparing supplementary materials for publications
- Creating reproducible research packages
- Archiving completed projects
- Handoff to other researchers
- Submitting to data repositories

## Core Principles

1. **Work on copies** - Never modify the working directory
2. **Choose appropriate level** - Match sharing depth to audience needs
3. **Document everything** - Include clear guides and metadata
4. **Clean before sharing** - Remove debug code, clear outputs, anonymize if needed
5. **Make it reproducible** - Include dependencies and instructions
6. **CRITICAL: After creating sharing folder, all future work happens in the main project directory, NOT in the sharing folder** - Sharing folders are read-only snapshots

---

## Three Sharing Levels

### Level 1: Summary Only

**Purpose:** Quick sharing for presentations, reports, or high-level review

**What to include:**
- PDF export of final notebook(s)
- Final data/results (CSV, Excel, figures) - optional
- Brief README

**Use when:**
- Sharing results with non-technical stakeholders
- Presentations or talks
- Quick review without reproduction needs
- Space/time constraints

**Structure:**
```
shared-summary/
├── README.md                          # Brief overview
├── analysis-YYYY-MM-DD.pdf           # Notebook as PDF
└── results/
    ├── figures/
    │   ├── fig1-main-result.png
    │   └── fig2-comparison.png
    └── tables/
        └── summary-statistics.csv
```

---

### Level 2: Reproducible

**Purpose:** Enable others to reproduce your analysis from processed data

**What to include:**
- Analysis notebooks (.ipynb) - cleaned
- Scripts for figure generation
- Processed/analysis-ready data
- Requirements file (requirements.txt or environment.yml)
- Detailed README with instructions

**Use when:**
- Sharing with collaborating researchers
- Peer review / manuscript supplementary materials
- Teaching or tutorials
- Standard collaboration needs

**Structure:**

For standard project structures, see the **folder-organization** skill. Reproducible packages should include:
- Processed data (in `data/processed/`)
- Cleaned notebooks (in `notebooks/`) with outputs cleared
- Scripts (in `scripts/`)
- Environment specification (`environment.yml` or `requirements.txt`)
- Documentation (`README.md`, `MANIFEST.md`)

```
shared-reproducible/
├── README.md                          # Setup and reproduction instructions
├── MANIFEST.md                        # File descriptions
├── environment.yml                    # Conda environment OR requirements.txt
├── notebooks/                         # Cleaned notebooks
├── scripts/                           # Standalone scripts
└── data/
    └── processed/                     # Analysis-ready data
```

---

### Level 3: Full Traceability

**Purpose:** Complete transparency from raw data through all processing steps

**What to include:**
- Starting/raw data
- All processing scripts and notebooks
- All intermediate files
- Final results
- Complete documentation
- Full dependency specification

**Use when:**
- Archiving for future reference
- Regulatory compliance
- High-stakes reproducibility (clinical, policy)
- Data repository submission (Zenodo, Dryad, etc.)
- Complete project handoff

**Structure:**

For standard project structures, see the **folder-organization** skill. Full traceability packages should include complete data hierarchy:

```
shared-complete/
├── README.md                          # Complete project guide
├── MANIFEST.md                        # Comprehensive file listing
├── environment.yml
├── data/
│   ├── raw/                          # Original, unmodified data
│   ├── intermediate/                 # Processing steps
│   └── processed/                    # Final analysis-ready
├── scripts/                           # All processing scripts
├── notebooks/                         # All notebooks (exploratory + final)
├── results/                           # All outputs
│   ├── figures/
│   ├── tables/
│   └── supplementary/
└── documentation/                     # Complete documentation
    ├── methods.md
    ├── changelog.md
    └── data-dictionary.md
```

---

## Preparation Workflow

### Step 1: Ask User for Sharing Level

**Questions to determine level:**

```
Which sharing level do you need?

1. Summary Only - PDF + final results (quick sharing)
2. Reproducible - Notebooks + scripts + data (standard sharing)
3. Full Traceability - Everything from raw data (archival/compliance)

Additional questions:
- Who is the audience? (colleagues, reviewers, public)
- Are there size constraints?
- Any sensitive data to handle?
- Timeline for sharing?
```

### Step 2: Identify Files to Include

**Level 1 - Summary:**
- Main analysis notebook(s)
- Key figures (publication-quality)
- Summary tables/statistics

**Level 2 - Reproducible:**
- All analysis notebooks (not exploratory)
- Figure generation scripts
- Processed/cleaned data
- Environment specification
- Any utility functions/modules

**Level 3 - Full:**
- Raw data (or links if too large)
- All processing scripts
- All notebooks (including exploratory)
- All intermediate files
- Complete documentation

### Step 3: Create Sharing Directory

```bash
# Create dated directory
SHARE_DIR="shared-$(date +%Y%m%d)-[level]"
mkdir -p "$SHARE_DIR"
```

### Step 4: Copy and Clean Files

**For notebooks (.ipynb):**

```python
import nbformat
from nbconvert.preprocessors import ClearOutputPreprocessor

def clean_notebook(input_path, output_path):
    """Clean notebook: clear outputs, remove debug cells."""
    with open(input_path, 'r') as f:
        nb = nbformat.read(f, as_version=4)

    clear_output = ClearOutputPreprocessor()
    nb, _ = clear_output.preprocess(nb, {})

    nb.cells = [cell for cell in nb.cells
                if 'debug' not in cell.metadata.get('tags', [])
                and 'remove' not in cell.metadata.get('tags', [])]

    with open(output_path, 'w') as f:
        nbformat.write(nb, f)
```

**For data files:** Copy as-is for small files; compress large files; check for sensitive information.

**For scripts:** Remove debugging code; add docstrings if missing; ensure paths are relative.

For notebook streamlining and the abridge option, see [notebook-streamlining.md](notebook-streamlining.md).

### Step 4.5: Verify and Fix File Paths

**Problem**: Notebooks and scripts with broken file paths will fail when shared.

For complete path verification procedures, automated checking scripts, and correction patterns, see the **folder-organization** skill.

| Breaks when shared | Works when shared |
|---------------------|-------------------|
| `/Users/yourname/project/data.csv` | `data/data.csv` |
| `C:\Users\yourname\project\fig.png` | `figures/fig.png` |
| `/absolute/path/to/results/` | `results/` |

**Quick check commands:**
```bash
# Check for absolute paths in notebooks
grep -l "/Users/" *.ipynb
grep -l "C:\\\\" *.ipynb
```

### Step 5: Generate Documentation

#### README.md Template

```markdown
# Project: [Project Name]

**Date:** YYYY-MM-DD
**Author:** [Your Name]
**Sharing Level:** [Summary/Reproducible/Full]

## Overview
Brief description of the p