source-evaluation-bibliography

What this skill does

# Source Evaluation & Bibliography

Evaluate, document, and maintain curated collections of references and sources.

## Source Evaluation Framework

### The CRAAP Test

| Criterion | Questions | Weight |
|-----------|-----------|--------|
| **Currency** | When was it published/updated? Is the information still valid? | Medium |
| **Relevance** | Does it address the specific question? Who is the intended audience? | High |
| **Authority** | Who is the author? What credentials/affiliations? | High |
| **Accuracy** | Is it evidence-based? Can claims be verified? Peer-reviewed? | Critical |
| **Purpose** | Is it informative, persuasive, or commercial? Any bias? | Medium |

### Source Tier Classification

| Tier | Source Type | Reliability | Example |
|------|-----------|-------------|---------|
| **S** | Primary research, official specs | Highest | Peer-reviewed papers, RFC documents, official API docs |
| **A** | Authoritative secondary | High | Textbooks, reputable technical blogs, conference proceedings |
| **B** | Community knowledge | Moderate | Stack Overflow answers (high-vote), well-maintained wikis |
| **C** | Informal | Variable | Blog posts, tutorials, forum discussions |
| **D** | Unverified | Low | Social media, anonymous posts, undated content |

### Evaluation Record

```yaml
source:
  title: "Designing Data-Intensive Applications"
  author: "Martin Kleppmann"
  type: book
  year: 2017
  url: "https://dataintensive.net/"
  tier: A
  evaluation:
    currency: Good (concepts still current despite 2017 publication)
    relevance: High (directly addresses distributed systems patterns)
    authority: Strong (researcher at Cambridge, industry experience)
    accuracy: Peer-reviewed content, extensively cited
    purpose: Educational, no commercial bias
  notes: "Definitive reference for distributed data systems. Chapter 5-9 most relevant."
  tags: [distributed-systems, databases, architecture]
  last_verified: 2026-03-20
```

## Annotated Bibliography Format

### Entry Structure

```markdown
## {Author Last}, {First}. "{Title}." *{Publication}*, {Year}. {URL}

**Tier:** {S/A/B/C/D} | **Relevance:** {High/Medium/Low} | **Last verified:** {Date}

**Summary:** {2-3 sentences describing the content and main argument}

**Key contributions:**
- {Specific idea, framework, or finding #1}
- {Specific idea, framework, or finding #2}

**Limitations:** {Any caveats, biases, or gaps}

**Connection:** {How this source relates to other sources or the project}
```

### Example Entry

```markdown
## Kleppmann, Martin. "Designing Data-Intensive Applications." O'Reilly, 2017.

**Tier:** A | **Relevance:** High | **Last verified:** 2026-03-20

**Summary:** Comprehensive guide to distributed data systems covering replication,
partitioning, transactions, and stream processing. Bridges theoretical CS concepts
with practical engineering tradeoffs.

**Key contributions:**
- Clear taxonomy of consistency models (linearizability, causal, eventual)
- Practical comparison of batch vs. stream processing architectures
- The "unbundling the database" framing for microservice data patterns

**Limitations:** Pre-dates serverless and edge computing patterns. Some implementation
details are framework-specific and may be dated.

**Connection:** Foundation for resilience-patterns and data-pipeline-architect skills.
Complements the CAP theorem discussion in redis-patterns.
```

## Bibliography Management

### Collection Structure

```
references/
├── bibliography.yaml          # Machine-readable catalog
├── by-topic/
│   ├── distributed-systems.md # Topic-organized annotations
│   ├── security.md
│   └── ai-agents.md
├── by-tier/
│   ├── tier-s.md              # Primary sources only
│   └── tier-a.md              # Authoritative secondary
└── reading-log.md             # Chronological reading notes
```

### Machine-Readable Catalog

```yaml
bibliography:
  - id: kleppmann2017
    title: "Designing Data-Intensive Applications"
    author: "Martin Kleppmann"
    year: 2017
    type: book
    tier: A
    topics: [distributed-systems, databases]
    cited_by: [resilience-patterns, data-pipeline-architect]

  - id: fowler2002
    title: "Patterns of Enterprise Application Architecture"
    author: "Martin Fowler"
    year: 2002
    type: book
    tier: A
    topics: [architecture, patterns]
    cited_by: [backend-implementation-patterns]
```

### Citation in Documents

```markdown
The circuit breaker pattern [kleppmann2017, ch.8] provides fault isolation
between services. This aligns with the bulkhead pattern described in
[nygard2018, ch.5], which isolates failure domains at the resource level.
```

## Curation Workflow

### Adding New Sources

1. **Discover** — Find source through research, recommendation, or citation chain
2. **Evaluate** — Apply CRAAP test and assign tier
3. **Annotate** — Write summary, key contributions, limitations
4. **Connect** — Link to existing sources and project skills
5. **Catalog** — Add to bibliography.yaml and topic files

### Periodic Review

```markdown
## Quarterly Review Checklist

- [ ] Verify URLs still resolve (automated: link-checker script)
- [ ] Check for updated editions or superseding publications
- [ ] Re-evaluate tier for sources whose domains have evolved
- [ ] Remove or demote sources that are no longer current
- [ ] Add newly discovered sources from recent research
```

## Anti-Patterns

- **Uncritical acceptance** — Every source needs evaluation, even authoritative ones
- **Recency bias** — Older sources can be foundational; evaluate on merit, not date alone
- **Single-source reliance** — Triangulate claims across multiple independent sources
- **No annotation** — A bibliography without annotations is just a link list
- **Stale references** — Sources need periodic re-verification
- **Collecting without connecting** — Always link sources to the questions they help answer