marketplace-recsys-feature-engineering

What this skill does

# Marketplace Engineering Recsys Feature Engineering Best Practices

Comprehensive first-principles guide for deriving usable recommender features from the raw assets of a two-sided trust marketplace — listing photos, owner-supplied listing metadata, and sitter wizard responses — for item-to-item, user-to-item, and user-to-user solutions. Contains 44 rules across 8 categories ordered by cascade impact on the feature-engineering lifecycle, plus one playbook that composes the rules into an end-to-end feature discovery workflow.

This skill is the **upstream precursor** to `marketplace-personalisation` (AWS Personalize) and `marketplace-search-recsys-planning` (OpenSearch retrieval). Those skills treat features as inputs they already have; this skill is about deciding what features to *build* from the raw assets, which decisions they serve, and how to prove each one is worth its maintenance cost.

## When to Apply

Reference this skill when:

- Planning what to extract from listing photos, descriptions, or amenity lists to power i2i similarity or u2i ranking
- Designing or revising the sitter onboarding wizard with recsys features as the primary output
- Deciding whether to build a vision embedding pipeline, a text encoder, or neither — and in what order
- Composing existing base features into item-to-item, user-to-item, or user-to-user scoring
- Auditing an existing feature store for coverage, drift, PII, duplication, or orphan features
- Choosing a ship/kill criterion for a new recsys feature and designing the ablation A/B test
- Answering the question: "we want to improve the similar-homes shelf — what feature should we build?"

## Setup

This skill has no user-specific configuration — it is self-contained. References are live URLs to engineering blogs from Airbnb, Pinterest, DoorDash, Uber, Netflix, and Google, to open-source libraries (Feast, Sentence-Transformers, Hugging Face CLIP, H3), to foundational academic papers (Airbnb KDD 2018, Pinterest ItemSage, YouTube Semantic IDs, PinSage), and to Google's Rules of Machine Learning.

## Rule Categories

Categories are ordered by cascade impact on the feature-engineering lifecycle: auditing mistakes build features on data that does not exist, first-principles mistakes produce features that do not map to real decisions, extraction mistakes poison everything downstream, and so on. Fix earlier-stage problems before later-stage problems.

| # | Category | Prefix | Impact |
|---|----------|--------|--------|
| 1 | Asset Audit and Inventory | `audit-` | CRITICAL |
| 2 | First-Principles Feature Decomposition | `firstp-` | CRITICAL |
| 3 | Image Feature Extraction | `vision-` | HIGH |
| 4 | Listing Text and Metadata Extraction | `listing-` | HIGH |
| 5 | Sitter Wizard and Profile Extraction | `wizard-` | HIGH |
| 6 | Derived Similarity and Affinity | `derive-` | MEDIUM-HIGH |
| 7 | Feature Quality and Governance | `quality-` | MEDIUM-HIGH |
| 8 | Incremental Rollout and Value Proof | `prove-` | MEDIUM |

## Quick Reference

### 1. Asset Audit and Inventory (CRITICAL)

- [`audit-measure-coverage-before-modelling`](references/audit-measure-coverage-before-modelling.md) — reject fields below 80% coverage from the feature plan
- [`audit-sample-every-asset-type-end-to-end`](references/audit-sample-every-asset-type-end-to-end.md) — pull 100 real instances through the real fetch path before planning
- [`audit-verify-rights-and-privacy-before-extraction`](references/audit-verify-rights-and-privacy-before-extraction.md) — ToS, GDPR, consent, face blur before encoding
- [`audit-quantify-freshness-per-asset`](references/audit-quantify-freshness-per-asset.md) — age distribution + expiry + refresh bucket
- [`audit-separate-raw-assets-from-derived-features`](references/audit-separate-raw-assets-from-derived-features.md) — raw immutable in object store, derived versioned in feature store

### 2. First-Principles Feature Decomposition (CRITICAL)

- [`firstp-start-from-the-decision-not-the-algorithm`](references/firstp-start-from-the-decision-not-the-algorithm.md) — decision first, sub-judgments second, tools last
- [`firstp-ask-what-signal-a-human-uses`](references/firstp-ask-what-signal-a-human-uses.md) — interview 8-12 owners and sitters; features trace back to quotes
- [`firstp-tie-every-feature-to-a-specific-solution`](references/firstp-tie-every-feature-to-a-specific-solution.md) — no feature without a named i2i/u2i/u2u consumer
- [`firstp-prefer-directly-observed-over-learned`](references/firstp-prefer-directly-observed-over-learned.md) — observed columns first, learned embeddings second
- [`firstp-reject-features-you-cannot-serve-at-inference`](references/firstp-reject-features-you-cannot-serve-at-inference.md) — training-serving parity starts at design time
- [`firstp-kill-features-a-popularity-baseline-already-captures`](references/firstp-kill-features-a-popularity-baseline-already-captures.md) — correlation screen before registration

### 3. Image Feature Extraction (HIGH)

- [`vision-use-clip-for-zero-shot-listing-embeddings`](references/vision-use-clip-for-zero-shot-listing-embeddings.md) — zero-shot CLIP ships in a week
- [`vision-detect-room-types-before-detecting-amenities`](references/vision-detect-room-types-before-detecting-amenities.md) — room prior conditions the amenity threshold
- [`vision-quantify-image-quality-separately-from-content`](references/vision-quantify-image-quality-separately-from-content.md) — blur, lighting, aesthetic as their own features
- [`vision-extract-per-object-counts-not-just-presence`](references/vision-extract-per-object-counts-not-just-presence.md) — `n_bed = 4` beats `has_bed = true`
- [`vision-pool-embeddings-across-a-listings-photo-set`](references/vision-pool-embeddings-across-a-listings-photo-set.md) — pooled listing vector; per-photo stored alongside
- [`vision-fine-tune-on-your-domain-when-clip-underperforms`](references/vision-fine-tune-on-your-domain-when-clip-underperforms.md) — contrastive fine-tune only after zero-shot plateaus

### 4. Listing Text and Metadata Extraction (HIGH)

- [`listing-declare-categorical-fields-for-bounded-vocabularies`](references/listing-declare-categorical-fields-for-bounded-vocabularies.md) — bounded vocab → categorical, validated on write
- [`listing-multi-hot-encode-amenity-lists`](references/listing-multi-hot-encode-amenity-lists.md) — fixed amenity vocabulary → multi-hot vector
- [`listing-hash-geo-to-hierarchies-not-raw-lat-lon`](references/listing-hash-geo-to-hierarchies-not-raw-lat-lon.md) — H3 at multiple resolutions
- [`listing-embed-description-with-pretrained-sentence-encoder`](references/listing-embed-description-with-pretrained-sentence-encoder.md) — all-MiniLM-L6-v2 for cheap semantic text features
- [`listing-extract-stay-duration-shape-not-just-length`](references/listing-extract-stay-duration-shape-not-just-length.md) — bin + holiday overlap + flexibility, not raw day count
- [`listing-encode-pet-requirements-as-structured-triples`](references/listing-encode-pet-requirements-as-structured-triples.md) — `(species, count, special_needs)` triples plus free text alongside

### 5. Sitter Wizard and Profile Extraction (HIGH)

- [`wizard-order-questions-by-information-gain`](references/wizard-order-questions-by-information-gain.md) — discriminative questions first, narrative last
- [`wizard-prefer-multiple-choice-over-free-text`](references/wizard-prefer-multiple-choice-over-free-text.md) — categorical features by construction
- [`wizard-make-skips-genuine-and-log-them`](references/wizard-make-skips-genuine-and-log-them.md) — skip is signal; defaults destroy it
- [`wizard-capture-experience-as-counts-and-dates`](references/wizard-capture-experience-as-counts-and-dates.md) — numbers, not adjectives; platform history overrides self-declaration
- [`wizard-separate-hard-constraints-from-soft-preferences`](references/wizard-separate-hard-constraints-from-soft-preferences.md) — filters vs ranking features

### 6. Derived Similari