- src/gwshop.rs: pick the FIRST longest facet path on ties, matching the
original Python max(paths, key=len). Rust's max_by_key returned the last,
which silently reclassified ~2298 entries for products faceted under
several branches (e.g. a generic kit listed per chapter). Adds a regression
test.
- AGENTS.md: document the first-longest tie-break and the facet-partition
coverage shortfall (non-system merchandise left out; no model kits).
- data/gwshop_catalog.json: regenerate with the fixed binary (3491 -> 3459
entries; 34 removed, 2 added, 188 structural + 785 collections changes).
The Rust rewrite has replaced the original python implementation, so the transition layout is no longer needed:
- move version-rust/{Cargo.toml,Cargo.lock,README.md,.gitignore,src,data} to the repo root; the crate is now the repo.
- move docs/SORTING.md and data/gwshop_catalog.json to the root.
- remove version-python/ entirely, plus the dev-only version-rust/tools/dump_warhammer_data.py (its only job was regenerating data/warhammer_data.json from python).
- rewrite AGENTS.md and README.md for the single-project layout; strip all version-python/version-rust references from code comments.
- fix the gwshop credential lookup: algolia.json now sits at CARGO_MANIFEST_DIR (crate root == repo root), not its parent.
- restore the credentials/run-output ignores in .gitignore (algolia.json, .env, *.token, downloads/, state.json, manifest.jsonl, ...).
- docs/SORTING.md: retarget the extraction/sorter/dlp examples at the dlphelper binary instead of the deleted python scripts.
cargo test: 86/86 green.
- git mv of all existing files into version-python/ (history preserved)
- version-rust/: minimal cargo scaffold (edition 2024), no deps yet
- root AGENTS.md documents the transition rules and artifact/schema contract
- main is untouched; the python layout there stays as-is
- warhammer_data.py: GW naming throughout - Space Marine chapters are
their own factions under a 'Space Marines' group, systems match the
shop (Age of Sigmar, The Horus Heresy, Other Games, The Old World,
Middle-Earth), classes are GW shop categories (Battleline, Infantry,
Vehicle, Walker, Flyer, Monster, Mounted, Hero, Titanic), group map
extended to AoS grand alliances / Other Games / Old World / Middle-Earth
- gwshop.py: hierarchy remap - GW depth-4 paths are system/group/faction/
class (the shop's last level is the class layer), depth-5 sources keep
a subfaction before it; catalog snapshot regenerated accordingly
- sorter.py: catalog entry is hierarchy-authoritative (group, class, and
faction when GW moved a unit or keyword tables are stale); generic GW
branches ('Unit Type') and a Battleline default never degrade a
confident keyword classification; whole-catalog retry when the faction
pool has no confident match (handles units GW moved between factions);
whole-catalog matching also runs when keyword classification found no
faction; invalid (HTML) items never fuzzy-match; default catalog is
now data/gwshop_catalog.json
- catalogue.py: --apply moves include the group layer, same structure
as the sorter
- docs/SORTING.md: hierarchy rewritten to GW naming with examples
- tests: expectations updated to GW naming (chapters, unit types)
- verified live: 4/4 real files in the test batch now catalog-match,
e.g. The Twin Lance -> 40k/Xenos Armies/T'au Empire/Epic Hero (GW's
current placement) instead of the stale Drukhari keyword
- warhammer.com shop HTML/listing endpoints are behind AWS WAF (405/202
goku challenge) for non-browser clients; findings recorded in AGENTS.md
- the shop search runs on Algolia; public search-only key lets us dump the
whole product index with plain requests (no browser, no captcha)
- gwshop.py: credential loading (env or gitignored algolia.json), index
listing, facet-partitioned full retrieval (search-only key cannot /browse
and pagination caps at 1000, so the index is partitioned on the
GameSystemsRoot hierarchy facet), normalization into the sorter's
source-agnostic catalog schema (system/group/faction/subfaction/class)
- data/gwshop_catalog.json: snapshot of prod-lazarus-product-en-ww,
3491 unique products, every product with full hierarchy path
- tools/: one-shot browse + catalog builder + shape inspector
Ben_Kosytorz