Commit Graph
2 Commits
Author SHA1 Message Date
Ben_Kosytorz 5fd5dacaf0 Add GW webshop catalog source (Algolia): dlphelper.gwshop + full dump
- warhammer.com shop HTML/listing endpoints are behind AWS WAF (405/202
  goku challenge) for non-browser clients; findings recorded in AGENTS.md
- the shop search runs on Algolia; public search-only key lets us dump the
  whole product index with plain requests (no browser, no captcha)
- gwshop.py: credential loading (env or gitignored algolia.json), index
  listing, facet-partitioned full retrieval (search-only key cannot /browse
  and pagination caps at 1000, so the index is partitioned on the
  GameSystemsRoot hierarchy facet), normalization into the sorter's
  source-agnostic catalog schema (system/group/faction/subfaction/class)
- data/gwshop_catalog.json: snapshot of prod-lazarus-product-en-ww,
  3491 unique products, every product with full hierarchy path
- tools/: one-shot browse + catalog builder + shape inspector
2026-10-04 21:48:58 +02:00
Ben_Kosytorz cccf10a308 Sorter: fuzzysort files into the miniset-aligned library via hardlinks
- docs/SORTING.md: target hierarchy (game > faction group > faction >
  subfaction > class > unit, miniset.net naming), canonical-name policy,
  hardlink policy, unzip outlook
- data/library_catalog.json: canonical catalog (name/aliases/subfaction/class)
- dlphelper/sorter.py: inspect (zip via zipfile, 7z via 7z l), keyword
  classification incl. inner-name voting for hash-named archives, fuzzy
  matching (difflib), plan/apply split, hardlink preferred with copy
  fallback, append-only audit, existing targets skipped, run artifacts
  filtered, HTML-page detection for bad downloads
- sorter.py CLI (plan by default, --apply to act)
2026-10-04 21:17:52 +02:00