Add GW webshop catalog source (Algolia): dlphelper.gwshop + full dump
- warhammer.com shop HTML/listing endpoints are behind AWS WAF (405/202 goku challenge) for non-browser clients; findings recorded in AGENTS.md - the shop search runs on Algolia; public search-only key lets us dump the whole product index with plain requests (no browser, no captcha) - gwshop.py: credential loading (env or gitignored algolia.json), index listing, facet-partitioned full retrieval (search-only key cannot /browse and pagination caps at 1000, so the index is partitioned on the GameSystemsRoot hierarchy facet), normalization into the sorter's source-agnostic catalog schema (system/group/faction/subfaction/class) - data/gwshop_catalog.json: snapshot of prod-lazarus-product-en-ww, 3491 unique products, every product with full hierarchy path - tools/: one-shot browse + catalog builder + shape inspector
This commit is contained in:
@@ -18,3 +18,7 @@ manifest.jsonl
|
||||
# Never commit credentials
|
||||
*.token
|
||||
.env
|
||||
|
||||
# Algolia credentials (search-only key, still never commit)
|
||||
algolia.json
|
||||
gwshop_raw.json
|
||||
|
||||
Reference in New Issue
Block a user