Ben_Kosytorz
5fd5dacaf0
- warhammer.com shop HTML/listing endpoints are behind AWS WAF (405/202 goku challenge) for non-browser clients; findings recorded in AGENTS.md - the shop search runs on Algolia; public search-only key lets us dump the whole product index with plain requests (no browser, no captcha) - gwshop.py: credential loading (env or gitignored algolia.json), index listing, facet-partitioned full retrieval (search-only key cannot /browse and pagination caps at 1000, so the index is partitioned on the GameSystemsRoot hierarchy facet), normalization into the sorter's source-agnostic catalog schema (system/group/faction/subfaction/class) - data/gwshop_catalog.json: snapshot of prod-lazarus-product-en-ww, 3491 unique products, every product with full hierarchy path - tools/: one-shot browse + catalog builder + shape inspector
22 lines
644 B
Python
22 lines
644 B
Python
#!/usr/bin/env python3
|
|
"""One-shot browse of the GW Algolia index -> gwshop_raw.json (gitignored)."""
|
|
import json
|
|
import logging
|
|
import pathlib
|
|
import sys
|
|
|
|
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent.parent))
|
|
|
|
import dlphelper.gwshop as gw
|
|
|
|
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
|
|
|
|
app, key = gw.load_credentials()
|
|
client = gw.GwShopClient(app, key)
|
|
hits = client.browse("prod-lazarus-product-en-ww", hits_per_page=100)
|
|
print("total hits", len(hits))
|
|
pathlib.Path("gwshop_raw.json").write_text(
|
|
json.dumps(hits, ensure_ascii=False), encoding="utf-8"
|
|
)
|
|
print("written gwshop_raw.json")
|