Catalogue: classify collection into system/faction/force-org class
- dlphelper/warhammer_data.py: keyword taxonomy (40k factions incl. Chaos and Xenos super-faction units, plus AoS/Heresy/Kill Team/Necromunda/Epic/OW) - dlphelper/catalogue.py: manifest+directory scanner, conservative keyword classifier (plural-tolerant word boundaries, longest match wins), markdown and jsonl reports, overrides support, audited --apply moves - catalogue.py CLI; tests/test_catalogue.py offline classifier test - real-data check on the user's dirs: 1624/3333 files classified so far
This commit is contained in:
@@ -52,6 +52,29 @@ then run (e.g. from a daily cron/systemd timer):
|
||||
Subs mode re-exports each channel to pick up new links; `state.json` and the
|
||||
global store make sure only genuinely new URLs are downloaded.
|
||||
|
||||
## Cataloguing the collection
|
||||
|
||||
`catalogue.py` classifies every downloaded file into game system, faction and
|
||||
force-organization class (HQ / Troops / Elites / Fast Attack / Heavy Support /
|
||||
Flyer / Dedicated Transport / Lords of War / Fortification) using a keyword
|
||||
taxonomy in `dlphelper/warhammer_data.py`:
|
||||
|
||||
# preview a dlp.py run dir (manifest + downloads)
|
||||
python3 catalogue.py /path/to/run_dir
|
||||
|
||||
# include the manual-era directories
|
||||
python3 catalogue.py /path/to/run_dir --scan /big/storage/manual --out-md catalogue.md
|
||||
|
||||
# catalogue arbitrary dirs only
|
||||
python3 catalogue.py --scan /big/storage/manual
|
||||
|
||||
# actually reorganize into a library tree (audited, only classified files)
|
||||
python3 catalogue.py <run_dir> --apply --library /big/storage/warhammer_library
|
||||
|
||||
Outputs `catalogue.md` (readable) and `catalogue.jsonl` (machine, with source
|
||||
attribution per file). Unclassified files are listed, never guessed; fix them
|
||||
with an overrides JSON (`--overrides overrides.json`).
|
||||
|
||||
## Already downloaded the old way?
|
||||
|
||||
`--assume-done` imports a plain URL list into the global store so those links
|
||||
|
||||
Reference in New Issue
Block a user