extract stage + system-locked matching + provenance unit mode

- tools/extract_all.py: recursive extraction of all archives (incl.
  nested) into a plain-file tree; nested archives are deleted after
  successful extraction, plain files are hardlinked, failures go to
  .broken/, extract_manifest.jsonl records archive->dest->unit.
- sorter --provenance: classify each extraction directory once under
  the archive's name (files of one archive are one unit; inner
  structure kept), so loose STLs sort with their model set.
- matching discipline: a fuzzy match never leaves the file's system;
  system-less files need CROSS_SYSTEM_MIN (0.75) for a catalog unit,
  and a same-system faction jump below 0.75 is rejected. Fixes 40k
  files landing in Middle-earth/Blood Bowl/AoS buckets.
- speed for big trees: lru_cache on keyword matchers and pair scores,
  quick_ratio pruning; keywords: night lord, csm.
This commit is contained in:
2026-10-05 08:09:22 +02:00
parent 7e28518a52
commit 86fcb3aecc
7 changed files with 455 additions and 26 deletions
+2 -2
View File
@@ -133,13 +133,13 @@ SYSTEMS: dict[str, dict[str, list[str]]] = {
# -- Chaos -----------------------------------------------------------
"Chaos Space Marines": [
"chaos space marine", "black legion", "iron warriors", "word bearers",
"night lords", "alpha legion", "chaos terminator", "chosen", "havoc",
"night lords", "night lord", "alpha legion", "chaos terminator", "chosen", "havoc",
"raptor", "warp talon", "chaos cultist", "cultist", "daemon prince",
"chaos lord", "chaos sorcerer", "abaddon", "kharn", "ahriman",
"typhus", "fabius bile", "vashtorr", "defiler", "forgefiend",
"maulerfiend", "helldrake", "helbrute", "chaos spawn",
"obliterator", "chaos rhino", "chaos vindicator", "spartan assault",
"mastodon", "typhon",
"mastodon", "typhon", "csm",
],
"Death Guard": [
"death guard", "nurgle", "plague marine", "deathshroud", "mortarion",