Review pass: fix bugs, silence clippy, correct docs

Bugs:
- extract: pick the OUTERMOST archive suffix (sfx.first(), not last())
  so x.tar.gz uses the gz->tar path
- gwshop: truncate the error snippet by chars, not bytes (UTF-8 panic)
- sorter: py_stem_split now matches pathlib (0 < i < len-1) so a
  trailing dot is not treated as a suffix
- sorter::apply_plan: a per-file link/copy/mkdir failure is logged and
  skipped instead of aborting the whole apply (never-abort contract)
- orchestrate/state: add State::mark_logged and use it for every
  per-URL record so a state-save error cannot abort a run

Cleanups:
- remove dead is_archive_name and unused SourceInfo.manifest field
- mark kept-but-unused capabilities with allow(dead_code) + rationale
- let-chains / is_ok_and / is_multiple_of / replace([..]) / sort_by_key
  and other idiomatic simplifications across modules
- difflib: SequenceMatcher owns its char vecs (bit-identical parity kept)
- --only now enforces gofile|0807 via value_parser

Docs:
- SORTING.md: drop phantom sorter_plan.jsonl, os.link -> std::fs::hard_link,
  zipfile -> the zip crate
- README: add --since to the dlp synopsis, clarify --subs/--assume-done
- .gitignore: ignore catalogue/sorter/_logs/subscriptions run artifacts

Validation: cargo fmt --check clean, cargo clippy --all-targets 0 warnings,
cargo test 87 passed / 0 failed.
This commit is contained in:
2026-10-11 13:27:33 +02:00
parent 93b86d34af
commit f631e351f8
13 changed files with 294 additions and 245 deletions
+10 -2
View File
@@ -50,11 +50,16 @@ All original Python entry points are ported and live-validated (see below).
## CLI
```
dlphelper dlp <channel_id> <out_dir> [--subs FILE] [--urls-file FILE] [--token]
dlphelper dlp <channel_id> <out_dir> [--since WHEN] [--urls-file FILE] [--token]
[--password] [--retries N] [--retry-delay N] [--cooldown-failures N]
[--cooldown-seconds N] [--only gofile|0807] [--limit N] [--shuffle]
[--hash] [--extract] [--fresh|--no-fresh] [--dry-run]
[--global-state FILE] [--no-global] [--assume-done FILE]
[--global-state FILE] [--no-global]
dlphelper dlp --subs FILE # alternative mode: iterate every channel
# in the list (channel_id/out_dir unused)
dlphelper dlp --assume-done FILE # alternative mode: mark URLs as done, exit
dlphelper catalogue [run_dir] [--scan DIR]... [--manifest FILE]
[--overrides FILE] [--out-md FILE] [--out-jsonl FILE] [--apply --library DIR]
@@ -78,6 +83,9 @@ Notes:
(bookkeeping stays out of the library, which may be a seeded torrent).
- The sorter implements the rules in `docs/SORTING.md` — that document is the
working agreement; change it first, then the code.
- `dlp --since` limits a run to recent messages (e.g. `5d`, `12h`,
`2026-10-04`); `--subs` and `--assume-done` are alternative modes that ignore
`<channel_id>`/`<out_dir>`.
- Retry loops, cooldowns and the "one gofile guest account per run" rule are
ported verbatim from `dlp.py`; a failure in one URL never aborts the run.
- Ctrl-C: state.json is saved after every URL; re-run to resume (the python