Files

T

Ben_Kosytorz 137efa7e6c feat(rust): port discord, gofile, zero807, extract_all and catalogue

Parallel module ports (behavior parity with the python versions):

- src/discord.rs: DCE docker export (token via env only), base64/plain URL
  extraction with grab.sh-compatible regexes, host classification; 11 tests
- src/gofile.rs: GoFile API client — one guest account per run, linear
  backoff on account creation, sha256 website token, recursive content walk,
  streaming .part downloads; 7 tests (offline goldens from python hashlib)
- src/zero807.rs: curl downloader with UA+Referer contract, dead-link
  detection; 4 tests
- src/extract.rs: recursive extraction (sevenz-rust2 for 7z, zip crate,
  unrar for rar, system tar/gzip/bzip2/xz for tarballs), nested BFS
  expansion, .broken quarantine, extract_manifest.jsonl, idempotent re-runs;
  8 tests
- src/catalogue.rs: keyword classifier over the dumped taxonomy, manifest/
  directory scanning, overrides, md+jsonl reports, audited moves; 7 tests
  incl. all 17 python test cases

2026-10-09 14:17:02 +02:00

data

feat(rust): difflib port + taxonomy data dump

2026-10-09 13:44:28 +02:00

src

feat(rust): port discord, gofile, zero807, extract_all and catalogue

2026-10-09 14:17:02 +02:00

tools

feat(rust): difflib port + taxonomy data dump

2026-10-09 13:44:28 +02:00

.gitignore

chore: start transition layout — python into version-python/, scaffold version-rust/

2026-10-09 13:18:31 +02:00

Cargo.lock

feat(rust): port discord, gofile, zero807, extract_all and catalogue

2026-10-09 14:17:02 +02:00

Cargo.toml

feat(rust): port discord, gofile, zero807, extract_all and catalogue

2026-10-09 14:17:02 +02:00

README.md

feat(rust): port state.py, globalstore.py and manifest appending

2026-10-09 13:27:59 +02:00

README.md

version-rust

Rust rewrite of version-python (same repo, main branch still carries the flat python layout). Goal: one concise binary + webapp, eventually replacing the python tooling.

Why this structure

main stays untouched (flat python layout) until the rewrite reaches parity.
This branch re-homes the python code into version-python/ (via git mv, history preserved) and starts version-rust/ beside it.
version-python/AGENTS.md still documents the python stage — its rules (secrets, failure policy, idempotency, rate limiting) apply to the rewrite too and are the contract the Rust version must keep.

Planned crate choices (initial, adjust as modules get ported)

Concern	Python today	Rust plan
7z extraction	7z CLI, cp437 name repair, /var/tmp fallback	`sevenz-rust2` (pure Rust; LZMA/LZMA2/PPMD/AES-256 decrypt, v0.23, active)
zip extraction	7z CLI	`zip` crate
RAR extraction (critical)	7z CLI	`unrar.rs` (muja/unrar.rs — bindings to the official UnRAR library, RAR5 support, extraction only). Note: UnRAR license is freeware-not-OSI and forbids building a RAR compressor — acceptable, we only extract, and compression later is 7z-only (`sevenz-rust2` `ArchiveWriter`) anyway
HTTP client	`requests`	`reqwest` (blocking first, async where the webapp needs it)
sha256 (gofile website token, --hash manifest)	hashlib	`sha2`
Fuzzy matching (sorter)	difflib-style ratios	`strsim` — port thresholds exactly, add unit tests
CLI	argparse	`clap`
Webapp	—	`axum` (CLI parity first, webapp second)
Discord export	DCE docker container (external)	unchanged: shell out to docker

Port order (suggested)

state / globalstore / manifest — done, see src/state.rs, src/globalstore.rs, src/manifest.rs (schemas backward compatible).
hosters: gofile (one guest account per run!), zero807 (UA + Referer).
extraction + manifest appender.
catalogue / sorter (hardest logic; port the matching rules documented in version-python/AGENTS.md §7 with tests).
webapp on top.