# Warhammer DLP Helper

A small "radarr for Warhammer proxy files" tool: point it at a Discord
channel/thread and an output directory, and it exports the channel, extracts
the encoded download links, and downloads everything - automatically choosing
the right downloader per host:

| Host        | Downloader | Notes |
|-------------|------------|-------|
| gofile.io   | built-in GoFile API client | one guest account per run (rate-limit friendly) |
| 0807.st     | curl | needs browser-like headers (`-A`, `-e https://0807.st/`) |

## Install

    pip install -r requirements.txt   # just `requests`
    docker pull tyrrrz/discordchatexporter:stable

## Usage

    export DISCORD_TOKEN="..."
    python3 dlp.py <channel_or_thread_id> /path/to/output

The token can also be passed with `--token` (or you'll be prompted, input
hidden). Useful options:

    --urls-file failed_urls.txt   retry a previous failure list, no re-export
    --password <pw>               for password-protected gofile links
    --only gofile|0807            process only one host
    --extract                     unzip archives into per-archive folders
    --hash                        record sha256 per file in the manifest
    --dry-run                     just show what would be downloaded
    --fresh                       force a fresh channel export

## Several channels / watching for updates

Put your channels into `subscriptions.json`:

```json
{
  "subscriptions": [
    {"name": "some-eldar",
     "channel_id": "1455993943471820842",
     "out_dir": "/big/storage/warhammer/some-eldar"}
  ]
}
```

then run (e.g. from a daily cron/systemd timer):

    python3 dlp.py --subs subscriptions.json

Subs mode re-exports each channel to pick up new links; `state.json` and the
global store make sure only genuinely new URLs are downloaded.

## Already downloaded the old way?

`--assume-done` imports a plain URL list into the global store so those links
are never fetched again (no verification — use deliberately):

    python3 dlp.py --assume-done old_urls.txt

## Output layout

    OUT_DIR/
      export.json           Discord channel export (cached between runs)
      state.json            per-URL status; re-runs skip everything marked "ok"
      failed_urls.txt       URLs that failed after all retries
      manifest.jsonl        one line per downloaded file (url, source message, path, size)
      downloads/
        gofile/<content>/<folders>/<files>
        0807/<id>.zip
| `downloads/` | actual content; `.part` files mean "interrupted, will be resumed" |
| `~/.local/share/dlphelper/urls.json` | global cross-run store of finished URLs (never re-downloaded, in any channel) |

Runs are resumable and idempotent: interrupted downloads use `.part` files and
finished ones are skipped on the next run. Individual failures never stop the
run; after 3 consecutive failures it pauses for a cooldown (rate-limit
protection) and continues.

## Why not use Lysagxra/GoFileDownloader directly?

That tool creates a new GoFile guest account **per URL** and calls
`sys.exit(1)` on the first failure, which is exactly why runs died after ~6
downloads. This project reimplements its (working) API scheme - headers,
`X-Website-Token`, download links with the `accountToken` cookie - but creates
one guest account per run, records failures per file, and keeps going.

`parts/grab.sh` is the original groundstone script, kept for reference.