Compare commits

..
5 Commits
Author SHA1 Message Date
Ben_Kosytorz 246b7969d6 Document daemon and packaging; refresh status across docs
Root README and documentation README: daemon + Docker/Nix status.
Roadmap: daemon checked off, real overlay peering stays open.
pilot.md: docker/nix and sn-node daemon mentioned as the comfortable
path beside the manual steps. AGENTS.md server status line updated.
2026-09-24 20:13:02 +02:00
Ben_Kosytorz 57b9b44d8a Add Docker and Nix packaging for the node
server/docker/: multi-stage image (rust:1-slim build, trixie runtime)
with all four binaries plus the pinned yggdrasil 0.5.14 .deb
(sha256-verified, fetched from the official release), a compose
example per household (host network, NET_ADMIN, /dev/net/tun,
init: true), a German-commented node.toml.example (safe default:
leech mode) and a README covering privileges, two-household wiring,
leech mode and the kill switch. Image build and smoke tests verified
locally with docker 29.8.1.

flake.nix: builds the workspace as one spidernet package via
rustPlatform.buildRustPackage (cargoLock against the committed
Cargo.lock) plus an experimental NixOS module; written without a
local Nix and marked accordingly.
2026-09-24 20:12:51 +02:00
Ben_Kosytorz f2f3609b87 Add sn-node daemon: supervise yggdrasil sidecar + in-process exit
sn-node daemon [--config node.toml] [--check] runs one household node
from a validated TOML config: generates the yggdrasil config with a
persistent node key in the state dir (PrivateKeyPath takes precedence
per the yggdrasil config reference, so the identity survives restarts),
waits for the overlay address via the admin socket, then serves the
sn-exit proxy in-process (or runs overlay-only in leech mode).
SIGINT/SIGTERM stop the accept loop, SIGTERM the sidecar (5s grace,
then SIGKILL); unexpected sidecar death shuts the daemon down with an
error. sn-exit gains an additive serve_until() with a shutdown future;
serve() keeps its behavior. Workspace: tokio gains process+signal,
serde/toml added. 73 tests green, clippy clean.
2026-09-24 20:12:44 +02:00
Ben_Kosytorz d8bd14ab1b Add pilot handbook and document sn-fair v0
pilot.md: step-by-step for connecting two households (link-agnostic),
measurement protocol M0-M4 (link baseline, uplink baseline, pooled,
concurrent downloads, sn-fair Soll/Ist comparison), safety/legal
checklist, and a results section to be filled after the pilot.
Cross-linked from Home, documentation README, roadmap, architecture
and server README; roadmap marks sn-fair v0 done.
2026-09-24 18:46:47 +02:00
Ben_Kosytorz ecf8848ca4 Add sn-fair v0: weighted max-min fairness scheduler with leech policy
Two-class allocation engine: contributors (configurable weight) get
weighted max-min shares of the pooled capacity, leeches (receive-only,
explicitly allowed) share the leftover capacity. Deterministic u128
arithmetic, validated newtypes, demo CLI in Mbit/s for the pilot
Soll/Ist comparison. 19 tests, workspace clippy clean.
2026-09-24 18:46:38 +02:00
31 changed files with 3349 additions and 37 deletions
+3 -1
View File
@@ -42,7 +42,9 @@ Das Projekt besteht aus zwei Hälften:
- Kernfeatures: Multi-Exit-Bandbreiten-Pooling, segmentierte Downloads, - Kernfeatures: Multi-Exit-Bandbreiten-Pooling, segmentierte Downloads,
Fairness-Scheduler, Nachbarschafts-Cache, **Leech-Modus** (Empfangen ohne Fairness-Scheduler, Nachbarschafts-Cache, **Leech-Modus** (Empfangen ohne
Exit-Bereitstellung ist ausdrücklich erlaubt — Netz unter Nachbarn) Exit-Bereitstellung ist ausdrücklich erlaubt — Netz unter Nachbarn)
- Status: Konzeptphase, Details in `server/README.md` - Status: Code existiert und ist getestet (`sn-fetch`, `sn-node` inkl.
Daemon, `sn-exit`, `sn-fair`; Deployment via Docker/Nix) — Details in
`server/README.md`
## Git / Gitea ## Git / Gitea
+10 -2
View File
@@ -25,7 +25,7 @@ Datensammlung, keine Bezahlung.
| Ordner/Datei | Inhalt | Einstieg | | Ordner/Datei | Inhalt | Einstieg |
|---|---|---| |---|---|---|
| `documentation/` | Idee, Ethos, Architektur, Rechtslage, Roadmap (Deutsch, Obsidian-freundlich mit `[[Wikilinks]]`; `documentation/Home.md` ist die Map of Content) | [README](documentation/README.md) | | `documentation/` | Idee, Ethos, Architektur, Rechtslage, Roadmap (Deutsch, Obsidian-freundlich mit `[[Wikilinks]]`; `documentation/Home.md` ist die Map of Content) | [README](documentation/README.md) |
| `server/` | Die Software (Rust): `sn-fetch`, `sn-node`, `sn-exit`; geplant `sn-fair`, `sn-cache`, `sn-freenet` | [README](server/README.md) | | `server/` | Die Software (Rust): `sn-fetch`, `sn-node`, `sn-exit`, `sn-fair`; geplant `sn-cache`, `sn-freenet` | [README](server/README.md) |
| `scripts/` | Repo-Setup (`bootstrap-gitea.sh`) | — | | `scripts/` | Repo-Setup (`bootstrap-gitea.sh`) | — |
| `AGENTS.md` | Arbeitsanweisungen für Coding-Agents und Contributor:innen | [AGENTS.md](AGENTS.md) | | `AGENTS.md` | Arbeitsanweisungen für Coding-Agents und Contributor:innen | [AGENTS.md](AGENTS.md) |
| `LICENSE` | Code-Lizenz: AGPL-3.0-only | [LICENSE](LICENSE) | | `LICENSE` | Code-Lizenz: AGPL-3.0-only | [LICENSE](LICENSE) |
@@ -53,8 +53,16 @@ Prototyp (Konzeptphase + erster Code):
- **`sn-exit` v0** funktioniert: kontingentierter Uplink-Proxy (Byte-Relay, - **`sn-exit` v0** funktioniert: kontingentierter Uplink-Proxy (Byte-Relay,
CONNECT-Tunnel für HTTPS, SSRF-Schutz, Source-Allowlist) — End-to-End CONNECT-Tunnel für HTTPS, SSRF-Schutz, Source-Allowlist) — End-to-End
mit `sn-fetch` gegen einen echten Server verifiziert mit `sn-fetch` gegen einen echten Server verifiziert
- **`sn-fair` v0** funktioniert: Fairness-Engine (gewichtetes Max-Min,
Leech-Politik) + Demo-CLI — die Soll-Aufteilung für den Piloten
- **`sn-node` Daemon** funktioniert: ein Befehl orchestriert Yggdrasil-
Sidecar + Exit-Dienst (`sn-node daemon --config node.toml`, TOML-Konfig,
Kill-Switch-Semantik)
- **Deployment:** Docker-Image (gebaut + getestet) und Nix-Flake
(unverifiziert) — Details in [server/README.md](server/README.md)
- **Nächste Schritte:** Pilot mit 2–3 Haushalten (Yggdrasil-Overlay - **Nächste Schritte:** Pilot mit 2–3 Haushalten (Yggdrasil-Overlay
verkabeln, Messwerte), Fairness-Scheduler — siehe verkabeln, Messwerte — Handbuch:
[pilot.md](documentation/ideen/pilot.md)) — siehe
[Roadmap](documentation/ideen/roadmap.md) [Roadmap](documentation/ideen/roadmap.md)
## Doku lesen (Obsidian) ## Doku lesen (Obsidian)
+3
View File
@@ -26,6 +26,7 @@ Nachbarschaft betreibt ihre eigene Instanz nach eigenen Regeln.
| [[Splitting]] | Was sich splitten lässt: Ranges, Flow-Verteilung, Realtime | | [[Splitting]] | Was sich splitten lässt: Ranges, Flow-Verteilung, Realtime |
| [[Lizenz]] | Lizenz-Optionen (Code & Doku) | | [[Lizenz]] | Lizenz-Optionen (Code & Doku) |
| [[Roadmap]] | Phasen: Pilot → MVP → Rollout → Freenet → Backbone | | [[Roadmap]] | Phasen: Pilot → MVP → Rollout → Freenet → Backbone |
| [[Pilot]] | Pilot-Handbuch: 2 Haushalte verkabeln, messen, dokumentieren |
| [[Server-README|Server-Konzept]] | Software-Komponenten, Code-Standards | | [[Server-README|Server-Konzept]] | Software-Komponenten, Code-Standards |
## Lesereihenfolge zum Einstieg ## Lesereihenfolge zum Einstieg
@@ -44,6 +45,8 @@ Nachbarschaft betreibt ihre eigene Instanz nach eigenen Regeln.
- [[Splitting]] ↔ [[Architektur]] (Schicht 2) — was der Exit-Pool tatsächlich nutzen kann - [[Splitting]] ↔ [[Architektur]] (Schicht 2) — was der Exit-Pool tatsächlich nutzen kann
- [[Overlay]] ↔ [[Architektur]] (Schicht 1) — Routing-Entscheidung und deren Grenzen - [[Overlay]] ↔ [[Architektur]] (Schicht 1) — Routing-Entscheidung und deren Grenzen
- [[Overlay]] ↔ [[Server-README]] — `sn-node` generiert die Yggdrasil-Config, `sn-fetch` nutzt `Egress::Proxy` - [[Overlay]] ↔ [[Server-README]] — `sn-node` generiert die Yggdrasil-Config, `sn-fetch` nutzt `Egress::Proxy`
- [[Pilot]] ↔ [[Roadmap]] (Phase 1) und [[Overlay]] §Deployment — der Pilot ist der erste Real-Test
- [[Pilot]] §M4 ↔ [[Architektur]] §Fairness-Scheduler — `sn-fair` berechnet die Soll-Aufteilung
- [[Lizenz]] ↔ [[Ethos]] §6 (Transparenz) und `freenet-core` (AGPL) - [[Lizenz]] ↔ [[Ethos]] §6 (Transparenz) und `freenet-core` (AGPL)
## Außerhalb der Doku ## Außerhalb der Doku
+8 -4
View File
@@ -17,6 +17,7 @@ unabhängige SpiderNet-Instanzen geben; wir providen nur die Werkzeuge.
- [[Splitting]] — Was sich über mehrere Exits splitten/verteilen lässt (und was nicht) - [[Splitting]] — Was sich über mehrere Exits splitten/verteilen lässt (und was nicht)
- [[Lizenz]] — Lizenz-Optionen und Empfehlung - [[Lizenz]] — Lizenz-Optionen und Empfehlung
- [[Roadmap]] — Phasenplan vom Pilot bis zum Nachbarschafts-Backbone - [[Roadmap]] — Phasenplan vom Pilot bis zum Nachbarschafts-Backbone
- [[Pilot]] — Pilot-Handbuch: Schritt-für-Schritt + Messprotokoll (Phase 1)
- [[Server-README|Server-Konzept]] — geplante Software (`sn-node`, `sn-fetch`, …) - [[Server-README|Server-Konzept]] — geplante Software (`sn-node`, `sn-fetch`, …)
## Die Idee in einem Satz ## Die Idee in einem Satz
@@ -29,10 +30,13 @@ ist — ohne Mehrkosten und mit besserer Versorgung schwacher Haushalte.
## Status ## Status
Prototypphase (siehe [[Roadmap]]): `sn-fetch` (segmentierter Multi-Exit- Prototypphase (siehe [[Roadmap]]): `sn-fetch` (segmentierter Multi-Exit-
Download mit Retry + Egress), `sn-node` v0 (Yggdrasil-Config + Status) Download mit Retry + Egress), `sn-node` v0 + Daemon (Yggdrasil-Config,
und `sn-exit` v0 (kontingentierter Uplink-Proxy mit SSRF-Schutz) sind Status, Sidecar-/Exit-Orchestrierung), `sn-exit` v0 (kontingentierter
implementiert und getestet — Details in [`server/README.md`](../server/README.md). Uplink-Proxy mit SSRF-Schutz) und `sn-fair` v0 (Fairness-Engine +
Nächster Meilenstein: Pilot mit 2–3 Haushalten (echtes Overlay, Messwerte). Demo-CLI) sind implementiert und getestet — Details in
[`server/README.md`](../server/README.md), Deployment via Docker/Nix
(`server/docker/`, `flake.nix`). Nächster Meilenstein: Pilot mit
2–3 Haushalten (Handbuch: [[Pilot]]).
## Verwandte Projekte (Prior Art) ## Verwandte Projekte (Prior Art)
+5
View File
@@ -133,6 +133,11 @@ siehe [[Splitting]].
- Exit-Kontingente (Volumenlimit, Wartungszeit) werden respektiert - Exit-Kontingente (Volumenlimit, Wartungszeit) werden respektiert
- Leech-Nodes: niedrigste Priorität bei Knappheit, nie ausgeschlossen - Leech-Nodes: niedrigste Priorität bei Knappheit, nie ausgeschlossen
Status: **v0 implementiert** — gewichtetes Max-Min in zwei Klassen:
Contributors teilen die volle Kapazität nach Max-Min, Leeches teilen
die Restkapazität (bei Sättigung zuerst gedrosselt, nie ausgeschlossen).
Details in `server/README.md`; Soll-Ist-Vergleich im Piloten: [[Pilot]] §M4.
### Nachbarschafts-Cache (`sn-cache`) ### Nachbarschafts-Cache (`sn-cache`)
- Transparenter Shared-Cache (lancache-Prinzip): große, populäre Objekte - Transparenter Shared-Cache (lancache-Prinzip): große, populäre Objekte
+214
View File
@@ -0,0 +1,214 @@
# Pilot: zwei Haushalte verbinden und messen
> Prototyp-Betrieb, **keine Garantie** (AGPL §§ 15/16) — Nutzung auf eigene
> Verantwortung, siehe [[Recht]] und [[Server-README]]. Persönliche Daten
> (Namen, Adressen, Keys realer Nachbarn) bleiben **außerhalb des Repos**.
Ziel der Phase 1 (siehe [[Roadmap]]) ist der Beweis, dass Bandbreiten-
Pooling über zwei echte Anschlüsse real funktioniert — nicht auf dem
Papier. Der Pilot liefert:
1. **Messwerte:** 1-Exit-Baseline vs. 2-Exit-gepoolt (gleiche Datei)
2. **Fairness-Daten:** zwei gleichzeitige Downloads teilen sich den Pool
3. **Stolpersteine:** was die Doku/Software dadurch lernen muss
Der Pilot ist **link-agnostisch**: Ob die Häuser per Glasfaser,
SFP+-Direktkabel, Switch im Keller, 60-GHz-Richtfunk oder normalem
Ethernet verbunden sind, entscheidet die Nachbarschaft — die Software
misst, was da ist (siehe [[Architektur]], Schicht 0). Alle
Angaben zu Link-Typen in dieser Datei sind Beispiele.
## Voraussetzungen
### Teilnehmer
- 2 Haushalte (ideal 3, der dritte als **Leech** — Empfang ohne
Exit-Angebot, um genau diesen Modus mitzutesten)
- Ein persönlicher Termin; technisches Onboarding gesicht zu gesicht
(Nachbarn sind der Trust-Anchor, siehe [[Architektur]])
### Hardware (je Haushalt, Beispiele)
- Ein Mini-PC oder Einplatinenrechner (≥ 2 GB RAM, Debian o. ä.) als
Node-Host — je eine NIC Richtung Heim-Router und Richtung
Nachbarschafts-Link (übergangsweise reicht auch eine NIC am
Mesh-Switch)
- Verbindungsweg — **eins davon genügt:**
- Glasfaser/DAC-Direktkabel zwischen den SFP/SFP+-Ports
- gemeinsamer Switch im Keller/Treppenhaus
- 60-GHz-Richtfunk-Pair (Sichtlinie nötig, siehe [[Recht]] §3)
- vorhandenes Ethernet/WLAN (langsamer, aber null Aufwand)
### Software
- [yggdrasil](https://yggdrasil-network.github.io/) (Binary oder
Debian-Paket)
- SpiderNet-Binaries bauen und rüberkopieren:
```sh
cd server
cargo build --release --workspace
# → target/release/sn-node, sn-exit, sn-fetch, sn-fair
```
Alternativ (bequemer ab dem zweiten Haushalt): **Docker** —
`server/docker/README.md` (Image mit allen Binaries + Yggdrasil,
Compose-Beispiel) — oder **Nix** via `flake.nix`. Der Node-Daemon
(`sn-node daemon --config node.toml`, siehe `server/README.md`)
orchestriert Overlay + Exit in einem Befehl; die manuellen Schritte
unten zeigen trotzdem erst den Geräteweg, damit man sieht, was
gepoolt wird.
### Vor dem Termin (Checkliste)
- [ ] ISP-AGB der Teilnehmenden geprüft (Weitergabe an Dritte,
siehe [[Recht]] §2)
- [ ] Nachbarschafts-Abkommen geklärt: wer spendiert wie viel
(Quota), Leechen ausdrücklich erlaubt, alles jederzeit widerruflich
- [ ] Hardware vorhanden und bootfähig
- [ ] Binaries gebaut und auf beide Nodes kopiert
- [ ] Testdatei gewählt: Range-fähiger Server, ~750 MB, bekannte
SHA-256 (im PoC hat sich eine Debian-Netinst-ISO bewährt; die
offizielle `SHA256SUMS` der jeweiligen Version nutzen)
## Termintag: Schritt-für-Schritt
Die technischen Details (Config, Schlüssel, Admin-Socket) stehen in
[[Overlay]] §Deployment — hier die Reihenfolge mit Checkpunkten.
1. **Physik verbinden.** Link herstellen, L2-Test: `ping` über die
lokale Verbindung. Optional `iperf3` über den Link → das ist
**Messpunkt M0** (Kapazität des Nachbarschafts-Links).
2. **Yggdrasil einrichten (beide Häuser).**
```sh
sn-node genconf --listen "tcp://[::]:1337" \
--peer "tcp://[<Mesh-IP des Nachbarn>]:1337" \
--allow-key <Public-Key des Nachbarn> \
--private-key-path /etc/spidernet/node.key \
--out /etc/yggdrasil.conf
yggdrasil -useconffile /etc/yggdrasil.conf
sn-node status # eigene Overlay-Adresse + Peering-Status
```
- Im selben LAN reicht Multicast-Auto-Peering (ohne `--peer`).
- Config enthält den **privaten** Key (`chmod 600` macht
`genconf` selbst) — niemals teilen oder committen.
- Die eigenen Adressen/Keys werden **persönlich** ausgetauscht,
nicht per Chat/Repo.
3. **Overlay-Test.** `sn-node status` zeigt auf *beiden* Seiten den
Nachbarn als Peer; `ping <Overlay-Adresse des Nachbarn>` geht.
4. **Exit starten (Haus B spendiert):**
```sh
sn-exit --bind "[<Overlay-Adresse B>]:8080" \
--allow <Overlay-Adresse A> \
--quota-mib 1024
```
- `--allow` nie offen lassen (ohne Liste ist nur Loopback
erlaubt, das ist der Dev-Mode).
- Test von Haus A: ein HTTP-Request gegen einen Origin über
`http://[<Overlay-Adresse B>]:8080` als Proxy (z. B. mit
`sn-fetch --probe-only` oder `curl -x`).
5. **Erster gepoolter Download (Haus A):**
```sh
sn-fetch <url> -o testdatei.iso \
-e "1=1,2=1@http://[<Overlay-Adresse B>]:8080" \
--sha256 <erwarteter-hash>
```
6. **Kill-Switch-Test.** `sn-exit` auf B stoppen → laufender und
nächster Download fällt auf den eigenen Uplink zurück (Retry mit
Exit-Reassignment). Danach B wieder starten. Beides dokumentieren:
Dauer der Störung, verlorene Segmente.
Wichtig zur Beruhigung: SpiderNet v0 installiert **keine
Routing-Änderung** im Heimnetz. Geräte nutzen wie gewohnt den eigenen
Router; nur `sn-fetch`-Downloads gehen (explizit) in den Pool. Alles
stoppen (`sn-exit`, `yggdrasil`) = das Haus ist wie vorher.
## Messprotokoll
Regeln für belastbare Werte:
- Gleiche Testdatei, gleiche Uhrzeit ± 1 h (ISP-Congestion schwankt
tageszeitabhängig)
- Pro Szenario **3 Läufe**, Median notieren; zwischen Läufen Pause
- Nichts anderes auf den Nodes laufen lassen; Messwerte immer mit
Datum/Uhrzeit und den beidseitigen Uplink-Tarifen notieren
### Szenarien
| # | Szenario | Wo | Was gemessen wird |
|---|---|---|---|
| M0 | Nachbarschafts-Link-Kapazität | A↔B | `iperf3` über den lokalen Link |
| M1 | Uplink-Baseline | je Haus | nur eigener Uplink (`-e "1=1"`), ISP-Kapazität |
| M2 | Gepoolt, 1 Download | A | 2 Exits (`-e "1=1,2=1@http://[B]:8080"`), Gesamt-Durchsatz vs. M1 |
| M3 | Gepoolt, Konkurrenz | A + B | zwei `sn-fetch`-Läufe gleichzeitig, beide über beide Exits — wie teilen sich die Aufstellung |
| M4 | Fairness-Soll | (Simulation) | `sn-fair` mit den M1/M2-Werten → Soll-Aufteilung zum Vergleich mit M3 |
### Tabelle M2/M3 (Vorlage, pro Lauf eine Zeile)
| Lauf | Szenario | Datum/Zeit | Durchsatz Mbit/s | Dauer | Segmente | Retries | Bemerkung |
|---|---|---|---|---|---|---|---|
| 1 | M2 | | | | | | |
| 2 | M2 | | | | | | |
| 3 | M2 | | | | | | |
Zusätzlich notieren: Quota-Rest auf B, CPU/RAM der Node-Hosts,
(evtl.) Firmware/Treiber des Link-Wegs.
### Fairness-Soll mit `sn-fair` (M4)
Die gemessenen Uplink-Kapazitäten (M1) als Pool-Kapazität und die
gleichzeitigen Nachfragen (M3) einsetzen — das CLI arbeitet in
Mbit/s:
```sh
sn-fair --capacity <M1-A + M1-B, Mbit/s> \
--participant 1=contributing \
--participant 2=contributing \
--participant 3=leeching \
--demand 1=<Nachfrage A> \
--demand 2=<Nachfrage B> \
--demand 3=<Nachfrage Leech>
```
Erwartung (Regelwerk in [[Architektur]] §Fairness-Scheduler):
Contributors bekommen die Kapazität nach max-min Fairness (gewichtet),
Leeches profitieren von Restkapazität — bei Sättigung zuerst gedrosselt,
nie ausgeschlossen. Weicht M3 (Ist) deutlich von M4 (Soll) ab, landet
das als Erkenntnis in der Ergebnisse-Sektion unten (die echte
Runtime-Integration in `sn-fetch` ist Phase 2, siehe [[Roadmap]]).
## Sicherheit & Recht (Checkliste während des Piloten)
- [ ] `sn-exit` nur mit `--allow` (Allowlist) und Quota laufen lassen
- [ ] SSRF-Schutz nie deaktivieren (verhindert Nachbarn-Zugriff auf
Heim-LAN/Admin-Sockets des Exit-Haushalts)
- [ ] Keine Port-Forwards ins Internet, außer ein Peering braucht es
zwingend (siehe [[Overlay]] §Grenzen)
- [ ] Logs minimal und lokal (keine Inhaltslogs, siehe [[Recht]] §4)
- [ ] Haftungsausschluss ist den Teilnehmenden bekannt gezeigt
- [ ] Sperrbarkeit praktisch geübt: Exit stoppen = Kill-Switch,
Teilnehmersperre = `--allow`-Liste ändern
## Ergebnisse
*(wird nach dem Piloten gefüllt — Messwerte, Erkenntnisse und
Stolpersteine gehören hier ins Repo, nicht in den Chat)*
### Messwerte
*(Tabelle ergänzen — M0 bis M4, Mediane der 3 Läufe)*
### Erkenntnisse
*(kurz, ehrlich, mit Quellen/Logs wo sinnvoll)*
### Stolpersteine & Anpassungen
*(was in [[Architektur]], [[Overlay]], [[Roadmap]] oder der Software
geändert werden muss, damit der nächste Pilot weniger stolpert)*
+10 -4
View File
@@ -14,6 +14,7 @@ endet mit etwas, das man benutzen kann.
## Phase 1 — Pilot (2–3 Haushalte) ## Phase 1 — Pilot (2–3 Haushalte)
Ziel: Beweis, dass Pooling über zwei Anschlüsse real funktioniert. Ziel: Beweis, dass Pooling über zwei Anschlüsse real funktioniert.
Handbuch mit Messprotokoll: [[Pilot]].
- [x] `sn-fetch` PoC (lokal): Range-Probe, gewichteter Planner, parallele - [x] `sn-fetch` PoC (lokal): Range-Probe, gewichteter Planner, parallele
Segment-Downloads, Reassembly + SHA-256 — verifiziert gegen einen Segment-Downloads, Reassembly + SHA-256 — verifiziert gegen einen
@@ -23,14 +24,19 @@ Ziel: Beweis, dass Pooling über zwei Anschlüsse real funktioniert.
end-to-end getestet end-to-end getestet
- [x] `sn-node` v0: Yggdrasil-Config-Generierung (`genconf`) + Admin-Socket- - [x] `sn-node` v0: Yggdrasil-Config-Generierung (`genconf`) + Admin-Socket-
Client (`status`) — die Overlay-Anbindung beginnt Client (`status`) — die Overlay-Anbindung beginnt
- [x] `sn-node` Daemon v1: ein Befehl orchestriert Yggdrasil-Sidecar +
Exit-Dienst (TOML-Konfig, Node-Key-Persistenz, Kill-Switch-,
Supervisor-Semantik) — Deployment via Docker/Nix vorhanden
- [ ] Yggdrasil-Overlay zwischen 2 Haushalten (Kabel oder Kurzstrecke - [ ] Yggdrasil-Overlay zwischen 2 Haushalten (Kabel oder Kurzstrecke
WLAN/60-GHz), `sn-node`-Daemon orchestriert den Node WLAN/60-GHz) aufsetzen
- [x] `sn-exit` v0: HTTP-Proxy auf der Yggdrasil-Adresse (Kontingente, - [x] `sn-exit` v0: HTTP-Proxy auf der Yggdrasil-Adresse (Kontingente,
SSRF-Schutz, Kill-Switch = Prozess stoppen) SSRF-Schutz, Kill-Switch = Prozess stoppen)
- [x] `sn-fair` v0: Fairness-Engine (gewichtetes Max-Min über zwei
Klassen, Leech-Politik) + Demo-CLI; Runtime-Integration in
`sn-fetch` ist Phase 2
- [ ] `sn-fetch` PoC über 2 echte Exits, plus Baseline-Messung - [ ] `sn-fetch` PoC über 2 echte Exits, plus Baseline-Messung
(1-Exit vs. 2-Exit, gleiche Datei) (1-Exit vs. 2-Exit, gleiche Datei — Protokoll in [[Pilot]])
- [ ] Fair-Share-Scheduler v0: zwei gleichzeitige Downloads, faire Aufteilung - [ ] Doku: Messwerte, Erkenntnisse, Stolpersteine → in [[Pilot]] §Ergebnisse
- [ ] Doku: Messwerte, Erkenntnisse, Stolpersteine → hier in das Repo
## Phase 2 — MVP-Software ## Phase 2 — MVP-Software
+193
View File
@@ -0,0 +1,193 @@
# SpiderNet Nix flake — builds the Rust workspace in server/ and exposes
# one `spidernet` package containing all four binaries (sn-node, sn-exit,
# sn-fetch, sn-fair) in $out/bin, plus an experimental NixOS module.
#
# NO WARRANTY — SpiderNet is provided "as is" without warranty of any kind
# (AGPL-3.0, sections 15/16). Use is entirely at the operator's own
# responsibility.
#
# !!! NOT VERIFIED ON THIS MACHINE !!!
# Nix is not installed in the environment where this flake was written.
# `nix flake check` and `nix build` have NOT been run — treat everything
# below as best-effort and report build failures as issues.
#
# Usage (once Nix >= 2.19 is installed):
# nix build . # -> result/bin/{sn-node,sn-exit,...}
# nix shell .# -c sn-fair --capacity 100 --participant 1=contributing
#
# The nixpkgs input is pinned to the current stable NixOS release branch
# (nixos-26.05, released 2026-05-30; nixos-25.11 is already deprecated).
{
description =
"SpiderNet: pooled-bandwidth neighborhood network node software";
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-26.05";
};
outputs =
{ self, nixpkgs }:
let
systems = [
"x86_64-linux"
"aarch64-linux"
"x86_64-darwin"
"aarch64-darwin"
];
forAllSystems = nixpkgs.lib.genAttrs systems;
in
{
packages = forAllSystems (
system:
let
pkgs = nixpkgs.legacyPackages.${system};
inherit (pkgs) lib;
# Only the files cargo needs; keeps target/, docs and any other
# clutter out of the Nix store hash.
src = lib.cleanSourceWith {
src = lib.cleanSource ./server;
filter =
path: type:
# Prune cargo build dirs entirely.
!(type == "directory" && lib.hasSuffix "/target" path)
&& (
type == "directory"
|| lib.hasSuffix ".rs" path
|| lib.hasSuffix ".toml" path
|| lib.hasSuffix ".lock" path
);
};
spidernet = pkgs.rustPlatform.buildRustPackage {
pname = "spidernet";
version = "0.1.0";
inherit src;
cargoLock.lockFile = ./server/Cargo.lock;
# Integration tests talk to real servers; skip them when
# packaging. Run `cargo test` inside server/ for development.
doCheck = false;
meta = {
description =
"SpiderNet node software (sn-node, sn-exit, sn-fetch, sn-fair)";
longDescription = ''
SpiderNet pools the bandwidth of household internet
connections over a neighborhood mesh (Yggdrasil overlay).
NO WARRANTY — AGPL-3.0, sections 15/16.
'';
license = lib.licenses.agpl3Only;
mainProgram = "sn-node";
platforms = lib.platforms.unix;
};
};
in
{
inherit spidernet;
default = spidernet;
}
);
# ------------------------------------------------------------------
# EXPERIMENTAL NixOS module — never built or activated anywhere.
# Runs `sn-node daemon` (which manages the yggdrasil sidecar) as a
# hardened systemd service.
#
# Usage example (in a NixOS configuration that imports this flake):
#
# imports = [ spidernet.nixosModules.default ];
#
# services.spidernet = {
# enable = true;
# configFile = "/etc/spidernet/node.toml";
# # stateDir defaults to "spidernet" -> /var/lib/spidernet,
# # which the example config uses as state_dir.
# };
#
# The config file must set state_dir = "/var/lib/spidernet" (or
# leave the default and rely on WorkingDirectory below).
# ------------------------------------------------------------------
nixosModules.default =
{
config,
lib,
pkgs,
...
}:
let
cfg = config.services.spidernet;
in
{
options.services.spidernet = {
enable = lib.mkEnableOption "SpiderNet node daemon (sn-node)";
package = lib.mkOption {
type = lib.types.package;
# The flake package of the evaluating system; can be
# overridden with any build of spidernet.
default = self.packages.${pkgs.system}.default;
defaultText = lib.literalExpression "spidernet.packages.\${pkgs.system}.default";
description = "SpiderNet package to run (contains sn-node etc.).";
};
configFile = lib.mkOption {
type = lib.types.path;
description = ''
Path to the node.toml config file read by
`sn-node daemon --config`.
'';
};
stateDir = lib.mkOption {
type = lib.types.str;
default = "spidernet";
description = ''
State directory (systemd StateDirectory), holding node.key
and the generated yggdrasil.conf. Created as
/var/lib/<stateDir>.
'';
};
};
config = lib.mkIf cfg.enable {
systemd.services.spidernet = {
description = "SpiderNet node (Yggdrasil overlay, optional exit)";
wantedBy = [ "multi-user.target" ];
after = [ "network-online.target" ];
wants = [ "network-online.target" ];
# The daemon spawns yggdrasil (manage = true by default).
path = [ pkgs.yggdrasil ];
serviceConfig = {
ExecStart =
"${lib.getExe cfg.package} daemon --config ${cfg.configFile}";
StateDirectory = cfg.stateDir;
WorkingDirectory = "/var/lib/${cfg.stateDir}";
# TUN setup requires CAP_NET_ADMIN; nothing more.
AmbientCapabilities = [ "CAP_NET_ADMIN" ];
CapabilityBoundingSet = [ "CAP_NET_ADMIN" ];
DeviceAllow = [
"/dev/net/tun rw"
# Yggdrasil TUN devices are created at runtime.
"char-tun rwm"
];
Restart = "on-failure";
RestartSec = 5;
NoNewPrivileges = true;
ProtectSystem = "strict";
ProtectHome = true;
PrivateTmp = true;
# State dir must stay writable.
ReadWritePaths = [ "/var/lib/${cfg.stateDir}" ];
};
};
};
};
};
}
+9
View File
@@ -0,0 +1,9 @@
# Keep the build context small; everything needed is Cargo.toml,
# Cargo.lock and crates/ (see server/docker/Dockerfile).
target/
.git/
docker/node.toml.example
docker/node.toml
docker/README.md
docker/docker-compose.yml
spidernet-state/
+128
View File
@@ -228,6 +228,12 @@ dependencies = [
"syn 3.0.6", "syn 3.0.6",
] ]
[[package]]
name = "equivalent"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]] [[package]]
name = "errno" name = "errno"
version = "0.3.14" version = "0.3.14"
@@ -329,6 +335,12 @@ dependencies = [
"wasm-bindgen", "wasm-bindgen",
] ]
[[package]]
name = "hashbrown"
version = "0.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
[[package]] [[package]]
name = "heck" name = "heck"
version = "0.5.0" version = "0.5.0"
@@ -543,6 +555,16 @@ dependencies = [
"icu_properties", "icu_properties",
] ]
[[package]]
name = "indexmap"
version = "2.14.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cc4e190f5d26ca7051642629da2c52fc03bde85a03197c99408dcd291734c855"
dependencies = [
"equivalent",
"hashbrown",
]
[[package]] [[package]]
name = "ipnet" name = "ipnet"
version = "2.12.2" version = "2.12.2"
@@ -924,6 +946,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba" checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba"
dependencies = [ dependencies = [
"serde_core", "serde_core",
"serde_derive",
] ]
[[package]] [[package]]
@@ -959,6 +982,15 @@ dependencies = [
"zmij", "zmij",
] ]
[[package]]
name = "serde_spanned"
version = "0.6.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bf41e0cfaf7226dca15e8197172c295a782857fcb97fad1808a166870dee75a3"
dependencies = [
"serde",
]
[[package]] [[package]]
name = "serde_urlencoded" name = "serde_urlencoded"
version = "0.7.1" version = "0.7.1"
@@ -997,6 +1029,16 @@ version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba" checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
[[package]]
name = "signal-hook-registry"
version = "1.4.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c4db69cba1110affc0e9f7bcd48bbf87b3f4fc7c61fc9155afd4c469eb3d6c1b"
dependencies = [
"errno",
"libc",
]
[[package]] [[package]]
name = "slab" name = "slab"
version = "0.4.12" version = "0.4.12"
@@ -1009,6 +1051,25 @@ version = "1.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba467056f1b547ed52077911161fc86985becbc60e8e1857c8a144dab0def891" checksum = "ba467056f1b547ed52077911161fc86985becbc60e8e1857c8a144dab0def891"
[[package]]
name = "sn-exit"
version = "0.1.0"
dependencies = [
"clap",
"thiserror",
"tokio",
"tracing",
"tracing-subscriber",
]
[[package]]
name = "sn-fair"
version = "0.1.0"
dependencies = [
"clap",
"thiserror",
]
[[package]] [[package]]
name = "sn-fetch" name = "sn-fetch"
version = "0.1.0" version = "0.1.0"
@@ -1024,6 +1085,22 @@ dependencies = [
"tracing-subscriber", "tracing-subscriber",
] ]
[[package]]
name = "sn-node"
version = "0.1.0"
dependencies = [
"clap",
"serde",
"serde_json",
"sn-exit",
"tempfile",
"thiserror",
"tokio",
"toml",
"tracing",
"tracing-subscriber",
]
[[package]] [[package]]
name = "socket2" name = "socket2"
version = "0.6.5" version = "0.6.5"
@@ -1162,6 +1239,7 @@ dependencies = [
"libc", "libc",
"mio", "mio",
"pin-project-lite", "pin-project-lite",
"signal-hook-registry",
"socket2", "socket2",
"tokio-macros", "tokio-macros",
"windows-sys 0.61.2", "windows-sys 0.61.2",
@@ -1188,6 +1266,47 @@ dependencies = [
"tokio", "tokio",
] ]
[[package]]
name = "toml"
version = "0.8.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc1beb996b9d83529a9e75c17a1686767d148d70663143c7854d8b4a09ced362"
dependencies = [
"serde",
"serde_spanned",
"toml_datetime",
"toml_edit",
]
[[package]]
name = "toml_datetime"
version = "0.6.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "22cddaf88f4fbc13c51aebbf5f8eceb5c7c5a9da2ac40a13519eb5b0a0e8f11c"
dependencies = [
"serde",
]
[[package]]
name = "toml_edit"
version = "0.22.27"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41fe8c660ae4257887cf66394862d21dbca4a6ddd26f04a3560410406a2f819a"
dependencies = [
"indexmap",
"serde",
"serde_spanned",
"toml_datetime",
"toml_write",
"winnow",
]
[[package]]
name = "toml_write"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5d99f8c9a7727884afe522e9bd5edbfc91a3312b36a77b5fb8926e4c31a41801"
[[package]] [[package]]
name = "tower" name = "tower"
version = "0.5.3" version = "0.5.3"
@@ -1541,6 +1660,15 @@ version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec" checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec"
[[package]]
name = "winnow"
version = "0.7.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df79d97927682d2fd8adb29682d1140b343be4ac0f08fd68b7765d9c059d3945"
dependencies = [
"memchr",
]
[[package]] [[package]]
name = "writeable" name = "writeable"
version = "0.6.4" version = "0.6.4"
+4 -2
View File
@@ -1,6 +1,6 @@
[workspace] [workspace]
resolver = "2" resolver = "2"
members = ["crates/sn-fetch", "crates/sn-node", "crates/sn-exit"] members = ["crates/sn-fetch", "crates/sn-node", "crates/sn-exit", "crates/sn-fair"]
[workspace.package] [workspace.package]
version = "0.1.0" version = "0.1.0"
@@ -18,8 +18,10 @@ panic = "deny"
unreachable = "deny" unreachable = "deny"
[workspace.dependencies] [workspace.dependencies]
tokio = { version = "1", features = ["rt-multi-thread", "macros", "fs", "io-util", "net", "time"] } tokio = { version = "1", features = ["rt-multi-thread", "macros", "fs", "io-util", "net", "time", "process", "signal"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1" serde_json = "1"
toml = "0.8"
reqwest = { version = "0.12", default-features = false, features = ["rustls-tls"] } reqwest = { version = "0.12", default-features = false, features = ["rustls-tls"] }
sha2 = "0.10" sha2 = "0.10"
hex = "0.4" hex = "0.4"
+53 -5
View File
@@ -60,10 +60,58 @@ SHA-256 entspricht der offiziellen `SHA256SUMS`).
- Verifiziert end-to-end: `sn-fetch` über einen laufenden `sn-exit` - Verifiziert end-to-end: `sn-fetch` über einen laufenden `sn-exit`
lädt die Debian-ISO (756 MB, 4 Segmente, SHA-256 korrekt) lädt die Debian-ISO (756 MB, 4 Segmente, SHA-256 korrekt)
**Noch nicht drin:** `sn-node`-Daemon (Supervisor, der yggdrasil + sn-exit **`sn-fair` v0 ist implementiert** (siehe `crates/sn-fair/`):
+ sn-fetch orchestriert), Fairness-Scheduler.
Details zum Overlay: [[Overlay]]. - Gewichtetes Max-Min-Fairness-Engine (zwei Klassen): **Contributors**
(Haushalte mit Exit-Angebot, konfigurierbares Gewicht) teilen sich
die Gesamtkapazität nach Max-Min-Fairness; **Leeches** (Empfang ohne
Beitrag, ausdrücklich legitim) bekommen die Restkapazität — bei
Sättigung zuerst gedrosselt, nie ausgeschlossen
- Deterministische, overflow-sichere Berechnung (u128-Arithmetik,
Restverteilung in ID-Ordnung), validierte Newtypes
(`ParticipantId`, `Weight`, `Rate`, `Capacity`)
- Demo-CLI in Mbit/s für den Piloten: Soll-Aufteilung simulieren und
gegen die gemessene Ist-Aufteilung halten (siehe [[Pilot]], M4)
- Runtime-Integration in `sn-fetch` (echte gleichzeitige Downloads
drosseln) ist Phase 2, siehe [[Roadmap]]
**`sn-node` Daemon v1 ist implementiert** (`sn-node daemon`, siehe
`crates/sn-node/src/daemon.rs`):
- **Ein Befehl pro Haushalt:** `sn-node daemon --config node.toml`
orchestriert den Node — Yggdrasil-Sidecar (generierte Config +
persistenter Node-Key im `state_dir`, Identität bleibt über Neustarts
stabil: `PrivateKeyPath` hat Vorrang vor dem eingebetteten Key, siehe
[Config-Referenz](https://yggdrasil-network.github.io/configurationref.html))
+ optional den in-process `sn-exit`-Dienst
- **TOML-Konfiguration, validiert beim Laden** (`deny_unknown_fields`,
wiederverwendete validierte Typen; `sn-node daemon --check` prüft
ohne Start — für Packaging/Ops)
- **Leech-Modus per Konfig** (`[exit] enabled = false`): Overlay läuft,
kein Exit-Angebot; enabled + leere Allowlist = Start verweigert
(Ausnahme: Loopback-Dev-Bind)
- **Kill-Switch-Semantik:** SIGINT/SIGTERM → Accept-Loop stoppt,
Sidecar bekommt SIGTERM (5 s Gnade, dann SIGKILL); stirbt Yggdrasil
unerwartet, fährt der Daemon mit Fehler herunter (Supervisor-Pflicht)
- Getestet mit Fake-Yggdrasil (Script-Binary) und Fake-Admin-Socket:
Key-Persistenz, Exit-Bind, Shutdown, Sidecar-Tod
**Deployment: Docker & Nix** (siehe `server/docker/README.md` und
`flake.nix`):
- `server/docker/`: Multi-Stage-Image (alle 4 Binaries + Yggdrasil
v0.5.14, SHA-256-verifiziert), Compose-Beispiel pro Haushalt
(`network_mode: host`, `NET_ADMIN`, `/dev/net/tun`, `init: true`) —
Image gebaut und geraucht-getestet
- `flake.nix`: `nix build` baut das Workspace als ein Paket
(+ experimentelles NixOS-Modul) — **nicht lokal verifiziert**
(Nix nicht installiert), Feedback willkommen
**Noch nicht drin:** Freenet-Integration, Cache, Metriken/Onboarding
(Phase 2).
Details zum Overlay: [[Overlay]]. Pilot-Handbuch mit Messprotokoll:
[[Pilot]].
## Haftungsausschluss (muss im Produkt sichtbar sein) ## Haftungsausschluss (muss im Produkt sichtbar sein)
@@ -93,10 +141,10 @@ Einplatinenrechner), der:
| Crate/Binary | Zweck | | Crate/Binary | Zweck |
|---|---| |---|---|
| `sn-node` | Node-Glue: Yggdrasil-Config (`genconf`), Overlay-Status (`status`), später Daemon/Supervisor | | `sn-node` | Node-Glue: Yggdrasil-Config (`genconf`), Overlay-Status (`status`), Node-Daemon (`daemon`: Sidecar + Exit-Orchestrierung) |
| `sn-exit` | Exit-Dienst: kontingentierter Uplink-Proxy auf der Yggdrasil-Adresse (v0: Byte-Relay + Quota + SSRF-Schutz) | | `sn-exit` | Exit-Dienst: kontingentierter Uplink-Proxy auf der Yggdrasil-Adresse (v0: Byte-Relay + Quota + SSRF-Schutz) |
| `sn-fetch` | Segmentierter Multi-Exit-Downloader (HTTP-API + CLI) | | `sn-fetch` | Segmentierter Multi-Exit-Downloader (HTTP-API + CLI) |
| `sn-fair` | Fairness-Scheduler (max-min, Exit-Kontingente, Karma, Leech-Modus) | | `sn-fair` | Fairness-Scheduler (max-min, Exit-Kontingente, Karma, Leech-Modus) — v0: zwei-Klassen-Max-Min-Engine + Demo-CLI |
| `sn-cache` | Transparenter Nachbarschafts-Cache (DNS-basiert) | | `sn-cache` | Transparenter Nachbarschafts-Cache (DNS-basiert) |
| `sn-freenet` | Integration von `freenet-core` (Contracts für Nachbarschafts-Inhalte) | | `sn-freenet` | Integration von `freenet-core` (Contracts für Nachbarschafts-Inhalte) |
+29 -7
View File
@@ -6,6 +6,7 @@
//! the exit only sees which host is contacted and how many bytes flow. //! the exit only sees which host is contacted and how many bytes flow.
use std::collections::HashMap; use std::collections::HashMap;
use std::future::Future;
use std::net::IpAddr; use std::net::IpAddr;
use std::sync::{Arc, Mutex}; use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant}; use std::time::{Duration, Instant};
@@ -128,17 +129,38 @@ fn roll_window(
/// Accept loop. Runs until the listener errors fatally. /// Accept loop. Runs until the listener errors fatally.
pub async fn serve(config: ExitConfig, listener: TcpListener) -> Result<(), ExitError> { pub async fn serve(config: ExitConfig, listener: TcpListener) -> Result<(), ExitError> {
serve_until(config, listener, std::future::pending()).await
}
/// Like [`serve`], but stops accepting when `shutdown` completes (used
/// by the node daemon for kill-switch semantics). Connections already
/// accepted keep running in the background; the caller or process exit
/// owns them.
pub async fn serve_until(
config: ExitConfig,
listener: TcpListener,
shutdown: impl Future<Output = ()>,
) -> Result<(), ExitError> {
let config = Arc::new(config); let config = Arc::new(config);
let ledger = Arc::new(QuotaLedger::new()); let ledger = Arc::new(QuotaLedger::new());
tokio::pin!(shutdown);
loop { loop {
let (stream, peer) = listener.accept().await?; tokio::select! {
let config = Arc::clone(&config); () = &mut shutdown => {
let ledger = Arc::clone(&ledger); tracing::info!("exit: accept loop stopped (shutdown)");
tokio::spawn(async move { return Ok(());
if let Err(error) = handle_connection(stream, peer.ip(), config, ledger).await {
tracing::debug!(%peer, "connection ended: {error}");
} }
}); accepted = listener.accept() => {
let (stream, peer) = accepted?;
let config = Arc::clone(&config);
let ledger = Arc::clone(&ledger);
tokio::spawn(async move {
if let Err(error) = handle_connection(stream, peer.ip(), config, ledger).await {
tracing::debug!(%peer, "connection ended: {error}");
}
});
}
}
} }
} }
+13
View File
@@ -0,0 +1,13 @@
[package]
name = "sn-fair"
description = "Weighted max-min fairness scheduler for SpiderNet neighborhood networks"
version.workspace = true
edition.workspace = true
license.workspace = true
[dependencies]
thiserror = { workspace = true }
clap = { workspace = true }
[lints]
workspace = true
+24
View File
@@ -0,0 +1,24 @@
//! Error type for the fairness scheduler.
use crate::types::ParticipantId;
#[derive(Debug, thiserror::Error)]
pub enum FairnessError {
#[error("capacity must be greater than zero")]
ZeroCapacity,
#[error("invalid participant id `{0}`: expected a decimal number greater than zero")]
InvalidParticipantId(String),
#[error("demand references unknown participant {0}")]
UnknownParticipant(ParticipantId),
#[error("duplicate demand for participant {0}")]
DuplicateDemand(ParticipantId),
#[error("duplicate participant {0}")]
DuplicateParticipant(ParticipantId),
#[error("conflicting participation classes for participant {0}")]
ConflictingParticipation(ParticipantId),
}
+578
View File
@@ -0,0 +1,578 @@
//! Two-class weighted max-min fairness allocation.
//!
//! Participants are split into two priority classes:
//!
//! 1. **Contributors** ([`Participation::Contributing`]) — households that
//! offer their own uplink to the pool. They are served first, with
//! shares shaped by their [`Weight`] via weighted max-min fairness
//! over the *full* capacity.
//! 2. **Leeches** ([`Participation::Leeching`]) — receive-only
//! households. Leech mode is explicitly allowed (the network exists
//! under neighbors, not against them). Leeches have the lowest but
//! still fair priority: when contributors saturate the capacity,
//! leeches get (near) nothing; when contributors are undersubscribed,
//! leeches benefit from the leftovers. Nobody is dropped from the
//! system.
//!
//! See the crate root for the phase overview.
use std::collections::{BTreeMap, BTreeSet};
use crate::error::FairnessError;
use crate::types::{Capacity, ParticipantId, Rate, Weight};
/// How a household takes part in the neighborhood network.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Participation {
/// The household offers its own uplink to the pool; `weight` shapes
/// its share of the capacity (1 = equal share).
Contributing { weight: Weight },
/// The household only receives (leech mode, explicitly allowed).
/// Leeches share whatever contributors leave unused, with implicit
/// equal weights, capped by their demand.
Leeching,
}
/// A participant of the allocation: an id plus its participation class.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Participant {
id: ParticipantId,
participation: Participation,
}
impl Participant {
pub const fn new(id: ParticipantId, participation: Participation) -> Self {
Self { id, participation }
}
pub const fn id(&self) -> ParticipantId {
self.id
}
pub const fn participation(&self) -> Participation {
self.participation
}
}
/// Requested rate of a participant.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Demand {
participant: ParticipantId,
rate: Rate,
}
impl Demand {
pub const fn new(participant: ParticipantId, rate: Rate) -> Self {
Self { participant, rate }
}
pub const fn participant(&self) -> ParticipantId {
self.participant
}
pub const fn rate(&self) -> Rate {
self.rate
}
}
/// Granted rate for a participant.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Allocation {
participant: ParticipantId,
rate: Rate,
}
impl Allocation {
pub const fn new(participant: ParticipantId, rate: Rate) -> Self {
Self { participant, rate }
}
pub const fn participant(&self) -> ParticipantId {
self.participant
}
pub const fn rate(&self) -> Rate {
self.rate
}
}
/// Result of [`allocate`]: per-participant rates plus the totals.
///
/// Allocations are keyed by participant id in a `BTreeMap`, so iteration
/// is always in ascending id order (deterministic output).
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct FairShare {
allocations: BTreeMap<ParticipantId, Rate>,
total_allocated: Rate,
unused: Rate,
}
impl FairShare {
/// Allocations ordered by participant id. Participants without a
/// demand at all are absent; use [`FairShare::rate_of`] to read a
/// single participant's rate.
pub fn allocations(&self) -> &BTreeMap<ParticipantId, Rate> {
&self.allocations
}
/// Allocations as [`Allocation`] values, in participant-id order.
pub fn iter(&self) -> impl Iterator<Item = Allocation> + '_ {
self.allocations
.iter()
.map(|(&participant, &rate)| Allocation::new(participant, rate))
}
/// Allocated rate of one participant (zero if it received nothing).
pub fn rate_of(&self, participant: ParticipantId) -> Rate {
self.allocations
.get(&participant)
.copied()
.unwrap_or(Rate::new(0))
}
pub const fn total_allocated(&self) -> Rate {
self.total_allocated
}
pub const fn unused(&self) -> Rate {
self.unused
}
}
/// Allocates `capacity` among `participants` according to their `demands`.
///
/// Phase 1 runs weighted max-min fairness among contributors over the
/// full capacity; phase 2 runs weighted max-min among leeches (implicit
/// equal weights) over the capacity the contributors leave unused.
///
/// Empty `demands` is valid: nothing is allocated, everything stays
/// unused. Participants without a demand (or with demand 0) receive
/// nothing.
///
/// Errors:
/// - [`FairnessError::ZeroCapacity`] — capacity is zero (cannot happen
/// with a validated [`Capacity`])
/// - [`FairnessError::UnknownParticipant`] — a demand references an id
/// that is not among `participants`
/// - [`FairnessError::DuplicateDemand`] — two demands for the same id
/// - [`FairnessError::DuplicateParticipant`] — the same id appears twice
/// among `participants` with the same participation class
/// - [`FairnessError::ConflictingParticipation`] — the same id appears
/// twice among `participants` with different participation classes
pub fn allocate(
capacity: Capacity,
participants: &[Participant],
demands: &[Demand],
) -> Result<FairShare, FairnessError> {
let mut participation_of: BTreeMap<ParticipantId, Participation> = BTreeMap::new();
for participant in participants {
if let Some(previous) =
participation_of.insert(participant.id(), participant.participation())
{
if previous == participant.participation() {
return Err(FairnessError::DuplicateParticipant(participant.id()));
}
return Err(FairnessError::ConflictingParticipation(participant.id()));
}
}
for demand in demands {
if !participation_of.contains_key(&demand.participant()) {
return Err(FairnessError::UnknownParticipant(demand.participant()));
}
}
let mut demand_of: BTreeMap<ParticipantId, u64> = BTreeMap::new();
for demand in demands {
if demand_of
.insert(demand.participant(), demand.rate().value())
.is_some()
{
return Err(FairnessError::DuplicateDemand(demand.participant()));
}
}
let mut contributors = Vec::new();
let mut leeches = Vec::new();
for (&id, &participation) in &participation_of {
let demand = demand_of.get(&id).copied().unwrap_or(0);
match participation {
Participation::Contributing { weight } => contributors.push(Entry {
id,
demand,
weight: weight.value(),
}),
Participation::Leeching => leeches.push(Entry {
id,
demand,
weight: 1,
}),
}
}
let phase_one = weighted_max_min(contributors, capacity.value());
let allocated: u128 = phase_one.values().map(|rate| u128::from(*rate)).sum();
let leftover = capacity.value().saturating_sub(to_u64(allocated));
let phase_two = weighted_max_min(leeches, leftover);
let mut allocations = phase_one;
allocations.extend(phase_two);
let total: u128 = allocations.values().map(|rate| u128::from(*rate)).sum();
let total = to_u64(total);
let unused = capacity.value().saturating_sub(total);
Ok(FairShare {
allocations: allocations
.into_iter()
.map(|(id, rate)| (id, Rate::new(rate)))
.collect(),
total_allocated: Rate::new(total),
unused: Rate::new(unused),
})
}
/// Internal per-participant input for the max-min helper.
struct Entry {
id: ParticipantId,
demand: u64,
weight: u32,
}
/// Weighted max-min fairness over `entries` with (at most) `capacity` to
/// distribute. Deterministic: participants are processed in participant-id
/// order and the result is a `BTreeMap`.
///
/// Loop: each round hands out proportional shares
/// `share_i = capacity * weight_i / total_weight` (computed in `u128`,
/// floored). Participants whose demand fits their share are satisfied and
/// leave the active set together with their allocation; the rest re-share
/// the remaining capacity. When nobody's demand fits, everyone gets its
/// floor share and the integer-rounding remainder is distributed
/// deterministically in participant-id order, one unit at a time, only to
/// participants still below their demand.
///
/// Guarantees (asserted by tests): the sum of allocations never exceeds
/// `capacity`; when every remaining demand is at least its computed
/// share, the capacity is distributed in full.
fn weighted_max_min(entries: Vec<Entry>, capacity: u64) -> BTreeMap<ParticipantId, u64> {
let mut result = BTreeMap::new();
let mut active: Vec<Entry> = Vec::new();
// Zero demand is trivially satisfied with an allocation of zero.
for entry in entries {
if entry.demand == 0 {
result.insert(entry.id, 0);
} else {
active.push(entry);
}
}
active.sort_by_key(|entry| entry.id);
let mut capacity = capacity;
while capacity > 0 && !active.is_empty() {
let total_weight: u128 = active.iter().map(|entry| u128::from(entry.weight)).sum();
let shares: Vec<u128> = active
.iter()
.map(|entry| u128::from(capacity) * u128::from(entry.weight) / total_weight)
.collect();
let mut satisfied = Vec::new();
for (index, entry) in active.iter().enumerate() {
if u128::from(entry.demand) <= shares[index] {
satisfied.push(index);
}
}
if satisfied.is_empty() {
// Every demand is above its share: hand out the floor shares,
// then spread the rounding remainder one unit at a time. The
// remainder is smaller than the number of active participants
// and every one of them still has room, so it is distributed
// completely.
for (index, entry) in active.iter().enumerate() {
result.insert(entry.id, to_u64(shares[index]));
}
let distributed: u128 = shares.iter().sum();
let mut remainder = u128::from(capacity) - distributed;
while remainder > 0 {
let mut handed_out = 0;
for entry in &active {
if remainder == 0 {
break;
}
if let Some(allocation) = result.get_mut(&entry.id) {
if *allocation < entry.demand {
*allocation += 1;
remainder -= 1;
handed_out += 1;
}
}
}
if handed_out == 0 {
break;
}
}
active.clear();
break;
}
let allocated: u64 = satisfied.iter().map(|&index| active[index].demand).sum();
let satisfied_set: BTreeSet<usize> = satisfied.into_iter().collect();
let mut remaining = Vec::new();
for (index, entry) in active.into_iter().enumerate() {
if satisfied_set.contains(&index) {
result.insert(entry.id, entry.demand);
} else {
remaining.push(entry);
}
}
active = remaining;
capacity = capacity.saturating_sub(allocated);
}
// Capacity exhausted (or nothing to distribute): whoever is still
// active receives nothing.
for entry in active {
result.insert(entry.id, 0);
}
result
}
/// All values in the allocation arithmetic are bounded by the `u64`
/// capacity, so this conversion cannot truncate; `u64::MAX` is a safe
/// fallback that keeps the code panic-free.
fn to_u64(value: u128) -> u64 {
u64::try_from(value).unwrap_or(u64::MAX)
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used)]
use std::num::{NonZeroU16, NonZeroU32};
use super::*;
fn id(value: u16) -> ParticipantId {
ParticipantId::new(NonZeroU16::new(value).unwrap())
}
fn contributing(participant: u16, weight: u32) -> Participant {
Participant::new(
id(participant),
Participation::Contributing {
weight: Weight::new(NonZeroU32::new(weight).unwrap()),
},
)
}
fn leeching(participant: u16) -> Participant {
Participant::new(id(participant), Participation::Leeching)
}
fn demand(participant: u16, rate: u64) -> Demand {
Demand::new(id(participant), Rate::new(rate))
}
fn capacity(value: u64) -> Capacity {
Capacity::try_new(value).unwrap()
}
fn rate_of(share: &FairShare, participant: u16) -> u64 {
share.rate_of(id(participant)).value()
}
#[test]
fn equal_demands_equal_weights_split_equally() {
let share = allocate(
capacity(800),
&[contributing(1, 1), contributing(2, 1)],
&[demand(1, 800), demand(2, 800)],
)
.unwrap();
assert_eq!(rate_of(&share, 1), 400);
assert_eq!(rate_of(&share, 2), 400);
assert_eq!(share.total_allocated().value(), 800);
assert_eq!(share.unused().value(), 0);
}
#[test]
fn small_demand_is_satisfied_first_then_remainder_is_redistributed() {
let share = allocate(
capacity(900),
&[contributing(1, 1), contributing(2, 1)],
&[demand(1, 100), demand(2, 900)],
)
.unwrap();
assert_eq!(rate_of(&share, 1), 100);
assert_eq!(rate_of(&share, 2), 800);
}
#[test]
fn weights_shape_the_split_two_to_one() {
let share = allocate(
capacity(900),
&[contributing(1, 2), contributing(2, 1)],
&[demand(1, 900), demand(2, 900)],
)
.unwrap();
assert_eq!(rate_of(&share, 1), 600);
assert_eq!(rate_of(&share, 2), 300);
}
#[test]
fn rounding_remainder_is_distributed_in_id_order() {
let share = allocate(
capacity(1001),
&[contributing(1, 1), contributing(2, 1)],
&[demand(1, 1000), demand(2, 1000)],
)
.unwrap();
// Shares are 500 + 500, the extra unit goes to the lower id.
assert_eq!(rate_of(&share, 1), 501);
assert_eq!(rate_of(&share, 2), 500);
assert_eq!(share.total_allocated().value(), 1001);
}
#[test]
fn saturated_contributor_leaves_nothing_for_the_leech() {
let share = allocate(
capacity(100),
&[contributing(1, 1), leeching(2)],
&[demand(1, 200), demand(2, 50)],
)
.unwrap();
assert_eq!(rate_of(&share, 1), 100);
assert_eq!(rate_of(&share, 2), 0);
}
#[test]
fn undersubscribed_contributor_leaves_leftover_for_the_leech() {
let share = allocate(
capacity(100),
&[contributing(1, 1), leeching(2)],
&[demand(1, 60), demand(2, 100)],
)
.unwrap();
assert_eq!(rate_of(&share, 1), 60);
assert_eq!(rate_of(&share, 2), 40);
}
#[test]
fn two_leeches_split_leftover_equally_capped_by_demand() {
let share = allocate(
capacity(100),
&[contributing(1, 1), leeching(2), leeching(3)],
&[demand(1, 50), demand(2, 100), demand(3, 20)],
)
.unwrap();
assert_eq!(rate_of(&share, 1), 50);
// Leftover 50: leech 3 (demand 20) is satisfied first, leech 2
// gets the rest.
assert_eq!(rate_of(&share, 2), 30);
assert_eq!(rate_of(&share, 3), 20);
}
#[test]
fn capacity_below_all_demands_yields_proportional_weighted_split() {
let share = allocate(
capacity(30),
&[contributing(1, 2), contributing(2, 1)],
&[demand(1, 100), demand(2, 100)],
)
.unwrap();
assert_eq!(rate_of(&share, 1), 20);
assert_eq!(rate_of(&share, 2), 10);
assert_eq!(share.unused().value(), 0);
}
#[test]
fn zero_demand_receives_zero_without_consuming_capacity() {
let share = allocate(
capacity(800),
&[contributing(1, 1), contributing(2, 1)],
&[demand(1, 0), demand(2, 500)],
)
.unwrap();
assert_eq!(rate_of(&share, 1), 0);
assert_eq!(share.allocations().get(&id(1)), Some(&Rate::new(0)));
assert_eq!(rate_of(&share, 2), 500);
assert_eq!(share.unused().value(), 300);
}
#[test]
fn empty_demands_allocates_nothing_and_everything_is_unused() {
let share = allocate(capacity(500), &[contributing(1, 1), leeching(2)], &[]).unwrap();
assert_eq!(share.total_allocated().value(), 0);
assert_eq!(share.unused().value(), 500);
assert!(share.allocations().values().all(|rate| rate.value() == 0));
}
#[test]
fn unknown_participant_in_demand_is_rejected() {
let error = allocate(capacity(100), &[contributing(1, 1)], &[demand(9, 50)]).unwrap_err();
assert!(matches!(error, FairnessError::UnknownParticipant(_)));
}
#[test]
fn duplicate_demand_is_rejected() {
let error = allocate(
capacity(100),
&[contributing(1, 1)],
&[demand(1, 50), demand(1, 20)],
)
.unwrap_err();
assert!(matches!(error, FairnessError::DuplicateDemand(_)));
}
#[test]
fn duplicate_participant_with_same_class_is_rejected() {
let error = allocate(
capacity(100),
&[contributing(1, 1), contributing(1, 1)],
&[],
)
.unwrap_err();
assert!(matches!(error, FairnessError::DuplicateParticipant(_)));
}
#[test]
fn duplicate_participant_with_different_class_is_rejected() {
let error = allocate(capacity(100), &[contributing(1, 1), leeching(1)], &[]).unwrap_err();
assert!(matches!(error, FairnessError::ConflictingParticipation(_)));
}
#[test]
fn all_demands_at_least_share_distributes_capacity_in_full() {
let share = allocate(
capacity(1000),
&[contributing(1, 1), contributing(2, 1), contributing(3, 2)],
&[demand(1, 1000), demand(2, 1000), demand(3, 1000)],
)
.unwrap();
assert_eq!(rate_of(&share, 1), 250);
assert_eq!(rate_of(&share, 2), 250);
assert_eq!(rate_of(&share, 3), 500);
assert_eq!(share.total_allocated().value(), 1000);
assert_eq!(share.unused().value(), 0);
}
/// Roadmap pilot scenario: two household downloads (contributing,
/// equal) plus one leech competing for the pooled uplinks.
#[test]
fn pilot_scenario_two_households_and_a_leech() {
let share = allocate(
capacity(600),
&[contributing(1, 1), contributing(2, 1), leeching(3)],
&[demand(1, 250), demand(2, 250), demand(3, 150)],
)
.unwrap();
assert_eq!(rate_of(&share, 1), 250);
assert_eq!(rate_of(&share, 2), 250);
assert_eq!(rate_of(&share, 3), 100);
assert_eq!(share.total_allocated().value(), 600);
assert_eq!(share.unused().value(), 0);
}
}
+38
View File
@@ -0,0 +1,38 @@
//! sn-fair — weighted max-min fairness scheduler for `SpiderNet`.
//!
//! Allocates a shared capacity (e.g. the pooled uplink bandwidth of a
//! neighborhood) among participants with a two-class weighted max-min
//! fairness scheme:
//!
//! 1. **Contributors** (households that offer capacity to the network,
//! with a configurable weight) are served first, via weighted max-min
//! fairness over the full capacity.
//! 2. **Leeches** (receive-only households; explicitly allowed — the
//! network exists under neighbors, not against them) share whatever
//! the contributors leave unused, weighted max-min with implicit equal
//! weights, capped by their demand.
//!
//! Rationale: leeches have the lowest but still fair priority. When
//! contributors saturate the capacity, leeches get (near) nothing;
//! otherwise they benefit from the leftovers. No one is dropped from the
//! system.
//!
//! Pure computation: no I/O, no async runtime.
//!
//! `SpiderNet` software carries no warranty and accepts no responsibility
//! for its usage or for transported content (AGPL-3.0, sections 15/16 —
//! responsibility for usage lies with the users).
#![forbid(unsafe_code)]
#![warn(clippy::pedantic)]
// deny is the goal before the first release
// PoC scope: doc-annotation noise, not correctness. Revisit at release.
#![allow(clippy::must_use_candidate, clippy::missing_errors_doc)]
pub mod error;
pub mod fair;
pub mod types;
pub use error::FairnessError;
pub use fair::{allocate, Allocation, Demand, FairShare, Participant, Participation};
pub use types::{Capacity, ParticipantId, Rate, Weight};
+212
View File
@@ -0,0 +1,212 @@
//! CLI for the sn-fair fairness scheduler.
//!
//! KEINE GARANTIE — this software is provided "as is" without warranty of
//! any kind (AGPL-3.0, sections 15/16). Users are solely responsible for
//! what they transmit or receive with it.
//!
//! Pilot scope: simulates how pooled neighborhood capacity is split —
//! e.g. two household downloads plus a leech competing for the pooled
//! uplinks.
#![forbid(unsafe_code)]
#![warn(clippy::pedantic)]
// PoC scope: doc-annotation noise, not correctness. Revisit at release.
#![allow(clippy::must_use_candidate, clippy::missing_errors_doc)]
use std::collections::BTreeMap;
use std::num::NonZeroU32;
use std::process::ExitCode;
use clap::Parser;
use sn_fair::fair::{Demand, Participant, Participation};
use sn_fair::types::{Capacity, ParticipantId, Rate, Weight};
use sn_fair::{allocate, FairShare};
/// Parses `id=contributing[:weight]` or `id=leeching`.
fn parse_participant_spec(spec: &str) -> Result<(ParticipantId, Participation), String> {
let (id_raw, class_raw) = spec.split_once('=').ok_or_else(|| {
format!("invalid participant spec `{spec}`, expected `id=contributing[:weight]` or `id=leeching`")
})?;
let participant = id_raw
.trim()
.parse::<ParticipantId>()
.map_err(|err| format!("invalid participant spec `{spec}`: {err}"))?;
let class = class_raw.trim();
let participation = match class.strip_prefix("contributing") {
Some(rest) => {
let weight = if rest.is_empty() {
Weight::default()
} else if let Some(weight_raw) = rest.strip_prefix(':') {
weight_raw
.trim()
.parse::<u32>()
.ok()
.and_then(NonZeroU32::new)
.map(Weight::new)
.ok_or_else(|| {
format!("invalid weight `{weight_raw}` in `{spec}` (must be a non-zero number)")
})?
} else {
return Err(format!(
"invalid participation `{class}` in `{spec}`, expected `contributing[:weight]` or `leeching`"
));
};
Participation::Contributing { weight }
}
None if class == "leeching" => Participation::Leeching,
None => {
return Err(format!(
"invalid participation `{class}` in `{spec}`, expected `contributing[:weight]` or `leeching`"
))
}
};
Ok((participant, participation))
}
/// Parses `id=rate`.
fn parse_demand_spec(spec: &str) -> Result<(ParticipantId, Rate), String> {
let (id_raw, rate_raw) = spec
.split_once('=')
.ok_or_else(|| format!("invalid demand spec `{spec}`, expected `id=rate`"))?;
let participant = id_raw
.trim()
.parse::<ParticipantId>()
.map_err(|err| format!("invalid demand spec `{spec}`: {err}"))?;
let rate = rate_raw.trim().parse::<u64>().map(Rate::new).map_err(|_| {
format!("invalid rate `{rate_raw}` in `{spec}` (must be a non-negative number)")
})?;
Ok((participant, rate))
}
#[derive(Debug, Parser)]
#[command(
name = "sn-fair",
version,
about = "SpiderNet weighted max-min fairness scheduler (pilot)"
)]
struct Args {
/// Total pooled capacity, in Mbit/s.
#[arg(long)]
capacity: u64,
/// Participant: `id=contributing[:weight]` (offers its uplink,
/// weight shapes the share, default weight 1) or `id=leeching`
/// (receive-only). Repeatable.
#[arg(
long = "participant",
value_name = "ID=CLASS[:WEIGHT]",
value_parser = parse_participant_spec
)]
participants: Vec<(ParticipantId, Participation)>,
/// Demand: `id=rate` in Mbit/s. Repeatable; participants without a
/// demand receive nothing.
#[arg(
long = "demand",
value_name = "ID=RATE",
value_parser = parse_demand_spec
)]
demands: Vec<(ParticipantId, Rate)>,
}
fn run() -> Result<(), String> {
let args = Args::parse();
let capacity = Capacity::try_new(args.capacity).map_err(|err| err.to_string())?;
let participants: Vec<Participant> = args
.participants
.into_iter()
.map(|(id, participation)| Participant::new(id, participation))
.collect();
let demand_of: BTreeMap<ParticipantId, u64> = args
.demands
.iter()
.map(|(participant, rate)| (*participant, rate.value()))
.collect();
let demands: Vec<Demand> = args
.demands
.into_iter()
.map(|(participant, rate)| Demand::new(participant, rate))
.collect();
let share = allocate(capacity, &participants, &demands).map_err(|err| err.to_string())?;
print_report(capacity, &participants, &demand_of, &share);
Ok(())
}
fn print_report(
capacity: Capacity,
participants: &[Participant],
demand_of: &BTreeMap<ParticipantId, u64>,
share: &FairShare,
) {
println!("NO WARRANTY — SpiderNet sn-fair (AGPL-3.0, no liability)");
println!();
println!("pooled capacity: {} Mbit/s", capacity.value());
println!();
let headers = ["participant", "class", "weight", "demand", "allocation"];
let mut rows = Vec::new();
for participant in participants {
let (class, weight) = match participant.participation() {
Participation::Contributing { weight } => ("contributing", weight.value().to_string()),
Participation::Leeching => ("leeching", "-".to_string()),
};
let demand = demand_of
.get(&participant.id())
.map_or_else(|| "0".to_string(), ToString::to_string);
rows.push([
participant.id().to_string(),
class.to_string(),
weight,
demand,
share.rate_of(participant.id()).to_string(),
]);
}
let mut widths = [0usize; 5];
for (index, header) in headers.iter().enumerate() {
widths[index] = header.len();
}
for row in &rows {
for (index, cell) in row.iter().enumerate() {
widths[index] = widths[index].max(cell.len());
}
}
print_row(&headers.map(ToString::to_string), &widths);
for row in &rows {
print_row(row, &widths);
}
println!();
println!("total allocated: {}", share.total_allocated());
println!("unused: {}", share.unused());
}
fn print_row(cells: &[String], widths: &[usize]) {
println!(
"{:<w0$} {:<w1$} {:>w2$} {:>w3$} {:>w4$}",
cells[0],
cells[1],
cells[2],
cells[3],
cells[4],
w0 = widths[0],
w1 = widths[1],
w2 = widths[2],
w3 = widths[3],
w4 = widths[4],
);
}
fn main() -> ExitCode {
match run() {
Ok(()) => ExitCode::SUCCESS,
Err(report) => {
eprintln!("NO WARRANTY — SpiderNet sn-fair (AGPL-3.0, no liability)");
eprintln!("error: {report}");
ExitCode::FAILURE
}
}
}
+133
View File
@@ -0,0 +1,133 @@
//! Domain types: newtypes over raw primitives, validated at construction.
use std::fmt;
use std::num::{NonZeroU16, NonZeroU32, NonZeroU64};
use std::str::FromStr;
use crate::error::FairnessError;
/// Identifier of a participant (a household) in the fairness allocation.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct ParticipantId(NonZeroU16);
impl ParticipantId {
pub const fn new(value: NonZeroU16) -> Self {
Self(value)
}
pub const fn value(self) -> u16 {
self.0.get()
}
}
impl fmt::Display for ParticipantId {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}", self.0.get())
}
}
/// Parses a decimal id; rejects zero and non-numeric input.
impl FromStr for ParticipantId {
type Err = FairnessError;
fn from_str(s: &str) -> Result<Self, Self::Err> {
let parsed = s.trim().parse::<u16>().ok().and_then(NonZeroU16::new);
parsed
.map(Self::new)
.ok_or_else(|| FairnessError::InvalidParticipantId(s.to_string()))
}
}
/// Relative share shaping of a contributing participant. 1 means equal
/// share; a participant with weight 2 gets twice the share of one with
/// weight 1 (as far as demands allow).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Weight(NonZeroU32);
impl Weight {
pub const fn new(value: NonZeroU32) -> Self {
Self(value)
}
pub const fn value(self) -> u32 {
self.0.get()
}
}
impl Default for Weight {
fn default() -> Self {
Self(NonZeroU32::MIN)
}
}
/// A traffic rate, unit-agnostic: bytes per second, Mbit/s — whatever
/// unit the caller works with (the CLI treats it as Mbit/s).
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
pub struct Rate(u64);
impl Rate {
pub const fn new(value: u64) -> Self {
Self(value)
}
pub const fn value(self) -> u64 {
self.0
}
}
impl fmt::Display for Rate {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}", self.0)
}
}
/// Total poolable capacity shared by all participants; strictly positive.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
pub struct Capacity(NonZeroU64);
impl Capacity {
pub fn try_new(value: u64) -> Result<Self, FairnessError> {
NonZeroU64::new(value)
.map(Self)
.ok_or(FairnessError::ZeroCapacity)
}
pub const fn value(self) -> u64 {
self.0.get()
}
}
impl fmt::Display for Capacity {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}", self.0.get())
}
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used)]
use super::*;
#[test]
fn participant_id_parses_decimal_and_rejects_zero_and_garbage() {
assert_eq!("7".parse::<ParticipantId>().unwrap().value(), 7);
assert_eq!(" 42 ".parse::<ParticipantId>().unwrap().value(), 42);
assert!("0".parse::<ParticipantId>().is_err());
assert!("abc".parse::<ParticipantId>().is_err());
assert!("-1".parse::<ParticipantId>().is_err());
assert!("70000".parse::<ParticipantId>().is_err());
assert!("".parse::<ParticipantId>().is_err());
}
#[test]
fn weight_defaults_to_one() {
assert_eq!(Weight::default().value(), 1);
}
#[test]
fn capacity_rejects_zero() {
assert!(Capacity::try_new(0).is_err());
assert_eq!(Capacity::try_new(5).unwrap().value(), 5);
}
}
+8
View File
@@ -7,9 +7,17 @@ license.workspace = true
[dependencies] [dependencies]
tokio = { workspace = true } tokio = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true } serde_json = { workspace = true }
toml = { workspace = true }
thiserror = { workspace = true } thiserror = { workspace = true }
tracing = { workspace = true }
tracing-subscriber = { workspace = true }
clap = { workspace = true } clap = { workspace = true }
sn-exit = { path = "../sn-exit" }
[dev-dependencies]
tempfile = { workspace = true }
[lints] [lints]
workspace = true workspace = true
+11
View File
@@ -239,6 +239,17 @@ pub fn write_config(path: &Path, json: &str) -> Result<(), NodeError> {
Ok(()) Ok(())
} }
/// Parse a TUN interface setting: `auto`, `none` (headless router-only)
/// or an explicit interface name.
pub fn parse_ifname(input: &str) -> Result<IfName, NodeError> {
match input.trim() {
"auto" => Ok(IfName::Auto),
"none" => Ok(IfName::Headless),
name if !name.is_empty() => Ok(IfName::Named(name.to_string())),
_ => Err(NodeError::InvalidIfName(input.to_string())),
}
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used)] #![allow(clippy::unwrap_used, clippy::expect_used)]
+337
View File
@@ -0,0 +1,337 @@
//! The `SpiderNet` node daemon: supervises the Yggdrasil overlay sidecar
//! and runs the optional exit service in-process.
//!
//! Responsibilities (kill-switch semantics):
//!
//! - manage the yggdrasil sidecar when `[yggdrasil] manage = true`:
//! persistent node key at `<state_dir>/node.key`, generated config at
//! `<state_dir>/yggdrasil.conf`, child process with graceful stop;
//! - wait for the overlay address via the admin socket and log it;
//! - run the exit service (`sn-exit`) in-process when
//! `[exit] enabled = true`; otherwise leech mode (overlay only,
//! receive without offering an exit);
//! - on shutdown (SIGINT/SIGTERM wired in by the CLI): stop the accept
//! loop, then stop the sidecar — SIGTERM first, `start_kill` after a
//! bounded grace period;
//! - if the sidecar dies unexpectedly: shut down and exit with an error
//! (supervisor duty).
//!
//! [`run_daemon`] takes the shutdown as a plain future so tests can drive
//! it without signals.
use std::future::Future;
use std::path::Path;
use std::pin::{pin, Pin};
use std::time::Duration;
use tokio::net::TcpListener;
use tokio::process::Child;
use crate::config::{self, PublicKeyHex};
use crate::daemon_config::{ValidatedDaemonConfig, YGGDRASIL_CONF_FILE};
use crate::error::NodeError;
use crate::yggdrasil::{AdminClient, YggAddr};
/// Printed first by the daemon CLI; the software carries no warranty and
/// accepts no liability (AGPL-3.0, sections 15/16).
pub const NO_WARRANTY_BANNER: &str =
"NO WARRANTY — SpiderNet sn-node daemon (AGPL-3.0, no liability)";
/// How long the daemon waits for the overlay (admin socket) to come up.
const OVERLAY_WAIT: Duration = Duration::from_secs(60);
/// Interval between admin socket attempts while the overlay is down.
const OVERLAY_POLL: Duration = Duration::from_secs(1);
/// Grace period for SIGTERM-terminated yggdrasil before we kill it.
const TERMINATE_GRACE: Duration = Duration::from_secs(5);
/// Run the node daemon until `shutdown` completes or a fatal error
/// occurs. Returns `Ok(())` on a clean shutdown.
pub async fn run_daemon(
config: ValidatedDaemonConfig,
shutdown: impl Future<Output = ()> + Send,
) -> Result<(), NodeError> {
let mut shutdown = pin!(shutdown);
log_config_summary(&config);
let mut child = if config.manage() {
Some(spawn_sidecar(&config).await?)
} else {
None
};
let overlay = match wait_for_overlay(&config, &mut child, shutdown.as_mut()).await {
OverlayOutcome::Up(address) => address,
OverlayOutcome::Shutdown => {
terminate_sidecar(&mut child).await;
return Ok(());
}
OverlayOutcome::Failed(error) => {
kill_child(&mut child).await;
return Err(error);
}
};
if let Some(exit) = config.exit() {
let bind = exit.bind().resolve(overlay);
let listener = match TcpListener::bind(bind).await {
Ok(listener) => listener,
Err(source) => {
kill_child(&mut child).await;
return Err(NodeError::ExitListen { addr: bind, source });
}
};
tracing::info!("exit listening on {bind}");
if exit.allowlist().is_loopback_only() {
tracing::warn!(
"exit in DEV MODE: no neighbor allowlist — only loopback clients are accepted"
);
}
let exit_config =
sn_exit::config::ExitConfig::new(bind, exit.allowlist().clone(), exit.quota());
tokio::select! {
result = sn_exit::proxy::serve_until(exit_config, listener, shutdown.as_mut()) => {
if let Err(error) = result {
kill_child(&mut child).await;
return Err(NodeError::Exit(error));
}
}
status = child_exit(&mut child) => {
kill_child(&mut child).await;
return Err(NodeError::SidecarExited(
status.unwrap_or_else(|| "unknown".to_string()),
));
}
}
} else {
// Leech mode: overlay only, until signal or sidecar death.
tokio::select! {
() = shutdown.as_mut() => {}
status = child_exit(&mut child) => {
return Err(NodeError::SidecarExited(
status.unwrap_or_else(|| "unknown".to_string()),
));
}
}
}
terminate_sidecar(&mut child).await;
tracing::info!("SpiderNet node daemon stopped");
Ok(())
}
fn log_config_summary(config: &ValidatedDaemonConfig) {
let overlay = if config.manage() {
"managed sidecar"
} else {
"external (yggdrasil.manage = false)"
};
let exit = match config.exit() {
Some(_) => "enabled (sharing the uplink)",
None => "disabled (leech mode)",
};
tracing::info!(
admin = %config.admin_endpoint().config_value(),
state_dir = %config.state_dir().display(),
overlay,
exit,
"SpiderNet node daemon starting"
);
}
/// Prepare the state directory (node key + generated Yggdrasil config)
/// and spawn the sidecar process.
async fn spawn_sidecar(config: &ValidatedDaemonConfig) -> Result<Child, NodeError> {
let state_dir = config.state_dir().to_path_buf();
tokio::fs::create_dir_all(&state_dir).await?;
let key_path = state_dir.join(crate::daemon_config::NODE_KEY_FILE);
let settings = config.node_settings();
// One `yggdrasil -genconf -json` run provides both the base config
// and — on first boot — the fresh node key.
let json = config::generate_config(config.yggdrasil_binary(), &settings)?;
let base: serde_json::Value = serde_json::from_str(&json)?;
if key_path.exists() {
// The node identity must survive restarts: reuse the persisted
// key, never regenerate it.
//
// The base JSON still embeds a fresh `PrivateKey`, but per the
// official configuration reference PrivateKeyPath takes
// precedence when both are set
// (https://yggdrasil-network.github.io/configurationref.html),
// so yggdrasil runs with the persisted key.
let existing = tokio::fs::read_to_string(&key_path).await?;
let key = existing.trim();
PublicKeyHex::parse(key).map_err(|err| {
NodeError::DaemonConfig(format!(
"existing node key {} is invalid ({err}); the node identity \
must stay stable, so it is not regenerated",
key_path.display()
))
})?;
tracing::info!("node key loaded from {}", key_path.display());
} else {
let generated = base
.get("PrivateKey")
.and_then(serde_json::Value::as_str)
.ok_or_else(|| {
NodeError::DaemonConfig(format!(
"`{} -genconf -json` output has no `PrivateKey` field",
config.yggdrasil_binary()
))
})?;
let key = PublicKeyHex::parse(generated)
.map_err(|err| NodeError::DaemonConfig(format!("generated node key: {err}")))?;
write_private_key(&key_path, key.as_str())?;
tracing::info!(
"node key generated and stored at {} (mode 600)",
key_path.display()
);
}
let conf_path = state_dir.join(YGGDRASIL_CONF_FILE);
config::write_config(&conf_path, &json)?;
tracing::info!("yggdrasil config written to {}", conf_path.display());
let mut command = tokio::process::Command::new(config.yggdrasil_binary());
command.arg("-useconffile").arg(&conf_path);
let child = command.spawn()?;
tracing::info!("yggdrasil sidecar spawned (pid {:?})", child.id());
Ok(child)
}
/// Write the node's private key with owner-only permissions (mode 600).
fn write_private_key(path: &Path, key: &str) -> Result<(), NodeError> {
use std::io::Write as _;
let mut file = std::fs::File::create(path)?;
file.write_all(key.as_bytes())?;
file.flush()?;
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o600))?;
}
Ok(())
}
/// Result of waiting for the overlay.
enum OverlayOutcome {
Up(YggAddr),
/// Shutdown was requested while waiting.
Shutdown,
Failed(NodeError),
}
/// Poll the admin socket until the overlay reports the node's own
/// address. Aborts on shutdown and on unexpected sidecar death; the
/// sidecar child (if any) is reaped by the `child.wait()` arm.
async fn wait_for_overlay(
config: &ValidatedDaemonConfig,
child: &mut Option<Child>,
mut shutdown: Pin<&mut (impl Future<Output = ()> + Send)>,
) -> OverlayOutcome {
let admin = config.admin_endpoint();
let deadline = tokio::time::Instant::now() + OVERLAY_WAIT;
loop {
let remaining = deadline.saturating_duration_since(tokio::time::Instant::now());
if remaining.is_zero() {
return OverlayOutcome::Failed(NodeError::OverlayNotUp {
endpoint: admin.config_value(),
timeout_secs: OVERLAY_WAIT.as_secs(),
});
}
let attempt_timeout = remaining.min(OVERLAY_POLL);
tokio::select! {
biased;
() = shutdown.as_mut() => return OverlayOutcome::Shutdown,
status = child_exit(&mut *child) => {
return OverlayOutcome::Failed(NodeError::SidecarExited(
status.unwrap_or_else(|| "unknown".to_string()),
));
}
result = AdminClient::get_self(admin, attempt_timeout) => match result {
Ok(info) => {
tracing::info!("overlay is up: address {}", info.address);
if let Ok(peers) = AdminClient::get_peers(admin, attempt_timeout).await {
tracing::info!("overlay peers: {}", peers.len());
} else {
tracing::info!("overlay peers: not reported yet");
}
return OverlayOutcome::Up(info.address);
}
Err(_) => tokio::time::sleep(OVERLAY_POLL).await,
}
}
}
}
/// Resolve when the sidecar process exits; never resolves without one.
/// Returns a human-readable exit status description.
async fn child_exit(child: &mut Option<Child>) -> Option<String> {
if let Some(child) = child {
return Some(match child.wait().await {
Ok(status) => status.to_string(),
Err(error) => format!("wait failed: {error}"),
});
}
std::future::pending::<()>().await;
None
}
/// Stop the sidecar gracefully: SIGTERM via `kill`, bounded grace, then
/// a hard kill.
async fn terminate_sidecar(child: &mut Option<Child>) {
let Some(child) = child.as_mut() else {
return;
};
let Some(pid) = child.id() else {
tracing::info!("yggdrasil sidecar already gone");
return;
};
tracing::info!("stopping yggdrasil sidecar (pid {pid}): sending SIGTERM");
match std::process::Command::new("kill")
.arg("-TERM")
.arg(pid.to_string())
.status()
{
Ok(status) if status.success() => {}
Ok(status) => tracing::warn!("`kill -TERM {pid}` failed: {status}"),
Err(error) => tracing::warn!("could not run `kill -TERM {pid}`: {error}"),
}
match tokio::time::timeout(TERMINATE_GRACE, child.wait()).await {
Ok(Ok(status)) => tracing::info!("yggdrasil sidecar stopped ({status})"),
Ok(Err(error)) => tracing::warn!("waiting for yggdrasil failed: {error}"),
Err(_) => {
tracing::warn!(
"yggdrasil did not exit within {}s; killing",
TERMINATE_GRACE.as_secs()
);
if let Err(error) = child.start_kill() {
tracing::warn!("start_kill failed: {error}");
}
child.wait().await.ok();
tracing::info!("yggdrasil sidecar killed");
}
}
}
/// Kill the sidecar immediately (failure path); a no-op when it is
/// already gone.
async fn kill_child(child: &mut Option<Child>) {
let Some(child) = child.as_mut() else {
return;
};
if child.id().is_some() {
tracing::warn!("killing yggdrasil sidecar after failure");
if let Err(error) = child.start_kill() {
tracing::warn!("start_kill failed: {error}");
}
}
child.wait().await.ok();
}
+555
View File
@@ -0,0 +1,555 @@
//! Typed, validated configuration for the node daemon (`sn-node daemon`).
//!
//! The TOML file is parsed into raw optional sections and then validated
//! into types that carry their invariants ("parsed data is validated"):
//! URIs, keys, interface names and addresses reuse the existing parsers of
//! `sn-node` and `sn-exit`, so an invalid config fails with a clear error
//! at load time instead of a mid-run surprise.
//!
//! File schema (TOML):
//!
//! ```toml
//! [node]
//! admin_endpoint = "tcp://localhost:9001" # optional; also used as Yggdrasil AdminListen
//! state_dir = "spidernet-state" # optional; holds node.key + generated yggdrasil.conf
//!
//! [yggdrasil]
//! manage = true # optional, default true; false = an already-running yggdrasil daemon is used
//! binary = "yggdrasil" # optional, default "yggdrasil"; only relevant when manage = true
//! peers = [] # optional; list of PeerUri strings (tcp://, tls://, quic://, socks://)
//! listen = [] # optional; list of ListenUri strings (tcp://, tls://, quic://)
//! allow_keys = [] # optional; 64-hex neighbor public keys (AllowedPublicKeys ACL)
//! no_multicast = false # optional bool
//! ifname = "auto" # optional: "auto" | "none" | interface name
//!
//! [exit]
//! enabled = false # optional, default false (false = leech mode)
//! bind = "" # optional explicit bind address; default "[<yggdrasil-address>]:port"
//! port = 8080 # optional u16, default 8080 (used when bind is unset)
//! allow = [] # optional neighbor overlay addresses (sn-exit --allow semantics)
//! quota_mib = 1024 # optional u64, default 1024; 0 = no quota
//! window_hours = 24 # optional u64, default 24
//! ```
use std::net::{IpAddr, SocketAddr};
use std::path::{Path, PathBuf};
use std::time::Duration;
use serde::Deserialize;
use sn_exit::config::{parse_bind, parse_source, QuotaPolicy, SourceAllowlist};
use crate::config::{parse_ifname, IfName, ListenUri, NodeSettings, PeerUri, PublicKeyHex};
use crate::error::NodeError;
use crate::yggdrasil::{AdminEndpoint, YggAddr};
/// Default state directory (relative to the daemon's working directory).
pub const DEFAULT_STATE_DIR: &str = "spidernet-state";
/// Default yggdrasil binary (looked up in `PATH`).
pub const DEFAULT_YGGDRASIL_BINARY: &str = "yggdrasil";
/// Name of the persisted node private key inside the state directory.
pub const NODE_KEY_FILE: &str = "node.key";
/// Name of the generated Yggdrasil config inside the state directory.
pub const YGGDRASIL_CONF_FILE: &str = "yggdrasil.conf";
/// Default admin endpoint when `[node] admin_endpoint` is unset.
const DEFAULT_ADMIN_ENDPOINT: &str = "tcp://localhost:9001";
/// Default exit port when `[exit] bind` is unset.
const DEFAULT_EXIT_PORT: u16 = 8080;
/// Default per-neighbor quota in MiB.
const DEFAULT_QUOTA_MIB: u64 = 1024;
/// Default quota window in hours.
const DEFAULT_WINDOW_HOURS: u64 = 24;
const MIB: u64 = 1024 * 1024;
/// Where the exit service binds. Resolved at runtime because the overlay
/// address is only known once the overlay is up.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ExitBind {
/// An explicit address from the config.
Explicit(SocketAddr),
/// `[<overlay address>]:port` on the node's own overlay address.
Overlay { port: u16 },
}
impl ExitBind {
/// Resolve this bind against the node's overlay address.
pub const fn resolve(self, overlay: YggAddr) -> SocketAddr {
match self {
Self::Explicit(addr) => addr,
Self::Overlay { port } => SocketAddr::new(IpAddr::V6(overlay.inner()), port),
}
}
}
/// Validated settings for the exit service; present only when
/// `[exit] enabled = true`. Without it the daemon runs in leech mode
/// (overlay only, receive without offering an exit).
#[derive(Debug, Clone)]
pub struct ExitServiceSettings {
bind: ExitBind,
allowlist: SourceAllowlist,
quota: Option<QuotaPolicy>,
}
impl ExitServiceSettings {
/// Where the exit should bind.
pub const fn bind(&self) -> ExitBind {
self.bind
}
/// Which neighbor source addresses may use the exit.
pub const fn allowlist(&self) -> &SourceAllowlist {
&self.allowlist
}
/// Per-neighbor quota, or none when disabled (`quota_mib = 0`).
pub const fn quota(&self) -> Option<QuotaPolicy> {
self.quota
}
}
/// A fully validated daemon config. Construction goes through
/// [`parse`]/[`load`]; every field carries its invariant afterwards.
#[derive(Debug, Clone)]
pub struct ValidatedDaemonConfig {
admin_endpoint: AdminEndpoint,
state_dir: PathBuf,
manage: bool,
yggdrasil_binary: String,
peers: Vec<PeerUri>,
listen: Vec<ListenUri>,
allowed_keys: Vec<PublicKeyHex>,
no_multicast: bool,
if_name: IfName,
exit: Option<ExitServiceSettings>,
}
impl ValidatedDaemonConfig {
/// Admin socket endpoint; also used as Yggdrasil `AdminListen` when
/// the daemon manages the sidecar.
pub const fn admin_endpoint(&self) -> &AdminEndpoint {
&self.admin_endpoint
}
/// State directory: holds `node.key` and the generated yggdrasil
/// config.
pub fn state_dir(&self) -> &Path {
&self.state_dir
}
/// Whether the daemon manages the yggdrasil sidecar process.
pub const fn manage(&self) -> bool {
self.manage
}
/// Path or name of the yggdrasil binary.
pub fn yggdrasil_binary(&self) -> &str {
&self.yggdrasil_binary
}
/// The Yggdrasil settings derived from this config, with the node key
/// at `<state_dir>/node.key`.
pub fn node_settings(&self) -> NodeSettings {
NodeSettings {
peers: self.peers.clone(),
listen: self.listen.clone(),
multicast: Some(if self.no_multicast {
crate::config::MulticastMode::Disabled
} else {
crate::config::MulticastMode::PlatformDefault
}),
admin_listen: Some(self.admin_endpoint.clone()),
if_name: Some(self.if_name.clone()),
allowed_public_keys: self.allowed_keys.clone(),
private_key_path: Some(self.state_dir.join(NODE_KEY_FILE)),
}
}
/// Exit service settings, or `None` in leech mode.
pub const fn exit(&self) -> Option<&ExitServiceSettings> {
self.exit.as_ref()
}
}
/// Parse a daemon config from TOML text.
pub fn parse(input: &str) -> Result<ValidatedDaemonConfig, NodeError> {
let raw: RawDaemonConfig = toml::from_str(input)
.map_err(|err| NodeError::DaemonConfig(format!("cannot parse config: {err}")))?;
validate(raw)
}
/// Parse a daemon config from the file at `path`.
pub fn load(path: &Path) -> Result<ValidatedDaemonConfig, NodeError> {
let text = std::fs::read_to_string(path).map_err(|err| {
NodeError::DaemonConfig(format!("cannot read config file {}: {err}", path.display()))
})?;
parse(&text)
}
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct RawDaemonConfig {
#[serde(default)]
node: RawNodeSection,
#[serde(default)]
yggdrasil: RawYggdrasilSection,
#[serde(default)]
exit: RawExitSection,
}
#[derive(Debug, Default, Deserialize)]
#[serde(deny_unknown_fields)]
struct RawNodeSection {
admin_endpoint: Option<String>,
state_dir: Option<String>,
}
#[derive(Debug, Default, Deserialize)]
#[serde(deny_unknown_fields)]
struct RawYggdrasilSection {
manage: Option<bool>,
binary: Option<String>,
#[serde(default)]
peers: Option<Vec<String>>,
#[serde(default)]
listen: Option<Vec<String>>,
#[serde(default)]
allow_keys: Option<Vec<String>>,
no_multicast: Option<bool>,
ifname: Option<String>,
}
#[derive(Debug, Default, Deserialize)]
#[serde(deny_unknown_fields)]
struct RawExitSection {
enabled: Option<bool>,
bind: Option<String>,
port: Option<u16>,
#[serde(default)]
allow: Option<Vec<String>>,
quota_mib: Option<u64>,
window_hours: Option<u64>,
}
fn validate(raw: RawDaemonConfig) -> Result<ValidatedDaemonConfig, NodeError> {
let RawDaemonConfig {
node,
yggdrasil,
exit,
} = raw;
let admin_endpoint = match node.admin_endpoint.as_deref() {
None => AdminEndpoint::parse(DEFAULT_ADMIN_ENDPOINT)
.map_err(|err| NodeError::DaemonConfig(err.to_string()))?,
Some(value) => AdminEndpoint::parse(value)
.map_err(|err| NodeError::DaemonConfig(format!("node.admin_endpoint: {err}")))?,
};
let state_dir = node
.state_dir
.map_or_else(|| PathBuf::from(DEFAULT_STATE_DIR), PathBuf::from);
let manage = yggdrasil.manage.unwrap_or(true);
let yggdrasil_binary = yggdrasil
.binary
.unwrap_or_else(|| DEFAULT_YGGDRASIL_BINARY.to_string());
let peers = parse_each(
yggdrasil.peers.as_deref(),
"yggdrasil.peers",
PeerUri::parse,
)?;
let listen = parse_each(
yggdrasil.listen.as_deref(),
"yggdrasil.listen",
ListenUri::parse,
)?;
let allowed_keys = parse_each(
yggdrasil.allow_keys.as_deref(),
"yggdrasil.allow_keys",
PublicKeyHex::parse,
)?;
let no_multicast = yggdrasil.no_multicast.unwrap_or(false);
let if_name = yggdrasil.ifname.as_deref().unwrap_or("auto");
let if_name = parse_ifname(if_name)
.map_err(|err| NodeError::DaemonConfig(format!("yggdrasil.ifname: {err}")))?;
let exit = validate_exit(&exit)?;
if !manage && exit.is_none() {
return Err(NodeError::DaemonConfig(
"nothing to do: set `[yggdrasil] manage = true` or `[exit] enabled = true`".to_string(),
));
}
Ok(ValidatedDaemonConfig {
admin_endpoint,
state_dir,
manage,
yggdrasil_binary,
peers,
listen,
allowed_keys,
no_multicast,
if_name,
exit,
})
}
fn validate_exit(raw: &RawExitSection) -> Result<Option<ExitServiceSettings>, NodeError> {
let enabled = raw.enabled.unwrap_or(false);
let explicit_bind = match raw.bind.as_deref() {
None | Some("") => None,
Some(value) => Some(
parse_bind(value)
.map_err(|err| NodeError::DaemonConfig(format!("exit.bind: {err}")))?,
),
};
let allow_sources: Vec<IpAddr> = parse_each(raw.allow.as_deref(), "exit.allow", parse_source)?;
// Without a neighbor allowlist only loopback clients would be
// accepted — that is acceptable only for development, where an
// explicit loopback bind makes the intent unmistakable.
if enabled && allow_sources.is_empty() {
let dev_mode = explicit_bind.is_some_and(|addr| addr.ip().is_loopback());
if !dev_mode {
return Err(NodeError::DaemonConfig(
"exit enabled but no neighbor allowlist: add `exit.allow` entries, \
or (dev only) set `exit.bind` to a loopback address"
.to_string(),
));
}
}
let allowlist = if allow_sources.is_empty() {
SourceAllowlist::loopback_only()
} else {
SourceAllowlist::new(allow_sources)
};
let quota = match raw.quota_mib.unwrap_or(DEFAULT_QUOTA_MIB) {
0 => None,
mib => Some(
QuotaPolicy::try_new(
mib.saturating_mul(MIB),
Duration::from_secs(
raw.window_hours
.unwrap_or(DEFAULT_WINDOW_HOURS)
.saturating_mul(3600),
),
)
.map_err(|err| NodeError::DaemonConfig(format!("exit.quota: {err}")))?,
),
};
let bind = match explicit_bind {
Some(addr) => ExitBind::Explicit(addr),
None => ExitBind::Overlay {
port: raw.port.unwrap_or(DEFAULT_EXIT_PORT),
},
};
if enabled {
Ok(Some(ExitServiceSettings {
bind,
allowlist,
quota,
}))
} else {
Ok(None)
}
}
/// Parse every entry with `parse`, wrapping failures in a single
/// config error that names the offending section.
fn parse_each<T, E: std::fmt::Display>(
inputs: Option<&[String]>,
what: &str,
parse: fn(&str) -> Result<T, E>,
) -> Result<Vec<T>, NodeError> {
let inputs = inputs.unwrap_or(&[]);
inputs
.iter()
.map(|input| parse(input).map_err(|err| NodeError::DaemonConfig(format!("{what}: {err}"))))
.collect()
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used)]
use super::*;
fn error_for(toml_text: &str) -> String {
parse(toml_text)
.expect_err("config should be invalid")
.to_string()
}
#[test]
fn full_config_maps_into_validated_types() {
let config = parse(
r#"
[node]
admin_endpoint = "tcp://localhost:9001"
state_dir = "/var/lib/spidernet"
[yggdrasil]
manage = true
binary = "/usr/local/bin/yggdrasil"
peers = ["tcp://[200:1234::1]:1337", "socks://p:1/h:2"]
listen = ["tls://[::]:1337"]
allow_keys = ["00112233445566778899aabbccddeeff00112233445566778899aabbccddeeff"]
no_multicast = true
ifname = "eth0"
[exit]
enabled = true
bind = "127.0.0.1:18099"
allow = []
quota_mib = 2048
window_hours = 12
"#,
)
.unwrap();
assert_eq!(
config.admin_endpoint(),
&AdminEndpoint::parse("tcp://localhost:9001").unwrap()
);
assert_eq!(config.state_dir(), Path::new("/var/lib/spidernet"));
assert!(config.manage());
assert_eq!(config.yggdrasil_binary(), "/usr/local/bin/yggdrasil");
let settings = config.node_settings();
assert_eq!(settings.peers.len(), 2);
assert_eq!(settings.peers[0].as_str(), "tcp://[200:1234::1]:1337");
assert_eq!(settings.peers[1].as_str(), "socks://p:1/h:2");
assert_eq!(settings.listen.len(), 1);
assert_eq!(settings.listen[0].as_str(), "tls://[::]:1337");
assert_eq!(settings.allowed_public_keys.len(), 1);
assert!(matches!(
settings.if_name,
Some(IfName::Named(ref name)) if name == "eth0"
));
assert_eq!(
settings.private_key_path.as_deref(),
Some(Path::new("/var/lib/spidernet/node.key"))
);
let exit = config.exit().unwrap();
assert_eq!(
exit.bind(),
ExitBind::Explicit("127.0.0.1:18099".parse().unwrap())
);
assert!(exit.allowlist().is_loopback_only());
let quota = exit.quota().unwrap();
assert_eq!(quota.max_bytes(), 2048 * 1024 * 1024);
assert_eq!(quota.window(), Duration::from_hours(12));
}
#[test]
fn minimal_config_defaults_to_leech_mode() {
let config = parse("[exit]\nenabled = false\n").unwrap();
assert!(config.manage());
assert_eq!(config.yggdrasil_binary(), DEFAULT_YGGDRASIL_BINARY);
assert_eq!(config.state_dir(), Path::new(DEFAULT_STATE_DIR));
assert!(config.exit().is_none());
assert!(config.node_settings().peers.is_empty());
}
#[test]
fn load_missing_file_is_an_error() {
let error = load(Path::new("/nonexistent/spidernet-node.toml")).unwrap_err();
assert!(error.to_string().contains("cannot read config file"));
}
#[test]
fn load_valid_file_succeeds() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("node.toml");
std::fs::write(&path, "[yggdrasil]\nmanage = true\n").unwrap();
assert!(load(&path).is_ok());
}
#[test]
fn broken_toml_gives_readable_error() {
let error = error_for("[node]\nadmin_endpoint = \n");
assert!(error.contains("cannot parse config"), "{error}");
}
#[test]
fn exit_without_allowlist_requires_loopback_bind() {
let no_bind = error_for("[exit]\nenabled = true\n");
assert!(
no_bind.contains("exit enabled but no neighbor allowlist"),
"{no_bind}"
);
let non_loopback = error_for("[exit]\nenabled = true\nbind = \"192.168.1.5:8080\"\n");
assert!(
non_loopback.contains("exit enabled but no neighbor allowlist"),
"{non_loopback}"
);
}
#[test]
fn exit_with_loopback_bind_is_valid_dev_mode() {
let config = parse("[exit]\nenabled = true\nbind = \"127.0.0.1:18099\"\n").unwrap();
let exit = config.exit().unwrap();
assert!(exit.allowlist().is_loopback_only());
assert_eq!(
exit.bind(),
ExitBind::Explicit("127.0.0.1:18099".parse().unwrap())
);
}
#[test]
fn manage_false_and_exit_disabled_is_rejected() {
let error = error_for("[yggdrasil]\nmanage = false\n");
assert!(error.contains("nothing to do"), "{error}");
}
#[test]
fn quota_zero_disables_quota() {
let config =
parse("[exit]\nenabled = true\nbind = \"[::1]:8080\"\nquota_mib = 0\n").unwrap();
assert_eq!(config.exit().unwrap().quota(), None);
}
#[test]
fn overlay_bind_defaults_to_port_8080_and_can_be_overridden() {
let config = parse("[exit]\nenabled = true\nallow = [\"200:5678::2\"]\n").unwrap();
let bind = config.exit().unwrap().bind();
assert_eq!(bind, ExitBind::Overlay { port: 8080 });
let overlay: std::net::Ipv6Addr = "200:1111::1".parse().unwrap();
assert_eq!(
bind.resolve(YggAddr::new(overlay)),
SocketAddr::new(IpAddr::V6(overlay), 8080)
);
let config =
parse("[exit]\nenabled = true\nallow = [\"200:5678::2\"]\nport = 9099\n").unwrap();
let bind = config.exit().unwrap().bind();
assert_eq!(
bind.resolve(YggAddr::new(overlay)),
SocketAddr::new(IpAddr::V6(overlay), 9099)
);
}
#[test]
fn invalid_entries_surface_reused_type_errors() {
for invalid in [
"[yggdrasil]\npeers = [\"banana\"]\n",
"[yggdrasil]\nlisten = [\"socks://[::]:1337\"]\n",
"[yggdrasil]\nallow_keys = [\"nothex\"]\n",
"[yggdrasil]\nifname = \"\"\n",
"[node]\nadmin_endpoint = \"banana\"\n",
"[exit]\nallow = [\"banana\"]\n",
"[exit]\nbind = \"banana\"\nenabled = true\n",
"[exit]\nport = 70000\n",
] {
let error = parse(invalid).expect_err("should be invalid").to_string();
assert!(error.contains("invalid daemon config"), "{error}");
}
}
}
+21
View File
@@ -31,4 +31,25 @@ pub enum NodeError {
#[error("admin socket operation timed out")] #[error("admin socket operation timed out")]
Timeout, Timeout,
#[error("invalid daemon config: {0}")]
DaemonConfig(String),
#[error("overlay did not come up within {timeout_secs}s (admin endpoint {endpoint})")]
OverlayNotUp { endpoint: String, timeout_secs: u64 },
#[error("yggdrasil sidecar exited unexpectedly: {0}")]
SidecarExited(String),
#[error("exit service cannot listen on {addr}: {source}")]
ExitListen {
addr: std::net::SocketAddr,
source: std::io::Error,
},
#[error("exit service error: {0}")]
Exit(#[from] sn_exit::error::ExitError),
#[error("invalid interface name `{0}`: expected `auto`, `none` or an interface name")]
InvalidIfName(String),
} }
+5
View File
@@ -10,6 +10,9 @@
//! - [`yggdrasil`]: client for the Yggdrasil admin socket (JSON over //! - [`yggdrasil`]: client for the Yggdrasil admin socket (JSON over
//! TCP/unix, default `localhost:9001`) to query node status (`getSelf`, //! TCP/unix, default `localhost:9001`) to query node status (`getSelf`,
//! `getPeers`). //! `getPeers`).
//! - [`daemon_config`]: the validated TOML config of the node daemon.
//! - [`daemon`]: the node daemon — supervises the yggdrasil sidecar and
//! runs the optional `sn-exit` exit service in-process.
//! //!
//! Yggdrasil provides encrypted end-to-end routing between households. //! Yggdrasil provides encrypted end-to-end routing between households.
//! It is **not** an anonymity network, and `SpiderNet` never claims to //! It is **not** an anonymity network, and `SpiderNet` never claims to
@@ -24,5 +27,7 @@
#![allow(clippy::must_use_candidate, clippy::missing_errors_doc)] #![allow(clippy::must_use_candidate, clippy::missing_errors_doc)]
pub mod config; pub mod config;
pub mod daemon;
pub mod daemon_config;
pub mod error; pub mod error;
pub mod yggdrasil; pub mod yggdrasil;
+53 -12
View File
@@ -8,9 +8,9 @@ use std::path::PathBuf;
use std::time::Duration; use std::time::Duration;
use clap::{Parser, Subcommand}; use clap::{Parser, Subcommand};
use sn_node::config::{ use sn_node::config::{self, ListenUri, MulticastMode, NodeSettings, PeerUri, PublicKeyHex};
self, IfName, ListenUri, MulticastMode, NodeSettings, PeerUri, PublicKeyHex, use sn_node::daemon::{self, NO_WARRANTY_BANNER};
}; use sn_node::daemon_config;
use sn_node::yggdrasil::{self, AdminEndpoint}; use sn_node::yggdrasil::{self, AdminEndpoint};
#[derive(Debug, Parser)] #[derive(Debug, Parser)]
@@ -77,15 +77,17 @@ enum Command {
#[arg(long, default_value_t = 3)] #[arg(long, default_value_t = 3)]
timeout_secs: u64, timeout_secs: u64,
}, },
} /// Run the node daemon: supervise the Yggdrasil sidecar and the
/// optional in-process exit service. Stops on SIGINT/SIGTERM.
Daemon {
/// Path to the TOML config file.
#[arg(long, default_value = "node.toml")]
config: PathBuf,
fn parse_ifname(input: &str) -> Result<IfName, String> { /// Validate the config and exit without starting anything.
match input.trim() { #[arg(long)]
"auto" => Ok(IfName::Auto), check: bool,
"none" => Ok(IfName::Headless), },
name if !name.is_empty() => Ok(IfName::Named(name.to_string())),
_ => Err("--ifname must be `auto`, `none` or an interface name".to_string()),
}
} }
fn parse_uris<T>( fn parse_uris<T>(
@@ -139,6 +141,7 @@ async fn run() -> Result<(), String> {
admin, admin,
timeout_secs, timeout_secs,
} => run_status(&admin, timeout_secs).await, } => run_status(&admin, timeout_secs).await,
Command::Daemon { config, check } => run_daemon_command(&config, check).await,
} }
} }
@@ -161,7 +164,7 @@ async fn run_genconf(
.iter() .iter()
.map(|key| PublicKeyHex::parse(key).map_err(|err| err.to_string())) .map(|key| PublicKeyHex::parse(key).map_err(|err| err.to_string()))
.collect::<Result<Vec<_>, String>>()?; .collect::<Result<Vec<_>, String>>()?;
let if_name = parse_ifname(&ifname)?; let if_name = config::parse_ifname(&ifname).map_err(|err| err.to_string())?;
let settings = NodeSettings { let settings = NodeSettings {
peers, peers,
@@ -227,3 +230,41 @@ async fn run_status(admin: &str, timeout_secs: u64) -> Result<(), String> {
} }
Ok(()) Ok(())
} }
async fn run_daemon_command(config_path: &std::path::Path, check: bool) -> Result<(), String> {
println!("{NO_WARRANTY_BANNER}");
let validated = daemon_config::load(config_path).map_err(|err| err.to_string())?;
if check {
println!("config ok");
return Ok(());
}
tracing_subscriber::fmt()
.with_env_filter(
tracing_subscriber::EnvFilter::try_from_default_env()
.unwrap_or_else(|_| tracing_subscriber::EnvFilter::new("info")),
)
.init();
daemon::run_daemon(validated, shutdown_signal())
.await
.map_err(|err| err.to_string())
}
/// Future that resolves when SIGINT or SIGTERM (unix) arrives.
async fn shutdown_signal() {
#[cfg(unix)]
{
if let Ok(mut sigterm) =
tokio::signal::unix::signal(tokio::signal::unix::SignalKind::terminate())
{
tokio::select! {
_ = tokio::signal::ctrl_c() => {}
_ = sigterm.recv() => {}
}
return;
}
}
tokio::signal::ctrl_c().await.ok();
}
+331
View File
@@ -0,0 +1,331 @@
//! Integration tests for the node daemon, against a *fake* yggdrasil
//! (a shell script; real yggdrasil is not assumed to be installed) and
//! a fake admin socket server speaking the documented protocol.
//!
//! Unix-only: the fake yggdrasil is a shell script and the sidecar
//! handling relies on POSIX process semantics.
#![cfg(unix)]
#![forbid(unsafe_code)]
#![allow(clippy::unwrap_used, clippy::expect_used)]
use std::net::SocketAddr;
use std::os::unix::fs::PermissionsExt;
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::time::Duration;
use sn_node::daemon_config::{self, ExitBind, ValidatedDaemonConfig};
use sn_node::error::NodeError;
use tokio::net::{TcpListener, TcpStream};
const FAKE_PRIVATE_KEY: &str = "00112233445566778899aabbccddeeff00112233445566778899aabbccddeeff";
const SELF_RESPONSE: &str = concat!(
r#"{"request": {"request": "getself"}, "status": "success", "response": {"#,
r#""200:1111:2222:3333:4444:5555:6666:7777": {"#,
r#""address": "200:1111:2222:3333:4444:5555:6666:7777","#,
r#""key": "0000aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa1111","#,
r#""build_name": "yggdrasil", "build_version": "0.5.12","#,
r#""subnet": "300:1111:2222:3333::/64", "coords": [1, 2, 3]}}}"#
);
const PEERS_RESPONSE: &str = concat!(
r#"{"request": {"request": "getpeers"}, "status": "success", "response": {"#,
r#""200:1111:2222:3333:4444:5555:6666:7777": {"coords": [1]}, "#,
r#""200:9999:2222:3333:4444:5555:6666:7777": {"#,
r#""coords": [1, 2], "port": 1, "key": "ffff...", "#,
r#""remote": "tcp://[fe80::1%25eth0]:1337"}}}"#
);
/// What the fake yggdrasil does once started with `-useconffile`.
#[derive(Clone, Copy)]
enum SidecarBehavior {
/// Keep running until killed (test shutdown paths).
RunForever,
/// Exit shortly after start (test the supervisor death path).
DieSoon,
}
/// Fake admin server: answers `getself`/`getpeers` request lines, then
/// closes the connection (mirrors real yggdrasil's one-request behavior).
async fn fake_admin_server() -> SocketAddr {
use tokio::io::{AsyncReadExt, AsyncWriteExt};
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap();
tokio::spawn(async move {
loop {
let Ok((mut socket, _)) = listener.accept().await else {
break;
};
let mut line = Vec::new();
let mut byte = [0u8; 1];
while !line.ends_with(b"\n") {
if socket.read(&mut byte).await.unwrap_or(0) == 0 {
break;
}
line.push(byte[0]);
}
let request = String::from_utf8_lossy(&line);
let response = if request.contains("getpeers") {
PEERS_RESPONSE
} else {
SELF_RESPONSE
};
socket.write_all(response.as_bytes()).await.ok();
socket.shutdown().await.ok();
}
});
addr
}
/// Write a shell script that impersonates the yggdrasil binary: it
/// answers `-genconf -json` with a fixed minimal config and busy-waits
/// (or dies) on `-useconffile`. The script path is returned.
fn write_fake_yggdrasil(dir: &Path, behavior: SidecarBehavior) -> PathBuf {
let pidfile = dir.join("fake-yggdrasil.pid");
let wait = match behavior {
SidecarBehavior::RunForever => "exec sleep 600",
SidecarBehavior::DieSoon => "exec sleep 0.3",
};
let script = format!(
r#"#!/bin/sh
# fake yggdrasil for the SpiderNet daemon tests
if [ "$1" = "-genconf" ]; then
printf '%s' '{{"PrivateKey": "{FAKE_PRIVATE_KEY}", "Peers": [], "Listen": []}}'
exit 0
fi
if [ "$1" = "-useconffile" ]; then
echo $$ > '{pidfile}'
{wait}
fi
exit 1
"#,
pidfile = pidfile.display(),
);
let path = dir.join("fake-yggdrasil");
std::fs::write(&path, script).unwrap();
std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o755)).unwrap();
path
}
/// Build a validated daemon config pointing at the fake infrastructure.
fn test_config(
admin: SocketAddr,
binary: &Path,
state_dir: &Path,
exit_toml: &str,
) -> ValidatedDaemonConfig {
let text = format!(
"[node]\nadmin_endpoint = \"tcp://{}:{}\"\nstate_dir = \"{}\"\n\
[yggdrasil]\nbinary = \"{}\"\n\
[exit]\n{exit_toml}\n",
admin.ip(),
admin.port(),
state_dir.display(),
binary.display(),
);
daemon_config::parse(&text).unwrap()
}
/// Poll `check` until it returns true or a deadline passes.
async fn wait_until(check: impl Fn() -> bool) -> bool {
for _ in 0..200 {
if check() {
return true;
}
tokio::time::sleep(Duration::from_millis(50)).await;
}
false
}
/// Poll a TCP connect until the daemon's exit listener is up.
async fn exit_is_accepting(addr: SocketAddr) -> bool {
wait_until_async(|| async { TcpStream::connect(addr).await.is_ok() }).await
}
async fn wait_until_async<F>(check: impl Fn() -> F) -> bool
where
F: std::future::Future<Output = bool>,
{
for _ in 0..200 {
if check().await {
return true;
}
tokio::time::sleep(Duration::from_millis(50)).await;
}
false
}
/// A shutdown future driven by a shared flag (avoids a tokio `sync`
/// dependency in tests; the 10 ms polling is plenty for tests).
async fn flag_shutdown(flag: Arc<AtomicBool>) {
loop {
if flag.load(Ordering::Relaxed) {
return;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
}
/// A shutdown flag plus a setter closure that flips it.
fn shutdown_flag() -> (Arc<AtomicBool>, impl Fn() + Send + Clone + 'static) {
let flag = Arc::new(AtomicBool::new(false));
let setter = {
let flag = Arc::clone(&flag);
move || flag.store(true, Ordering::Relaxed)
};
(flag, setter)
}
#[tokio::test]
async fn node_key_is_generated_once_and_reused() {
let tmp = tempfile::tempdir().unwrap();
let admin = fake_admin_server().await;
let binary = write_fake_yggdrasil(tmp.path(), SidecarBehavior::RunForever);
let state_dir = tmp.path().join("state");
let config = test_config(admin, &binary, &state_dir, "enabled = false");
// First run: the daemon generates the node key, the yggdrasil config
// and spawns the fake sidecar, then waits for shutdown.
let (flag, shutdown) = shutdown_flag();
let handle = tokio::spawn(sn_node::daemon::run_daemon(
config.clone(),
flag_shutdown(flag),
));
let key_path = state_dir.join("node.key");
assert!(
wait_for_file(&key_path).await,
"node key was not generated in time"
);
let first_key = std::fs::read_to_string(&key_path).unwrap();
assert_eq!(first_key.trim(), FAKE_PRIVATE_KEY);
assert_eq!(
std::fs::metadata(&key_path).unwrap().permissions().mode() & 0o777,
0o600,
"node key must be owner-only"
);
let conf = std::fs::read_to_string(state_dir.join("yggdrasil.conf")).unwrap();
assert!(
conf.contains("\"PrivateKeyPath\"") && conf.contains(&key_path.display().to_string()),
"generated config must reference the node key file:\n{conf}"
);
assert!(
conf.contains(&format!(
"\"AdminListen\": \"tcp://{}:{}\"",
admin.ip(),
admin.port()
)),
"generated config must use the configured admin endpoint:\n{conf}"
);
shutdown();
let result = tokio::time::timeout(Duration::from_secs(30), handle)
.await
.unwrap()
.unwrap();
assert!(result.is_ok(), "first run failed: {result:?}");
// Second run: the key must be reused, not regenerated.
let (flag, shutdown) = shutdown_flag();
let handle = tokio::spawn(sn_node::daemon::run_daemon(config, flag_shutdown(flag)));
shutdown();
let result = tokio::time::timeout(Duration::from_secs(30), handle)
.await
.unwrap()
.unwrap();
assert!(result.is_ok(), "second run failed: {result:?}");
let second_key = std::fs::read_to_string(&key_path).unwrap();
assert_eq!(second_key, first_key, "node identity must survive restarts");
}
async fn wait_for_file(path: &Path) -> bool {
wait_until(|| path.exists()).await
}
#[tokio::test]
async fn exit_service_listens_and_accepts_connections() {
let tmp = tempfile::tempdir().unwrap();
let admin = fake_admin_server().await;
let binary = write_fake_yggdrasil(tmp.path(), SidecarBehavior::RunForever);
let state_dir = tmp.path().join("state");
// Grab a free port, then release it for the daemon to bind.
let scratch = TcpListener::bind("127.0.0.1:0").await.unwrap();
let exit_addr = scratch.local_addr().unwrap();
drop(scratch);
let config = test_config(
admin,
&binary,
&state_dir,
&format!("enabled = true\nbind = \"127.0.0.1:{}\"", exit_addr.port()),
);
let (flag, shutdown) = shutdown_flag();
let handle = tokio::spawn(sn_node::daemon::run_daemon(config, flag_shutdown(flag)));
assert!(
exit_is_accepting(exit_addr).await,
"the exit service did not start listening in time"
);
shutdown();
let result = tokio::time::timeout(Duration::from_secs(30), handle)
.await
.unwrap()
.unwrap();
assert!(result.is_ok(), "daemon run failed: {result:?}");
}
#[tokio::test]
async fn immediate_shutdown_ends_run_daemon_cleanly() {
// No sidecar (manage = false): an already-running yggdrasil is
// assumed; the daemon must still exit cleanly on shutdown.
let config = daemon_config::parse(
"[yggdrasil]\nmanage = false\n[exit]\nenabled = true\nbind = \"127.0.0.1:0\"\n",
)
.unwrap();
assert!(matches!(
config.exit().unwrap().bind(),
ExitBind::Explicit(_)
));
let result = tokio::time::timeout(
Duration::from_secs(30),
sn_node::daemon::run_daemon(config, std::future::ready(())),
)
.await
.unwrap();
assert!(result.is_ok(), "daemon should end cleanly: {result:?}");
}
#[tokio::test]
async fn sidecar_death_fails_the_daemon() {
let tmp = tempfile::tempdir().unwrap();
// Admin endpoint points nowhere: only the supervisor's child-death
// detection can end the run.
let config = daemon_config::parse(&format!(
"[node]\nadmin_endpoint = \"tcp://127.0.0.1:1\"\nstate_dir = \"{}\"\n\
[yggdrasil]\nbinary = \"{}\"\n[exit]\nenabled = false\n",
tmp.path().join("state").display(),
write_fake_yggdrasil(tmp.path(), SidecarBehavior::DieSoon).display(),
))
.unwrap();
let result = tokio::time::timeout(
Duration::from_secs(30),
sn_node::daemon::run_daemon(config, std::future::pending()),
)
.await
.unwrap();
assert!(
matches!(result, Err(NodeError::SidecarExited(_))),
"expected SidecarExited error, got {result:?}"
);
}
+67
View File
@@ -0,0 +1,67 @@
# syntax=docker/dockerfile:1
#
# SpiderNet node container image (multi-stage).
#
# NO WARRANTY — SpiderNet is provided "as is", without warranty of any kind
# (AGPL-3.0, sections 15/16). Running a node is entirely at the operator's
# own responsibility, including all traffic sent and received with it.
#
# Build context is the `server/` directory (see docker-compose.yml):
# docker build -t spidernet-node -f docker/Dockerfile ..
# --- Stage 1: build the Rust workspace --------------------------------------
FROM rust:1-slim AS build
WORKDIR /server
COPY Cargo.toml Cargo.lock ./
COPY crates ./crates
# --locked: the build must match the committed Cargo.lock exactly.
RUN cargo build --release --locked --workspace
# --- Stage 2: runtime --------------------------------------------------------
# trixie, not bookworm: rust:1-slim is trixie-based, and binaries built
# against its glibc (2.41) need a matching runtime (bookworm only has 2.36).
FROM debian:trixie-slim
# Yggdrasil sidecar: current stable release of yggdrasil-go. Release v0.5.14
# ships official .deb packages (no raw tarballs). Binary inside: /usr/bin/yggdrasil.
# Digests from https://api.github.com/repos/yggdrasil-network/yggdrasil-go/releases/latest
ARG YGGDRASIL_VERSION=0.5.14
ARG TARGETARCH
# ca-certificates: TLS for sn-exit/sn-fetch (rustls) and tls:// peers.
# curl is only needed to fetch the yggdrasil .deb and is removed again.
# TARGETARCH is only set by BuildKit; fall back to the host arch otherwise
# (both use the same amd64/arm64 naming as the yggdrasil .deb assets).
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates curl \
&& ARCH="${TARGETARCH:-$(dpkg --print-architecture)}" \
&& case "$ARCH" in \
amd64) YGGDRASIL_SHA256=72745496b09baa1c3f5a6741e1257a704cfdb9d00cdf3ddbef025987b239699d ;; \
arm64) YGGDRASIL_SHA256=ac8c9a9a76c54b878fd251aef5a2bbd20375c6109b28b32435683441462360c9 ;; \
*) echo "unsupported architecture: $ARCH" >&2; exit 1 ;; \
esac \
&& curl -fsSL -o /tmp/yggdrasil.deb \
"https://github.com/yggdrasil-network/yggdrasil-go/releases/download/v${YGGDRASIL_VERSION}/yggdrasil-${YGGDRASIL_VERSION}-${ARCH}.deb" \
&& echo "${YGGDRASIL_SHA256} /tmp/yggdrasil.deb" | sha256sum -c - \
&& dpkg-deb -x /tmp/yggdrasil.deb /tmp/yggdrasil-root \
&& cp /tmp/yggdrasil-root/usr/bin/yggdrasil /usr/local/bin/yggdrasil \
&& rm -rf /tmp/yggdrasil.deb /tmp/yggdrasil-root \
&& apt-get purge -y curl \
&& apt-get autoremove -y \
&& rm -rf /var/lib/apt/lists/*
COPY --from=build /server/target/release/sn-node /usr/local/bin/sn-node
COPY --from=build /server/target/release/sn-exit /usr/local/bin/sn-exit
COPY --from=build /server/target/release/sn-fetch /usr/local/bin/sn-fetch
COPY --from=build /server/target/release/sn-fair /usr/local/bin/sn-fair
# WORKDIR matches the state volume in docker-compose.yml; a config that
# leaves `state_dir` at its default then resolves to the volume as well.
WORKDIR /var/lib/spidernet
ENTRYPOINT ["sn-node", "daemon", "--config", "/etc/spidernet/node.toml"]
+125
View File
@@ -0,0 +1,125 @@
# SpiderNet-Node mit Docker
Dieser Ordner enthält alles, um den SpiderNet-Node eines Haushalts als
Container zu betreiben: Multi-Stage-Dockerfile, ein Compose-Beispiel für
einen Host und eine kommentierte Beispiel-Konfiguration.
## Haftungsausschluss
> **KEINE GARANTIE.** Diese Software wird "as is" ohne jegliche Garantie
> bereitgestellt (AGPL §§ 15/16). Die Betreiber und Contributor:innen dieses
> Projekts übernehmen **keinerlei Verantwortung oder Haftung** dafür, wozu
> Nutzer:innen die Software verwenden oder welche Inhalte über sie
> übertragen werden. Die Verantwortung für gesendete und empfangene Inhalte
> liegt vollständig bei den Nutzer:innen.
## Quickstart
```sh
# 1. Konfiguration anlegen und anpassen
cp node.toml.example node.toml
$EDITOR node.toml
# 2. Image bauen und starten (Build-Kontext ist server/, daher "context: ..")
docker compose up --build -d
# 3. Beobachten
docker compose logs -f
# 4. Overlay-Status ansehen (Admin-Socket lauscht auf localhost:9001)
docker compose exec node sn-node status
# 5. Einen großen Download über die gepoolte Bandbreite laden
docker compose exec node sn-fetch "https://…/große-datei.iso" \
-o /tmp/out.iso -e "1=2,2=1@http://[<ygg-adresse-des-nachbarn>]:8080"
```
Image direkt bauen (ohne Compose), Build-Kontext `server/`:
```sh
docker build -t spidernet-node -f docker/Dockerfile ..
```
## Was die Container-Privilegien bedeuten
| Einstellung | Warum |
|---|---|
| `network_mode: host` | Yggdrasil muss auf dem realen LAN peeren und auf der Overlay-Seite erreichbar sein. Mit Bridge-NAT würden Peer-Verbindungen und der Exit-Port nicht stimmen. |
| `cap_add: [NET_ADMIN]` | Yggdrasil legt das TUN-Interface (`ifname`) an und konfiguriert es — das braucht `CAP_NET_ADMIN`. |
| `/dev/net/tun` | Das Kernel-TUN-Gerät, das Yggdrasil öffnet. |
| `init: true` | Sauberer PID 1: Reaping + SIGTERM erreicht den Daemon, damit `docker compose stop` als sauberer Kill-Switch funktioniert. |
Der Container läuft als `root` — für TUN im Container üblich. `NET_ADMIN`
ist bewusst nur als Capability gesetzt, ohne `privileged`.
## Zwei Haushalte verbinden
Beide Hosts laufen mit demselben Image; die Rolle (Exit oder Leech) steht
nur in der jeweiligen `node.toml`.
- **Die Mesh-Verbindung liegt außerhalb des Containers:** Glasfaser,
Switch oder WiFi zwischen den Haushalten. Der Container peeret über das
Host-Netzwerk.
- **Auto-Peering:** Im selben LAN (Multicast aktiv) finden sich
Yggdrasil-Nodes selbst — `peers = []` reicht.
- **Explizit:** Falls Multicast nicht geht (z. B. VLANs), trägt man in
`node.toml` bei `peers` URIs wie `tcp://<nachbar-lan-ip>:1337` ein.
- **ACL:** Die 64-Hex-Public-Keys der erlaubten Nachbarn gehören in
`allow_keys`, sonst kann jeder Node im Overlay peeren.
Die `node.toml` jedes Haushalts enthält persönliche Daten der Nachbarschaft
(Public Keys, Adressen) und später den Node-Private-Key im State-Verzeichnis:
**niemals committen**, außerhalb des Repos aufbewahren.
## Leech-Modus (Empfang ohne Exit)
Die Beispiel-Konfiguration startet sicher im **Leech-Modus** — ein
Haushalt, der nur empfangen will, muss nichts ändern:
```toml
[exit]
enabled = false
```
Das ist ausdrücklich legitim — Leeches werden bei Sättigung zuerst
gedrosselt, aber nie ausgeschlossen. Es gibt kein separates Leech-Image;
das Compose-Beispiel funktioniert unverändert.
Wer seinen Uplink als Exit anbieten will (opt-in), setzt:
```toml
[exit]
enabled = true
allow = ["<overlay-adresse-des-nachbarn>"]
```
`enabled = true` mit leerer `allow`-Liste ist ungültig — der Daemon
verweigert dann den Start (Ausnahme: Dev-Modus mit Loopback-bind).
## Kill-Switch
```sh
docker compose stop
```
Stoppen des Exits bzw. des Daemons ist der Kill-Switch: Sobald der Daemon
nicht läuft, stellt der Haushalt keinen Uplink mehr bereit. Der eigene Node
fällt auf den eigenen Internetanschluss zurück. Wichtig zum Verständnis:
SpiderNet v0 leitet **normalen Verkehr nicht um** — kein Routing, kein
NAT-Trick. Nur Downloads, die `sn-fetch` explizit über das Netz verteilt,
nutzen die gepoolte Bandbreite; alles andere geht wie gewohnt über den
eigenen Anschluss.
## Zustand
`node.key` und die generierte `yggdrasil.conf` liegen im State-Verzeichnis
(`/var/lib/spidernet`, Named Volume `spidernet-state` im Compose-Beispiel).
Der Node behält damit seine Identität über Container-Neustarts. Volume
löschen = neue Node-Identität.
## Noch offen / Hinweise
- Der `daemon`-Subcommand ist implementiert und die Beispiel-Konfiguration
wurde damit verifiziert (`sn-node daemon --check` → `config ok`).
- Images werden lokal gebaut, nicht in ein Registry gepusht (das Projekt
kennt keine zentrale Infrastruktur).
+74
View File
@@ -0,0 +1,74 @@
# SpiderNet node — single-household example (one node per host).
#
# NO WARRANTY — SpiderNet is provided "as is" without warranty of any kind
# (AGPL-3.0, sections 15/16). Running a node is entirely at the operator's
# own responsibility.
#
# The same image serves both roles:
# - exit household: [exit] enabled = true in node.toml (plus neighbor
# addresses in [exit].allow — enabled with an empty allowlist is
# rejected by the daemon)
# - leech household: [exit] enabled = false in node.toml (the example
# config default — safe, receive-only)
# There is no separate leech image; only the config differs.
#
# Quickstart:
# cp node.toml.example node.toml # then edit node.toml
# docker compose up --build -d
services:
node:
build:
# Build context is server/, matching the Dockerfile paths.
context: ..
dockerfile: docker/Dockerfile
image: spidernet-node
container_name: spidernet-node
# Proper PID 1: reaps zombies and makes sure SIGTERM reaches sn-node
# so `docker compose stop` behaves as the documented kill switch.
init: true
# Yggdrasil must bind its TUN interface and peer on the real LAN;
# both require the host network stack (no NAT translation).
network_mode: host
cap_add:
# Needed by yggdrasil to create/configure the TUN interface.
- NET_ADMIN
devices:
# The kernel TUN device used for the overlay interface.
- /dev/net/tun:/dev/net/tun
restart: unless-stopped
volumes:
# Config is read-only; state (node.key, yggdrasil.conf) goes to a
# named volume so the node identity survives container upgrades.
- ./node.toml:/etc/spidernet/node.toml:ro
- spidernet-state:/var/lib/spidernet
# ---------------------------------------------------------------------------
# EXIT VARIANT (offering the household's uplink as exit):
#
# The example config defaults to leech mode ([exit] enabled = false). To
# become an exit household set enabled = true in node.toml and add the
# neighbor overlay addresses to [exit].allow. This compose file works
# unchanged for both roles (kill switch: `docker compose stop`).
#
# If you prefer two service definitions (e.g. to toggle via profiles), note
# that `network_mode: host` allows only ONE node per host — stop the other
# service first:
#
# node-leech:
# build:
# context: ..
# dockerfile: docker/Dockerfile
# image: spidernet-node
# container_name: spidernet-node-leech
# init: true
# network_mode: host
# cap_add: [NET_ADMIN]
# devices: ["/dev/net/tun:/dev/net/tun"]
# restart: unless-stopped
# volumes:
# - ./node-leech.toml:/etc/spidernet/node.toml:ro
# - spidernet-state-leech:/var/lib/spidernet
volumes:
spidernet-state:
+95
View File
@@ -0,0 +1,95 @@
# Beispielkonfiguration für den SpiderNet-Node-Daemon (`sn-node daemon`).
#
# Vorlage: diese Datei nach `node.toml` kopieren und anpassen. Die
# Compose-Datei in diesem Ordner bindet `./node.toml` ein.
#
# KEINE GARANTIE — SpiderNet wird "as is" ohne jegliche Garantie
# bereitgestellt (AGPL §§ 15/16). Nutzung auf eigene Verantwortung;
# die Verantwortung für gesendete und empfangene Inhalte liegt bei den
# Nutzer:innen.
#
# WICHTIG: Die echte node.toml enthält persönliche Daten der Nachbarschaft
# (Public Keys, Adressen) und später den Node-Private-Key im state_dir.
# node.toml NIEMALS ins Repository committen.
[node]
# Admin-Socket des Yggdrasil-Knotens (tcp://host:port oder unix:///pfad).
# Wird auch als Yggdrasil-AdminListen verwendet, wenn der Daemon das
# Yggdrasil-Sidecar selbst verwaltet (siehe [yggdrasil].manage).
admin_endpoint = "tcp://localhost:9001"
# Verzeichnis für den Zustand des Nodes (node.key, generierte
# yggdrasil.conf). Optional, Standard "spidernet-state" — relativ zum
# Arbeitsverzeichnis des Daemons. Im Container besser absolut setzen,
# damit es auf das State-Volume fällt:
state_dir = "/var/lib/spidernet"
[yggdrasil]
# true = der Daemon startet und überwacht das Yggdrasil-Sidecar selbst.
# false = es wird ein bereits laufender Yggdrasil-Daemon verwendet.
manage = true
# Pfad/Name des Yggdrasil-Binary; nur relevant, wenn manage = true.
# Im Container ist yggdrasil unter /usr/local/bin/yggdrasil installiert.
binary = "yggdrasil"
# Peer-URIs (tcp://, tls://, quic://, socks://) für manuelles Peering.
# Über das Nachbarschafts-LAN peert Yggdrasil per Multicast automatisch;
# explizite Peers braucht man z. B. über WLAN-Brücken hinweg.
peers = []
# Listener-URIs für eingehende Peerings (tcp://, tls://, quic://).
listen = []
# Access-Control: 64-stellige Hex-Public-Keys der Nachbarn, die mit uns
# peeren dürfen (AllowedPublicKeys). Leere Liste = jeder darf peeren.
allow_keys = []
# true = Multicast-Auto-Peering im LAN deaktivieren.
no_multicast = false
# TUN-Interface: "auto" (Standard), "none" (headless, nur Router-Modus)
# oder ein fester Interface-Name.
ifname = "auto"
[exit]
# true = dieser Haushalt stellt seinen Uplink als Exit bereit (opt-in).
# false = Leech-Modus (Standard hier): der Node empfängt nur, bietet
# keinen Exit an — ausdrücklich legitim, niemand wird gedrosselt, nur
# fair priorisiert.
#
# Wer Exit anbieten will: enabled = true setzen UND in `allow` die
# Overlay-Adressen der Nachbarn eintragen — enabled = true mit leerer
# `allow`-Liste ist ungültig (der Daemon verweigert den Start; Ausnahme:
# Dev-Modus mit Loopback-bind).
enabled = false
# Explizite Bind-Adresse. Leer (Standard) = "[<Yggdrasil-Adresse>]:port",
# d. h. der Exit lauscht auf der eigenen Overlay-Adresse.
# UNGÜLTIG: allow = [] zusammen mit einem nicht-Loopback-bind — der Exit
# wäre sonst für beliebig viele Quellen offen (Schutz: ohne Allowlist nur
# Loopback, das ist der Dev-Modus).
bind = ""
# TCP-Port des Exit-Dienstes (u16, Standard 8080).
port = 8080
# Overlay-Adressen der Nachbarn, die diesen Exit nutzen dürfen.
# Leere Liste = nur Loopback (Dev-Modus); bei nicht-Loopback-bind Pflicht.
# Beispiel:
# allow = ["200:abcd:ef01::2", "200:abcd:ef01::3"]
allow = []
# Kontingent pro Nachbar: Bytes pro Zeitfenster (u64, Standard 1024 MiB).
# 0 = unbegrenzt (kein Kontingent).
quota_mib = 1024
# Länge des Kontingent-Zeitfensters in Stunden (fixed window).
window_hours = 24
# KILL-SWITCH: Wer nichts mehr teilen will, stoppt einfach den Exit bzw.
# den Daemon (Container: `docker compose stop`). Damit endet die
# Bereitstellung des eigenen Uplinks sofort — der eigene Node fällt auf
# den eigenen Internetanschluss zurück. SpiderNet v0 leitet normalen
# Verkehr nicht um: nur Downloads, die sn-fetch explizit über das Netz
# verteilt, nutzen die gepoolte Bandbreite.