Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b7577c4552 | ||
|
|
43b634a135 | ||
|
|
660bb70c5d | ||
|
|
e59932ab75 | ||
|
|
cf7a3bcc88 | ||
|
|
579a2c164f | ||
|
|
f5ea5840d4 | ||
|
|
893fe0c95d | ||
|
|
5786b1e206 |
@@ -8,6 +8,70 @@ Die Version steht an genau einer Stelle: `src/netmover/__init__.py`.
|
|||||||
|
|
||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
|
||||||
|
## [0.9.0] – 2026-10-15
|
||||||
|
|
||||||
|
### Hinzugefügt
|
||||||
|
- **Regeln kopieren** (Knopf „Kopieren“): alle Einstellungen inklusive Archiv
|
||||||
|
übernommen, Name „… (Kopie)“. Die Kopie ist zunächst **deaktiviert**, damit sie nicht
|
||||||
|
mit dem Original um dieselben Dateien konkurriert; Protokoll und Buchführung
|
||||||
|
beginnen neu.
|
||||||
|
|
||||||
|
## [0.8.0] – 2026-10-12
|
||||||
|
|
||||||
|
### Hinzugefügt
|
||||||
|
- **Anbindungen kopieren** (Knopf „Kopieren“): Formular vorausgefüllt, Name
|
||||||
|
„… (Kopie)“. Eigene Zugangsdaten der Vorlage werden übernommen (eigenes Exemplar),
|
||||||
|
solange kein neues Passwort eingegeben wird; ein eigener SSH-Schlüssel wird
|
||||||
|
weiterverwendet; gespeicherte Zugangsdaten bleiben ausgewählt. „Testen“ im
|
||||||
|
Kopie-Formular nutzt die übernommenen Zugangsdaten. Der SFTP-Hostschlüssel wird nur
|
||||||
|
bei gleichem Server und Port übernommen.
|
||||||
|
|
||||||
|
## [0.7.0] – 2026-10-12
|
||||||
|
|
||||||
|
### Hinzugefügt
|
||||||
|
- **Archiv je Regel:** zusätzliche Kopie in ein Archiv (Anbindung SMB/NFS/SFTP oder
|
||||||
|
lokaler Pfad), Ablage „wie in der Quelle“ oder „nach Datum“ (`Jahr/Monat/…`).
|
||||||
|
Reihenfolge: erst Archiv, dann Ziel, **erst dann wird die Quelle gelöscht**.
|
||||||
|
Scheitert das Ziel nach dem Archiv, verhindert eine Buchführung
|
||||||
|
(`archived_files`) eine zweite Archivkopie. Im Archiv wird nie überschrieben.
|
||||||
|
Nicht erreichbares Archiv → Regel wartet. Platzprüfung auch fürs Archiv,
|
||||||
|
Probelauf zeigt das Archivziel. Migration 0004.
|
||||||
|
- Export/Import enthalten die Archiv-Einstellungen; eine als Archiv benutzte
|
||||||
|
Anbindung lässt sich nicht löschen.
|
||||||
|
|
||||||
|
## [0.6.0] – 2026-10-11
|
||||||
|
|
||||||
|
### Geändert
|
||||||
|
- **Anmeldung je Netz konfigurierbar** (*Einstellungen → Anmeldung*): „Ohne Passwort
|
||||||
|
aus diesen Netzen“ und „Mit Passwort aus diesen Netzen“ (leer = alle übrigen).
|
||||||
|
Ist die zweite Liste ausgefüllt, werden alle anderen Netze abgewiesen (403, auch die
|
||||||
|
Anmeldeseite); `/healthz` und statische Dateien bleiben erreichbar. Ersetzt die
|
||||||
|
Auswahl „immer / vertraute Netze / nie“ aus 0.5.0 (`0.0.0.0/0, ::/0` = ganz aus;
|
||||||
|
gespeicherter Modus „nie“ gilt weiter).
|
||||||
|
- Schutz vor Aussperren: Eine Einstellung, mit der die eigene Adresse keinen Zugang
|
||||||
|
mehr hätte, wird nicht gespeichert. `netmover reset-password` hebt auch Netz-Sperren auf.
|
||||||
|
|
||||||
|
## [0.5.0] – 2026-10-11
|
||||||
|
|
||||||
|
### Hinzugefügt
|
||||||
|
- **SFTP als Quelle** mit denselben Garantien wie bei SMB/NFS: Stabilitätsprüfung,
|
||||||
|
atomares Schreiben, Löschen der Quelle erst nach vollständiger Übertragung,
|
||||||
|
Kopier-Buchführung, Quarantäne. Verbindungsabbrüche brechen den Lauf als gestörte
|
||||||
|
Anbindung ab. Bei SFTP-Quellen mindestens 2 s Stabilitätsfenster (Zeiten nur
|
||||||
|
sekundengenau).
|
||||||
|
- **Gespeicherte Zugangsdaten** für SMB und SFTP-Passwörter (Benutzer, Passwort,
|
||||||
|
Domäne); in Anbindungen auswählbar, ein neues Passwort gilt sofort für alle.
|
||||||
|
Die Seite „Schlüssel“ heißt jetzt **„Zugangsdaten“** (Passwörter + SSH-Schlüssel);
|
||||||
|
`/keys` leitet weiter.
|
||||||
|
- **Anmeldung abschaltbar** (*Einstellungen → Anmeldung*, Bestätigung mit Passwort):
|
||||||
|
immer / nicht aus vertrauenswürdigen Netzen / nie (z. B. hinter Reverse Proxy mit
|
||||||
|
eigener Anmeldung). Deutlicher Hinweis in der Kopfzeile; CSRF-Schutz bleibt;
|
||||||
|
`netmover reset-password` schaltet die Anmeldung wieder ein.
|
||||||
|
|
||||||
|
### Geändert
|
||||||
|
- Mover intern auf eine Quell-Schnittstelle umgestellt (`LocalSource`, `SftpSource`);
|
||||||
|
bestehendes Verhalten unverändert (alle bisherigen Tests gelten weiter).
|
||||||
|
|
||||||
## [0.4.0] – 2026-10-11
|
## [0.4.0] – 2026-10-11
|
||||||
|
|
||||||
### Hinzugefügt
|
### Hinzugefügt
|
||||||
@@ -213,7 +277,12 @@ Erste Version.
|
|||||||
- Löschen von Zugangsdaten, die noch von einem Mount benutzt werden, trennt den
|
- Löschen von Zugangsdaten, die noch von einem Mount benutzt werden, trennt den
|
||||||
Mount nicht mehr stillschweigend ab, sondern wird von der Datenbank abgelehnt.
|
Mount nicht mehr stillschweigend ab, sondern wird von der Datenbank abgelehnt.
|
||||||
|
|
||||||
[Unreleased]: https://git.wilde-server.de/florian.wilde/netmover/compare/v0.4.0...HEAD
|
[Unreleased]: https://git.wilde-server.de/florian.wilde/netmover/compare/v0.9.0...HEAD
|
||||||
|
[0.9.0]: https://git.wilde-server.de/florian.wilde/netmover/compare/v0.8.0...v0.9.0
|
||||||
|
[0.8.0]: https://git.wilde-server.de/florian.wilde/netmover/compare/v0.7.0...v0.8.0
|
||||||
|
[0.7.0]: https://git.wilde-server.de/florian.wilde/netmover/compare/v0.6.0...v0.7.0
|
||||||
|
[0.6.0]: https://git.wilde-server.de/florian.wilde/netmover/compare/v0.5.0...v0.6.0
|
||||||
|
[0.5.0]: https://git.wilde-server.de/florian.wilde/netmover/compare/v0.4.0...v0.5.0
|
||||||
[0.4.0]: https://git.wilde-server.de/florian.wilde/netmover/compare/v0.3.0...v0.4.0
|
[0.4.0]: https://git.wilde-server.de/florian.wilde/netmover/compare/v0.3.0...v0.4.0
|
||||||
[0.3.0]: https://git.wilde-server.de/florian.wilde/netmover/compare/v0.2.1...v0.3.0
|
[0.3.0]: https://git.wilde-server.de/florian.wilde/netmover/compare/v0.2.1...v0.3.0
|
||||||
[0.2.1]: https://git.wilde-server.de/florian.wilde/netmover/compare/v0.2.0...v0.2.1
|
[0.2.1]: https://git.wilde-server.de/florian.wilde/netmover/compare/v0.2.0...v0.2.1
|
||||||
|
|||||||
@@ -18,10 +18,102 @@ Zeiten in Ortszeit (Europe/Berlin).
|
|||||||
| 2026-10-10 | 23:31 | 23:58 | 0:27 | – | Schlüsselverwaltung, Update-Skript, 0.2.x |
|
| 2026-10-10 | 23:31 | 23:58 | 0:27 | – | Schlüsselverwaltung, Update-Skript, 0.2.x |
|
||||||
| 2026-10-11 | 00:13 | 00:32 | 0:19 | – | Menü, Anleitung, Roadmap, Uptime Kuma (0.3.0) |
|
| 2026-10-11 | 00:13 | 00:32 | 0:19 | – | Menü, Anleitung, Roadmap, Uptime Kuma (0.3.0) |
|
||||||
| 2026-10-11 | 00:31 | 00:50 | 0:19 | – | Pausieren, Wartungsmodus, E-Mail (0.4.0) |
|
| 2026-10-11 | 00:31 | 00:50 | 0:19 | – | Pausieren, Wartungsmodus, E-Mail (0.4.0) |
|
||||||
| **Summe** | | | **2:55** | | |
|
| 2026-10-11 | 00:54 | 01:16 | 0:22 | – | SFTP-Quelle, Zugangsdaten, Anmeldung (0.5.0) |
|
||||||
|
| 2026-10-11 | 08:43 | 08:55 | 0:12 | – | Anmeldung je Netz (0.6.0) |
|
||||||
|
| 2026-10-12 | 00:14 | 00:34 | 0:20 | – | Archiv je Regel (0.7.0) |
|
||||||
|
| 2026-10-12 | 00:46 | 00:55 | 0:09 | – | Anbindungen kopieren (0.8.0) |
|
||||||
|
| 2026-10-15 | 09:17 | 09:26 | 0:09 | – | Regeln kopieren (0.9.0) |
|
||||||
|
| **Summe** | | | **4:07** | | |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## 2026-10-15 · 09:17–09:26 · Regeln kopieren (0.9.0)
|
||||||
|
|
||||||
|
**Erledigt** – Knopf „Kopieren“, Route `/rules/{id}/copy` (GET Formular, POST
|
||||||
|
speichern), `rule_copy_name`. 5 Tests (362 gesamt); Gegenprobe „Kopie aktiv“ schlägt
|
||||||
|
an (2 Tests).
|
||||||
|
|
||||||
|
**Entscheidung** – Kopie startet deaktiviert: Mit identischer Quelle würden Original
|
||||||
|
und Kopie sonst sofort parallel dieselben Dateien bearbeiten (doppelte Lieferung bzw.
|
||||||
|
Kollisionen). Protokoll und Buchführung (`copied_files`, `archived_files`) werden
|
||||||
|
bewusst nicht mitkopiert – die Kopie ist eine neue Regel.
|
||||||
|
|
||||||
|
## 2026-10-12 · 00:46–00:55 · Anbindungen kopieren (0.8.0)
|
||||||
|
|
||||||
|
**Erledigt** – Knopf „Kopieren“, Route `/mounts/{id}/copy` (GET Formular, POST
|
||||||
|
speichern), `save_mount(copy_from=…)`, Formular-Test mit Vorlage. 7 Tests (357 gesamt);
|
||||||
|
Gegenprobe ohne Übernahme der Zugangsdaten: 3 Tests schlagen an.
|
||||||
|
|
||||||
|
**Entscheidungen**
|
||||||
|
- Eigene Zugangsdaten werden als eigenes Exemplar kopiert (nicht geteilt) – eigene
|
||||||
|
Zugangsdaten verschwinden mit ihrer Anbindung, eine geteilte Nutzung wäre
|
||||||
|
dafür der Fall „Zugangsdaten“.
|
||||||
|
- Eigenes Schlüsselpaar bleibt gleich (schon in `authorized_keys`).
|
||||||
|
- Hostschlüssel nur bei gleichem Server/Port übernehmen, sonst würde der Test an
|
||||||
|
einem anderen Server mit „Hostschlüssel geändert“ scheitern.
|
||||||
|
|
||||||
|
## 2026-10-12 · 00:14–00:34 · Archiv je Regel (0.7.0)
|
||||||
|
|
||||||
|
**Erledigt**
|
||||||
|
- Archivfelder an der Regel, Tabelle `archived_files` (Migration 0004).
|
||||||
|
- Engine: Archiv zuerst, dann Ziel, dann Quelle löschen; Archiv immer „umbenennen“.
|
||||||
|
- Runner: Ziel- und Archivaufbau über eine gemeinsame Fabrik (`_target`).
|
||||||
|
- 14 Tests (350 gesamt); Mutationsprobe 4/4 erkannt (Buchführung, Archiv
|
||||||
|
übersprungen, Kollisionsregel, Platzprüfung).
|
||||||
|
|
||||||
|
**Entscheidungen**
|
||||||
|
- Archiv *vor* Ziel: Paperless holt Dateien sofort ab – andersherum würde ein
|
||||||
|
gescheitertes Archiv beim Neuversuch zur doppelten Lieferung führen.
|
||||||
|
- Archiv überschreibt nie (unabhängig von der Kollisionsregel des Ziels).
|
||||||
|
- Ablage „nach Datum“ nach Verarbeitungsdatum (nicht Datei-mtime) – eindeutig
|
||||||
|
nachvollziehbar, wann eine Datei durchgelaufen ist.
|
||||||
|
|
||||||
|
**Stolperstein**
|
||||||
|
- Downgrade von 0004 scheiterte an den CHECK-Regeln der neuen Auswahlfelder
|
||||||
|
(verweisen nach dem Entfernen der Spalten ins Leere) – nur dank der Migrations-
|
||||||
|
tests mit Daten aufgefallen; Downgrade entfernt sie jetzt zuerst.
|
||||||
|
|
||||||
|
## 2026-10-11 · 08:43–08:55 · Anmeldung je Netz (0.6.0)
|
||||||
|
|
||||||
|
**Anlass** – Wunsch: Anmeldung abschalten, und zwar konfigurierbar, aus welchen Netzen
|
||||||
|
mit und ohne Passwort. Zugriffe kamen aus 10.0.10.0/24 und 10.1.5.0/24 (Access-Log).
|
||||||
|
|
||||||
|
**Erledigt**
|
||||||
|
- `auth.access_decision()` als einzige Entscheidung (ohne PW / mit PW / abgewiesen),
|
||||||
|
benutzt von Middleware (403 für fremde Netze), Guard und Anmeldeseite.
|
||||||
|
- Schutz vor Aussperren beim Speichern; `reset-password` hebt Sperren auf.
|
||||||
|
- 19 Tests für die Anmeldung (336 gesamt).
|
||||||
|
|
||||||
|
**Entscheidungen**
|
||||||
|
- `/healthz` und `/static` nie sperren – Uptime Kuma kann in einem anderen Netz stehen.
|
||||||
|
- Nicht-IP-Clients (z. B. Unix-Socket) gelten als „in keinem Netz“.
|
||||||
|
|
||||||
|
## 2026-10-11 · 00:54–01:16 · SFTP-Quelle, Zugangsdaten, Anmeldung abschaltbar (0.5.0)
|
||||||
|
|
||||||
|
**Erledigt**
|
||||||
|
- `mover/sources.py`: `Source` mit `LocalSource`/`SftpSource`; Engine greift nur noch
|
||||||
|
darüber auf die Quelle zu (Scan, frischer Zustand, Lesen, Löschen, Quarantäne).
|
||||||
|
Ziele bekommen statt `Path` ein „Readable“ (Adapter) – Destinations fast unverändert.
|
||||||
|
- `credentials.py` (vorher `keys.py`): gespeicherte Passwörter; Seite „Zugangsdaten“.
|
||||||
|
- Anmeldung: Modus immer / vertraute Netze / nie, zentral im `guard`.
|
||||||
|
- 26 neue Tests (325 gesamt); alle bisherigen Mover-Tests bestanden den Umbau
|
||||||
|
unverändert (nur ein Monkeypatch-Ziel verschoben).
|
||||||
|
|
||||||
|
**Entscheidungen**
|
||||||
|
- SFTP-Quelle: Stabilität über Größe + sekundengenaue mtime, Mindestfenster 2 s;
|
||||||
|
Verbindungsabbruch → `ENOTCONN` → Lauf abgebrochen (Anbindung gestört).
|
||||||
|
- Gespeicherte Passwörter tragen den Benutzer selbst (anders als Schlüssel, deren
|
||||||
|
Benutzer zur Anbindung gehört).
|
||||||
|
- „Passworteingabe deaktivieren“ als Login-Modus umgesetzt, mit Bestätigung durch
|
||||||
|
das Passwort, Warnhinweis und Notausgang über `reset-password`. Ein Passwort bleibt
|
||||||
|
Pflicht (Ersteinrichtung), sonst ließe sich die Anmeldung nie wieder einschalten.
|
||||||
|
|
||||||
|
**Stolpersteine**
|
||||||
|
- paramiko `prefetch()` braucht `fstat`; der Test-Server konnte es nicht – jetzt
|
||||||
|
kann er es, und die Quelle übergibt die bekannte Größe (spart einen Umlauf).
|
||||||
|
- Zwei TestClients mit eigenem Lebenszyklus starteten den Scheduler doppelt →
|
||||||
|
zweiter Client ohne Lebenszyklus auf derselben App.
|
||||||
|
|
||||||
## 2026-10-11 · 00:31–00:50 · Pausieren, Wartungsmodus, E-Mail (0.4.0)
|
## 2026-10-11 · 00:31–00:50 · Pausieren, Wartungsmodus, E-Mail (0.4.0)
|
||||||
|
|
||||||
**Erledigt**
|
**Erledigt**
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
Selbst gehosteter Dienst mit Weboberfläche, der **SMB/CIFS- und NFS-Freigaben
|
Selbst gehosteter Dienst mit Weboberfläche, der **SMB/CIFS- und NFS-Freigaben
|
||||||
verwaltet** und **Dateien zeitgesteuert verschiebt** – „alle N Sekunden Ordner 1
|
verwaltet** und **Dateien zeitgesteuert verschiebt** – „alle N Sekunden Ordner 1
|
||||||
prüfen, passende Dateien nach Ordner 2 bringen“. Quelle und Ziel sind
|
prüfen, passende Dateien nach Ordner 2 bringen“. Quelle und Ziel sind
|
||||||
unabhängig voneinander per SMB, NFS, lokal oder (nur als Ziel) per SFTP angebunden.
|
unabhängig voneinander per SMB, NFS, SFTP oder (als Ziel) lokal angebunden.
|
||||||
|
|
||||||
Typischer Einsatz: Ein Synology-NAS nimmt Scans entgegen, NetMover schiebt sie alle
|
Typischer Einsatz: Ein Synology-NAS nimmt Scans entgegen, NetMover schiebt sie alle
|
||||||
30 Sekunden in den Eingangsordner von Paperless-ngx.
|
30 Sekunden in den Eingangsordner von Paperless-ngx.
|
||||||
@@ -176,7 +176,7 @@ Lege in NetMover die *Anbindung → Neue Anbindung* an:
|
|||||||
| Feld | Wert |
|
| Feld | Wert |
|
||||||
|---|---|
|
|---|---|
|
||||||
| Name | `Paperless` |
|
| Name | `Paperless` |
|
||||||
| Typ | SFTP (nur Ziel) |
|
| Typ | SFTP |
|
||||||
| Server / SSH-Port | IP des LXC / `22` |
|
| Server / SSH-Port | IP des LXC / `22` |
|
||||||
| Freigabe / Pfad | `/opt/paperless_data/consume` |
|
| Freigabe / Pfad | `/opt/paperless_data/consume` |
|
||||||
| Anmeldeart | SSH-Schlüssel |
|
| Anmeldeart | SSH-Schlüssel |
|
||||||
|
|||||||
@@ -6,13 +6,6 @@ stehen mit Version im [CHANGELOG](CHANGELOG.md); Details zu Entscheidungen im
|
|||||||
|
|
||||||
Legende: ✅ erledigt · 🔨 in Arbeit · 📋 geplant · 💡 Idee (noch nicht entschieden)
|
Legende: ✅ erledigt · 🔨 in Arbeit · 📋 geplant · 💡 Idee (noch nicht entschieden)
|
||||||
|
|
||||||
## In Arbeit (0.5.0)
|
|
||||||
|
|
||||||
| Status | Thema | Warum |
|
|
||||||
|---|---|---|
|
|
||||||
| 🔨 | **SFTP als Quelle**: Scan, Stabilitätsprüfung und Löschen der Quelle über SFTP, mit denselben Garantien wie bei SMB/NFS | Quellen, die nur per SSH erreichbar sind |
|
|
||||||
| 🔨 | **Gespeicherte Zugangsdaten** für SMB und SFTP-Passwörter, zusammen mit den SSH-Schlüsseln auf einer Seite „Zugangsdaten“ | Mehrere Freigaben mit demselben NAS-Benutzer an einer Stelle pflegen |
|
|
||||||
|
|
||||||
## Geplant
|
## Geplant
|
||||||
|
|
||||||
| Status | Thema | Warum |
|
| Status | Thema | Warum |
|
||||||
@@ -35,6 +28,11 @@ Legende: ✅ erledigt · 🔨 in Arbeit · 📋 geplant · 💡 Idee (noch nicht
|
|||||||
|
|
||||||
| Version | Thema |
|
| Version | Thema |
|
||||||
|---|---|
|
|---|---|
|
||||||
|
| 0.9.0 | Regeln kopieren (Kopie zunächst deaktiviert) |
|
||||||
|
| 0.8.0 | Anbindungen kopieren inklusive Zugangsdaten |
|
||||||
|
| 0.7.0 | Archiv je Regel: Kopie ins Ziel und ins Archiv, Quelle erst danach löschen |
|
||||||
|
| 0.6.0 | Anmeldung je Netz: ohne Passwort / mit Passwort / abgewiesen, Schutz vor Aussperren |
|
||||||
|
| 0.5.0 | SFTP als Quelle; gespeicherte Zugangsdaten für SMB/SFTP-Passwörter (Seite „Zugangsdaten“); Anmeldung abschaltbar (vertrauenswürdige Netze / ganz) |
|
||||||
| 0.4.0 | Automatisch pausieren/fortsetzen bei nicht erreichbaren Anbindungen; Wartungsmodus; E-Mail-Benachrichtigung; eine Meldung pro ausgefallener Anbindung |
|
| 0.4.0 | Automatisch pausieren/fortsetzen bei nicht erreichbaren Anbindungen; Wartungsmodus; E-Mail-Benachrichtigung; eine Meldung pro ausgefallener Anbindung |
|
||||||
| 0.3.0 | Neues Menü (gruppierte Seitenleiste, Statusampel, Handy-Menü); Anleitung im Repo und als Hilfe-Seite; Roadmap; Uptime Kuma (Einzel-Endpunkte, Push-Monitor) |
|
| 0.3.0 | Neues Menü (gruppierte Seitenleiste, Statusampel, Handy-Menü); Anleitung im Repo und als Hilfe-Seite; Roadmap; Uptime Kuma (Einzel-Endpunkte, Push-Monitor) |
|
||||||
| 0.2.1 | Migrationen in einer Transaktion und ohne Fremdschlüsselfehler auf befüllten Datenbanken |
|
| 0.2.1 | Migrationen in einer Transaktion und ohne Fremdschlüsselfehler auf befüllten Datenbanken |
|
||||||
|
|||||||
@@ -4,7 +4,7 @@
|
|||||||
services:
|
services:
|
||||||
netmover:
|
netmover:
|
||||||
build: .
|
build: .
|
||||||
image: netmover:0.4.0
|
image: netmover:0.9.0
|
||||||
container_name: netmover
|
container_name: netmover
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
cap_add:
|
cap_add:
|
||||||
|
|||||||
@@ -8,21 +8,21 @@ Diese Anleitung erklärt die Bedienung. Installation, Updates und technische Det
|
|||||||
stehen im [README](../README.md).
|
stehen im [README](../README.md).
|
||||||
|
|
||||||
**Inhalt:** [Begriffe](#begriffe) · [Erste Anmeldung](#erste-anmeldung) ·
|
**Inhalt:** [Begriffe](#begriffe) · [Erste Anmeldung](#erste-anmeldung) ·
|
||||||
[Anbindungen](#anbindungen) · [Schlüssel](#schluessel) · [Regeln](#regeln) ·
|
[Zugangsdaten](#zugangsdaten) · [Anbindungen](#anbindungen) · [Regeln](#regeln) · [Archiv](#archiv) ·
|
||||||
[Probelauf](#probelauf) · [Status](#status) · [Wartungsmodus](#wartungsmodus) · [Protokoll](#protokoll) ·
|
[Probelauf](#probelauf) · [Status](#status) · [Wartungsmodus](#wartungsmodus) · [Protokoll](#protokoll) ·
|
||||||
[Einstellungen](#einstellungen) · [Uptime Kuma](#uptime-kuma) · [Beispiele](#beispiele) · [Häufige Fragen](#fragen)
|
[Einstellungen](#einstellungen) · [Anmeldung](#anmeldung) · [Uptime Kuma](#uptime-kuma) · [Beispiele](#beispiele) · [Häufige Fragen](#fragen)
|
||||||
|
|
||||||
## Die vier Begriffe {#begriffe}
|
## Die vier Begriffe {#begriffe}
|
||||||
|
|
||||||
| Begriff | Was es ist | Beispiel |
|
| Begriff | Was es ist | Beispiel |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| **Anbindung** | Ein Ort, an den NetMover herankommt: eine SMB-Freigabe, ein NFS-Export oder ein Server per SFTP | „NAS“ = `//10.0.10.6/Scans` |
|
| **Anbindung** | Ein Ort, an den NetMover herankommt: eine SMB-Freigabe, ein NFS-Export oder ein Server per SFTP | „NAS“ = `//10.0.10.6/Scans` |
|
||||||
| **Schlüssel** | Ein gespeicherter SSH-Schlüssel für SFTP-Anbindungen | „paperless-lxc“ |
|
| **Zugangsdaten** | Gespeicherter Benutzer mit Passwort oder SSH-Schlüssel, in mehreren Anbindungen nutzbar | „NAS-Scanner“, „paperless-lxc“ |
|
||||||
| **Regel** | „Alle N Sekunden aus Anbindung A nach Anbindung B“ – mit Filter, Sicherheitsprüfungen und Verhalten bei gleichen Namen | „Scans → Paperless“ |
|
| **Regel** | „Alle N Sekunden aus Anbindung A nach Anbindung B“ – mit Filter, Sicherheitsprüfungen und Verhalten bei gleichen Namen | „Scans → Paperless“ |
|
||||||
| **Lauf** | Eine Ausführung einer Regel; steht mit allen Dateien im Protokoll | 23:41:07, 3 Dateien verschoben |
|
| **Lauf** | Eine Ausführung einer Regel; steht mit allen Dateien im Protokoll | 23:41:07, 3 Dateien verschoben |
|
||||||
|
|
||||||
Die Reihenfolge beim Einrichten ist immer dieselbe – und so ist auch das Menü
|
Die Reihenfolge beim Einrichten ist immer dieselbe – und so ist auch das Menü
|
||||||
aufgebaut: **Anbindungen → (Schlüssel) → Regeln → Status und Protokoll beobachten.**
|
aufgebaut: **(Zugangsdaten) → Anbindungen → Regeln → Status und Protokoll beobachten.**
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
@@ -50,11 +50,13 @@ Passwort festlegt. Passwort ändern: *System → Einstellungen*.
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| **SMB/CIFS** | Windows-Freigaben, Synology, QNAP, … | Quelle und Ziel |
|
| **SMB/CIFS** | Windows-Freigaben, Synology, QNAP, … | Quelle und Ziel |
|
||||||
| **NFS 4 / NFS 3** | Linux-Server, NAS mit NFS | Quelle und Ziel |
|
| **NFS 4 / NFS 3** | Linux-Server, NAS mit NFS | Quelle und Ziel |
|
||||||
| **SFTP** | Server, die nur per SSH erreichbar sind (z. B. ein LXC mit Paperless) | nur Ziel |
|
| **SFTP** | Server, die nur per SSH erreichbar sind (z. B. ein LXC mit Paperless) | Quelle und Ziel |
|
||||||
|
|
||||||
SMB- und NFS-Anbindungen werden **eingehängt** und erscheinen unter
|
SMB- und NFS-Anbindungen werden **eingehängt** und erscheinen unter
|
||||||
`/mnt/netmover/<name>`. SFTP wird nicht eingehängt, NetMover verbindet sich bei
|
`/mnt/netmover/<name>`. SFTP wird nicht eingehängt, NetMover verbindet sich bei
|
||||||
jedem Lauf.
|
jedem Lauf. Als **Quelle** gelten bei SFTP dieselben Sicherheitsprüfungen; weil
|
||||||
|
SFTP-Server Änderungszeiten nur sekundengenau melden, sollte das
|
||||||
|
Stabilitätsfenster dort mindestens 2 Sekunden betragen.
|
||||||
|
|
||||||
### Felder {#anbindung-felder}
|
### Felder {#anbindung-felder}
|
||||||
|
|
||||||
@@ -64,12 +66,28 @@ jedem Lauf.
|
|||||||
| **Server** | IP-Adresse oder Hostname, z. B. `10.0.10.6` |
|
| **Server** | IP-Adresse oder Hostname, z. B. `10.0.10.6` |
|
||||||
| **Freigabe / Pfad** | SMB: Freigabename, optional mit Unterordner (`Scans` oder `Scans/privat`). NFS: Exportpfad (`/volume1/scans`). SFTP: absoluter Ordner auf dem Server (`/opt/paperless_data/consume`). |
|
| **Freigabe / Pfad** | SMB: Freigabename, optional mit Unterordner (`Scans` oder `Scans/privat`). NFS: Exportpfad (`/volume1/scans`). SFTP: absoluter Ordner auf dem Server (`/opt/paperless_data/consume`). |
|
||||||
| **Benutzername / Passwort** | Bei SMB leer lassen für Gastzugang. Beim Bearbeiten heißt ein leeres Passwortfeld „unverändert“ – Passwörter werden nie angezeigt. |
|
| **Benutzername / Passwort** | Bei SMB leer lassen für Gastzugang. Beim Bearbeiten heißt ein leeres Passwortfeld „unverändert“ – Passwörter werden nie angezeigt. |
|
||||||
| **Anmeldeart, Schlüssel** (SFTP) | SSH-Schlüssel (gespeichert oder nur für diese Anbindung) oder Passwort, siehe [Schlüssel](#schluessel). |
|
| **Zugangsdaten / Anmeldeart** | Gespeicherte Zugangsdaten auswählen oder eigene Angaben nur für diese Anbindung; bei SFTP zusätzlich SSH-Schlüssel oder Passwort, siehe [Zugangsdaten](#zugangsdaten). |
|
||||||
| **uid, gid, file_mode, dir_mode** (SMB) | Wem die Dateien im eingehängten Ordner gehören. Meist leer lassen. |
|
| **uid, gid, file_mode, dir_mode** (SMB) | Wem die Dateien im eingehängten Ordner gehören. Meist leer lassen. |
|
||||||
| **Zusatzoptionen** | Für Sonderfälle, z. B. `vers=3.0` oder `nohandlecache`. Nur erlaubte Optionen; Zugangsdaten setzt NetMover selbst. |
|
| **Zusatzoptionen** | Für Sonderfälle, z. B. `vers=3.0` oder `nohandlecache`. Nur erlaubte Optionen; Zugangsdaten setzt NetMover selbst. |
|
||||||
| **Beim Start automatisch einhängen** | Nach einem Neustart des Servers wieder einhängen. Fällt das NAS aus, versucht NetMover es außerdem jede Minute erneut. |
|
| **Beim Start automatisch einhängen** | Nach einem Neustart des Servers wieder einhängen. Fällt das NAS aus, versucht NetMover es außerdem jede Minute erneut. |
|
||||||
| **Aktiv** | Deaktivierte Anbindungen werden von Regeln nicht benutzt. |
|
| **Aktiv** | Deaktivierte Anbindungen werden von Regeln nicht benutzt. |
|
||||||
|
|
||||||
|
### Kopieren {#anbindung-kopieren}
|
||||||
|
|
||||||
|
Für eine ähnliche Anbindung – etwa eine zweite Freigabe auf demselben NAS – bei der
|
||||||
|
Vorlage auf **Kopieren** klicken. Das Formular ist mit allen Werten vorausgefüllt
|
||||||
|
(Name „… (Kopie)“); meist muss nur die Freigabe geändert werden.
|
||||||
|
|
||||||
|
- **Zugangsdaten werden übernommen**, solange das Passwortfeld leer bleibt. Die Kopie
|
||||||
|
bekommt ein eigenes Exemplar – das Löschen der Vorlage betrifft sie nicht.
|
||||||
|
Gespeicherte Zugangsdaten sind einfach vorausgewählt.
|
||||||
|
- Ein **eigener SSH-Schlüssel** der Vorlage wird weiter benutzt; ein neuer Eintrag in
|
||||||
|
`authorized_keys` ist nicht nötig.
|
||||||
|
- **Testen** funktioniert im Kopie-Formular bereits mit den übernommenen Zugangsdaten.
|
||||||
|
|
||||||
|
Tipp: Nutzen mehrere Anbindungen dieselben Zugangsdaten, lohnt sich ein Eintrag unter
|
||||||
|
[Zugangsdaten](#zugangsdaten) – dann gilt ein neues Passwort sofort überall.
|
||||||
|
|
||||||
### Testen {#anbindung-testen}
|
### Testen {#anbindung-testen}
|
||||||
|
|
||||||
**Testen** prüft drei Dinge einzeln und sagt im Klartext, was nicht klappt:
|
**Testen** prüft drei Dinge einzeln und sagt im Klartext, was nicht klappt:
|
||||||
@@ -102,14 +120,28 @@ Es wird nie in ein leeres Verzeichnis geschrieben, nur weil die Freigabe fehlt.
|
|||||||
Solche Regeln **warten** automatisch, bis die Anbindung zurück ist (siehe
|
Solche Regeln **warten** automatisch, bis die Anbindung zurück ist (siehe
|
||||||
[Status](#status)), und laufen dann von selbst weiter.
|
[Status](#status)), und laufen dann von selbst weiter.
|
||||||
|
|
||||||
## Schlüssel {#schluessel}
|
## Zugangsdaten {#zugangsdaten}
|
||||||
|
|
||||||
*Verbindungen → Schlüssel*
|
*Verbindungen → Zugangsdaten*
|
||||||
|
|
||||||
Für SFTP-Ziele ist ein SSH-Schlüssel sicherer als ein Passwort. Einen Schlüssel kann
|
Benutzer und Passwörter für Freigaben sowie SSH-Schlüssel einmal hinterlegen und in
|
||||||
man für mehrere Anbindungen verwenden; den Benutzernamen legt jede Anbindung fest.
|
beliebig vielen Anbindungen auswählen. Passwörter und private Schlüssel liegen
|
||||||
|
verschlüsselt in der Datenbank und werden nie angezeigt.
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
|
### Passwörter {#zugangsdaten-passwoerter}
|
||||||
|
|
||||||
|
Für SMB-Freigaben und SFTP-Server mit Passwort: Name, Benutzer, Passwort und
|
||||||
|
optional Domäne. In der Anbindung unter *Zugangsdaten* auswählen statt „eigene
|
||||||
|
Angaben“. **Ändern** setzt Benutzer, Domäne oder ein neues Passwort – das gilt sofort
|
||||||
|
für alle Anbindungen, die den Eintrag nutzen (praktisch, wenn das NAS-Passwort
|
||||||
|
wechselt).
|
||||||
|
|
||||||
|
### SSH-Schlüssel {#schluessel}
|
||||||
|
|
||||||
|
Für SFTP ist ein Schlüssel sicherer als ein Passwort. Einen Schlüssel kann man für
|
||||||
|
mehrere Anbindungen verwenden; den Benutzernamen legt jede Anbindung fest.
|
||||||
|
|
||||||
**Neuen Schlüssel erzeugen** – Namen eingeben, *Ed25519-Schlüssel erzeugen*. Dann:
|
**Neuen Schlüssel erzeugen** – Namen eingeben, *Ed25519-Schlüssel erzeugen*. Dann:
|
||||||
|
|
||||||
@@ -117,17 +149,15 @@ man für mehrere Anbindungen verwenden; den Benutzernamen legt jede Anbindung fe
|
|||||||
**authorized_keys-Zeile** kopieren. Sie beginnt mit
|
**authorized_keys-Zeile** kopieren. Sie beginnt mit
|
||||||
`restrict,command="/usr/lib/openssh/sftp-server"` – damit darf der Schlüssel
|
`restrict,command="/usr/lib/openssh/sftp-server"` – damit darf der Schlüssel
|
||||||
**nur SFTP**, keine Shell.
|
**nur SFTP**, keine Shell.
|
||||||
2. Auf dem Zielserver an die Datei `~/.ssh/authorized_keys` des Benutzers anhängen
|
2. Auf dem Server an die Datei `~/.ssh/authorized_keys` des Benutzers anhängen
|
||||||
(Ordner `~/.ssh` mit Rechten `700`, Datei `600`, Eigentümer der Benutzer).
|
(Ordner `~/.ssh` mit Rechten `700`, Datei `600`, Eigentümer der Benutzer).
|
||||||
3. In der Anbindung *Anmeldeart: SSH-Schlüssel* und den Schlüssel wählen, **Testen**.
|
3. In der Anbindung *Anmeldeart: SSH-Schlüssel* und den Schlüssel wählen, **Testen**.
|
||||||
|
|
||||||
**Vorhandenen Schlüssel importieren** – privaten Schlüssel einfügen (OpenSSH- oder
|
**Vorhandenen Schlüssel importieren** – privaten Schlüssel einfügen (OpenSSH- oder
|
||||||
PEM-Format; Ed25519, ECDSA oder RSA ab 2048 Bit). Hat er eine Passphrase, diese
|
PEM-Format; Ed25519, ECDSA oder RSA ab 2048 Bit). Hat er eine Passphrase, diese
|
||||||
angeben: Sie wird nur zum Entschlüsseln benutzt, der Schlüssel liegt danach
|
angeben: Sie wird nur zum Entschlüsseln benutzt.
|
||||||
verschlüsselt in NetMovers Datenbank.
|
|
||||||
|
|
||||||
Private Schlüssel zeigt NetMover nie an. Ein Schlüssel lässt sich nur löschen, wenn
|
Gespeicherte Zugangsdaten lassen sich nur löschen, wenn sie keine Anbindung mehr nutzt.
|
||||||
ihn keine Anbindung mehr nutzt.
|
|
||||||
|
|
||||||
## Regeln {#regeln}
|
## Regeln {#regeln}
|
||||||
|
|
||||||
@@ -146,6 +176,34 @@ ihn keine Anbindung mehr nutzt.
|
|||||||
| **Aktion** | **verschieben** (Quelle wird nach erfolgreicher Übertragung gelöscht) oder **kopieren** (Quelle bleibt; jede Dateiversion wird nur einmal kopiert). |
|
| **Aktion** | **verschieben** (Quelle wird nach erfolgreicher Übertragung gelöscht) oder **kopieren** (Quelle bleibt; jede Dateiversion wird nur einmal kopiert). |
|
||||||
| **Wenn Datei existiert** | **umbenennen** (`scan.pdf` → `scan-1.pdf`, `scan-2.pdf` …), **überspringen** oder **überschreiben**. |
|
| **Wenn Datei existiert** | **umbenennen** (`scan.pdf` → `scan-1.pdf`, `scan-2.pdf` …), **überspringen** oder **überschreiben**. |
|
||||||
|
|
||||||
|
### Archiv {#archiv}
|
||||||
|
|
||||||
|
Optional bekommt jede Datei zusätzlich eine **Kopie ins Archiv** – z. B. Paperless als
|
||||||
|
Ziel und ein Archivordner auf dem NAS. **Die Quelle wird erst gelöscht, wenn die Datei
|
||||||
|
im Ziel UND im Archiv angekommen ist.**
|
||||||
|
|
||||||
|
| Feld | Bedeutung |
|
||||||
|
|---|---|
|
||||||
|
| **Archiv** | kein Archiv / eine Anbindung (SMB, NFS, SFTP) / ein lokaler Pfad |
|
||||||
|
| **Archivordner** | Anbindung: relativer Unterordner (wird angelegt). Lokal: absoluter Pfad, muss existieren. |
|
||||||
|
| **Ablage** | **wie in der Quelle** oder **nach Datum** – `2026/10/scan001.pdf` (Monat der Verarbeitung) |
|
||||||
|
|
||||||
|
So läuft es pro Datei:
|
||||||
|
|
||||||
|
1. **Zuerst ins Archiv** – mit denselben Sicherungen wie beim Ziel.
|
||||||
|
2. **Dann ins Ziel.**
|
||||||
|
3. **Dann erst** wird die Quelle gelöscht (bei „verschieben“).
|
||||||
|
|
||||||
|
Das Archiv kommt zuerst, weil ein Ziel wie Paperless Dateien sofort abholt: Scheitert
|
||||||
|
das Archiv, wurde noch nichts geliefert, und die Datei bleibt in der Quelle. Scheitert
|
||||||
|
danach das Ziel, merkt sich NetMover, dass die Datei schon archiviert ist – der nächste
|
||||||
|
Versuch legt keine zweite Archivkopie an.
|
||||||
|
|
||||||
|
Im Archiv wird **nie überschrieben**: Gleiche Namen werden immer umbenannt
|
||||||
|
(`scan-1.pdf`), egal was die Regel fürs Ziel vorsieht. Ist das Archiv nicht erreichbar,
|
||||||
|
wartet die Regel wie bei Quelle und Ziel. Im Kopiermodus bekommen Ziel und Archiv je
|
||||||
|
eine Kopie, die Quelle bleibt.
|
||||||
|
|
||||||
### Sicherheit und Grenzen {#regel-sicherheit}
|
### Sicherheit und Grenzen {#regel-sicherheit}
|
||||||
|
|
||||||
| Feld | Bedeutung | Empfehlung |
|
| Feld | Bedeutung | Empfehlung |
|
||||||
@@ -168,6 +226,17 @@ Synology-Verwaltungsordner (`@eaDir`, `#recycle`).
|
|||||||
| **Modus setzen** | z. B. `0644`; leer = wie die Quelldatei |
|
| **Modus setzen** | z. B. `0644`; leer = wie die Quelldatei |
|
||||||
| **Quarantäneordner** | Dateien, die mehrmals hintereinander scheitern, werden dorthin verschoben (relativ zur Quell-Anbindung), statt es endlos zu versuchen. Leer = keine Quarantäne. |
|
| **Quarantäneordner** | Dateien, die mehrmals hintereinander scheitern, werden dorthin verschoben (relativ zur Quell-Anbindung), statt es endlos zu versuchen. Leer = keine Quarantäne. |
|
||||||
|
|
||||||
|
### Regel kopieren {#regel-kopieren}
|
||||||
|
|
||||||
|
Für eine ähnliche Regel – etwa für einen zweiten Quell-Unterordner – bei der Vorlage
|
||||||
|
auf **Kopieren** klicken. Alle Einstellungen (Filter, Ziel, Archiv, Sicherheit,
|
||||||
|
Rechte) sind übernommen, der Name lautet „… (Kopie)“.
|
||||||
|
|
||||||
|
Die Kopie ist zunächst **deaktiviert**: Mit gleicher Quelle würde sie sonst sofort mit
|
||||||
|
dem Original um dieselben Dateien konkurrieren. Also Quelle bzw. Ziel anpassen,
|
||||||
|
speichern, **Probelauf**, dann **Aktivieren**. Protokoll und Buchführung beginnen bei
|
||||||
|
der Kopie neu.
|
||||||
|
|
||||||
### Probelauf {#probelauf}
|
### Probelauf {#probelauf}
|
||||||
|
|
||||||
**Probelauf** zeigt in einer Tabelle, welche Datei wohin ginge – **ohne etwas zu
|
**Probelauf** zeigt in einer Tabelle, welche Datei wohin ginge – **ohne etwas zu
|
||||||
@@ -317,6 +386,36 @@ Als Problem gilt: Scheduler steht, eine aktive Anbindung ist im Fehlerzustand, e
|
|||||||
aktive Regel hatte länger als das Dreifache ihres Intervalls keinen Erfolg oder ist
|
aktive Regel hatte länger als das Dreifache ihres Intervalls keinen Erfolg oder ist
|
||||||
dreimal hintereinander gescheitert.
|
dreimal hintereinander gescheitert.
|
||||||
|
|
||||||
|
### Anmeldung {#anmeldung}
|
||||||
|
|
||||||
|
Unter *Einstellungen → Anmeldung* legst du fest, **aus welchen Netzen** es mit und
|
||||||
|
ohne Passwort geht (Bestätigung mit dem aktuellen Passwort):
|
||||||
|
|
||||||
|
| Feld | Wirkung |
|
||||||
|
|---|---|
|
||||||
|
| **Ohne Passwort aus diesen Netzen** | z. B. `10.0.10.0/24, 10.1.5.0/24` – von dort keine Anmeldung. Leer = niemand. `0.0.0.0/0, ::/0` = Anmeldung ganz aus. |
|
||||||
|
| **Mit Passwort aus diesen Netzen** | Leer = **alle übrigen** Netze mit Passwort. Ausgefüllt = nur von dort mit Passwort, **alle anderen Netze werden abgewiesen** (auch die Anmeldeseite). |
|
||||||
|
|
||||||
|
Beispiel: Ohne Passwort aus dem Heimnetz `10.0.10.0/24`, mit Passwort aus dem
|
||||||
|
VPN-Netz `10.8.0.0/24`, alles andere gesperrt.
|
||||||
|
|
||||||
|
- Einzelne Adressen gehen auch (`192.168.1.20`).
|
||||||
|
- Die Seite zeigt deine aktuelle Adresse. Eine Einstellung, mit der du dich selbst
|
||||||
|
aussperren würdest, wird nicht gespeichert.
|
||||||
|
- `/healthz` bleibt für die Überwachung von überall erreichbar.
|
||||||
|
- Solange du ohne Passwort arbeitest, steht oben gelb „ohne Anmeldung“. Der Schutz
|
||||||
|
gegen untergeschobene Formulare bleibt immer aktiv.
|
||||||
|
|
||||||
|
> NetMover läuft als root und kennt die Zugangsdaten deiner Freigaben. Wer die
|
||||||
|
> Oberfläche ohne Passwort erreicht, kann alles ändern – Netze „ohne Passwort“ daher
|
||||||
|
> so eng wie möglich wählen. Läuft ein Reverse Proxy auf demselben Server, kommen
|
||||||
|
> alle Anfragen von `127.0.0.1`: dann `127.0.0.1` nicht eintragen und
|
||||||
|
> `NETMOVER_FORWARDED_ALLOW_IPS` setzen.
|
||||||
|
|
||||||
|
Ausgesperrt oder Passwort vergessen? Auf dem Server
|
||||||
|
`/opt/netmover/venv/bin/netmover reset-password` – setzt das Passwort zurück, schaltet
|
||||||
|
die Anmeldung überall wieder ein und hebt Netz-Sperren auf.
|
||||||
|
|
||||||
### Export und Import {#export-import}
|
### Export und Import {#export-import}
|
||||||
|
|
||||||
*Konfiguration exportieren* lädt Anbindungen, Regeln und Einstellungen als JSON
|
*Konfiguration exportieren* lädt Anbindungen, Regeln und Einstellungen als JSON
|
||||||
@@ -329,17 +428,20 @@ deaktiviert, Zugangsdaten müssen neu eingegeben werden.
|
|||||||
### Scanner → NAS → Paperless {#beispiel-paperless}
|
### Scanner → NAS → Paperless {#beispiel-paperless}
|
||||||
|
|
||||||
1. Anbindung **NAS** (SMB, Freigabe `Scans`), testen, einhängen.
|
1. Anbindung **NAS** (SMB, Freigabe `Scans`), testen, einhängen.
|
||||||
2. Schlüssel **paperless-lxc** erzeugen, Zeile in `authorized_keys` im Paperless-LXC
|
2. Unter *Zugangsdaten* den Schlüssel **paperless-lxc** erzeugen, Zeile in
|
||||||
eintragen.
|
`authorized_keys` im Paperless-LXC eintragen.
|
||||||
3. Anbindung **Paperless** (SFTP, Pfad `/opt/paperless_data/consume`, Schlüssel
|
3. Anbindung **Paperless** (SFTP, Pfad `/opt/paperless_data/consume`, Schlüssel
|
||||||
wählen, Benutzer = der Paperless-Benutzer), testen.
|
wählen, Benutzer = der Paperless-Benutzer), testen.
|
||||||
4. Regel **Scans → Paperless**: Quelle NAS, Filter `*.pdf`, Ziel Paperless,
|
4. Regel **Scans → Paperless**: Quelle NAS, Filter `*.pdf`, Ziel Paperless,
|
||||||
verschieben, umbenennen, 30 s. **Probelauf**, dann aktivieren.
|
verschieben, umbenennen, 30 s. Optional **Archiv**: Anbindung NAS, Ordner
|
||||||
|
`Archiv/Scans`, Ablage nach Datum. **Probelauf**, dann aktivieren.
|
||||||
|
|
||||||
### Jede Person in einen eigenen Ordner {#beispiel-unterordner}
|
### Jede Person in einen eigenen Ordner {#beispiel-unterordner}
|
||||||
|
|
||||||
Scannt jede Person in einen eigenen Unterordner (`Scans/max`, `Scans/erika`): pro
|
Scannt jede Person in einen eigenen Unterordner (`Scans/max`, `Scans/erika`): pro
|
||||||
Person eine Regel mit eigenem **Unterordner** in der Quelle und eigenem Ziel.
|
Person eine Regel mit eigenem **Unterordner** in der Quelle und eigenem Ziel. Am
|
||||||
|
schnellsten: die erste Regel einrichten, dann **Kopieren** und nur Unterordner und
|
||||||
|
Ziel ändern.
|
||||||
|
|
||||||
### Kopie als zusätzliche Sicherung {#beispiel-kopie}
|
### Kopie als zusätzliche Sicherung {#beispiel-kopie}
|
||||||
|
|
||||||
|
|||||||
|
Before Width: | Height: | Size: 33 KiB After Width: | Height: | Size: 33 KiB |
|
Before Width: | Height: | Size: 36 KiB After Width: | Height: | Size: 36 KiB |
|
Before Width: | Height: | Size: 36 KiB |
|
Before Width: | Height: | Size: 45 KiB After Width: | Height: | Size: 46 KiB |
|
After Width: | Height: | Size: 41 KiB |
@@ -1,3 +1,3 @@
|
|||||||
"""NetMover – NFS/SMB-Mounts verwalten und Dateien zeitgesteuert verschieben."""
|
"""NetMover – NFS/SMB-Mounts verwalten und Dateien zeitgesteuert verschieben."""
|
||||||
|
|
||||||
__version__ = "0.4.0"
|
__version__ = "0.9.0"
|
||||||
|
|||||||
@@ -18,6 +18,8 @@ from sqlalchemy.orm import Session
|
|||||||
|
|
||||||
from netmover.crypto import SecretBox, generate_ssh_keypair
|
from netmover.crypto import SecretBox, generate_ssh_keypair
|
||||||
from netmover.models import (
|
from netmover.models import (
|
||||||
|
ArchiveKind,
|
||||||
|
ArchiveLayout,
|
||||||
CollisionPolicy,
|
CollisionPolicy,
|
||||||
Credential,
|
Credential,
|
||||||
CredentialKind,
|
CredentialKind,
|
||||||
@@ -85,6 +87,7 @@ def _int(
|
|||||||
|
|
||||||
|
|
||||||
NEW_KEY = "new"
|
NEW_KEY = "new"
|
||||||
|
OWN = "own"
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
@@ -101,6 +104,8 @@ class MountInput:
|
|||||||
# nur sftp + Schlüssel: "new" = eigener Schlüssel nur für diese Anbindung,
|
# nur sftp + Schlüssel: "new" = eigener Schlüssel nur für diese Anbindung,
|
||||||
# sonst ID eines Schlüssels aus der Schlüsselverwaltung
|
# sonst ID eines Schlüssels aus der Schlüsselverwaltung
|
||||||
key_id: str = NEW_KEY
|
key_id: str = NEW_KEY
|
||||||
|
# Passwort-Anmeldung (SMB, SFTP): "own" = eigene Angaben, sonst ID gespeicherter Zugangsdaten
|
||||||
|
cred_id: str = OWN
|
||||||
uid: int | None = None
|
uid: int | None = None
|
||||||
gid: int | None = None
|
gid: int | None = None
|
||||||
file_mode: str | None = None
|
file_mode: str | None = None
|
||||||
@@ -135,7 +140,12 @@ def mount_input_from_model(m: Mount) -> MountInput:
|
|||||||
username=m.username or (c.username if c else ""),
|
username=m.username or (c.username if c else ""),
|
||||||
domain=(c.domain or "") if c else "",
|
domain=(c.domain or "") if c else "",
|
||||||
auth=c.kind if c else CredentialKind.PASSWORD,
|
auth=c.kind if c else CredentialKind.PASSWORD,
|
||||||
key_id=str(c.id) if c is not None and c.managed else NEW_KEY,
|
key_id=str(c.id)
|
||||||
|
if c is not None and c.managed and c.kind is CredentialKind.SSH_KEY
|
||||||
|
else NEW_KEY,
|
||||||
|
cred_id=str(c.id)
|
||||||
|
if c is not None and c.managed and c.kind is CredentialKind.PASSWORD
|
||||||
|
else OWN,
|
||||||
uid=m.uid,
|
uid=m.uid,
|
||||||
gid=m.gid,
|
gid=m.gid,
|
||||||
file_mode=m.file_mode,
|
file_mode=m.file_mode,
|
||||||
@@ -159,10 +169,14 @@ def parse_mount_form(form: Mapping[str, str]) -> tuple[MountInput, Errors]:
|
|||||||
auto_mount=_bool(form, "auto_mount"),
|
auto_mount=_bool(form, "auto_mount"),
|
||||||
enabled=_bool(form, "enabled"),
|
enabled=_bool(form, "enabled"),
|
||||||
key_id=(form.get("key_id") or NEW_KEY).strip(),
|
key_id=(form.get("key_id") or NEW_KEY).strip(),
|
||||||
|
cred_id=(form.get("cred_id") or OWN).strip(),
|
||||||
)
|
)
|
||||||
if data.key_id != NEW_KEY and not data.key_id.isdigit():
|
if data.key_id != NEW_KEY and not data.key_id.isdigit():
|
||||||
errors["key_id"] = "Ungültige Schlüsselauswahl."
|
errors["key_id"] = "Ungültige Schlüsselauswahl."
|
||||||
data.key_id = NEW_KEY
|
data.key_id = NEW_KEY
|
||||||
|
if data.cred_id != OWN and not data.cred_id.isdigit():
|
||||||
|
errors["cred_id"] = "Ungültige Auswahl."
|
||||||
|
data.cred_id = OWN
|
||||||
try:
|
try:
|
||||||
data.type = MountType(form.get("type", "cifs"))
|
data.type = MountType(form.get("type", "cifs"))
|
||||||
except ValueError:
|
except ValueError:
|
||||||
@@ -210,7 +224,14 @@ def parse_mount_form(form: Mapping[str, str]) -> tuple[MountInput, Errors]:
|
|||||||
data.username = check("username", lambda: validate_username(data.username), data.username)
|
data.username = check("username", lambda: validate_username(data.username), data.username)
|
||||||
if data.domain:
|
if data.domain:
|
||||||
check("domain", lambda: validate_username(data.domain), "")
|
check("domain", lambda: validate_username(data.domain), "")
|
||||||
if data.type is MountType.SFTP and not data.username:
|
if data.auth is CredentialKind.SSH_KEY or data.type in (MountType.NFS3, MountType.NFS4):
|
||||||
|
data.cred_id = OWN
|
||||||
|
uses_stored = data.cred_id != OWN
|
||||||
|
if uses_stored:
|
||||||
|
data.username = data.domain = data.password = "" # kommen aus den Zugangsdaten
|
||||||
|
errors.pop("username", None)
|
||||||
|
errors.pop("password", None)
|
||||||
|
if data.type is MountType.SFTP and not data.username and not uses_stored:
|
||||||
errors["username"] = "SFTP braucht einen Benutzernamen."
|
errors["username"] = "SFTP braucht einen Benutzernamen."
|
||||||
if data.type is not MountType.SFTP:
|
if data.type is not MountType.SFTP:
|
||||||
data.auth = CredentialKind.PASSWORD
|
data.auth = CredentialKind.PASSWORD
|
||||||
@@ -224,7 +245,19 @@ def credential_for_test(
|
|||||||
) -> CredentialData | None:
|
) -> CredentialData | None:
|
||||||
"""Zugangsdaten für „Testen“ aus dem Formular – leeres Passwort = gespeichertes,
|
"""Zugangsdaten für „Testen“ aus dem Formular – leeres Passwort = gespeichertes,
|
||||||
gewählter Schlüssel aus der Schlüsselverwaltung."""
|
gewählter Schlüssel aus der Schlüsselverwaltung."""
|
||||||
if data.type in (MountType.NFS3, MountType.NFS4) or not data.username:
|
if data.type in (MountType.NFS3, MountType.NFS4):
|
||||||
|
return None
|
||||||
|
if data.cred_id != OWN and s is not None:
|
||||||
|
stored = s.get(Credential, int(data.cred_id))
|
||||||
|
if stored is None or not stored.managed or stored.kind is not CredentialKind.PASSWORD:
|
||||||
|
return None
|
||||||
|
return CredentialData(
|
||||||
|
CredentialKind.PASSWORD,
|
||||||
|
stored.username,
|
||||||
|
secrets.decrypt(stored.secret_enc),
|
||||||
|
stored.domain,
|
||||||
|
)
|
||||||
|
if not data.username:
|
||||||
return None
|
return None
|
||||||
if data.auth is CredentialKind.SSH_KEY and data.key_id != NEW_KEY and s is not None:
|
if data.auth is CredentialKind.SSH_KEY and data.key_id != NEW_KEY and s is not None:
|
||||||
key = s.get(Credential, int(data.key_id))
|
key = s.get(Credential, int(data.key_id))
|
||||||
@@ -261,7 +294,11 @@ def save_mount(
|
|||||||
mounts: MountManager,
|
mounts: MountManager,
|
||||||
data: MountInput,
|
data: MountInput,
|
||||||
mount_id: int | None = None,
|
mount_id: int | None = None,
|
||||||
|
copy_from: Mount | None = None,
|
||||||
) -> tuple[Mount | None, Errors]:
|
) -> tuple[Mount | None, Errors]:
|
||||||
|
"""``copy_from``: Vorlage beim Kopieren – eigene Zugangsdaten (Passwort bzw.
|
||||||
|
Schlüsselpaar) werden übernommen, solange kein neues Passwort eingegeben ist;
|
||||||
|
die Kopie bekommt ein eigenes Exemplar."""
|
||||||
errors: Errors = {}
|
errors: Errors = {}
|
||||||
m = s.get(Mount, mount_id) if mount_id is not None else None
|
m = s.get(Mount, mount_id) if mount_id is not None else None
|
||||||
if mount_id is not None and m is None:
|
if mount_id is not None and m is None:
|
||||||
@@ -316,13 +353,25 @@ def save_mount(
|
|||||||
setattr(m, f, getattr(data, f))
|
setattr(m, f, getattr(data, f))
|
||||||
if not data.type.is_kernel_mount:
|
if not data.type.is_kernel_mount:
|
||||||
m.auto_mount = False
|
m.auto_mount = False
|
||||||
|
if (
|
||||||
|
mount_id is None
|
||||||
|
and copy_from is not None
|
||||||
|
and data.type is MountType.SFTP
|
||||||
|
and copy_from.type is MountType.SFTP
|
||||||
|
and (copy_from.server, copy_from.port) == (data.server, data.port)
|
||||||
|
):
|
||||||
|
m.host_key = copy_from.host_key # gleicher Server – Hostschlüssel bleibt gültig
|
||||||
|
|
||||||
# Zugangsdaten: entweder ein gespeicherter Schlüssel aus der Schlüsselverwaltung
|
# Zugangsdaten: entweder ein gespeicherter Schlüssel aus der Schlüsselverwaltung
|
||||||
# (geteilt, wird hier nie verändert) oder eigene Zugangsdaten nur dieser Anbindung.
|
# (geteilt, wird hier nie verändert) oder eigene Zugangsdaten nur dieser Anbindung.
|
||||||
current = m.credential
|
current = m.credential
|
||||||
own = current if current is not None and not current.managed else None
|
own = current if current is not None and not current.managed else None
|
||||||
|
stored_password = data.cred_id != OWN and (
|
||||||
|
data.type is MountType.CIFS
|
||||||
|
or (data.type is MountType.SFTP and data.auth is CredentialKind.PASSWORD)
|
||||||
|
)
|
||||||
needs_cred = data.type is MountType.SFTP or (
|
needs_cred = data.type is MountType.SFTP or (
|
||||||
data.type is MountType.CIFS and bool(data.username)
|
data.type is MountType.CIFS and (bool(data.username) or stored_password)
|
||||||
)
|
)
|
||||||
use_managed = (
|
use_managed = (
|
||||||
data.type is MountType.SFTP
|
data.type is MountType.SFTP
|
||||||
@@ -332,6 +381,13 @@ def save_mount(
|
|||||||
if not needs_cred:
|
if not needs_cred:
|
||||||
m.credential = None
|
m.credential = None
|
||||||
m.username = None
|
m.username = None
|
||||||
|
elif stored_password:
|
||||||
|
stored = s.get(Credential, int(data.cred_id))
|
||||||
|
if stored is None or not stored.managed or stored.kind is not CredentialKind.PASSWORD:
|
||||||
|
s.rollback()
|
||||||
|
return None, {"cred_id": "Zugangsdaten existieren nicht."}
|
||||||
|
m.credential = stored
|
||||||
|
m.username = None
|
||||||
elif use_managed:
|
elif use_managed:
|
||||||
key = s.get(Credential, int(data.key_id))
|
key = s.get(Credential, int(data.key_id))
|
||||||
if key is None or not key.managed or key.kind is not CredentialKind.SSH_KEY:
|
if key is None or not key.managed or key.kind is not CredentialKind.SSH_KEY:
|
||||||
@@ -342,21 +398,32 @@ def save_mount(
|
|||||||
else:
|
else:
|
||||||
m.username = None
|
m.username = None
|
||||||
cred = own
|
cred = own
|
||||||
|
template = (
|
||||||
|
copy_from.credential
|
||||||
|
if copy_from is not None
|
||||||
|
and copy_from.credential is not None
|
||||||
|
and not copy_from.credential.managed
|
||||||
|
and copy_from.credential.kind is data.auth
|
||||||
|
else None
|
||||||
|
)
|
||||||
if cred is None or cred.kind is not data.auth:
|
if cred is None or cred.kind is not data.auth:
|
||||||
if data.auth is CredentialKind.SSH_KEY:
|
if template is not None and (data.auth is CredentialKind.SSH_KEY or not data.password):
|
||||||
private, public = generate_ssh_keypair(f"netmover@{m.slug}")
|
# Kopie: dasselbe Passwort bzw. Schlüsselpaar (steht schon in authorized_keys)
|
||||||
secret, pub = private, public
|
secret_enc, pub = template.secret_enc, template.public_key
|
||||||
|
elif data.auth is CredentialKind.SSH_KEY:
|
||||||
|
private, pub = generate_ssh_keypair(f"netmover@{m.slug}")
|
||||||
|
secret_enc = secrets.encrypt(private)
|
||||||
else:
|
else:
|
||||||
if not data.password:
|
if not data.password:
|
||||||
errors["password"] = "Passwort fehlt." # noqa: S105
|
errors["password"] = "Passwort fehlt." # noqa: S105
|
||||||
s.rollback()
|
s.rollback()
|
||||||
return None, errors
|
return None, errors
|
||||||
secret, pub = data.password, None
|
secret_enc, pub = secrets.encrypt(data.password), None
|
||||||
if cred is None:
|
if cred is None:
|
||||||
cred = Credential(name=f"mount:{m.slug}", managed=False)
|
cred = Credential(name=f"mount:{m.slug}", managed=False)
|
||||||
s.add(cred)
|
s.add(cred)
|
||||||
cred.kind = data.auth
|
cred.kind = data.auth
|
||||||
cred.secret_enc = secrets.encrypt(secret)
|
cred.secret_enc = secret_enc
|
||||||
cred.public_key = pub
|
cred.public_key = pub
|
||||||
elif data.password and data.auth is CredentialKind.PASSWORD:
|
elif data.password and data.auth is CredentialKind.PASSWORD:
|
||||||
cred.secret_enc = secrets.encrypt(data.password)
|
cred.secret_enc = secrets.encrypt(data.password)
|
||||||
@@ -377,12 +444,26 @@ def save_mount(
|
|||||||
return m, errors
|
return m, errors
|
||||||
|
|
||||||
|
|
||||||
|
def copy_name(s: Session, name: str) -> str:
|
||||||
|
"""„NAS (Kopie)“, „NAS (Kopie 2)“, … – frei in der Datenbank."""
|
||||||
|
base = f"{name} (Kopie)"[:100]
|
||||||
|
candidate, n = base, 1
|
||||||
|
while s.scalar(select(Mount.id).where(Mount.name == candidate)) is not None:
|
||||||
|
n += 1
|
||||||
|
candidate = f"{name} (Kopie {n})"[:100]
|
||||||
|
return candidate
|
||||||
|
|
||||||
|
|
||||||
def delete_mount(s: Session, mounts: MountManager, mount_id: int) -> None:
|
def delete_mount(s: Session, mounts: MountManager, mount_id: int) -> None:
|
||||||
m = s.get(Mount, mount_id)
|
m = s.get(Mount, mount_id)
|
||||||
if m is None:
|
if m is None:
|
||||||
return
|
return
|
||||||
used = s.scalars(
|
used = s.scalars(
|
||||||
select(Rule.name).where((Rule.src_mount_id == mount_id) | (Rule.dst_mount_id == mount_id))
|
select(Rule.name).where(
|
||||||
|
(Rule.src_mount_id == mount_id)
|
||||||
|
| (Rule.dst_mount_id == mount_id)
|
||||||
|
| (Rule.archive_mount_id == mount_id)
|
||||||
|
)
|
||||||
).all()
|
).all()
|
||||||
if used:
|
if used:
|
||||||
raise AdminError(f"Wird noch von Regeln benutzt: {', '.join(used)}")
|
raise AdminError(f"Wird noch von Regeln benutzt: {', '.join(used)}")
|
||||||
@@ -423,6 +504,10 @@ class RuleInput:
|
|||||||
dry_run: bool = False
|
dry_run: bool = False
|
||||||
quarantine_path: str = ""
|
quarantine_path: str = ""
|
||||||
quarantine_after: int = 3
|
quarantine_after: int = 3
|
||||||
|
archive_kind: ArchiveKind = ArchiveKind.NONE
|
||||||
|
archive_mount_id: int | None = None
|
||||||
|
archive_path: str = ""
|
||||||
|
archive_layout: ArchiveLayout = ArchiveLayout.FLAT
|
||||||
extra: dict[str, str] = field(default_factory=dict)
|
extra: dict[str, str] = field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
@@ -450,6 +535,10 @@ def rule_input_from_model(r: Rule) -> RuleInput:
|
|||||||
dry_run=r.dry_run,
|
dry_run=r.dry_run,
|
||||||
quarantine_path=r.quarantine_path or "",
|
quarantine_path=r.quarantine_path or "",
|
||||||
quarantine_after=r.quarantine_after,
|
quarantine_after=r.quarantine_after,
|
||||||
|
archive_kind=r.archive_kind,
|
||||||
|
archive_mount_id=r.archive_mount_id,
|
||||||
|
archive_path=r.archive_path,
|
||||||
|
archive_layout=r.archive_layout,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -496,8 +585,6 @@ def parse_rule_form(s: Session, form: Mapping[str, str]) -> tuple[RuleInput, Err
|
|||||||
src = s.get(Mount, d.src_mount_id) if d.src_mount_id else None
|
src = s.get(Mount, d.src_mount_id) if d.src_mount_id else None
|
||||||
if src is None:
|
if src is None:
|
||||||
errors["src_mount_id"] = "Quelle wählen."
|
errors["src_mount_id"] = "Quelle wählen."
|
||||||
elif not src.type.is_kernel_mount:
|
|
||||||
errors["src_mount_id"] = "SFTP-Anbindungen sind nur als Ziel möglich."
|
|
||||||
try:
|
try:
|
||||||
d.src_subpath = safe_relpath("src_subpath", d.src_subpath)
|
d.src_subpath = safe_relpath("src_subpath", d.src_subpath)
|
||||||
except ValidationError as exc:
|
except ValidationError as exc:
|
||||||
@@ -530,6 +617,10 @@ def parse_rule_form(s: Session, form: Mapping[str, str]) -> tuple[RuleInput, Err
|
|||||||
errors["pattern"] = f"Ungültiger regulärer Ausdruck: {exc}"
|
errors["pattern"] = f"Ungültiger regulärer Ausdruck: {exc}"
|
||||||
if d.stability_seconds >= d.op_timeout_seconds:
|
if d.stability_seconds >= d.op_timeout_seconds:
|
||||||
errors["stability_seconds"] = "Muss kleiner als das Zeitlimit sein."
|
errors["stability_seconds"] = "Muss kleiner als das Zeitlimit sein."
|
||||||
|
elif src is not None and src.type is MountType.SFTP and d.stability_seconds < 2:
|
||||||
|
errors["stability_seconds"] = (
|
||||||
|
"Bei SFTP-Quellen mindestens 2 s – der Server meldet Zeiten nur sekundengenau."
|
||||||
|
)
|
||||||
if d.set_owner and not _OWNER_RE.fullmatch(d.set_owner):
|
if d.set_owner and not _OWNER_RE.fullmatch(d.set_owner):
|
||||||
errors["set_owner"] = "Als „uid:gid“ angeben, z. B. „1000:1000“."
|
errors["set_owner"] = "Als „uid:gid“ angeben, z. B. „1000:1000“."
|
||||||
try:
|
try:
|
||||||
@@ -541,9 +632,52 @@ def parse_rule_form(s: Session, form: Mapping[str, str]) -> tuple[RuleInput, Err
|
|||||||
d.quarantine_path = safe_relpath("quarantine_path", d.quarantine_path)
|
d.quarantine_path = safe_relpath("quarantine_path", d.quarantine_path)
|
||||||
except ValidationError as exc:
|
except ValidationError as exc:
|
||||||
errors["quarantine_path"] = str(exc)
|
errors["quarantine_path"] = str(exc)
|
||||||
|
|
||||||
|
# Archiv
|
||||||
|
try:
|
||||||
|
d.archive_kind = ArchiveKind(form.get("archive_kind") or "none")
|
||||||
|
d.archive_layout = ArchiveLayout(form.get("archive_layout") or "flat")
|
||||||
|
except ValueError:
|
||||||
|
errors["archive_kind"] = "Ungültige Auswahl."
|
||||||
|
d.archive_path = (form.get("archive_path") or "").strip()
|
||||||
|
if d.archive_kind is ArchiveKind.MOUNT:
|
||||||
|
d.archive_mount_id = _int(form, "archive_mount_id", errors)
|
||||||
|
if d.archive_mount_id is None or s.get(Mount, d.archive_mount_id) is None:
|
||||||
|
errors["archive_mount_id"] = "Archiv-Anbindung wählen."
|
||||||
|
try:
|
||||||
|
d.archive_path = safe_relpath("archive_path", d.archive_path)
|
||||||
|
except ValidationError as exc:
|
||||||
|
errors["archive_path"] = str(exc)
|
||||||
|
if (
|
||||||
|
"archive_path" not in errors
|
||||||
|
and d.archive_mount_id == d.dst_mount_id
|
||||||
|
and d.dst_kind == "mount"
|
||||||
|
and d.archive_path == d.dst_path
|
||||||
|
):
|
||||||
|
errors["archive_path"] = "Archiv und Ziel sind derselbe Ordner."
|
||||||
|
elif d.archive_kind is ArchiveKind.LOCAL:
|
||||||
|
d.archive_mount_id = None
|
||||||
|
try:
|
||||||
|
d.archive_path = safe_abspath("archive_path", d.archive_path)
|
||||||
|
except ValidationError as exc:
|
||||||
|
errors["archive_path"] = str(exc)
|
||||||
|
if d.dst_kind == "local" and d.archive_path == d.dst_path:
|
||||||
|
errors["archive_path"] = "Archiv und Ziel sind derselbe Ordner."
|
||||||
|
else:
|
||||||
|
d.archive_mount_id, d.archive_path = None, ""
|
||||||
return d, errors
|
return d, errors
|
||||||
|
|
||||||
|
|
||||||
|
def rule_copy_name(s: Session, name: str) -> str:
|
||||||
|
"""„Scans (Kopie)“, „Scans (Kopie 2)“, … – frei in der Datenbank."""
|
||||||
|
base = f"{name} (Kopie)"[:100]
|
||||||
|
candidate, n = base, 1
|
||||||
|
while s.scalar(select(Rule.id).where(Rule.name == candidate)) is not None:
|
||||||
|
n += 1
|
||||||
|
candidate = f"{name} (Kopie {n})"[:100]
|
||||||
|
return candidate
|
||||||
|
|
||||||
|
|
||||||
def save_rule(s: Session, d: RuleInput, rule_id: int | None = None) -> tuple[Rule | None, Errors]:
|
def save_rule(s: Session, d: RuleInput, rule_id: int | None = None) -> tuple[Rule | None, Errors]:
|
||||||
dup = s.scalar(select(Rule.id).where(Rule.name == d.name))
|
dup = s.scalar(select(Rule.id).where(Rule.name == d.name))
|
||||||
if dup is not None and dup != rule_id:
|
if dup is not None and dup != rule_id:
|
||||||
@@ -573,6 +707,10 @@ def save_rule(s: Session, d: RuleInput, rule_id: int | None = None) -> tuple[Rul
|
|||||||
"on_collision",
|
"on_collision",
|
||||||
"dry_run",
|
"dry_run",
|
||||||
"quarantine_after",
|
"quarantine_after",
|
||||||
|
"archive_kind",
|
||||||
|
"archive_mount_id",
|
||||||
|
"archive_path",
|
||||||
|
"archive_layout",
|
||||||
):
|
):
|
||||||
setattr(r, f, getattr(d, f))
|
setattr(r, f, getattr(d, f))
|
||||||
r.set_owner = d.set_owner or None
|
r.set_owner = d.set_owner or None
|
||||||
|
|||||||
@@ -64,7 +64,16 @@ def main(argv: Sequence[str] | None = None) -> int:
|
|||||||
row = s.get(Setting, PASSWORD_KEY)
|
row = s.get(Setting, PASSWORD_KEY)
|
||||||
if row is not None:
|
if row is not None:
|
||||||
s.delete(row)
|
s.delete(row)
|
||||||
s.commit()
|
# Anmeldung wieder überall einschalten und Netz-Sperren aufheben
|
||||||
|
for key, value in (
|
||||||
|
("app.auth_mode", '"always"'),
|
||||||
|
("app.trusted_networks", '""'),
|
||||||
|
("app.allowed_networks", '""'),
|
||||||
|
):
|
||||||
|
setting = s.get(Setting, key)
|
||||||
|
if setting is not None:
|
||||||
|
setting.value = value
|
||||||
|
s.commit()
|
||||||
engine.dispose()
|
engine.dispose()
|
||||||
print("Passwort gelöscht. Dienst neu starten; der Einrichtungscode steht im Log.")
|
print("Passwort gelöscht. Dienst neu starten; der Einrichtungscode steht im Log.")
|
||||||
return 0
|
return 0
|
||||||
|
|||||||
@@ -1,4 +1,9 @@
|
|||||||
"""Schlüsselverwaltung: gespeicherte SSH-Schlüssel für SFTP-Anbindungen.
|
"""Zugangsdaten-Verwaltung: gespeicherte Passwörter und SSH-Schlüssel.
|
||||||
|
|
||||||
|
Passwörter (SMB und SFTP): Benutzer, Passwort, optional Domäne – einmal gepflegt,
|
||||||
|
von beliebig vielen Anbindungen genutzt. Ein neues Passwort gilt sofort für alle.
|
||||||
|
|
||||||
|
SSH-Schlüssel:
|
||||||
|
|
||||||
* Erzeugen (Ed25519) oder Importieren (Ed25519, ECDSA, RSA ≥ 2048 Bit; OpenSSH-
|
* Erzeugen (Ed25519) oder Importieren (Ed25519, ECDSA, RSA ≥ 2048 Bit; OpenSSH-
|
||||||
oder PEM-Format, optional mit Passphrase). Eine Passphrase wird beim Import
|
oder PEM-Format, optional mit Passphrase). Eine Passphrase wird beim Import
|
||||||
@@ -63,7 +68,7 @@ def _validate_name(s: Session, name: str, exclude_id: int | None = None) -> str:
|
|||||||
raise AdminError("Namen mit „mount:“ sind für NetMover reserviert.")
|
raise AdminError("Namen mit „mount:“ sind für NetMover reserviert.")
|
||||||
dup = s.scalar(select(Credential.id).where(Credential.name == name))
|
dup = s.scalar(select(Credential.id).where(Credential.name == name))
|
||||||
if dup is not None and dup != exclude_id:
|
if dup is not None and dup != exclude_id:
|
||||||
raise AdminError(f"Ein Schlüssel „{name}“ existiert bereits.")
|
raise AdminError(f"Zugangsdaten mit dem Namen „{name}“ existieren bereits.")
|
||||||
return name
|
return name
|
||||||
|
|
||||||
|
|
||||||
@@ -153,9 +158,115 @@ def _users(s: Session, key_id: int) -> list[str]:
|
|||||||
return sorted(s.scalars(select(Mount.name).where(Mount.cred_id == key_id)))
|
return sorted(s.scalars(select(Mount.name).where(Mount.cred_id == key_id)))
|
||||||
|
|
||||||
|
|
||||||
|
# -- Passwörter -------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class PasswordInfo:
|
||||||
|
id: int
|
||||||
|
name: str
|
||||||
|
username: str
|
||||||
|
domain: str | None
|
||||||
|
created_at: datetime
|
||||||
|
used_by: list[str] = field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_login(username: str, password: str | None, domain: str) -> None:
|
||||||
|
from netmover.validation import ValidationError, validate_secret, validate_username
|
||||||
|
|
||||||
|
try:
|
||||||
|
validate_username(username)
|
||||||
|
if domain:
|
||||||
|
validate_username(domain)
|
||||||
|
if password is not None:
|
||||||
|
validate_secret(password)
|
||||||
|
except ValidationError as exc:
|
||||||
|
raise AdminError(str(exc)) from None
|
||||||
|
|
||||||
|
|
||||||
|
def create_password(
|
||||||
|
s: Session, secrets: SecretBox, name: str, username: str, password: str, domain: str = ""
|
||||||
|
) -> Credential:
|
||||||
|
name = _validate_name(s, name)
|
||||||
|
username, domain = username.strip(), domain.strip()
|
||||||
|
if not password:
|
||||||
|
raise AdminError("Passwort fehlt.")
|
||||||
|
_validate_login(username, password, domain)
|
||||||
|
cred = Credential(
|
||||||
|
name=name,
|
||||||
|
kind=CredentialKind.PASSWORD,
|
||||||
|
username=username,
|
||||||
|
domain=domain or None,
|
||||||
|
secret_enc=secrets.encrypt(password),
|
||||||
|
managed=True,
|
||||||
|
)
|
||||||
|
s.add(cred)
|
||||||
|
s.flush()
|
||||||
|
return cred
|
||||||
|
|
||||||
|
|
||||||
|
def update_password(
|
||||||
|
s: Session, secrets: SecretBox, cred_id: int, username: str, password: str, domain: str = ""
|
||||||
|
) -> Credential:
|
||||||
|
"""Benutzer/Domäne ändern; leeres Passwort = unverändert."""
|
||||||
|
cred = managed_password(s, cred_id)
|
||||||
|
username, domain = username.strip(), domain.strip()
|
||||||
|
_validate_login(username, password or None, domain)
|
||||||
|
cred.username = username
|
||||||
|
cred.domain = domain or None
|
||||||
|
if password:
|
||||||
|
cred.secret_enc = secrets.encrypt(password)
|
||||||
|
return cred
|
||||||
|
|
||||||
|
|
||||||
|
def managed_password(s: Session, cred_id: int) -> Credential:
|
||||||
|
cred = s.get(Credential, cred_id)
|
||||||
|
if cred is None or not cred.managed or cred.kind is not CredentialKind.PASSWORD:
|
||||||
|
raise AdminError("Zugangsdaten existieren nicht.")
|
||||||
|
return cred
|
||||||
|
|
||||||
|
|
||||||
|
def list_passwords(s: Session) -> list[PasswordInfo]:
|
||||||
|
rows = s.scalars(
|
||||||
|
select(Credential)
|
||||||
|
.where(Credential.managed.is_(True), Credential.kind == CredentialKind.PASSWORD)
|
||||||
|
.order_by(Credential.name)
|
||||||
|
).all()
|
||||||
|
return [
|
||||||
|
PasswordInfo(c.id, c.name, c.username, c.domain, c.created_at, _users(s, c.id))
|
||||||
|
for c in rows
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def delete_credential(s: Session, cred_id: int) -> str:
|
||||||
|
"""Gespeicherte Zugangsdaten (Passwort oder Schlüssel) löschen – nur unbenutzt."""
|
||||||
|
cred = s.get(Credential, cred_id)
|
||||||
|
if cred is None or not cred.managed:
|
||||||
|
raise AdminError("Zugangsdaten existieren nicht.")
|
||||||
|
users = _users(s, cred_id)
|
||||||
|
if users:
|
||||||
|
raise AdminError(f"„{cred.name}“ wird noch benutzt von: {', '.join(users)}")
|
||||||
|
name = cred.name
|
||||||
|
s.delete(cred)
|
||||||
|
s.flush()
|
||||||
|
return name
|
||||||
|
|
||||||
|
|
||||||
|
def rename_credential(s: Session, cred_id: int, name: str) -> None:
|
||||||
|
cred = s.get(Credential, cred_id)
|
||||||
|
if cred is None or not cred.managed:
|
||||||
|
raise AdminError("Zugangsdaten existieren nicht.")
|
||||||
|
cred.name = _validate_name(s, name, exclude_id=cred_id)
|
||||||
|
|
||||||
|
|
||||||
|
# -- SSH-Schlüssel ------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
def list_keys(s: Session) -> list[KeyInfo]:
|
def list_keys(s: Session) -> list[KeyInfo]:
|
||||||
keys = s.scalars(
|
keys = s.scalars(
|
||||||
select(Credential).where(Credential.managed.is_(True)).order_by(Credential.name)
|
select(Credential)
|
||||||
|
.where(Credential.managed.is_(True), Credential.kind == CredentialKind.SSH_KEY)
|
||||||
|
.order_by(Credential.name)
|
||||||
).all()
|
).all()
|
||||||
result = []
|
result = []
|
||||||
for k in keys:
|
for k in keys:
|
||||||
@@ -0,0 +1,103 @@
|
|||||||
|
"""archive
|
||||||
|
|
||||||
|
Revision ID: 0004
|
||||||
|
Revises: 0003
|
||||||
|
Create Date: 2026-10-12
|
||||||
|
"""
|
||||||
|
|
||||||
|
from collections.abc import Sequence
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from alembic import op
|
||||||
|
|
||||||
|
import netmover.models
|
||||||
|
|
||||||
|
revision: str = "0004"
|
||||||
|
down_revision: str | None = "0003"
|
||||||
|
branch_labels: str | Sequence[str] | None = None
|
||||||
|
depends_on: str | Sequence[str] | None = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
op.create_table(
|
||||||
|
"archived_files",
|
||||||
|
sa.Column("id", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("rule_id", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("src_relpath", sa.String(length=2048), nullable=False),
|
||||||
|
sa.Column("size", sa.BigInteger(), nullable=False),
|
||||||
|
sa.Column("mtime_ns", sa.BigInteger(), nullable=False),
|
||||||
|
sa.Column("archive_path", sa.String(length=2048), nullable=False),
|
||||||
|
sa.Column("archived_at", netmover.models.UTCDateTime(), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(
|
||||||
|
["rule_id"],
|
||||||
|
["rules.id"],
|
||||||
|
name=op.f("fk_archived_files_rule_id_rules"),
|
||||||
|
ondelete="CASCADE",
|
||||||
|
),
|
||||||
|
sa.PrimaryKeyConstraint("id", name=op.f("pk_archived_files")),
|
||||||
|
sa.UniqueConstraint(
|
||||||
|
"rule_id",
|
||||||
|
"src_relpath",
|
||||||
|
"size",
|
||||||
|
"mtime_ns",
|
||||||
|
name=op.f("uq_archived_files_rule_id_src_relpath_size_mtime_ns"),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
with op.batch_alter_table("rules", schema=None) as batch_op:
|
||||||
|
batch_op.add_column(
|
||||||
|
sa.Column(
|
||||||
|
"archive_kind",
|
||||||
|
sa.Enum(
|
||||||
|
"none",
|
||||||
|
"mount",
|
||||||
|
"local",
|
||||||
|
name="archive_kind",
|
||||||
|
native_enum=False,
|
||||||
|
create_constraint=True,
|
||||||
|
length=24,
|
||||||
|
),
|
||||||
|
server_default="none",
|
||||||
|
nullable=False,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
batch_op.add_column(sa.Column("archive_mount_id", sa.Integer(), nullable=True))
|
||||||
|
batch_op.add_column(
|
||||||
|
sa.Column("archive_path", sa.String(length=1024), server_default="", nullable=False)
|
||||||
|
)
|
||||||
|
batch_op.add_column(
|
||||||
|
sa.Column(
|
||||||
|
"archive_layout",
|
||||||
|
sa.Enum(
|
||||||
|
"flat",
|
||||||
|
"date",
|
||||||
|
name="archive_layout",
|
||||||
|
native_enum=False,
|
||||||
|
create_constraint=True,
|
||||||
|
length=24,
|
||||||
|
),
|
||||||
|
server_default="flat",
|
||||||
|
nullable=False,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
batch_op.create_foreign_key(
|
||||||
|
batch_op.f("fk_rules_archive_mount_id_mounts"),
|
||||||
|
"mounts",
|
||||||
|
["archive_mount_id"],
|
||||||
|
["id"],
|
||||||
|
ondelete="RESTRICT",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
with op.batch_alter_table("rules", schema=None) as batch_op:
|
||||||
|
batch_op.drop_constraint(batch_op.f("fk_rules_archive_mount_id_mounts"), type_="foreignkey")
|
||||||
|
# CHECK-Regeln der Auswahlfelder zuerst entfernen – sonst verweisen sie beim
|
||||||
|
# Neuaufbau der Tabelle auf Spalten, die es nicht mehr gibt.
|
||||||
|
batch_op.drop_constraint(batch_op.f("ck_rules_archive_kind"), type_="check")
|
||||||
|
batch_op.drop_constraint(batch_op.f("ck_rules_archive_layout"), type_="check")
|
||||||
|
batch_op.drop_column("archive_layout")
|
||||||
|
batch_op.drop_column("archive_path")
|
||||||
|
batch_op.drop_column("archive_mount_id")
|
||||||
|
batch_op.drop_column("archive_kind")
|
||||||
|
|
||||||
|
op.drop_table("archived_files")
|
||||||
@@ -10,6 +10,7 @@ Abweichungen bzw. Ergänzungen gegenüber dem Mindestumfang aus der Spezifikatio
|
|||||||
* ``credentials.managed`` + ``mounts.username`` (Migration 0002) –
|
* ``credentials.managed`` + ``mounts.username`` (Migration 0002) –
|
||||||
Schlüsselverwaltung: gespeicherte SSH-Schlüssel, die mehrere Anbindungen mit
|
Schlüsselverwaltung: gespeicherte SSH-Schlüssel, die mehrere Anbindungen mit
|
||||||
jeweils eigenem Benutzernamen nutzen.
|
jeweils eigenem Benutzernamen nutzen.
|
||||||
|
* ``rules.archive_*`` + Tabelle ``archived_files`` (Migration 0004) – Archivkopie.
|
||||||
* ``rules.waiting_since`` / ``rules.waiting_reason`` (Migration 0003) – Regel wartet
|
* ``rules.waiting_since`` / ``rules.waiting_reason`` (Migration 0003) – Regel wartet
|
||||||
auf eine nicht erreichbare Anbindung (automatisch pausiert).
|
auf eine nicht erreichbare Anbindung (automatisch pausiert).
|
||||||
* ``mounts.state_since`` – für „hängt seit …“ / „gestört seit …“ auf der Statusseite.
|
* ``mounts.state_since`` – für „hängt seit …“ / „gestört seit …“ auf der Statusseite.
|
||||||
@@ -138,6 +139,17 @@ class CollisionPolicy(enum.StrEnum):
|
|||||||
OVERWRITE = "overwrite"
|
OVERWRITE = "overwrite"
|
||||||
|
|
||||||
|
|
||||||
|
class ArchiveKind(enum.StrEnum):
|
||||||
|
NONE = "none"
|
||||||
|
MOUNT = "mount"
|
||||||
|
LOCAL = "local"
|
||||||
|
|
||||||
|
|
||||||
|
class ArchiveLayout(enum.StrEnum):
|
||||||
|
FLAT = "flat" # wie in der Quelle
|
||||||
|
DATE = "date" # Jahr/Monat/…, nach Tag der Verarbeitung
|
||||||
|
|
||||||
|
|
||||||
class RunStatus(enum.StrEnum):
|
class RunStatus(enum.StrEnum):
|
||||||
RUNNING = "running"
|
RUNNING = "running"
|
||||||
OK = "ok" # keine Dateifehler
|
OK = "ok" # keine Dateifehler
|
||||||
@@ -259,6 +271,18 @@ class Rule(Base):
|
|||||||
quarantine_path: Mapped[str | None] = mapped_column(String(1024))
|
quarantine_path: Mapped[str | None] = mapped_column(String(1024))
|
||||||
quarantine_after: Mapped[int] = mapped_column(Integer, default=3)
|
quarantine_after: Mapped[int] = mapped_column(Integer, default=3)
|
||||||
|
|
||||||
|
# Archiv (Migration 0004): zusätzliche Kopie; Quelle erst löschen, wenn Ziel UND Archiv da
|
||||||
|
archive_kind: Mapped[ArchiveKind] = mapped_column(
|
||||||
|
_enum(ArchiveKind, "archive_kind"), default=ArchiveKind.NONE, server_default="none"
|
||||||
|
)
|
||||||
|
archive_mount_id: Mapped[int | None] = mapped_column(
|
||||||
|
ForeignKey("mounts.id", ondelete="RESTRICT")
|
||||||
|
)
|
||||||
|
archive_path: Mapped[str] = mapped_column(String(1024), default="", server_default="")
|
||||||
|
archive_layout: Mapped[ArchiveLayout] = mapped_column(
|
||||||
|
_enum(ArchiveLayout, "archive_layout"), default=ArchiveLayout.FLAT, server_default="flat"
|
||||||
|
)
|
||||||
|
|
||||||
last_run_at: Mapped[datetime | None] = mapped_column(UTCDateTime)
|
last_run_at: Mapped[datetime | None] = mapped_column(UTCDateTime)
|
||||||
last_success_at: Mapped[datetime | None] = mapped_column(UTCDateTime)
|
last_success_at: Mapped[datetime | None] = mapped_column(UTCDateTime)
|
||||||
consecutive_failures: Mapped[int] = mapped_column(Integer, default=0)
|
consecutive_failures: Mapped[int] = mapped_column(Integer, default=0)
|
||||||
@@ -269,6 +293,7 @@ class Rule(Base):
|
|||||||
|
|
||||||
src_mount: Mapped[Mount] = relationship(foreign_keys=[src_mount_id])
|
src_mount: Mapped[Mount] = relationship(foreign_keys=[src_mount_id])
|
||||||
dst_mount: Mapped[Mount | None] = relationship(foreign_keys=[dst_mount_id])
|
dst_mount: Mapped[Mount | None] = relationship(foreign_keys=[dst_mount_id])
|
||||||
|
archive_mount: Mapped[Mount | None] = relationship(foreign_keys=[archive_mount_id])
|
||||||
runs: Mapped[list[Run]] = relationship(
|
runs: Mapped[list[Run]] = relationship(
|
||||||
back_populates="rule", cascade="all, delete-orphan", passive_deletes=True
|
back_populates="rule", cascade="all, delete-orphan", passive_deletes=True
|
||||||
)
|
)
|
||||||
@@ -330,6 +355,22 @@ class CopiedFile(Base):
|
|||||||
copied_at: Mapped[datetime] = mapped_column(UTCDateTime, default=utcnow)
|
copied_at: Mapped[datetime] = mapped_column(UTCDateTime, default=utcnow)
|
||||||
|
|
||||||
|
|
||||||
|
class ArchivedFile(Base):
|
||||||
|
"""Welche Dateiversion schon im Archiv liegt – damit ein Neuversuch nach einem
|
||||||
|
gescheiterten Ziel keine zweite Archivkopie anlegt."""
|
||||||
|
|
||||||
|
__tablename__ = "archived_files"
|
||||||
|
__table_args__ = (UniqueConstraint("rule_id", "src_relpath", "size", "mtime_ns"),)
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
||||||
|
rule_id: Mapped[int] = mapped_column(ForeignKey("rules.id", ondelete="CASCADE"))
|
||||||
|
src_relpath: Mapped[str] = mapped_column(String(2048))
|
||||||
|
size: Mapped[int] = mapped_column(BigInteger)
|
||||||
|
mtime_ns: Mapped[int] = mapped_column(BigInteger)
|
||||||
|
archive_path: Mapped[str] = mapped_column(String(2048))
|
||||||
|
archived_at: Mapped[datetime] = mapped_column(UTCDateTime, default=utcnow)
|
||||||
|
|
||||||
|
|
||||||
class FileFailure(Base):
|
class FileFailure(Base):
|
||||||
"""Fehlversuche pro Datei, Grundlage für die Quarantäne."""
|
"""Fehlversuche pro Datei, Grundlage für die Quarantäne."""
|
||||||
|
|
||||||
|
|||||||
@@ -30,12 +30,13 @@ from abc import ABC, abstractmethod
|
|||||||
from collections.abc import Callable
|
from collections.abc import Callable
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Literal
|
from typing import Any, Literal, Protocol
|
||||||
|
|
||||||
from netmover.models import CollisionPolicy
|
from netmover.models import CollisionPolicy
|
||||||
from netmover.mounting.sftp import SftpSession
|
from netmover.mounting.sftp import SftpSession
|
||||||
from netmover.mover.fsops import rename_noreplace
|
from netmover.mover.fsops import rename_noreplace
|
||||||
from netmover.mover.naming import collision_candidates
|
from netmover.mover.naming import collision_candidates
|
||||||
|
from netmover.mover.sources import network_errors
|
||||||
from netmover.timeouts import CancelToken
|
from netmover.timeouts import CancelToken
|
||||||
|
|
||||||
CHUNK = 1024 * 1024
|
CHUNK = 1024 * 1024
|
||||||
@@ -81,6 +82,13 @@ class CommitResult:
|
|||||||
Verify = Callable[[], None]
|
Verify = Callable[[], None]
|
||||||
|
|
||||||
|
|
||||||
|
class Readable(Protocol):
|
||||||
|
"""Quelle einer Übertragung: ein lokaler ``Path`` oder eine Datei einer
|
||||||
|
:class:`~netmover.mover.sources.Source` (über einen kleinen Adapter)."""
|
||||||
|
|
||||||
|
def open(self, mode: str = ...) -> Any: ...
|
||||||
|
|
||||||
|
|
||||||
class Destination(ABC):
|
class Destination(ABC):
|
||||||
kind: FsKind
|
kind: FsKind
|
||||||
|
|
||||||
@@ -107,7 +115,7 @@ class Destination(ABC):
|
|||||||
def _ensure_parent(self, rel: str) -> None: ...
|
def _ensure_parent(self, rel: str) -> None: ...
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def _upload(self, src: Path, tmp: str, size: int, token: CancelToken) -> list[str]: ...
|
def _upload(self, src: Readable, tmp: str, size: int, token: CancelToken) -> list[str]: ...
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def _apply_meta(self, rel: str, meta: MetaOptions) -> list[str]: ...
|
def _apply_meta(self, rel: str, meta: MetaOptions) -> list[str]: ...
|
||||||
@@ -140,7 +148,7 @@ class Destination(ABC):
|
|||||||
|
|
||||||
def commit(
|
def commit(
|
||||||
self,
|
self,
|
||||||
src: Path,
|
src: Readable,
|
||||||
size: int,
|
size: int,
|
||||||
rel: str,
|
rel: str,
|
||||||
policy: CollisionPolicy,
|
policy: CollisionPolicy,
|
||||||
@@ -234,7 +242,7 @@ class LocalDestination(Destination):
|
|||||||
def _ensure_parent(self, rel: str) -> None:
|
def _ensure_parent(self, rel: str) -> None:
|
||||||
self._p(rel).parent.mkdir(mode=0o755, parents=True, exist_ok=True)
|
self._p(rel).parent.mkdir(mode=0o755, parents=True, exist_ok=True)
|
||||||
|
|
||||||
def _upload(self, src: Path, tmp: str, size: int, token: CancelToken) -> list[str]:
|
def _upload(self, src: Readable, tmp: str, size: int, token: CancelToken) -> list[str]:
|
||||||
written = 0
|
written = 0
|
||||||
fd = os.open(self._p(tmp), os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
|
fd = os.open(self._p(tmp), os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
|
||||||
try:
|
try:
|
||||||
@@ -429,10 +437,10 @@ class SftpDestination(Destination):
|
|||||||
except FileNotFoundError:
|
except FileNotFoundError:
|
||||||
self.client.mkdir(current, mode=0o755)
|
self.client.mkdir(current, mode=0o755)
|
||||||
|
|
||||||
def _upload(self, src: Path, tmp: str, size: int, token: CancelToken) -> list[str]:
|
def _upload(self, src: Readable, tmp: str, size: int, token: CancelToken) -> list[str]:
|
||||||
notes: list[str] = []
|
notes: list[str] = []
|
||||||
written = 0
|
written = 0
|
||||||
with self.client.open(self._p(tmp), "wb") as fh, src.open("rb") as fin:
|
with network_errors(), self.client.open(self._p(tmp), "wb") as fh, src.open("rb") as fin:
|
||||||
fh.set_pipelined(True)
|
fh.set_pipelined(True)
|
||||||
while chunk := fin.read(CHUNK // 4):
|
while chunk := fin.read(CHUNK // 4):
|
||||||
token.check()
|
token.check()
|
||||||
|
|||||||
@@ -15,12 +15,13 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import dataclasses
|
import dataclasses
|
||||||
import errno
|
import errno
|
||||||
import os
|
import posixpath
|
||||||
import stat
|
|
||||||
import time
|
import time
|
||||||
from collections.abc import Callable
|
from collections.abc import Callable
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import datetime
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
from netmover.logging_setup import event, get_logger
|
from netmover.logging_setup import event, get_logger
|
||||||
from netmover.models import (
|
from netmover.models import (
|
||||||
@@ -40,7 +41,8 @@ from netmover.mover.destinations import (
|
|||||||
)
|
)
|
||||||
from netmover.mover.ledger import FailureInfo, Ledger
|
from netmover.mover.ledger import FailureInfo, Ledger
|
||||||
from netmover.mover.naming import collision_candidates
|
from netmover.mover.naming import collision_candidates
|
||||||
from netmover.mover.scan import BUSY, Candidate, FileState, Matcher, current_state, scan
|
from netmover.mover.scan import BUSY, Candidate, FileState, Matcher
|
||||||
|
from netmover.mover.sources import LocalSource, Source, SourceMissing
|
||||||
from netmover.timeouts import Cancelled, CancelToken
|
from netmover.timeouts import Cancelled, CancelToken
|
||||||
|
|
||||||
log = get_logger(__name__)
|
log = get_logger(__name__)
|
||||||
@@ -71,6 +73,7 @@ class RuleSnapshot:
|
|||||||
set_mode: int | None = None
|
set_mode: int | None = None
|
||||||
quarantine_after: int = 3
|
quarantine_after: int = 3
|
||||||
dst_root_squash: bool | None = None # nur NFS-Ziele: Ergebnis des Schreibtests
|
dst_root_squash: bool | None = None # nur NFS-Ziele: Ergebnis des Schreibtests
|
||||||
|
archive_layout: str = "flat" # "flat" | "date" (Jahr/Monat/…)
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
@@ -114,6 +117,17 @@ def _mb(n: int) -> str:
|
|||||||
return f"{n / 1024 / 1024:.1f} MB"
|
return f"{n / 1024 / 1024:.1f} MB"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class _SourceFile:
|
||||||
|
"""Adapter: eine Datei einer :class:`Source` mit ``open()`` wie ein ``Path``."""
|
||||||
|
|
||||||
|
source: Source
|
||||||
|
rel: str
|
||||||
|
|
||||||
|
def open(self, mode: str = "rb") -> Any:
|
||||||
|
return self.source.open(self.rel)
|
||||||
|
|
||||||
|
|
||||||
class Engine:
|
class Engine:
|
||||||
def __init__(
|
def __init__(
|
||||||
self,
|
self,
|
||||||
@@ -140,20 +154,28 @@ class Engine:
|
|||||||
def execute(
|
def execute(
|
||||||
self,
|
self,
|
||||||
rule: RuleSnapshot,
|
rule: RuleSnapshot,
|
||||||
src_root: Path,
|
source: Source | Path,
|
||||||
dest: Destination,
|
dest: Destination,
|
||||||
token: CancelToken,
|
token: CancelToken,
|
||||||
*,
|
*,
|
||||||
dry_run: bool = False,
|
dry_run: bool = False,
|
||||||
|
quarantine_rel: str | None = None,
|
||||||
quarantine_root: Path | None = None,
|
quarantine_root: Path | None = None,
|
||||||
|
archive: Destination | None = None,
|
||||||
) -> RunResult:
|
) -> RunResult:
|
||||||
|
"""``source``: eine :class:`Source` oder (Kurzform) ein lokales Verzeichnis.
|
||||||
|
Quarantäne relativ zur Wurzel der Anbindung (``quarantine_rel``) oder – bei
|
||||||
|
der Kurzform – als Pfad innerhalb des Verzeichnisses (``quarantine_root``)."""
|
||||||
result = RunResult(dry_run=dry_run)
|
result = RunResult(dry_run=dry_run)
|
||||||
|
src = source if isinstance(source, Source) else LocalSource(source)
|
||||||
|
if quarantine_root is not None and isinstance(src, LocalSource):
|
||||||
|
quarantine_rel = str(quarantine_root.relative_to(src.root))
|
||||||
try:
|
try:
|
||||||
self._execute(rule, src_root, dest, token, result, dry_run, quarantine_root)
|
self._execute(rule, src, dest, token, result, dry_run, quarantine_rel, archive)
|
||||||
except Cancelled:
|
except Cancelled:
|
||||||
result.status = RunStatus.ERROR
|
result.status = RunStatus.ERROR
|
||||||
result.message = "Lauf abgebrochen"
|
result.message = "Lauf abgebrochen"
|
||||||
except RunAborted as exc:
|
except (RunAborted, SourceMissing) as exc:
|
||||||
result.status = RunStatus.ERROR
|
result.status = RunStatus.ERROR
|
||||||
result.message = str(exc)
|
result.message = str(exc)
|
||||||
except TransferError as exc:
|
except TransferError as exc:
|
||||||
@@ -170,34 +192,38 @@ class Engine:
|
|||||||
def _execute(
|
def _execute(
|
||||||
self,
|
self,
|
||||||
rule: RuleSnapshot,
|
rule: RuleSnapshot,
|
||||||
src_root: Path,
|
src: Source,
|
||||||
dest: Destination,
|
dest: Destination,
|
||||||
token: CancelToken,
|
token: CancelToken,
|
||||||
result: RunResult,
|
result: RunResult,
|
||||||
dry_run: bool,
|
dry_run: bool,
|
||||||
quarantine_root: Path | None,
|
quarantine_rel: str | None,
|
||||||
|
archive: Destination | None = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
if not src_root.is_dir():
|
src.check_root()
|
||||||
raise RunAborted(f"Quellordner {src_root} existiert nicht")
|
|
||||||
if not dry_run:
|
if not dry_run:
|
||||||
dest.prepare()
|
dest.prepare()
|
||||||
|
if archive is not None:
|
||||||
|
archive.prepare()
|
||||||
token.check()
|
token.check()
|
||||||
|
|
||||||
exclude: set[Path] = set()
|
exclude = frozenset({quarantine_rel}) if quarantine_rel else frozenset()
|
||||||
if quarantine_root is not None:
|
if isinstance(dest, LocalDestination) and isinstance(src, LocalSource):
|
||||||
exclude.add(quarantine_root.resolve())
|
src.exclude_paths.add(dest.root.resolve()) # Ziel innerhalb der Quelle auslassen
|
||||||
if isinstance(dest, LocalDestination):
|
|
||||||
exclude.add(dest.root.resolve()) # Ziel innerhalb der Quelle nie erneut scannen
|
|
||||||
matcher = Matcher(rule.pattern, rule.pattern_type)
|
matcher = Matcher(rule.pattern, rule.pattern_type)
|
||||||
candidates = list(scan(src_root, matcher, rule.recursive, frozenset(exclude)))
|
candidates = src.scan(matcher, rule.recursive, exclude)
|
||||||
result.seen = len(candidates)
|
result.seen = len(candidates)
|
||||||
token.check()
|
token.check()
|
||||||
|
|
||||||
# Kopier-Buchführung (Fallstrick 11). Gilt auch für „verschieben“, falls
|
# Kopier-Buchführung (Fallstrick 11). Gilt auch für „verschieben“, falls
|
||||||
# dort einmal die Quelle nicht gelöscht werden konnte.
|
# dort einmal die Quelle nicht gelöscht werden konnte.
|
||||||
done = self.ledger.copied_keys(rule.id)
|
done = self.ledger.copied_keys(rule.id)
|
||||||
|
archived = self.ledger.archived_keys(rule.id) if archive is not None else set()
|
||||||
if not dry_run:
|
if not dry_run:
|
||||||
self.ledger.prune_copied(rule.id, {c.relpath for c in candidates})
|
present = {c.relpath for c in candidates}
|
||||||
|
self.ledger.prune_copied(rule.id, present)
|
||||||
|
if archive is not None:
|
||||||
|
self.ledger.prune_archived(rule.id, present)
|
||||||
fresh = [c for c in candidates if (c.relpath, c.state.size, c.state.mtime_ns) not in done]
|
fresh = [c for c in candidates if (c.relpath, c.state.size, c.state.mtime_ns) not in done]
|
||||||
unchanged = len(candidates) - len(fresh)
|
unchanged = len(candidates) - len(fresh)
|
||||||
|
|
||||||
@@ -227,13 +253,13 @@ class Engine:
|
|||||||
|
|
||||||
# Stabilität (Fallstrick 1): zwei frische Messungen (öffnen + fstat, nicht
|
# Stabilität (Fallstrick 1): zwei frische Messungen (öffnen + fstat, nicht
|
||||||
# der Verzeichnis-Cache) im Abstand des Stabilitätsfensters.
|
# der Verzeichnis-Cache) im Abstand des Stabilitätsfensters.
|
||||||
first: dict[str, FileState | None] = {c.relpath: current_state(c.path) for c in pending}
|
first: dict[str, FileState | None] = {c.relpath: src.state(c.relpath) for c in pending}
|
||||||
if pending and rule.stability_seconds > 0:
|
if pending and rule.stability_seconds > 0:
|
||||||
self._sleep(rule.stability_seconds)
|
self._sleep(rule.stability_seconds)
|
||||||
token.check()
|
token.check()
|
||||||
stable: list[Candidate] = []
|
stable: list[Candidate] = []
|
||||||
for c in pending:
|
for c in pending:
|
||||||
before, st = first[c.relpath], current_state(c.path)
|
before, st = first[c.relpath], src.state(c.relpath)
|
||||||
if before is None or st is None:
|
if before is None or st is None:
|
||||||
continue # inzwischen verschwunden – nichts zu tun
|
continue # inzwischen verschwunden – nichts zu tun
|
||||||
if st != before or st == BUSY:
|
if st != before or st == BUSY:
|
||||||
@@ -255,7 +281,11 @@ class Engine:
|
|||||||
failures = self.ledger.failures(rule.id)
|
failures = self.ledger.failures(rule.id)
|
||||||
for c in stable:
|
for c in stable:
|
||||||
token.check()
|
token.check()
|
||||||
result.add(self._process(rule, c, dest, token, dry_run, quarantine_root, failures))
|
result.add(
|
||||||
|
self._process(
|
||||||
|
rule, c, src, dest, token, dry_run, quarantine_rel, failures, archive, archived
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
if unchanged:
|
if unchanged:
|
||||||
result.message = f"{unchanged} bereits übertragen"
|
result.message = f"{unchanged} bereits übertragen"
|
||||||
@@ -263,9 +293,9 @@ class Engine:
|
|||||||
# -- einzelne Datei ------------------------------------------------------------------
|
# -- einzelne Datei ------------------------------------------------------------------
|
||||||
|
|
||||||
def _meta(
|
def _meta(
|
||||||
self, rule: RuleSnapshot, path: Path, dest: Destination
|
self, rule: RuleSnapshot, src: Source, rel: str, dest: Destination
|
||||||
) -> tuple[MetaOptions, list[str]]:
|
) -> tuple[MetaOptions, list[str]]:
|
||||||
st = os.stat(path)
|
st = src.stat(rel)
|
||||||
notes: list[str] = []
|
notes: list[str] = []
|
||||||
owner = rule.set_owner
|
owner = rule.set_owner
|
||||||
if owner is not None:
|
if owner is not None:
|
||||||
@@ -280,25 +310,32 @@ class Engine:
|
|||||||
if self._warn_once(f"{rule.id}:nfs-chown"):
|
if self._warn_once(f"{rule.id}:nfs-chown"):
|
||||||
reason = "Root-Squash aktiv" if rule.dst_root_squash else "Schreibtest fehlt"
|
reason = "Root-Squash aktiv" if rule.dst_root_squash else "Schreibtest fehlt"
|
||||||
notes.append(f"NFS-Ziel: chown übersprungen ({reason})")
|
notes.append(f"NFS-Ziel: chown übersprungen ({reason})")
|
||||||
mode = rule.set_mode if rule.set_mode is not None else stat.S_IMODE(st.st_mode)
|
mode = rule.set_mode if rule.set_mode is not None else st.mode
|
||||||
return MetaOptions(st.st_atime_ns, st.st_mtime_ns, mode, owner), notes
|
return MetaOptions(st.atime_ns, st.mtime_ns, mode, owner), notes
|
||||||
|
|
||||||
def _process(
|
def _process(
|
||||||
self,
|
self,
|
||||||
rule: RuleSnapshot,
|
rule: RuleSnapshot,
|
||||||
c: Candidate,
|
c: Candidate,
|
||||||
|
src: Source,
|
||||||
dest: Destination,
|
dest: Destination,
|
||||||
token: CancelToken,
|
token: CancelToken,
|
||||||
dry_run: bool,
|
dry_run: bool,
|
||||||
quarantine_root: Path | None,
|
quarantine_rel: str | None,
|
||||||
failures: dict[str, FailureInfo],
|
failures: dict[str, FailureInfo],
|
||||||
|
archive: Destination | None = None,
|
||||||
|
archived: set[tuple[str, int, int]] | None = None,
|
||||||
) -> ItemResult:
|
) -> ItemResult:
|
||||||
size = c.state.size
|
size = c.state.size
|
||||||
|
arch_rel = self._archive_rel(c.relpath, rule.archive_layout)
|
||||||
|
need_archive = archive is not None and (
|
||||||
|
(c.relpath, size, c.state.mtime_ns) not in (archived or set())
|
||||||
|
)
|
||||||
failure = failures.get(c.relpath)
|
failure = failures.get(c.relpath)
|
||||||
|
|
||||||
# Quarantäne für wiederholt scheiternde Dateien
|
# Quarantäne für wiederholt scheiternde Dateien
|
||||||
if (
|
if (
|
||||||
quarantine_root is not None
|
quarantine_rel is not None
|
||||||
and failure is not None
|
and failure is not None
|
||||||
and failure.attempts >= rule.quarantine_after
|
and failure.attempts >= rule.quarantine_after
|
||||||
and (failure.size, failure.mtime_ns) == (c.state.size, c.state.mtime_ns)
|
and (failure.size, failure.mtime_ns) == (c.state.size, c.state.mtime_ns)
|
||||||
@@ -307,11 +344,11 @@ class Engine:
|
|||||||
return ItemResult(
|
return ItemResult(
|
||||||
c.relpath,
|
c.relpath,
|
||||||
ItemStatus.PLANNED,
|
ItemStatus.PLANNED,
|
||||||
str(quarantine_root / c.relpath),
|
src.describe_root_rel(posixpath.join(quarantine_rel, c.relpath)),
|
||||||
size,
|
size,
|
||||||
"würde in Quarantäne verschoben",
|
"würde in Quarantäne verschoben",
|
||||||
)
|
)
|
||||||
return self._quarantine(rule, c, quarantine_root, failure.attempts)
|
return self._quarantine(rule, c, src, quarantine_rel, failure.attempts)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
planned = dest.plan(c.relpath, rule.on_collision)
|
planned = dest.plan(c.relpath, rule.on_collision)
|
||||||
@@ -332,9 +369,14 @@ class Engine:
|
|||||||
else None
|
else None
|
||||||
)
|
)
|
||||||
verb = "kopieren" if rule.action is RuleAction.COPY else "verschieben"
|
verb = "kopieren" if rule.action is RuleAction.COPY else "verschieben"
|
||||||
return ItemResult(
|
msg = msg or f"würde {verb}"
|
||||||
c.relpath, ItemStatus.PLANNED, dest.describe(planned), size, msg or f"würde {verb}"
|
if archive is not None:
|
||||||
)
|
if need_archive:
|
||||||
|
arch_planned = archive.plan(arch_rel, CollisionPolicy.RENAME)
|
||||||
|
msg += f"; Archiv: {archive.describe(arch_planned or arch_rel)}"
|
||||||
|
else:
|
||||||
|
msg += "; bereits archiviert"
|
||||||
|
return ItemResult(c.relpath, ItemStatus.PLANNED, dest.describe(planned), size, msg)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
free = dest.free_bytes()
|
free = dest.free_bytes()
|
||||||
@@ -350,17 +392,67 @@ class Engine:
|
|||||||
f"benötigt {_mb(int(size * SPACE_RESERVE))})",
|
f"benötigt {_mb(int(size * SPACE_RESERVE))})",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
if need_archive and archive is not None:
|
||||||
|
try:
|
||||||
|
afree = archive.free_bytes()
|
||||||
|
except OSError:
|
||||||
|
afree = None
|
||||||
|
if afree is not None and afree < size * SPACE_RESERVE:
|
||||||
|
return ItemResult(
|
||||||
|
c.relpath,
|
||||||
|
ItemStatus.FAILED,
|
||||||
|
archive.describe(arch_rel),
|
||||||
|
size,
|
||||||
|
f"zu wenig Platz im Archiv (frei {_mb(afree)}, "
|
||||||
|
f"benötigt {_mb(int(size * SPACE_RESERVE))})",
|
||||||
|
)
|
||||||
|
|
||||||
def verify() -> None:
|
def verify() -> None:
|
||||||
if current_state(c.path) != c.state:
|
if src.state(c.relpath) != c.state:
|
||||||
raise SourceChanged("Quelle hat sich während der Übertragung geändert")
|
raise SourceChanged("Quelle hat sich während der Übertragung geändert")
|
||||||
|
|
||||||
|
# Archiv ZUERST: scheitert es, ist noch nichts ans Ziel (z. B. Paperless)
|
||||||
|
# geliefert. Scheitert danach das Ziel, verhindert die Buchführung eine
|
||||||
|
# zweite Archivkopie beim nächsten Versuch.
|
||||||
|
archive_note: str | None = None
|
||||||
|
if need_archive and archive is not None:
|
||||||
|
try:
|
||||||
|
meta, _ = self._meta(rule, src, c.relpath, dest)
|
||||||
|
ares = archive.commit(
|
||||||
|
_SourceFile(src, c.relpath),
|
||||||
|
size,
|
||||||
|
arch_rel,
|
||||||
|
CollisionPolicy.RENAME,
|
||||||
|
MetaOptions(meta.atime_ns, meta.mtime_ns, meta.mode, None),
|
||||||
|
verify,
|
||||||
|
token,
|
||||||
|
)
|
||||||
|
except CollisionSkip: # kann bei RENAME nicht auftreten
|
||||||
|
return ItemResult(c.relpath, ItemStatus.SKIPPED, None, size, "Archiv belegt")
|
||||||
|
except SourceChanged as exc:
|
||||||
|
return ItemResult(c.relpath, ItemStatus.SKIPPED, None, size, str(exc))
|
||||||
|
except (TransferError, OSError) as exc:
|
||||||
|
return self._fail(rule, c, exc, dry_run, prefix="Archiv: ")
|
||||||
|
archive_path = archive.describe(ares.final_rel)
|
||||||
|
self.ledger.record_archive(rule.id, c.relpath, size, c.state.mtime_ns, archive_path)
|
||||||
|
archive_note = f"archiviert: {archive_path}"
|
||||||
|
|
||||||
try:
|
try:
|
||||||
meta, notes = self._meta(rule, c.path, dest)
|
meta, notes = self._meta(rule, src, c.relpath, dest)
|
||||||
res = None
|
res = None
|
||||||
if rule.action is RuleAction.MOVE:
|
local = src.local_path(c.relpath)
|
||||||
res = dest.move_same_device(c.path, c.relpath, rule.on_collision, meta)
|
if rule.action is RuleAction.MOVE and local is not None:
|
||||||
|
res = dest.move_same_device(local, c.relpath, rule.on_collision, meta)
|
||||||
if res is None:
|
if res is None:
|
||||||
res = dest.commit(c.path, size, c.relpath, rule.on_collision, meta, verify, token)
|
res = dest.commit(
|
||||||
|
_SourceFile(src, c.relpath),
|
||||||
|
size,
|
||||||
|
c.relpath,
|
||||||
|
rule.on_collision,
|
||||||
|
meta,
|
||||||
|
verify,
|
||||||
|
token,
|
||||||
|
)
|
||||||
except CollisionSkip:
|
except CollisionSkip:
|
||||||
return ItemResult(
|
return ItemResult(
|
||||||
c.relpath,
|
c.relpath,
|
||||||
@@ -375,6 +467,8 @@ class Engine:
|
|||||||
return self._fail(rule, c, exc, dry_run)
|
return self._fail(rule, c, exc, dry_run)
|
||||||
|
|
||||||
notes += res.notes
|
notes += res.notes
|
||||||
|
if archive_note:
|
||||||
|
notes.insert(0, archive_note)
|
||||||
target = dest.describe(res.final_rel)
|
target = dest.describe(res.final_rel)
|
||||||
if c.relpath in failures:
|
if c.relpath in failures:
|
||||||
self.ledger.clear_failure(rule.id, c.relpath)
|
self.ledger.clear_failure(rule.id, c.relpath)
|
||||||
@@ -385,7 +479,7 @@ class Engine:
|
|||||||
|
|
||||||
if not res.source_consumed:
|
if not res.source_consumed:
|
||||||
try:
|
try:
|
||||||
os.unlink(c.path)
|
src.delete(c.relpath)
|
||||||
except OSError as exc:
|
except OSError as exc:
|
||||||
# Ziel ist vollständig – Buchführung verhindert Duplikate im nächsten Lauf.
|
# Ziel ist vollständig – Buchführung verhindert Duplikate im nächsten Lauf.
|
||||||
self.ledger.record_copy(rule.id, c.relpath, size, c.state.mtime_ns, target)
|
self.ledger.record_copy(rule.id, c.relpath, size, c.state.mtime_ns, target)
|
||||||
@@ -399,10 +493,12 @@ class Engine:
|
|||||||
)
|
)
|
||||||
return ItemResult(c.relpath, ItemStatus.MOVED, target, size, "; ".join(notes) or None)
|
return ItemResult(c.relpath, ItemStatus.MOVED, target, size, "; ".join(notes) or None)
|
||||||
|
|
||||||
def _fail(self, rule: RuleSnapshot, c: Candidate, exc: Exception, dry_run: bool) -> ItemResult:
|
def _fail(
|
||||||
|
self, rule: RuleSnapshot, c: Candidate, exc: Exception, dry_run: bool, prefix: str = ""
|
||||||
|
) -> ItemResult:
|
||||||
if isinstance(exc, OSError) and exc.errno in MOUNT_ERRNOS:
|
if isinstance(exc, OSError) and exc.errno in MOUNT_ERRNOS:
|
||||||
raise RunAborted(f"Anbindung gestört: {_describe_os_error(exc)}") from exc
|
raise RunAborted(f"{prefix}Anbindung gestört: {_describe_os_error(exc)}") from exc
|
||||||
msg = _describe_os_error(exc) if isinstance(exc, OSError) else str(exc)
|
msg = prefix + (_describe_os_error(exc) if isinstance(exc, OSError) else str(exc))
|
||||||
if not dry_run:
|
if not dry_run:
|
||||||
attempts = self.ledger.record_failure(
|
attempts = self.ledger.record_failure(
|
||||||
rule.id, c.relpath, c.state.size, c.state.mtime_ns, msg
|
rule.id, c.relpath, c.state.size, c.state.mtime_ns, msg
|
||||||
@@ -411,16 +507,25 @@ class Engine:
|
|||||||
event(log, "file.failed", level=30, rule=rule.name, file=c.relpath, error=msg)
|
event(log, "file.failed", level=30, rule=rule.name, file=c.relpath, error=msg)
|
||||||
return ItemResult(c.relpath, ItemStatus.FAILED, None, c.state.size, msg)
|
return ItemResult(c.relpath, ItemStatus.FAILED, None, c.state.size, msg)
|
||||||
|
|
||||||
|
def _archive_rel(self, relpath: str, layout: str) -> str:
|
||||||
|
if layout == "date":
|
||||||
|
t = datetime.fromtimestamp(self._clock())
|
||||||
|
return f"{t:%Y}/{t:%m}/{relpath}"
|
||||||
|
return relpath
|
||||||
|
|
||||||
def _quarantine(
|
def _quarantine(
|
||||||
self, rule: RuleSnapshot, c: Candidate, root: Path, attempts: int
|
self, rule: RuleSnapshot, c: Candidate, src: Source, q_rel: str, attempts: int
|
||||||
) -> ItemResult:
|
) -> ItemResult:
|
||||||
|
target = ""
|
||||||
try:
|
try:
|
||||||
for cand in collision_candidates(c.relpath):
|
for cand in collision_candidates(c.relpath):
|
||||||
target = root / cand
|
target = posixpath.join(q_rel, cand)
|
||||||
if os.path.lexists(target):
|
if src.exists_root_rel(target):
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
src.move_to(c.relpath, target)
|
||||||
|
except FileExistsError:
|
||||||
continue
|
continue
|
||||||
target.parent.mkdir(mode=0o755, parents=True, exist_ok=True)
|
|
||||||
os.rename(c.path, target)
|
|
||||||
break
|
break
|
||||||
else:
|
else:
|
||||||
raise TransferError("kein freier Name im Quarantäneordner")
|
raise TransferError("kein freier Name im Quarantäneordner")
|
||||||
@@ -434,7 +539,7 @@ class Engine:
|
|||||||
return ItemResult(
|
return ItemResult(
|
||||||
c.relpath,
|
c.relpath,
|
||||||
ItemStatus.QUARANTINED,
|
ItemStatus.QUARANTINED,
|
||||||
str(target),
|
src.describe_root_rel(target),
|
||||||
c.state.size,
|
c.state.size,
|
||||||
f"nach {attempts} Fehlversuchen in Quarantäne verschoben",
|
f"nach {attempts} Fehlversuchen in Quarantäne verschoben",
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ from sqlalchemy import delete, select
|
|||||||
from sqlalchemy.orm import Session, sessionmaker
|
from sqlalchemy.orm import Session, sessionmaker
|
||||||
|
|
||||||
from netmover.db import session_scope
|
from netmover.db import session_scope
|
||||||
from netmover.models import CopiedFile, FileFailure, utcnow
|
from netmover.models import ArchivedFile, CopiedFile, FileFailure, utcnow
|
||||||
|
|
||||||
CopyKey = tuple[str, int, int] # (relpath, size, mtime_ns)
|
CopyKey = tuple[str, int, int] # (relpath, size, mtime_ns)
|
||||||
|
|
||||||
@@ -72,6 +72,53 @@ class Ledger:
|
|||||||
s.execute(delete(CopiedFile).where(CopiedFile.id.in_(stale)))
|
s.execute(delete(CopiedFile).where(CopiedFile.id.in_(stale)))
|
||||||
return len(stale)
|
return len(stale)
|
||||||
|
|
||||||
|
# -- Archiv ------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def archived_keys(self, rule_id: int) -> set[CopyKey]:
|
||||||
|
with session_scope(self.session_factory) as s:
|
||||||
|
rows = s.execute(
|
||||||
|
select(ArchivedFile.src_relpath, ArchivedFile.size, ArchivedFile.mtime_ns).where(
|
||||||
|
ArchivedFile.rule_id == rule_id
|
||||||
|
)
|
||||||
|
).all()
|
||||||
|
return {(r[0], r[1], r[2]) for r in rows}
|
||||||
|
|
||||||
|
def record_archive(
|
||||||
|
self, rule_id: int, relpath: str, size: int, mtime_ns: int, archive_path: str
|
||||||
|
) -> None:
|
||||||
|
with session_scope(self.session_factory) as s:
|
||||||
|
exists = s.scalar(
|
||||||
|
select(ArchivedFile.id).where(
|
||||||
|
ArchivedFile.rule_id == rule_id,
|
||||||
|
ArchivedFile.src_relpath == relpath,
|
||||||
|
ArchivedFile.size == size,
|
||||||
|
ArchivedFile.mtime_ns == mtime_ns,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if exists is None:
|
||||||
|
s.add(
|
||||||
|
ArchivedFile(
|
||||||
|
rule_id=rule_id,
|
||||||
|
src_relpath=relpath,
|
||||||
|
size=size,
|
||||||
|
mtime_ns=mtime_ns,
|
||||||
|
archive_path=archive_path,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
def prune_archived(self, rule_id: int, present: set[str]) -> int:
|
||||||
|
"""Wie :meth:`prune_copied`: nur nach einem vollständigen Scan aufrufen."""
|
||||||
|
with session_scope(self.session_factory) as s:
|
||||||
|
rows = s.execute(
|
||||||
|
select(ArchivedFile.id, ArchivedFile.src_relpath).where(
|
||||||
|
ArchivedFile.rule_id == rule_id
|
||||||
|
)
|
||||||
|
).all()
|
||||||
|
stale = [r[0] for r in rows if r[1] not in present]
|
||||||
|
if stale:
|
||||||
|
s.execute(delete(ArchivedFile).where(ArchivedFile.id.in_(stale)))
|
||||||
|
return len(stale)
|
||||||
|
|
||||||
def failures(self, rule_id: int) -> dict[str, FailureInfo]:
|
def failures(self, rule_id: int) -> dict[str, FailureInfo]:
|
||||||
with session_scope(self.session_factory) as s:
|
with session_scope(self.session_factory) as s:
|
||||||
rows = s.scalars(select(FileFailure).where(FileFailure.rule_id == rule_id)).all()
|
rows = s.scalars(select(FileFailure).where(FileFailure.rule_id == rule_id)).all()
|
||||||
|
|||||||
@@ -34,6 +34,7 @@ from netmover.mover.destinations import (
|
|||||||
)
|
)
|
||||||
from netmover.mover.engine import Engine, RuleSnapshot, RunResult
|
from netmover.mover.engine import Engine, RuleSnapshot, RunResult
|
||||||
from netmover.mover.ledger import Ledger
|
from netmover.mover.ledger import Ledger
|
||||||
|
from netmover.mover.sources import LocalSource, SftpSource, Source
|
||||||
from netmover.timeouts import CancelToken, OperationTimeout, run_with_watchdog
|
from netmover.timeouts import CancelToken, OperationTimeout, run_with_watchdog
|
||||||
from netmover.validation import ValidationError, safe_abspath, safe_relpath
|
from netmover.validation import ValidationError, safe_abspath, safe_relpath
|
||||||
|
|
||||||
@@ -46,6 +47,7 @@ class RulePlan:
|
|||||||
name: str
|
name: str
|
||||||
src_mount_id: int
|
src_mount_id: int
|
||||||
src_type: MountType
|
src_type: MountType
|
||||||
|
src_name: str
|
||||||
src_subpath: str
|
src_subpath: str
|
||||||
dst_mount_id: int | None
|
dst_mount_id: int | None
|
||||||
dst_type: MountType | None
|
dst_type: MountType | None
|
||||||
@@ -54,6 +56,11 @@ class RulePlan:
|
|||||||
quarantine_path: str | None
|
quarantine_path: str | None
|
||||||
op_timeout: int
|
op_timeout: int
|
||||||
dry_run: bool
|
dry_run: bool
|
||||||
|
archive_kind: str = "none"
|
||||||
|
archive_mount_id: int | None = None
|
||||||
|
archive_type: MountType | None = None
|
||||||
|
archive_name: str | None = None
|
||||||
|
archive_path: str = ""
|
||||||
|
|
||||||
|
|
||||||
def parse_owner(value: str | None) -> tuple[int, int] | None:
|
def parse_owner(value: str | None) -> tuple[int, int] | None:
|
||||||
@@ -90,12 +97,14 @@ def load_plan(session: Session, rule_id: int) -> RulePlan:
|
|||||||
set_mode=parse_mode(r.set_mode),
|
set_mode=parse_mode(r.set_mode),
|
||||||
quarantine_after=r.quarantine_after,
|
quarantine_after=r.quarantine_after,
|
||||||
dst_root_squash=dst.root_squash if dst is not None else None,
|
dst_root_squash=dst.root_squash if dst is not None else None,
|
||||||
|
archive_layout=r.archive_layout.value,
|
||||||
)
|
)
|
||||||
return RulePlan(
|
return RulePlan(
|
||||||
snapshot=snap,
|
snapshot=snap,
|
||||||
name=r.name,
|
name=r.name,
|
||||||
src_mount_id=r.src_mount_id,
|
src_mount_id=r.src_mount_id,
|
||||||
src_type=r.src_mount.type,
|
src_type=r.src_mount.type,
|
||||||
|
src_name=r.src_mount.name,
|
||||||
src_subpath=safe_relpath("src_subpath", r.src_subpath),
|
src_subpath=safe_relpath("src_subpath", r.src_subpath),
|
||||||
dst_mount_id=r.dst_mount_id,
|
dst_mount_id=r.dst_mount_id,
|
||||||
dst_type=dst.type if dst is not None else None,
|
dst_type=dst.type if dst is not None else None,
|
||||||
@@ -106,6 +115,11 @@ def load_plan(session: Session, rule_id: int) -> RulePlan:
|
|||||||
else None,
|
else None,
|
||||||
op_timeout=r.op_timeout_seconds,
|
op_timeout=r.op_timeout_seconds,
|
||||||
dry_run=r.dry_run,
|
dry_run=r.dry_run,
|
||||||
|
archive_kind=r.archive_kind.value,
|
||||||
|
archive_mount_id=r.archive_mount_id if r.archive_kind.value == "mount" else None,
|
||||||
|
archive_type=r.archive_mount.type if r.archive_mount is not None else None,
|
||||||
|
archive_name=r.archive_mount.name if r.archive_mount is not None else None,
|
||||||
|
archive_path=r.archive_path,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -166,10 +180,6 @@ class RuleRunner:
|
|||||||
rule_id: int,
|
rule_id: int,
|
||||||
handed_over: list[bool],
|
handed_over: list[bool],
|
||||||
) -> RunResult:
|
) -> RunResult:
|
||||||
if not plan.src_type.is_kernel_mount:
|
|
||||||
return RunResult(
|
|
||||||
RunStatus.ERROR, dry, message="SFTP-Anbindungen sind nur als Ziel möglich"
|
|
||||||
)
|
|
||||||
try:
|
try:
|
||||||
src_mp = self.mounts.check_alive(plan.src_mount_id)
|
src_mp = self.mounts.check_alive(plan.src_mount_id)
|
||||||
dst_mp = (
|
dst_mp = (
|
||||||
@@ -177,31 +187,64 @@ class RuleRunner:
|
|||||||
if plan.dst_mount_id is not None
|
if plan.dst_mount_id is not None
|
||||||
else None
|
else None
|
||||||
)
|
)
|
||||||
|
arch_mp = (
|
||||||
|
self.mounts.check_alive(plan.archive_mount_id)
|
||||||
|
if plan.archive_mount_id is not None
|
||||||
|
else None
|
||||||
|
)
|
||||||
except MountUnavailable as exc:
|
except MountUnavailable as exc:
|
||||||
return RunResult(RunStatus.ERROR, dry, message=str(exc), unavailable=True)
|
return RunResult(RunStatus.ERROR, dry, message=str(exc), unavailable=True)
|
||||||
if src_mp is None: # nur bei SFTP, oben bereits ausgeschlossen
|
|
||||||
return RunResult(RunStatus.ERROR, dry, message="Quelle ist kein Kernel-Mount")
|
|
||||||
src_root = src_mp / plan.src_subpath if plan.src_subpath else src_mp
|
|
||||||
quarantine_root = src_mp / plan.quarantine_path if plan.quarantine_path else None
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
make_dest = self._dest_factory(plan, dst_mp, src_root)
|
make_source = self._source_factory(plan, src_mp)
|
||||||
|
src_root = src_mp / plan.src_subpath if src_mp and plan.src_subpath else src_mp
|
||||||
|
make_dest = self._target(
|
||||||
|
"dst_path",
|
||||||
|
plan.dst_mount_id,
|
||||||
|
plan.dst_type,
|
||||||
|
plan.dst_name,
|
||||||
|
plan.dst_path,
|
||||||
|
dst_mp,
|
||||||
|
src_root,
|
||||||
|
)
|
||||||
|
make_archive = (
|
||||||
|
self._target(
|
||||||
|
"archive_path",
|
||||||
|
plan.archive_mount_id,
|
||||||
|
plan.archive_type,
|
||||||
|
plan.archive_name,
|
||||||
|
plan.archive_path,
|
||||||
|
arch_mp,
|
||||||
|
src_root,
|
||||||
|
)
|
||||||
|
if plan.archive_kind != "none"
|
||||||
|
else None
|
||||||
|
)
|
||||||
except (ValidationError, TransferError, SftpError) as exc:
|
except (ValidationError, TransferError, SftpError) as exc:
|
||||||
return RunResult(RunStatus.ERROR, dry, message=str(exc))
|
return RunResult(RunStatus.ERROR, dry, message=str(exc))
|
||||||
|
|
||||||
def work(token: CancelToken) -> RunResult:
|
def work(token: CancelToken) -> RunResult:
|
||||||
dest = make_dest()
|
opened: list[Source | Destination] = []
|
||||||
try:
|
try:
|
||||||
|
source = make_source()
|
||||||
|
opened.append(source)
|
||||||
|
dest = make_dest()
|
||||||
|
opened.append(dest)
|
||||||
|
archive = make_archive() if make_archive is not None else None
|
||||||
|
if archive is not None:
|
||||||
|
opened.append(archive)
|
||||||
return self.engine.execute(
|
return self.engine.execute(
|
||||||
plan.snapshot,
|
plan.snapshot,
|
||||||
src_root,
|
source,
|
||||||
dest,
|
dest,
|
||||||
token,
|
token,
|
||||||
dry_run=dry,
|
dry_run=dry,
|
||||||
quarantine_root=quarantine_root,
|
quarantine_rel=plan.quarantine_path,
|
||||||
|
archive=archive,
|
||||||
)
|
)
|
||||||
finally:
|
finally:
|
||||||
dest.close()
|
for item in reversed(opened):
|
||||||
|
item.close()
|
||||||
|
|
||||||
def finish() -> None:
|
def finish() -> None:
|
||||||
self._started.pop(rule_id, None)
|
self._started.pop(rule_id, None)
|
||||||
@@ -230,41 +273,68 @@ class RuleRunner:
|
|||||||
paths = {path.absolute(), path.resolve()}
|
paths = {path.absolute(), path.resolve()}
|
||||||
return any(p == b or b in p.parents for p in paths for b in bases)
|
return any(p == b or b in p.parents for p in paths for b in bases)
|
||||||
|
|
||||||
def _dest_factory(
|
def _source_factory(self, plan: RulePlan, src_mp: Path | None) -> Callable[[], Source]:
|
||||||
self, plan: RulePlan, dst_mp: Path | None, src_root: Path
|
if plan.src_type is MountType.SFTP:
|
||||||
|
scfg = self.mounts.sftp_config_for(plan.src_mount_id)
|
||||||
|
label = plan.src_name or "sftp"
|
||||||
|
|
||||||
|
def make_sftp() -> Source:
|
||||||
|
return SftpSource(connect(scfg), scfg.base_path, plan.src_subpath, label=label)
|
||||||
|
|
||||||
|
return make_sftp
|
||||||
|
if src_mp is None:
|
||||||
|
raise TransferError("Quell-Anbindung ist nicht eingehängt")
|
||||||
|
root = src_mp
|
||||||
|
|
||||||
|
def make_local() -> Source:
|
||||||
|
return LocalSource(root, plan.src_subpath)
|
||||||
|
|
||||||
|
return make_local
|
||||||
|
|
||||||
|
def _target(
|
||||||
|
self,
|
||||||
|
field: str,
|
||||||
|
mount_id: int | None,
|
||||||
|
mount_type: MountType | None,
|
||||||
|
mount_name: str | None,
|
||||||
|
path: str,
|
||||||
|
mp: Path | None,
|
||||||
|
src_root: Path | None,
|
||||||
) -> Callable[[], Destination]:
|
) -> Callable[[], Destination]:
|
||||||
if plan.dst_mount_id is None:
|
"""Ziel oder Archiv: lokaler Pfad (``mount_id`` None), SFTP oder Kernel-Mount."""
|
||||||
root = Path(safe_abspath("dst_path", plan.dst_path))
|
what = "Archiv" if field == "archive_path" else "Ziel"
|
||||||
|
if mount_id is None:
|
||||||
|
root = Path(safe_abspath(field, path))
|
||||||
if self._under_mount_base(root):
|
if self._under_mount_base(root):
|
||||||
base = self.mounts.mount_base
|
base = self.mounts.mount_base
|
||||||
raise ValidationError(
|
raise ValidationError(
|
||||||
"dst_path",
|
field,
|
||||||
f"Lokales Ziel liegt unter {base} – bitte die Anbindung als Ziel wählen, "
|
f"Lokales {what} liegt unter {base} – bitte die Anbindung als {what} wählen, "
|
||||||
"damit geprüft wird, ob sie eingehängt ist.",
|
"damit geprüft wird, ob sie eingehängt ist.",
|
||||||
)
|
)
|
||||||
kind = "local"
|
kind = "local"
|
||||||
local_root = root
|
local_root = root
|
||||||
create = False
|
create = False
|
||||||
elif plan.dst_type is MountType.SFTP:
|
elif mount_type is MountType.SFTP:
|
||||||
scfg = self.mounts.sftp_config_for(plan.dst_mount_id)
|
scfg = self.mounts.sftp_config_for(mount_id)
|
||||||
sub = safe_relpath("dst_path", plan.dst_path)
|
sub = safe_relpath(field, path)
|
||||||
remote = posixpath.join(scfg.base_path, sub) if sub else scfg.base_path
|
remote = posixpath.join(scfg.base_path, sub) if sub else scfg.base_path
|
||||||
label = plan.dst_name or "sftp"
|
label = mount_name or "sftp"
|
||||||
|
|
||||||
def make_sftp() -> Destination:
|
def make_sftp() -> Destination:
|
||||||
return SftpDestination(connect(scfg), remote, label=label)
|
return SftpDestination(connect(scfg), remote, label=label)
|
||||||
|
|
||||||
return make_sftp
|
return make_sftp
|
||||||
else:
|
else:
|
||||||
if dst_mp is None:
|
if mp is None:
|
||||||
raise TransferError("Ziel-Anbindung ist nicht eingehängt")
|
raise TransferError(f"{what}-Anbindung ist nicht eingehängt")
|
||||||
sub = safe_relpath("dst_path", plan.dst_path)
|
sub = safe_relpath(field, path)
|
||||||
local_root = dst_mp / sub if sub else dst_mp
|
local_root = mp / sub if sub else mp
|
||||||
kind = "cifs" if plan.dst_type is MountType.CIFS else "nfs"
|
kind = "cifs" if mount_type is MountType.CIFS else "nfs"
|
||||||
create = True
|
create = True
|
||||||
|
|
||||||
if local_root.resolve() == src_root.resolve():
|
if src_root is not None and local_root.resolve() == src_root.resolve():
|
||||||
raise ValidationError("dst_path", "Quelle und Ziel sind derselbe Ordner")
|
raise ValidationError(field, f"Quelle und {what} sind derselbe Ordner")
|
||||||
|
|
||||||
def make_local() -> Destination:
|
def make_local() -> Destination:
|
||||||
return LocalDestination(local_root, kind, create_root=create) # type: ignore[arg-type]
|
return LocalDestination(local_root, kind, create_root=create) # type: ignore[arg-type]
|
||||||
|
|||||||
@@ -34,7 +34,7 @@ class FileState:
|
|||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class Candidate:
|
class Candidate:
|
||||||
relpath: str # POSIX, relativ zum Quellordner
|
relpath: str # POSIX, relativ zum Quellordner
|
||||||
path: Path
|
path: Path | None # nur bei lokalen Quellen
|
||||||
state: FileState
|
state: FileState
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,311 @@
|
|||||||
|
"""Quellen: eingehängtes Verzeichnis (SMB/NFS) oder SFTP.
|
||||||
|
|
||||||
|
Die Engine greift nur über diese Schnittstelle auf die Quelle zu – scannen,
|
||||||
|
frischen Zustand messen, lesen, löschen, in die Quarantäne verschieben. So gelten
|
||||||
|
für SFTP-Quellen dieselben Garantien wie für eingehängte Freigaben.
|
||||||
|
|
||||||
|
Adressierung: ``root`` ist die Wurzel der Anbindung, ``subdir`` der Quell-
|
||||||
|
Unterordner der Regel. Kandidaten tragen Pfade relativ zu ``subdir`` (so landen sie
|
||||||
|
im Ziel und in der Buchführung); die Quarantäne liegt relativ zu ``root``.
|
||||||
|
|
||||||
|
SFTP-Besonderheit: Änderungszeiten kommen nur sekundengenau. Die Stabilitäts-
|
||||||
|
prüfung stützt sich dort vor allem auf die Größe – mit einem Fenster ab 2 s
|
||||||
|
unproblematisch.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import contextlib
|
||||||
|
import errno
|
||||||
|
import os
|
||||||
|
import posixpath
|
||||||
|
import socket
|
||||||
|
import stat
|
||||||
|
from abc import ABC, abstractmethod
|
||||||
|
from collections.abc import Iterator
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import IO, Any
|
||||||
|
|
||||||
|
import paramiko
|
||||||
|
|
||||||
|
from netmover.mounting.sftp import SftpSession
|
||||||
|
from netmover.mover.fsops import rename_noreplace
|
||||||
|
from netmover.mover.scan import (
|
||||||
|
IGNORED_DIRS,
|
||||||
|
Candidate,
|
||||||
|
FileState,
|
||||||
|
Matcher,
|
||||||
|
current_state,
|
||||||
|
is_ignored_name,
|
||||||
|
scan,
|
||||||
|
)
|
||||||
|
|
||||||
|
_NET_ERRORS = (paramiko.SSHException, EOFError, socket.timeout, ConnectionError)
|
||||||
|
|
||||||
|
|
||||||
|
class SourceMissing(Exception):
|
||||||
|
"""Quellordner existiert nicht (Lauf abbrechen, nichts anfassen)."""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class SourceStat:
|
||||||
|
size: int
|
||||||
|
mtime_ns: int
|
||||||
|
atime_ns: int
|
||||||
|
mode: int # Rechte-Bits
|
||||||
|
|
||||||
|
|
||||||
|
def _join(*parts: str) -> str:
|
||||||
|
return posixpath.join(*[p for p in parts if p]) if any(parts) else ""
|
||||||
|
|
||||||
|
|
||||||
|
@contextlib.contextmanager
|
||||||
|
def network_errors() -> Iterator[None]:
|
||||||
|
"""Verbindungsabbrüche bei SFTP als ``ENOTCONN`` melden – die Engine bricht den
|
||||||
|
Lauf dann ab (Anbindung gestört), statt jede Datei einzeln scheitern zu lassen."""
|
||||||
|
try:
|
||||||
|
yield
|
||||||
|
except _NET_ERRORS as exc:
|
||||||
|
raise OSError(errno.ENOTCONN, f"SFTP-Verbindung unterbrochen: {exc}") from exc
|
||||||
|
|
||||||
|
|
||||||
|
class Source(ABC):
|
||||||
|
kind: str
|
||||||
|
|
||||||
|
def __init__(self, subdir: str = "") -> None:
|
||||||
|
self.subdir = subdir.strip("/")
|
||||||
|
|
||||||
|
def full(self, rel: str) -> str:
|
||||||
|
"""Pfad relativ zur Wurzel der Anbindung."""
|
||||||
|
return _join(self.subdir, rel)
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def describe(self, rel: str = "") -> str: ...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def check_root(self) -> None:
|
||||||
|
"""Wirft :class:`SourceMissing`, wenn der Quellordner fehlt."""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def scan(
|
||||||
|
self, matcher: Matcher, recursive: bool, exclude: frozenset[str] = frozenset()
|
||||||
|
) -> list[Candidate]:
|
||||||
|
"""``exclude``: Verzeichnisse relativ zur Wurzel der Anbindung."""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def state(self, rel: str) -> FileState | None:
|
||||||
|
"""Frischer Zustand; ``None`` = verschwunden, ``BUSY`` = gerade gesperrt."""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def stat(self, rel: str) -> SourceStat: ...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def open(self, rel: str) -> contextlib.AbstractContextManager[IO[bytes]]: ...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def delete(self, rel: str) -> None: ...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def exists_root_rel(self, root_rel: str) -> bool: ...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def describe_root_rel(self, root_rel: str) -> str: ...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def move_to(self, rel: str, root_rel_target: str) -> None:
|
||||||
|
"""Innerhalb der Anbindung verschieben, ohne zu überschreiben
|
||||||
|
(``FileExistsError``, wenn das Ziel existiert)."""
|
||||||
|
|
||||||
|
def local_path(self, rel: str) -> Path | None:
|
||||||
|
"""Lokaler Pfad, falls vorhanden (für den Schnellweg ohne Kopie)."""
|
||||||
|
return None
|
||||||
|
|
||||||
|
def close(self) -> None: # noqa: B027 – absichtlich optional
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class LocalSource(Source):
|
||||||
|
"""Eingehängte Freigabe (oder lokales Verzeichnis)."""
|
||||||
|
|
||||||
|
kind = "local"
|
||||||
|
|
||||||
|
def __init__(self, root: Path, subdir: str = "") -> None:
|
||||||
|
super().__init__(subdir)
|
||||||
|
self.root = root
|
||||||
|
self.base = root / self.subdir if self.subdir else root
|
||||||
|
# zusätzlich auszulassende Verzeichnisse, z. B. ein Ziel innerhalb der Quelle
|
||||||
|
self.exclude_paths: set[Path] = set()
|
||||||
|
|
||||||
|
def _p(self, rel: str) -> Path:
|
||||||
|
return self.base / rel
|
||||||
|
|
||||||
|
def describe(self, rel: str = "") -> str:
|
||||||
|
return str(self._p(rel)) if rel else str(self.base)
|
||||||
|
|
||||||
|
def check_root(self) -> None:
|
||||||
|
if not self.base.is_dir():
|
||||||
|
raise SourceMissing(f"Quellordner {self.base} existiert nicht")
|
||||||
|
|
||||||
|
def scan(
|
||||||
|
self, matcher: Matcher, recursive: bool, exclude: frozenset[str] = frozenset()
|
||||||
|
) -> list[Candidate]:
|
||||||
|
paths = {(self.root / e).resolve() for e in exclude} | set(self.exclude_paths)
|
||||||
|
return list(scan(self.base, matcher, recursive, frozenset(paths)))
|
||||||
|
|
||||||
|
def state(self, rel: str) -> FileState | None:
|
||||||
|
return current_state(self._p(rel))
|
||||||
|
|
||||||
|
def stat(self, rel: str) -> SourceStat:
|
||||||
|
st = os.stat(self._p(rel))
|
||||||
|
return SourceStat(st.st_size, st.st_mtime_ns, st.st_atime_ns, stat.S_IMODE(st.st_mode))
|
||||||
|
|
||||||
|
def open(self, rel: str) -> contextlib.AbstractContextManager[IO[bytes]]:
|
||||||
|
return self._p(rel).open("rb")
|
||||||
|
|
||||||
|
def delete(self, rel: str) -> None:
|
||||||
|
os.unlink(self._p(rel))
|
||||||
|
|
||||||
|
def exists_root_rel(self, root_rel: str) -> bool:
|
||||||
|
return os.path.lexists(self.root / root_rel)
|
||||||
|
|
||||||
|
def describe_root_rel(self, root_rel: str) -> str:
|
||||||
|
return str(self.root / root_rel)
|
||||||
|
|
||||||
|
def move_to(self, rel: str, root_rel_target: str) -> None:
|
||||||
|
target = self.root / root_rel_target
|
||||||
|
target.parent.mkdir(mode=0o755, parents=True, exist_ok=True)
|
||||||
|
if rename_noreplace(self._p(rel), target):
|
||||||
|
return
|
||||||
|
if os.path.lexists(target):
|
||||||
|
raise FileExistsError(errno.EEXIST, "exists", str(target))
|
||||||
|
os.rename(self._p(rel), target)
|
||||||
|
|
||||||
|
def local_path(self, rel: str) -> Path:
|
||||||
|
return self._p(rel)
|
||||||
|
|
||||||
|
|
||||||
|
class SftpSource(Source):
|
||||||
|
"""Quelle auf einem SFTP-Server (z. B. ein Server, der nur per SSH erreichbar ist)."""
|
||||||
|
|
||||||
|
kind = "sftp"
|
||||||
|
|
||||||
|
def __init__(self, session: SftpSession, base: str, subdir: str = "", label: str = ""):
|
||||||
|
super().__init__(subdir)
|
||||||
|
self.session = session
|
||||||
|
self.client = session.client
|
||||||
|
self.root = base.rstrip("/") or "/"
|
||||||
|
self.label = label or "sftp"
|
||||||
|
|
||||||
|
def _p(self, rel: str) -> str:
|
||||||
|
return posixpath.join(self.root, self.full(rel)) if self.full(rel) else self.root
|
||||||
|
|
||||||
|
def describe(self, rel: str = "") -> str:
|
||||||
|
return f"{self.label}:{self._p(rel)}"
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
self.session.close()
|
||||||
|
|
||||||
|
def check_root(self) -> None:
|
||||||
|
with network_errors():
|
||||||
|
try:
|
||||||
|
attrs = self.client.stat(self._p(""))
|
||||||
|
except FileNotFoundError:
|
||||||
|
raise SourceMissing(f"Quellordner {self.describe()} existiert nicht") from None
|
||||||
|
if attrs.st_mode is None or not stat.S_ISDIR(attrs.st_mode):
|
||||||
|
raise SourceMissing(f"{self.describe()} ist kein Verzeichnis")
|
||||||
|
|
||||||
|
def scan(
|
||||||
|
self, matcher: Matcher, recursive: bool, exclude: frozenset[str] = frozenset()
|
||||||
|
) -> list[Candidate]:
|
||||||
|
excluded = {e.strip("/") for e in exclude}
|
||||||
|
found: list[Candidate] = []
|
||||||
|
|
||||||
|
def walk(rel_dir: str) -> None:
|
||||||
|
with network_errors():
|
||||||
|
entries = sorted(
|
||||||
|
self.client.listdir_attr(self._p(rel_dir)), key=lambda a: a.filename
|
||||||
|
)
|
||||||
|
for a in entries:
|
||||||
|
name = a.filename
|
||||||
|
if name.startswith(".") or a.st_mode is None:
|
||||||
|
continue
|
||||||
|
rel = _join(rel_dir, name)
|
||||||
|
if stat.S_ISDIR(a.st_mode):
|
||||||
|
if recursive and name not in IGNORED_DIRS and self.full(rel) not in excluded:
|
||||||
|
walk(rel)
|
||||||
|
continue
|
||||||
|
if not stat.S_ISREG(a.st_mode) or is_ignored_name(name):
|
||||||
|
continue # Symlinks, Geräte … und unfertige Übertragungen
|
||||||
|
if matcher(rel):
|
||||||
|
found.append(Candidate(rel, None, _state(a)))
|
||||||
|
|
||||||
|
walk("")
|
||||||
|
return found
|
||||||
|
|
||||||
|
def state(self, rel: str) -> FileState | None:
|
||||||
|
with network_errors():
|
||||||
|
try:
|
||||||
|
a = self.client.lstat(self._p(rel))
|
||||||
|
except FileNotFoundError:
|
||||||
|
return None
|
||||||
|
if a.st_mode is None or not stat.S_ISREG(a.st_mode):
|
||||||
|
return None
|
||||||
|
return _state(a)
|
||||||
|
|
||||||
|
def stat(self, rel: str) -> SourceStat:
|
||||||
|
with network_errors():
|
||||||
|
a = self.client.lstat(self._p(rel))
|
||||||
|
mtime = int(a.st_mtime or 0) * 1_000_000_000
|
||||||
|
atime = int(a.st_atime or a.st_mtime or 0) * 1_000_000_000
|
||||||
|
return SourceStat(int(a.st_size or 0), mtime, atime, stat.S_IMODE(a.st_mode or 0o644))
|
||||||
|
|
||||||
|
@contextlib.contextmanager
|
||||||
|
def _reader(self, rel: str) -> Iterator[IO[bytes]]:
|
||||||
|
with network_errors():
|
||||||
|
size = self.client.lstat(self._p(rel)).st_size
|
||||||
|
fh: Any = self.client.open(self._p(rel), "rb")
|
||||||
|
try:
|
||||||
|
fh.prefetch(size) # Größe mitgeben: spart ein fstat und liest vorab
|
||||||
|
yield fh
|
||||||
|
finally:
|
||||||
|
with contextlib.suppress(*_NET_ERRORS, OSError):
|
||||||
|
fh.close()
|
||||||
|
|
||||||
|
def open(self, rel: str) -> contextlib.AbstractContextManager[IO[bytes]]:
|
||||||
|
return self._reader(rel)
|
||||||
|
|
||||||
|
def delete(self, rel: str) -> None:
|
||||||
|
with network_errors():
|
||||||
|
self.client.remove(self._p(rel))
|
||||||
|
|
||||||
|
def exists_root_rel(self, root_rel: str) -> bool:
|
||||||
|
with network_errors():
|
||||||
|
try:
|
||||||
|
self.client.lstat(posixpath.join(self.root, root_rel))
|
||||||
|
except FileNotFoundError:
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
def describe_root_rel(self, root_rel: str) -> str:
|
||||||
|
return f"{self.label}:{posixpath.join(self.root, root_rel)}"
|
||||||
|
|
||||||
|
def move_to(self, rel: str, root_rel_target: str) -> None:
|
||||||
|
target = posixpath.join(self.root, root_rel_target)
|
||||||
|
with network_errors():
|
||||||
|
current = self.root
|
||||||
|
for part in posixpath.dirname(root_rel_target).split("/"):
|
||||||
|
if not part:
|
||||||
|
continue
|
||||||
|
current = posixpath.join(current, part)
|
||||||
|
try:
|
||||||
|
self.client.stat(current)
|
||||||
|
except FileNotFoundError:
|
||||||
|
self.client.mkdir(current, mode=0o755)
|
||||||
|
if self.exists_root_rel(root_rel_target):
|
||||||
|
raise FileExistsError(errno.EEXIST, "exists", target)
|
||||||
|
self.client.rename(self._p(rel), target) # SFTPv3-rename überschreibt nicht
|
||||||
|
|
||||||
|
|
||||||
|
def _state(a: paramiko.SFTPAttributes) -> FileState:
|
||||||
|
return FileState(int(a.st_size or 0), int(a.st_mtime or 0) * 1_000_000_000)
|
||||||
@@ -46,6 +46,12 @@ class AppSettings:
|
|||||||
smtp_password_enc: str = ""
|
smtp_password_enc: str = ""
|
||||||
smtp_from: str = ""
|
smtp_from: str = ""
|
||||||
smtp_to: str = ""
|
smtp_to: str = ""
|
||||||
|
# Anmeldung: Netze ohne Passwort (trusted_networks), Netze mit Passwort
|
||||||
|
# (allowed_networks, leer = alle übrigen); alle anderen Netze werden abgewiesen.
|
||||||
|
# auth_mode "off" = ohne Anmeldung von überall (aus 0.5.0, weiterhin gültig).
|
||||||
|
auth_mode: str = "always"
|
||||||
|
trusted_networks: str = ""
|
||||||
|
allowed_networks: str = ""
|
||||||
|
|
||||||
|
|
||||||
_PREFIX = "app."
|
_PREFIX = "app."
|
||||||
|
|||||||
@@ -16,6 +16,8 @@ from sqlalchemy.orm import Session
|
|||||||
|
|
||||||
from netmover import __version__
|
from netmover import __version__
|
||||||
from netmover.models import (
|
from netmover.models import (
|
||||||
|
ArchiveKind,
|
||||||
|
ArchiveLayout,
|
||||||
CollisionPolicy,
|
CollisionPolicy,
|
||||||
Mount,
|
Mount,
|
||||||
MountState,
|
MountState,
|
||||||
@@ -72,6 +74,9 @@ RULE_FIELDS = (
|
|||||||
"dry_run",
|
"dry_run",
|
||||||
"quarantine_path",
|
"quarantine_path",
|
||||||
"quarantine_after",
|
"quarantine_after",
|
||||||
|
"archive_kind",
|
||||||
|
"archive_path",
|
||||||
|
"archive_layout",
|
||||||
)
|
)
|
||||||
SETTING_FIELDS = (
|
SETTING_FIELDS = (
|
||||||
"retention_days",
|
"retention_days",
|
||||||
@@ -99,6 +104,7 @@ def export_config(s: Session, settings: AppSettings) -> dict[str, Any]:
|
|||||||
**{f: _plain(getattr(r, f)) for f in RULE_FIELDS},
|
**{f: _plain(getattr(r, f)) for f in RULE_FIELDS},
|
||||||
"src_mount": names[r.src_mount_id],
|
"src_mount": names[r.src_mount_id],
|
||||||
"dst_mount": names.get(r.dst_mount_id) if r.dst_mount_id else None,
|
"dst_mount": names.get(r.dst_mount_id) if r.dst_mount_id else None,
|
||||||
|
"archive_mount": names.get(r.archive_mount_id) if r.archive_mount_id else None,
|
||||||
}
|
}
|
||||||
for r in s.scalars(select(Rule).order_by(Rule.name))
|
for r in s.scalars(select(Rule).order_by(Rule.name))
|
||||||
],
|
],
|
||||||
@@ -165,13 +171,16 @@ def import_config(s: Session, mounts: MountManager, data: dict[str, Any]) -> Imp
|
|||||||
try:
|
try:
|
||||||
src = ids[raw["src_mount"]]
|
src = ids[raw["src_mount"]]
|
||||||
dst = ids[raw["dst_mount"]] if raw.get("dst_mount") else None
|
dst = ids[raw["dst_mount"]] if raw.get("dst_mount") else None
|
||||||
r = Rule(src_mount_id=src, dst_mount_id=dst)
|
arch = ids[raw["archive_mount"]] if raw.get("archive_mount") else None
|
||||||
|
r = Rule(src_mount_id=src, dst_mount_id=dst, archive_mount_id=arch)
|
||||||
for f in RULE_FIELDS:
|
for f in RULE_FIELDS:
|
||||||
if f in raw:
|
if f in raw:
|
||||||
setattr(r, f, raw[f])
|
setattr(r, f, raw[f])
|
||||||
r.pattern_type = PatternType(r.pattern_type)
|
r.pattern_type = PatternType(r.pattern_type)
|
||||||
r.action = RuleAction(r.action)
|
r.action = RuleAction(r.action)
|
||||||
r.on_collision = CollisionPolicy(r.on_collision)
|
r.on_collision = CollisionPolicy(r.on_collision)
|
||||||
|
r.archive_kind = ArchiveKind(r.archive_kind or "none")
|
||||||
|
r.archive_layout = ArchiveLayout(r.archive_layout or "flat")
|
||||||
r.src_subpath = safe_relpath("src_subpath", r.src_subpath or "")
|
r.src_subpath = safe_relpath("src_subpath", r.src_subpath or "")
|
||||||
if int(r.interval_seconds) < 10:
|
if int(r.interval_seconds) < 10:
|
||||||
raise ValueError("Intervall < 10 s")
|
raise ValueError("Intervall < 10 s")
|
||||||
|
|||||||
@@ -38,19 +38,30 @@ from netmover.admin import (
|
|||||||
AdminError,
|
AdminError,
|
||||||
MountInput,
|
MountInput,
|
||||||
RuleInput,
|
RuleInput,
|
||||||
|
copy_name,
|
||||||
credential_for_test,
|
credential_for_test,
|
||||||
delete_mount,
|
delete_mount,
|
||||||
mount_input_from_model,
|
mount_input_from_model,
|
||||||
parse_mount_form,
|
parse_mount_form,
|
||||||
parse_rule_form,
|
parse_rule_form,
|
||||||
|
rule_copy_name,
|
||||||
rule_input_from_model,
|
rule_input_from_model,
|
||||||
save_mount,
|
save_mount,
|
||||||
save_rule,
|
save_rule,
|
||||||
)
|
)
|
||||||
from netmover.app_state import NetMover
|
from netmover.app_state import NetMover
|
||||||
from netmover.config import Settings, get_settings
|
from netmover.config import Settings, get_settings
|
||||||
|
from netmover.credentials import (
|
||||||
|
create_password,
|
||||||
|
delete_credential,
|
||||||
|
generate_key,
|
||||||
|
import_key,
|
||||||
|
list_keys,
|
||||||
|
list_passwords,
|
||||||
|
rename_credential,
|
||||||
|
update_password,
|
||||||
|
)
|
||||||
from netmover.db import session_scope
|
from netmover.db import session_scope
|
||||||
from netmover.keys import delete_key, generate_key, import_key, list_keys, rename_key
|
|
||||||
from netmover.kuma import push as kuma_push
|
from netmover.kuma import push as kuma_push
|
||||||
from netmover.logging_setup import event, get_logger
|
from netmover.logging_setup import event, get_logger
|
||||||
from netmover.mail import SECURITY_MODES, MailConfigError, parse_recipients, valid_address
|
from netmover.mail import SECURITY_MODES, MailConfigError, parse_recipients, valid_address
|
||||||
@@ -171,7 +182,7 @@ def flash(request: Request, message: str, kind: str = "ok") -> None:
|
|||||||
HELP_ANCHORS = {
|
HELP_ANCHORS = {
|
||||||
"status": "status",
|
"status": "status",
|
||||||
"mounts": "anbindungen",
|
"mounts": "anbindungen",
|
||||||
"keys": "schluessel",
|
"keys": "zugangsdaten",
|
||||||
"rules": "regeln",
|
"rules": "regeln",
|
||||||
"log": "protokoll",
|
"log": "protokoll",
|
||||||
"settings": "einstellungen",
|
"settings": "einstellungen",
|
||||||
@@ -191,7 +202,7 @@ class NavState:
|
|||||||
|
|
||||||
|
|
||||||
def _nav_state(request: Request) -> NavState | None:
|
def _nav_state(request: Request) -> NavState | None:
|
||||||
if not auth.is_logged_in(request):
|
if not _authorized(request):
|
||||||
return None
|
return None
|
||||||
state = nm(request)
|
state = nm(request)
|
||||||
health = state.health()
|
health = state.health()
|
||||||
@@ -228,7 +239,9 @@ def render(request: Request, name: str, status_code: int = 200, **ctx: Any) -> H
|
|||||||
{
|
{
|
||||||
"csrf": auth.csrf_token(request),
|
"csrf": auth.csrf_token(request),
|
||||||
"flashes": flashes,
|
"flashes": flashes,
|
||||||
"logged_in": auth.is_logged_in(request),
|
"logged_in": _authorized(request),
|
||||||
|
"session_login": auth.is_logged_in(request),
|
||||||
|
"auth_bypass": getattr(request.state, "auth_bypass", None),
|
||||||
"active": ctx.pop("active", ""),
|
"active": ctx.pop("active", ""),
|
||||||
**ctx,
|
**ctx,
|
||||||
},
|
},
|
||||||
@@ -254,13 +267,22 @@ def _form_dict(form: Any) -> dict[str, str]:
|
|||||||
|
|
||||||
async def guard(request: Request) -> None:
|
async def guard(request: Request) -> None:
|
||||||
state = nm(request)
|
state = nm(request)
|
||||||
|
# Ein Passwort ist immer Pflicht – sonst ließe sich die Anmeldung nie wieder einschalten.
|
||||||
if not auth.password_is_set(state.session_factory):
|
if not auth.password_is_set(state.session_factory):
|
||||||
raise auth.LoginRequired(setup=True)
|
raise auth.LoginRequired(setup=True)
|
||||||
if not auth.is_logged_in(request):
|
cfg = load_settings(state.session_factory)
|
||||||
|
bypass = auth.login_bypass(request, cfg.auth_mode, cfg.trusted_networks, cfg.allowed_networks)
|
||||||
|
if bypass:
|
||||||
|
request.state.auth_bypass = bypass
|
||||||
|
elif not auth.is_logged_in(request):
|
||||||
raise auth.LoginRequired()
|
raise auth.LoginRequired()
|
||||||
await auth.check_csrf(request)
|
await auth.check_csrf(request)
|
||||||
|
|
||||||
|
|
||||||
|
def _authorized(request: Request) -> bool:
|
||||||
|
return auth.is_logged_in(request) or bool(getattr(request.state, "auth_bypass", None))
|
||||||
|
|
||||||
|
|
||||||
protected = APIRouter(dependencies=[Depends(guard)])
|
protected = APIRouter(dependencies=[Depends(guard)])
|
||||||
public = APIRouter()
|
public = APIRouter()
|
||||||
|
|
||||||
@@ -340,8 +362,12 @@ async def setup_submit(request: Request) -> Response:
|
|||||||
|
|
||||||
@public.get("/login", response_class=HTMLResponse)
|
@public.get("/login", response_class=HTMLResponse)
|
||||||
async def login_form(request: Request) -> Response:
|
async def login_form(request: Request) -> Response:
|
||||||
if not auth.password_is_set(nm(request).session_factory):
|
state = nm(request)
|
||||||
|
if not auth.password_is_set(state.session_factory):
|
||||||
return redirect("/setup")
|
return redirect("/setup")
|
||||||
|
cfg = load_settings(state.session_factory)
|
||||||
|
if auth.login_bypass(request, cfg.auth_mode, cfg.trusted_networks, cfg.allowed_networks):
|
||||||
|
return redirect("/")
|
||||||
return render(request, "login.html")
|
return render(request, "login.html")
|
||||||
|
|
||||||
|
|
||||||
@@ -398,13 +424,24 @@ class RuleView:
|
|||||||
target: str
|
target: str
|
||||||
|
|
||||||
|
|
||||||
|
def _archive_target(r: Rule) -> str | None:
|
||||||
|
if r.archive_kind.value == "none":
|
||||||
|
return None
|
||||||
|
if r.archive_kind.value == "local" or r.archive_mount is None:
|
||||||
|
place = r.archive_path
|
||||||
|
else:
|
||||||
|
place = f"{r.archive_mount.name}:/{r.archive_path}"
|
||||||
|
return place + (" (nach Datum)" if r.archive_layout.value == "date" else "")
|
||||||
|
|
||||||
|
|
||||||
def _rule_target(r: Rule) -> str:
|
def _rule_target(r: Rule) -> str:
|
||||||
src = f"{r.src_mount.name}:/{r.src_subpath}" if r.src_subpath else f"{r.src_mount.name}:/"
|
src = f"{r.src_mount.name}:/{r.src_subpath}" if r.src_subpath else f"{r.src_mount.name}:/"
|
||||||
if r.dst_mount is None:
|
if r.dst_mount is None:
|
||||||
dst = r.dst_path
|
dst = r.dst_path
|
||||||
else:
|
else:
|
||||||
dst = f"{r.dst_mount.name}:/{r.dst_path}" if r.dst_path else f"{r.dst_mount.name}:/"
|
dst = f"{r.dst_mount.name}:/{r.dst_path}" if r.dst_path else f"{r.dst_mount.name}:/"
|
||||||
return f"{src} → {dst}"
|
archive = _archive_target(r)
|
||||||
|
return f"{src} → {dst}" + (f" + Archiv {archive}" if archive else "")
|
||||||
|
|
||||||
|
|
||||||
async def _status_context(request: Request) -> dict[str, Any]:
|
async def _status_context(request: Request) -> dict[str, Any]:
|
||||||
@@ -511,7 +548,7 @@ async def mounts_page(request: Request) -> Response:
|
|||||||
m.id: m.credential.name
|
m.id: m.credential.name
|
||||||
for m in mounts
|
for m in mounts
|
||||||
if m.credential is not None and m.credential.managed
|
if m.credential is not None and m.credential.managed
|
||||||
}
|
} # Name gespeicherter Zugangsdaten (Passwort oder Schlüssel)
|
||||||
return render(
|
return render(
|
||||||
request,
|
request,
|
||||||
"mounts.html",
|
"mounts.html",
|
||||||
@@ -533,6 +570,15 @@ def _report_data(raw: str | None) -> dict[str, Any]:
|
|||||||
return data
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
def _password_options(request: Request) -> list[tuple[str, str]]:
|
||||||
|
with nm(request).session_factory() as s:
|
||||||
|
stored = list_passwords(s)
|
||||||
|
return [("own", "eigene Angaben für diese Anbindung")] + [
|
||||||
|
(str(p.id), f"{p.name} ({p.domain + chr(92) if p.domain else ''}{p.username})")
|
||||||
|
for p in stored
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
def _key_options(request: Request) -> list[tuple[str, str]]:
|
def _key_options(request: Request) -> list[tuple[str, str]]:
|
||||||
with nm(request).session_factory() as s:
|
with nm(request).session_factory() as s:
|
||||||
keys = list_keys(s)
|
keys = list_keys(s)
|
||||||
@@ -547,10 +593,12 @@ def _mount_form(
|
|||||||
errors: dict[str, str],
|
errors: dict[str, str],
|
||||||
mount: Mount | None = None,
|
mount: Mount | None = None,
|
||||||
status_code: int = 200,
|
status_code: int = 200,
|
||||||
|
copy_from: Mount | None = None,
|
||||||
) -> Response:
|
) -> Response:
|
||||||
|
owner = mount or copy_from
|
||||||
pub = (
|
pub = (
|
||||||
mount.credential.public_key
|
owner.credential.public_key
|
||||||
if mount is not None and mount.credential and not mount.credential.managed
|
if owner is not None and owner.credential and not owner.credential.managed
|
||||||
else None
|
else None
|
||||||
)
|
)
|
||||||
return render(
|
return render(
|
||||||
@@ -563,9 +611,11 @@ def _mount_form(
|
|||||||
mount=mount,
|
mount=mount,
|
||||||
public_key=pub,
|
public_key=pub,
|
||||||
has_secret=bool(
|
has_secret=bool(
|
||||||
mount is not None and mount.credential is not None and not mount.credential.managed
|
owner is not None and owner.credential is not None and not owner.credential.managed
|
||||||
),
|
),
|
||||||
|
copy_from=copy_from,
|
||||||
key_options=_key_options(request),
|
key_options=_key_options(request),
|
||||||
|
password_options=_password_options(request),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -583,16 +633,33 @@ async def mount_edit(request: Request, mount_id: int) -> Response:
|
|||||||
return _mount_form(request, mount_input_from_model(m), {}, m)
|
return _mount_form(request, mount_input_from_model(m), {}, m)
|
||||||
|
|
||||||
|
|
||||||
|
@protected.get("/mounts/{source_id}/copy", response_class=HTMLResponse)
|
||||||
|
async def mount_copy(request: Request, source_id: int) -> Response:
|
||||||
|
"""Formular für eine neue Anbindung, vorausgefüllt aus einer bestehenden."""
|
||||||
|
with nm(request).session_factory() as s:
|
||||||
|
src = s.get(Mount, source_id)
|
||||||
|
if src is None:
|
||||||
|
return redirect("/mounts")
|
||||||
|
data = mount_input_from_model(src)
|
||||||
|
data.name = copy_name(s, src.name)
|
||||||
|
return _mount_form(request, data, {}, copy_from=src)
|
||||||
|
|
||||||
|
|
||||||
@protected.post("/mounts/new", response_class=HTMLResponse)
|
@protected.post("/mounts/new", response_class=HTMLResponse)
|
||||||
@protected.post("/mounts/{mount_id}/edit", response_class=HTMLResponse)
|
@protected.post("/mounts/{mount_id}/edit", response_class=HTMLResponse)
|
||||||
async def mount_save(request: Request, mount_id: int | None = None) -> Response:
|
@protected.post("/mounts/{source_id}/copy", response_class=HTMLResponse)
|
||||||
|
async def mount_save(
|
||||||
|
request: Request, mount_id: int | None = None, source_id: int | None = None
|
||||||
|
) -> Response:
|
||||||
state = nm(request)
|
state = nm(request)
|
||||||
data, errors = parse_mount_form(_form_dict(await request.form()))
|
data, errors = parse_mount_form(_form_dict(await request.form()))
|
||||||
with session_scope(state.session_factory) as s:
|
with session_scope(state.session_factory) as s:
|
||||||
existing = s.get(Mount, mount_id) if mount_id is not None else None
|
copy_from = s.get(Mount, source_id) if source_id is not None else None
|
||||||
if not errors:
|
if not errors:
|
||||||
try:
|
try:
|
||||||
m, errors = save_mount(s, state.secrets, state.mounts, data, mount_id)
|
m, errors = save_mount(
|
||||||
|
s, state.secrets, state.mounts, data, mount_id, copy_from=copy_from
|
||||||
|
)
|
||||||
except AdminError as exc:
|
except AdminError as exc:
|
||||||
errors = {"_": str(exc)}
|
errors = {"_": str(exc)}
|
||||||
m = None
|
m = None
|
||||||
@@ -601,7 +668,10 @@ async def mount_save(request: Request, mount_id: int | None = None) -> Response:
|
|||||||
if errors or m is None:
|
if errors or m is None:
|
||||||
s.rollback()
|
s.rollback()
|
||||||
existing = s.get(Mount, mount_id) if mount_id is not None else None
|
existing = s.get(Mount, mount_id) if mount_id is not None else None
|
||||||
return _mount_form(request, data, errors, existing, status_code=400)
|
copy_from = s.get(Mount, source_id) if source_id is not None else None
|
||||||
|
return _mount_form(
|
||||||
|
request, data, errors, existing, status_code=400, copy_from=copy_from
|
||||||
|
)
|
||||||
saved_id, name = m.id, m.name
|
saved_id, name = m.id, m.name
|
||||||
is_sftp_key = m.type is MountType.SFTP and data.auth is CredentialKind.SSH_KEY
|
is_sftp_key = m.type is MountType.SFTP and data.auth is CredentialKind.SSH_KEY
|
||||||
flash(request, f"Anbindung „{name}“ gespeichert.")
|
flash(request, f"Anbindung „{name}“ gespeichert.")
|
||||||
@@ -623,12 +693,17 @@ async def mount_test_form(request: Request) -> Response:
|
|||||||
data, errors = parse_mount_form(form)
|
data, errors = parse_mount_form(form)
|
||||||
if errors:
|
if errors:
|
||||||
return render(request, "partials/check_result.html", errors=errors, report=None)
|
return render(request, "partials/check_result.html", errors=errors, report=None)
|
||||||
existing_id = int(form["mount_id"]) if form.get("mount_id", "").isdigit() else None
|
ref = form.get("mount_id") or form.get("copy_from") or ""
|
||||||
|
existing_id = int(ref) if ref.isdigit() else None
|
||||||
with state.session_factory() as s:
|
with state.session_factory() as s:
|
||||||
existing = s.get(Mount, existing_id) if existing_id else None
|
existing = s.get(Mount, existing_id) if existing_id else None
|
||||||
cred = credential_for_test(data, existing, state.secrets, s)
|
cred = credential_for_test(data, existing, state.secrets, s)
|
||||||
cfg = data.config()
|
cfg = data.config()
|
||||||
if data.type is MountType.SFTP and existing is not None:
|
if (
|
||||||
|
data.type is MountType.SFTP
|
||||||
|
and existing is not None
|
||||||
|
and (existing.server, existing.port) == (data.server, data.port)
|
||||||
|
):
|
||||||
cfg = dataclasses.replace(cfg, host_key=existing.host_key)
|
cfg = dataclasses.replace(cfg, host_key=existing.host_key)
|
||||||
report = await run_in_threadpool(state.mounts.check, cfg, cred, None)
|
report = await run_in_threadpool(state.mounts.check, cfg, cred, None)
|
||||||
return render(request, "partials/check_result.html", report=report, errors={})
|
return render(request, "partials/check_result.html", report=report, errors={})
|
||||||
@@ -702,26 +777,75 @@ async def mount_reset_hostkey(request: Request, mount_id: int) -> Response:
|
|||||||
return redirect(f"/mounts#mount-{mount_id}")
|
return redirect(f"/mounts#mount-{mount_id}")
|
||||||
|
|
||||||
|
|
||||||
# -- Schlüsselverwaltung ----------------------------------------------------------------------
|
# -- Zugangsdaten ------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
@protected.get("/keys", response_class=HTMLResponse)
|
@protected.get("/credentials", response_class=HTMLResponse)
|
||||||
async def keys_page(request: Request) -> Response:
|
async def credentials_page(request: Request) -> Response:
|
||||||
with nm(request).session_factory() as s:
|
with nm(request).session_factory() as s:
|
||||||
return render(request, "keys.html", active="keys", keys=list_keys(s))
|
return render(
|
||||||
|
request,
|
||||||
|
"credentials.html",
|
||||||
|
active="keys",
|
||||||
|
keys=list_keys(s),
|
||||||
|
passwords=list_passwords(s),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
async def _key_action(request: Request, work: Callable[[Any], str]) -> Response:
|
@protected.get("/keys")
|
||||||
|
async def keys_redirect(request: Request) -> Response:
|
||||||
|
return RedirectResponse("/credentials", status_code=301)
|
||||||
|
|
||||||
|
|
||||||
|
async def _cred_action(request: Request, work: Callable[[Any], str]) -> Response:
|
||||||
state = nm(request)
|
state = nm(request)
|
||||||
try:
|
try:
|
||||||
with session_scope(state.session_factory) as s:
|
with session_scope(state.session_factory) as s:
|
||||||
flash(request, work(s))
|
flash(request, work(s))
|
||||||
except AdminError as exc:
|
except AdminError as exc:
|
||||||
flash(request, str(exc), "error")
|
flash(request, str(exc), "error")
|
||||||
return redirect("/keys")
|
return redirect("/credentials")
|
||||||
|
|
||||||
|
|
||||||
@protected.post("/keys/generate")
|
@protected.post("/credentials/passwords")
|
||||||
|
async def password_create(request: Request) -> Response:
|
||||||
|
form = _form_dict(await request.form())
|
||||||
|
secrets = nm(request).secrets
|
||||||
|
|
||||||
|
def work(s: Any) -> str:
|
||||||
|
cred = create_password(
|
||||||
|
s,
|
||||||
|
secrets,
|
||||||
|
form.get("name", ""),
|
||||||
|
form.get("username", ""),
|
||||||
|
form.get("password", ""),
|
||||||
|
form.get("domain", ""),
|
||||||
|
)
|
||||||
|
return f"Zugangsdaten „{cred.name}“ gespeichert."
|
||||||
|
|
||||||
|
return await _cred_action(request, work)
|
||||||
|
|
||||||
|
|
||||||
|
@protected.post("/credentials/{cred_id}/password")
|
||||||
|
async def password_update(request: Request, cred_id: int) -> Response:
|
||||||
|
form = _form_dict(await request.form())
|
||||||
|
secrets = nm(request).secrets
|
||||||
|
|
||||||
|
def work(s: Any) -> str:
|
||||||
|
cred = update_password(
|
||||||
|
s,
|
||||||
|
secrets,
|
||||||
|
cred_id,
|
||||||
|
form.get("username", ""),
|
||||||
|
form.get("password", ""),
|
||||||
|
form.get("domain", ""),
|
||||||
|
)
|
||||||
|
return f"Zugangsdaten „{cred.name}“ geändert – gilt ab sofort für alle Anbindungen."
|
||||||
|
|
||||||
|
return await _cred_action(request, work)
|
||||||
|
|
||||||
|
|
||||||
|
@protected.post("/credentials/keys/generate")
|
||||||
async def key_generate(request: Request) -> Response:
|
async def key_generate(request: Request) -> Response:
|
||||||
form = _form_dict(await request.form())
|
form = _form_dict(await request.form())
|
||||||
secrets = nm(request).secrets
|
secrets = nm(request).secrets
|
||||||
@@ -730,10 +854,10 @@ async def key_generate(request: Request) -> Response:
|
|||||||
key = generate_key(s, secrets, form.get("name", ""))
|
key = generate_key(s, secrets, form.get("name", ""))
|
||||||
return f"Schlüssel „{key.name}“ erzeugt."
|
return f"Schlüssel „{key.name}“ erzeugt."
|
||||||
|
|
||||||
return await _key_action(request, work)
|
return await _cred_action(request, work)
|
||||||
|
|
||||||
|
|
||||||
@protected.post("/keys/import")
|
@protected.post("/credentials/keys/import")
|
||||||
async def key_import(request: Request) -> Response:
|
async def key_import(request: Request) -> Response:
|
||||||
form = _form_dict(await request.form())
|
form = _form_dict(await request.form())
|
||||||
secrets = nm(request).secrets
|
secrets = nm(request).secrets
|
||||||
@@ -748,26 +872,26 @@ async def key_import(request: Request) -> Response:
|
|||||||
)
|
)
|
||||||
return f"Schlüssel „{key.name}“ importiert."
|
return f"Schlüssel „{key.name}“ importiert."
|
||||||
|
|
||||||
return await _key_action(request, work)
|
return await _cred_action(request, work)
|
||||||
|
|
||||||
|
|
||||||
@protected.post("/keys/{key_id}/rename")
|
@protected.post("/credentials/{cred_id}/rename")
|
||||||
async def key_rename(request: Request, key_id: int) -> Response:
|
async def credential_rename(request: Request, cred_id: int) -> Response:
|
||||||
form = _form_dict(await request.form())
|
form = _form_dict(await request.form())
|
||||||
|
|
||||||
def work(s: Any) -> str:
|
def work(s: Any) -> str:
|
||||||
rename_key(s, key_id, form.get("name", ""))
|
rename_credential(s, cred_id, form.get("name", ""))
|
||||||
return "Schlüssel umbenannt."
|
return "Umbenannt."
|
||||||
|
|
||||||
return await _key_action(request, work)
|
return await _cred_action(request, work)
|
||||||
|
|
||||||
|
|
||||||
@protected.post("/keys/{key_id}/delete")
|
@protected.post("/credentials/{cred_id}/delete")
|
||||||
async def key_delete(request: Request, key_id: int) -> Response:
|
async def credential_delete(request: Request, cred_id: int) -> Response:
|
||||||
def work(s: Any) -> str:
|
def work(s: Any) -> str:
|
||||||
return f"Schlüssel „{delete_key(s, key_id)}“ gelöscht."
|
return f"„{delete_credential(s, cred_id)}“ gelöscht."
|
||||||
|
|
||||||
return await _key_action(request, work)
|
return await _cred_action(request, work)
|
||||||
|
|
||||||
|
|
||||||
# -- Regeln -----------------------------------------------------------------------------------
|
# -- Regeln -----------------------------------------------------------------------------------
|
||||||
@@ -779,6 +903,7 @@ def _rule_form(
|
|||||||
errors: dict[str, str],
|
errors: dict[str, str],
|
||||||
rule_id: int | None,
|
rule_id: int | None,
|
||||||
status_code: int = 200,
|
status_code: int = 200,
|
||||||
|
copy_of: str | None = None,
|
||||||
) -> Response:
|
) -> Response:
|
||||||
with nm(request).session_factory() as s:
|
with nm(request).session_factory() as s:
|
||||||
mounts = s.scalars(select(Mount).order_by(Mount.name)).all()
|
mounts = s.scalars(select(Mount).order_by(Mount.name)).all()
|
||||||
@@ -792,9 +917,9 @@ def _rule_form(
|
|||||||
d=d,
|
d=d,
|
||||||
errors=errors,
|
errors=errors,
|
||||||
rule_id=rule_id,
|
rule_id=rule_id,
|
||||||
src_mounts=[m for m in mounts if m.type.is_kernel_mount],
|
copy_of=copy_of,
|
||||||
src_options=[("", "– wählen –")]
|
src_mounts=mounts,
|
||||||
+ [(m.id, m.name) for m in mounts if m.type.is_kernel_mount],
|
src_options=[("", "– wählen –")] + [(m.id, f"{m.name} ({m.type.value})") for m in mounts],
|
||||||
dst_options=[("", "– wählen –")] + [(m.id, f"{m.name} ({m.type.value})") for m in mounts],
|
dst_options=[("", "– wählen –")] + [(m.id, f"{m.name} ({m.type.value})") for m in mounts],
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -820,25 +945,47 @@ async def rule_edit(request: Request, rule_id: int) -> Response:
|
|||||||
return _rule_form(request, d, {}, rule_id)
|
return _rule_form(request, d, {}, rule_id)
|
||||||
|
|
||||||
|
|
||||||
|
@protected.get("/rules/{source_id}/copy", response_class=HTMLResponse)
|
||||||
|
async def rule_copy(request: Request, source_id: int) -> Response:
|
||||||
|
"""Neue Regel, vorausgefüllt aus einer bestehenden – zunächst deaktiviert, damit sie
|
||||||
|
nicht sofort mit dem Original um dieselben Dateien konkurriert."""
|
||||||
|
with nm(request).session_factory() as s:
|
||||||
|
src = s.get(Rule, source_id)
|
||||||
|
if src is None:
|
||||||
|
return redirect("/rules")
|
||||||
|
d = rule_input_from_model(src)
|
||||||
|
d.name = rule_copy_name(s, src.name)
|
||||||
|
d.enabled = False
|
||||||
|
source_name = src.name
|
||||||
|
return _rule_form(request, d, {}, None, copy_of=source_name)
|
||||||
|
|
||||||
|
|
||||||
@protected.post("/rules/new", response_class=HTMLResponse)
|
@protected.post("/rules/new", response_class=HTMLResponse)
|
||||||
@protected.post("/rules/{rule_id}/edit", response_class=HTMLResponse)
|
@protected.post("/rules/{rule_id}/edit", response_class=HTMLResponse)
|
||||||
async def rule_save(request: Request, rule_id: int | None = None) -> Response:
|
@protected.post("/rules/{source_id}/copy", response_class=HTMLResponse)
|
||||||
|
async def rule_save(
|
||||||
|
request: Request, rule_id: int | None = None, source_id: int | None = None
|
||||||
|
) -> Response:
|
||||||
state = nm(request)
|
state = nm(request)
|
||||||
form = _form_dict(await request.form())
|
form = _form_dict(await request.form())
|
||||||
with session_scope(state.session_factory) as s:
|
with session_scope(state.session_factory) as s:
|
||||||
|
source = s.get(Rule, source_id) if source_id is not None else None
|
||||||
|
copy_of = source.name if source is not None else None
|
||||||
d, errors = parse_rule_form(s, form)
|
d, errors = parse_rule_form(s, form)
|
||||||
r = None
|
r = None
|
||||||
if not errors:
|
if not errors:
|
||||||
try:
|
try:
|
||||||
r, errors = save_rule(s, d, rule_id)
|
r, errors = save_rule(s, d, rule_id) # Kopie: rule_id ist None → neue Regel
|
||||||
except AdminError as exc:
|
except AdminError as exc:
|
||||||
errors = {"_": str(exc)}
|
errors = {"_": str(exc)}
|
||||||
if errors or r is None:
|
if errors or r is None:
|
||||||
s.rollback()
|
s.rollback()
|
||||||
return _rule_form(request, d, errors, rule_id, status_code=400)
|
return _rule_form(request, d, errors, rule_id, status_code=400, copy_of=copy_of)
|
||||||
name = r.name
|
name, enabled = r.name, r.enabled
|
||||||
state.scheduler.sync()
|
state.scheduler.sync()
|
||||||
flash(request, f"Regel „{name}“ gespeichert.")
|
flash(request, f"Regel „{name}“ gespeichert.")
|
||||||
|
if copy_of and not enabled:
|
||||||
|
flash(request, "Die Kopie ist deaktiviert. Probelauf machen, dann aktivieren.", "info")
|
||||||
return redirect("/rules")
|
return redirect("/rules")
|
||||||
|
|
||||||
|
|
||||||
@@ -994,6 +1141,8 @@ async def settings_page(request: Request) -> Response:
|
|||||||
cfg=cfg,
|
cfg=cfg,
|
||||||
errors={},
|
errors={},
|
||||||
monitor_urls=_monitor_urls(request),
|
monitor_urls=_monitor_urls(request),
|
||||||
|
client_ip=request.client.host if request.client else "",
|
||||||
|
loopback_trusted=auth.includes_loopback(cfg.trusted_networks),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -1149,6 +1298,58 @@ async def settings_password(request: Request) -> Response:
|
|||||||
return redirect("/settings")
|
return redirect("/settings")
|
||||||
|
|
||||||
|
|
||||||
|
@protected.post("/settings/auth", response_class=HTMLResponse)
|
||||||
|
async def settings_auth(request: Request) -> Response:
|
||||||
|
"""Netze ohne / mit Passwort festlegen. Nur mit dem aktuellen Passwort und nie so,
|
||||||
|
dass man sich mit der eigenen Adresse aussperrt."""
|
||||||
|
state = nm(request)
|
||||||
|
form = _form_dict(await request.form())
|
||||||
|
back = redirect("/settings#anmeldung")
|
||||||
|
if not await run_in_threadpool(
|
||||||
|
auth.verify_password, state.session_factory, form.get("current", "")
|
||||||
|
):
|
||||||
|
flash(request, "Aktuelles Passwort falsch – Anmeldung unverändert.", "error")
|
||||||
|
return back
|
||||||
|
trusted = (form.get("trusted_networks") or "").strip()
|
||||||
|
allowed = (form.get("allowed_networks") or "").strip()
|
||||||
|
try:
|
||||||
|
auth.parse_networks(trusted)
|
||||||
|
auth.parse_networks(allowed)
|
||||||
|
except ValueError as exc:
|
||||||
|
flash(request, str(exc), "error")
|
||||||
|
return back
|
||||||
|
me = auth.client_host(request)
|
||||||
|
if auth.access_decision(me, auth.AUTH_ALWAYS, trusted, allowed) == auth.ACCESS_DENY:
|
||||||
|
flash(
|
||||||
|
request,
|
||||||
|
f"Nicht gespeichert: Damit wärst du selbst ausgesperrt (deine Adresse {me}). "
|
||||||
|
"Trage dein Netz bei „mit“ oder „ohne Passwort“ ein.",
|
||||||
|
"error",
|
||||||
|
)
|
||||||
|
return back
|
||||||
|
cfg = load_settings(state.session_factory)
|
||||||
|
cfg.trusted_networks, cfg.allowed_networks = trusted, allowed
|
||||||
|
cfg.auth_mode = auth.AUTH_TRUSTED if trusted else auth.AUTH_ALWAYS
|
||||||
|
save_settings(state.session_factory, cfg)
|
||||||
|
event(
|
||||||
|
log,
|
||||||
|
"auth.access_changed",
|
||||||
|
level=30,
|
||||||
|
without_password=trusted,
|
||||||
|
with_password=allowed or "alle übrigen",
|
||||||
|
)
|
||||||
|
parts = [f"ohne Passwort aus: {trusted}" if trusted else "überall mit Passwort"]
|
||||||
|
parts.append(
|
||||||
|
f"mit Passwort nur aus: {allowed} – alle anderen Netze abgewiesen"
|
||||||
|
if allowed
|
||||||
|
else "alle übrigen Netze mit Passwort"
|
||||||
|
)
|
||||||
|
flash(
|
||||||
|
request, "Anmeldung gespeichert: " + "; ".join(parts) + ".", "warn" if trusted else "info"
|
||||||
|
)
|
||||||
|
return back
|
||||||
|
|
||||||
|
|
||||||
@protected.get("/settings/export")
|
@protected.get("/settings/export")
|
||||||
async def settings_export(request: Request) -> Response:
|
async def settings_export(request: Request) -> Response:
|
||||||
state = nm(request)
|
state = nm(request)
|
||||||
@@ -1250,6 +1451,35 @@ def create_app(settings: Settings | None = None, state: NetMover | None = None)
|
|||||||
openapi_url=None,
|
openapi_url=None,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
@app.middleware("http")
|
||||||
|
async def network_access(
|
||||||
|
request: Request, call_next: Callable[[Request], Awaitable[Response]]
|
||||||
|
) -> Response:
|
||||||
|
path = request.url.path
|
||||||
|
if not (path.startswith("/healthz") or path.startswith("/static/")):
|
||||||
|
app_state = getattr(request.app.state, "nm", None)
|
||||||
|
if app_state is not None:
|
||||||
|
cfg = load_settings(app_state.session_factory)
|
||||||
|
decision = auth.access_decision(
|
||||||
|
auth.client_host(request),
|
||||||
|
cfg.auth_mode,
|
||||||
|
cfg.trusted_networks,
|
||||||
|
cfg.allowed_networks,
|
||||||
|
)
|
||||||
|
if decision == auth.ACCESS_DENY:
|
||||||
|
event(
|
||||||
|
log,
|
||||||
|
"auth.denied_network",
|
||||||
|
level=30,
|
||||||
|
client=auth.client_host(request) or "?",
|
||||||
|
)
|
||||||
|
return HTMLResponse(
|
||||||
|
"<!doctype html><meta charset=utf-8><title>NetMover</title>"
|
||||||
|
"<p>Zugriff aus diesem Netz ist nicht erlaubt.</p>",
|
||||||
|
status_code=403,
|
||||||
|
)
|
||||||
|
return await call_next(request)
|
||||||
|
|
||||||
@app.middleware("http")
|
@app.middleware("http")
|
||||||
async def security_headers(
|
async def security_headers(
|
||||||
request: Request, call_next: Callable[[Request], Awaitable[Response]]
|
request: Request, call_next: Callable[[Request], Awaitable[Response]]
|
||||||
|
|||||||
@@ -12,6 +12,8 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import hmac
|
import hmac
|
||||||
|
import ipaddress
|
||||||
|
import re
|
||||||
import secrets
|
import secrets
|
||||||
import threading
|
import threading
|
||||||
import time
|
import time
|
||||||
@@ -32,6 +34,88 @@ SAFE_METHODS = frozenset({"GET", "HEAD", "OPTIONS"})
|
|||||||
_hasher = PasswordHasher()
|
_hasher = PasswordHasher()
|
||||||
|
|
||||||
|
|
||||||
|
AUTH_ALWAYS, AUTH_TRUSTED, AUTH_OFF = "always", "trusted", "off"
|
||||||
|
AUTH_MODES = (AUTH_ALWAYS, AUTH_TRUSTED, AUTH_OFF)
|
||||||
|
|
||||||
|
|
||||||
|
def parse_networks(raw: str) -> list[ipaddress.IPv4Network | ipaddress.IPv6Network]:
|
||||||
|
"""„10.0.10.0/24, 192.168.1.5“ → Netze; ``ValueError`` mit verständlicher Meldung."""
|
||||||
|
nets = []
|
||||||
|
for part in re.split(r"[,;\s]+", raw.strip()):
|
||||||
|
if not part:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
nets.append(ipaddress.ip_network(part, strict=False))
|
||||||
|
except ValueError:
|
||||||
|
raise ValueError(f"„{part}“ ist kein gültiges Netz (z. B. 10.0.10.0/24)") from None
|
||||||
|
return nets
|
||||||
|
|
||||||
|
|
||||||
|
ACCESS_NOPW, ACCESS_PW, ACCESS_DENY = "nopw", "pw", "deny"
|
||||||
|
|
||||||
|
|
||||||
|
def _address(host: str | None) -> ipaddress.IPv4Address | ipaddress.IPv6Address | None:
|
||||||
|
if not host:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return ipaddress.ip_address(host)
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _networks_or_empty(raw: str) -> list[ipaddress.IPv4Network | ipaddress.IPv6Network]:
|
||||||
|
try:
|
||||||
|
return parse_networks(raw)
|
||||||
|
except ValueError:
|
||||||
|
return [] # gespeicherte Werte sind beim Speichern geprüft
|
||||||
|
|
||||||
|
|
||||||
|
def access_decision(host: str | None, mode: str, trusted: str, allowed: str) -> str:
|
||||||
|
"""Wie darf diese Adresse auf die Oberfläche zugreifen?
|
||||||
|
|
||||||
|
1. aus ``trusted`` (oder Modus „off“): **ohne Passwort**,
|
||||||
|
2. ``allowed`` leer oder Adresse darin: **mit Passwort**,
|
||||||
|
3. sonst: **abgewiesen**.
|
||||||
|
|
||||||
|
Hinter einem Reverse Proxy ist die Client-Adresse nur dann echt, wenn
|
||||||
|
``NETMOVER_FORWARDED_ALLOW_IPS`` den Proxy kennt – sonst sähe jede Anfrage aus,
|
||||||
|
als käme sie vom Proxy selbst."""
|
||||||
|
if mode == AUTH_OFF:
|
||||||
|
return ACCESS_NOPW
|
||||||
|
addr = _address(host)
|
||||||
|
if addr is not None and any(addr in n for n in _networks_or_empty(trusted)):
|
||||||
|
return ACCESS_NOPW
|
||||||
|
allowed_nets = _networks_or_empty(allowed)
|
||||||
|
if allowed_nets and (addr is None or not any(addr in n for n in allowed_nets)):
|
||||||
|
return ACCESS_DENY
|
||||||
|
return ACCESS_PW
|
||||||
|
|
||||||
|
|
||||||
|
def client_host(request: Request) -> str | None:
|
||||||
|
return request.client.host if request.client else None
|
||||||
|
|
||||||
|
|
||||||
|
def login_bypass(request: Request, mode: str, trusted: str, allowed: str = "") -> str | None:
|
||||||
|
"""Grund, warum keine Anmeldung nötig ist (``"off"``/``"trusted"``), sonst ``None``."""
|
||||||
|
if access_decision(client_host(request), mode, trusted, allowed) != ACCESS_NOPW:
|
||||||
|
return None
|
||||||
|
return AUTH_OFF if mode == AUTH_OFF or _covers_everything(trusted) else AUTH_TRUSTED
|
||||||
|
|
||||||
|
|
||||||
|
def _covers_everything(raw: str) -> bool:
|
||||||
|
return any(n.prefixlen == 0 for n in _networks_or_empty(raw))
|
||||||
|
|
||||||
|
|
||||||
|
def includes_loopback(raw: str) -> bool:
|
||||||
|
try:
|
||||||
|
nets = parse_networks(raw)
|
||||||
|
except ValueError:
|
||||||
|
return False
|
||||||
|
return any(
|
||||||
|
ipaddress.ip_address("127.0.0.1") in n or ipaddress.ip_address("::1") in n for n in nets
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class LoginRequired(Exception):
|
class LoginRequired(Exception):
|
||||||
def __init__(self, setup: bool = False) -> None:
|
def __init__(self, setup: bool = False) -> None:
|
||||||
self.setup = setup
|
self.setup = setup
|
||||||
|
|||||||
@@ -23,6 +23,7 @@
|
|||||||
{% if nav %}
|
{% if nav %}
|
||||||
<a class="health {{ 'ok' if nav.healthy else 'bad' }}" href="/" title="Gesundheitszustand">
|
<a class="health {{ 'ok' if nav.healthy else 'bad' }}" href="/" title="Gesundheitszustand">
|
||||||
<span class="dot"></span>{{ 'Alles in Ordnung' if nav.healthy else (nav.problems ~ (' Problem' if nav.problems == 1 else ' Probleme')) }}</a>
|
<span class="dot"></span>{{ 'Alles in Ordnung' if nav.healthy else (nav.problems ~ (' Problem' if nav.problems == 1 else ' Probleme')) }}</a>
|
||||||
|
{% if auth_bypass %}<a class="health warn" href="/settings#anmeldung" title="Ohne Passwort bedienbar">{{ 'ohne Anmeldung' if auth_bypass == 'off' else 'ohne Anmeldung (vertrautes Netz)' }}</a>{% endif %}
|
||||||
{% if nav.maintenance %}<a class="health warn" href="/" title="Geplante Läufe sind pausiert"><span class="dot"></span>Wartungsmodus</a>{% endif %}
|
{% if nav.maintenance %}<a class="health warn" href="/" title="Geplante Läufe sind pausiert"><span class="dot"></span>Wartungsmodus</a>{% endif %}
|
||||||
{% endif %}
|
{% endif %}
|
||||||
<span class="spacer"></span>
|
<span class="spacer"></span>
|
||||||
@@ -36,7 +37,7 @@
|
|||||||
<div class="nav-group">
|
<div class="nav-group">
|
||||||
<div class="nav-title">Verbindungen</div>
|
<div class="nav-title">Verbindungen</div>
|
||||||
{{ item('mounts', '/mounts', 'Anbindungen', (nav.mounts_bad ~ ' gestört') if nav and nav.mounts_bad else (nav.mounts if nav and nav.mounts else ''), 'bad' if nav and nav.mounts_bad else '') }}
|
{{ item('mounts', '/mounts', 'Anbindungen', (nav.mounts_bad ~ ' gestört') if nav and nav.mounts_bad else (nav.mounts if nav and nav.mounts else ''), 'bad' if nav and nav.mounts_bad else '') }}
|
||||||
{{ item('keys', '/keys', 'Schlüssel', nav.keys if nav and nav.keys else '') }}
|
{{ item('keys', '/credentials', 'Zugangsdaten', nav.keys if nav and nav.keys else '') }}
|
||||||
</div>
|
</div>
|
||||||
<div class="nav-group">
|
<div class="nav-group">
|
||||||
<div class="nav-title">Automatisierung</div>
|
<div class="nav-title">Automatisierung</div>
|
||||||
@@ -49,10 +50,12 @@
|
|||||||
{{ item('help', '/help', 'Hilfe') }}
|
{{ item('help', '/help', 'Hilfe') }}
|
||||||
</div>
|
</div>
|
||||||
<div class="nav-foot">
|
<div class="nav-foot">
|
||||||
|
{% if session_login %}
|
||||||
<form method="post" action="/logout">
|
<form method="post" action="/logout">
|
||||||
<input type="hidden" name="csrf_token" value="{{ csrf }}">
|
<input type="hidden" name="csrf_token" value="{{ csrf }}">
|
||||||
<button class="nav-button">{{ icon('logout') }}<span>Abmelden</span></button>
|
<button class="nav-button">{{ icon('logout') }}<span>Abmelden</span></button>
|
||||||
</form>
|
</form>
|
||||||
|
{% endif %}
|
||||||
<div class="version">NetMover {{ version }}</div>
|
<div class="version">NetMover {{ version }}</div>
|
||||||
</div>
|
</div>
|
||||||
</nav>
|
</nav>
|
||||||
|
|||||||
@@ -0,0 +1,97 @@
|
|||||||
|
{% extends "base.html" %}
|
||||||
|
{% from "macros.html" import csrf_field %}
|
||||||
|
{% block title %}Zugangsdaten{% endblock %}
|
||||||
|
{% block content %}
|
||||||
|
<h1>Zugangsdaten</h1>
|
||||||
|
<p class="muted small">Einmal hinterlegen, in beliebig vielen Anbindungen auswählen. Passwörter und private Schlüssel
|
||||||
|
liegen verschlüsselt in der Datenbank und werden nie angezeigt.</p>
|
||||||
|
|
||||||
|
<h2>Passwörter <a class="legend-help" href="/help#zugangsdaten" title="Hilfe">?</a></h2>
|
||||||
|
<p class="muted small">Für SMB-Freigaben und SFTP-Server mit Passwort. Ein neues Passwort gilt sofort für alle Anbindungen, die es nutzen.</p>
|
||||||
|
{% if not passwords %}<p class="muted">Noch keine Passwörter gespeichert.</p>{% endif %}
|
||||||
|
<div class="stack">
|
||||||
|
{% for p in passwords %}
|
||||||
|
<div class="card" id="cred-{{ p.id }}">
|
||||||
|
<div class="row spread">
|
||||||
|
<div><strong>{{ p.name }}</strong> <span class="badge">{% if p.domain %}{{ p.domain }}\{% endif %}{{ p.username }}</span>
|
||||||
|
{% if p.used_by %}<span class="badge ok">benutzt von {{ p.used_by|join(', ') }}</span>{% else %}<span class="badge">unbenutzt</span>{% endif %}</div>
|
||||||
|
<div class="row">
|
||||||
|
<details class="rename"><summary class="btn small">Ändern</summary>
|
||||||
|
<form class="stack" method="post" action="/credentials/{{ p.id }}/password">{{ csrf_field(csrf) }}
|
||||||
|
<input type="text" name="username" value="{{ p.username }}" aria-label="Benutzer" required>
|
||||||
|
<input type="text" name="domain" value="{{ p.domain or '' }}" placeholder="Domäne (optional)" aria-label="Domäne">
|
||||||
|
<input type="password" name="password" placeholder="neues Passwort (leer = unverändert)" aria-label="Passwort" autocomplete="new-password">
|
||||||
|
<button class="small primary">Speichern</button></form></details>
|
||||||
|
<details class="rename"><summary class="btn small">Umbenennen</summary>
|
||||||
|
<form class="row" method="post" action="/credentials/{{ p.id }}/rename">{{ csrf_field(csrf) }}
|
||||||
|
<input type="text" name="name" value="{{ p.name }}" aria-label="Neuer Name" required maxlength="100">
|
||||||
|
<button class="small primary">Speichern</button></form></details>
|
||||||
|
<form class="inline" method="post" action="/credentials/{{ p.id }}/delete" data-confirm="„{{ p.name }}“ löschen?">{{ csrf_field(csrf) }}
|
||||||
|
<button class="small danger"{% if p.used_by %} disabled title="Wird noch benutzt"{% endif %}>Löschen</button></form>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="small muted">angelegt {{ p.created_at|localdt }}</div>
|
||||||
|
</div>
|
||||||
|
{% endfor %}
|
||||||
|
</div>
|
||||||
|
<form class="form card stack" method="post" action="/credentials/passwords">{{ csrf_field(csrf) }}
|
||||||
|
<div class="fields">
|
||||||
|
<div class="field"><label for="p-name">Name</label><input id="p-name" type="text" name="name" placeholder="z. B. NAS-Scanner" required maxlength="100"></div>
|
||||||
|
<div class="field"><label for="p-user">Benutzer</label><input id="p-user" type="text" name="username" required autocomplete="off"></div>
|
||||||
|
<div class="field"><label for="p-pass">Passwort</label><input id="p-pass" type="password" name="password" required autocomplete="new-password"></div>
|
||||||
|
<div class="field"><label for="p-dom">Domäne (optional)</label><input id="p-dom" type="text" name="domain"></div>
|
||||||
|
</div>
|
||||||
|
<button class="primary">Passwort speichern</button>
|
||||||
|
</form>
|
||||||
|
|
||||||
|
<h2>SSH-Schlüssel</h2>
|
||||||
|
<p class="muted small">Für SFTP-Anbindungen. Ein Schlüssel kann von mehreren Anbindungen benutzt werden; den Benutzernamen legt jede Anbindung selbst fest.</p>
|
||||||
|
|
||||||
|
{% if not keys %}<p class="muted">Noch keine Schlüssel gespeichert.</p>{% endif %}
|
||||||
|
<div class="stack">
|
||||||
|
{% for k in keys %}
|
||||||
|
<div class="card" id="key-{{ k.id }}">
|
||||||
|
<div class="row spread">
|
||||||
|
<div><strong>{{ k.name }}</strong> <span class="badge">{{ k.key_type.removeprefix('ssh-') }}</span>
|
||||||
|
{% if k.used_by %}<span class="badge ok">benutzt von {{ k.used_by|join(', ') }}</span>{% else %}<span class="badge">unbenutzt</span>{% endif %}</div>
|
||||||
|
<div class="row">
|
||||||
|
<details class="rename"><summary class="btn small">Umbenennen</summary>
|
||||||
|
<form class="row" method="post" action="/credentials/{{ k.id }}/rename">{{ csrf_field(csrf) }}
|
||||||
|
<input type="text" name="name" value="{{ k.name }}" aria-label="Neuer Name" required maxlength="100">
|
||||||
|
<button class="small primary">Speichern</button></form></details>
|
||||||
|
<form class="inline" method="post" action="/credentials/{{ k.id }}/delete" data-confirm="Schlüssel „{{ k.name }}“ löschen?">{{ csrf_field(csrf) }}
|
||||||
|
<button class="small danger"{% if k.used_by %} disabled title="Wird noch benutzt"{% endif %}>Löschen</button></form>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="small muted">Fingerabdruck <span class="mono">{{ k.fingerprint }}</span> · angelegt {{ k.created_at|localdt }}</div>
|
||||||
|
<details class="small"><summary>Öffentlicher Schlüssel / authorized_keys</summary>
|
||||||
|
<div class="field"><label>Zeile für <code>~/.ssh/authorized_keys</code> (nur SFTP erlaubt, empfohlen)</label>
|
||||||
|
<pre class="mono card" id="auth-{{ k.id }}">{{ k.authorized_keys_line }}</pre>
|
||||||
|
<button class="small" type="button" data-copy="auth-{{ k.id }}">kopieren</button></div>
|
||||||
|
<div class="field"><label>Öffentlicher Schlüssel</label>
|
||||||
|
<pre class="mono card" id="pub-{{ k.id }}">{{ k.public_key }}</pre>
|
||||||
|
<button class="small" type="button" data-copy="pub-{{ k.id }}">kopieren</button></div>
|
||||||
|
</details>
|
||||||
|
</div>
|
||||||
|
{% endfor %}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<h3>Neuen Schlüssel erzeugen</h3>
|
||||||
|
<form class="form card row" method="post" action="/credentials/keys/generate">{{ csrf_field(csrf) }}
|
||||||
|
<input class="w-auto" type="text" name="name" placeholder="Name, z. B. paperless-lxc" aria-label="Name" required maxlength="100">
|
||||||
|
<button class="primary">Ed25519-Schlüssel erzeugen</button>
|
||||||
|
</form>
|
||||||
|
|
||||||
|
<h3>Vorhandenen Schlüssel importieren</h3>
|
||||||
|
<form class="form card stack" method="post" action="/credentials/keys/import">{{ csrf_field(csrf) }}
|
||||||
|
<div class="fields">
|
||||||
|
<div class="field"><label for="k-name">Name</label><input id="k-name" type="text" name="name" required maxlength="100"></div>
|
||||||
|
<div class="field"><label for="k-pass">Passphrase (falls vorhanden)</label><input id="k-pass" type="password" name="passphrase" autocomplete="off"></div>
|
||||||
|
<div class="field wide"><label for="k-priv">Privater Schlüssel</label>
|
||||||
|
<textarea id="k-priv" name="private_key" class="mono" rows="8" autocomplete="off" spellcheck="false"
|
||||||
|
placeholder="-----BEGIN OPENSSH PRIVATE KEY----- … -----END OPENSSH PRIVATE KEY-----" required></textarea>
|
||||||
|
<span class="hint">Ed25519, ECDSA oder RSA (mind. 2048 Bit), OpenSSH- oder PEM-Format. Eine Passphrase wird nur zum Entschlüsseln benutzt.</span></div>
|
||||||
|
</div>
|
||||||
|
<button>Importieren</button>
|
||||||
|
</form>
|
||||||
|
{% endblock %}
|
||||||
@@ -1,57 +0,0 @@
|
|||||||
{% extends "base.html" %}
|
|
||||||
{% from "macros.html" import csrf_field %}
|
|
||||||
{% block title %}Schlüssel{% endblock %}
|
|
||||||
{% block content %}
|
|
||||||
<h1>Schlüssel</h1>
|
|
||||||
<p class="muted small">Gespeicherte Schlüssel für SFTP-Anbindungen. Ein Schlüssel kann von mehreren
|
|
||||||
Anbindungen benutzt werden; den Benutzernamen legt jede Anbindung selbst fest. Private Schlüssel
|
|
||||||
liegen verschlüsselt in der Datenbank und werden nie angezeigt.</p>
|
|
||||||
|
|
||||||
{% if not keys %}<p class="muted">Noch keine Schlüssel gespeichert.</p>{% endif %}
|
|
||||||
<div class="stack">
|
|
||||||
{% for k in keys %}
|
|
||||||
<div class="card" id="key-{{ k.id }}">
|
|
||||||
<div class="row spread">
|
|
||||||
<div><strong>{{ k.name }}</strong> <span class="badge">{{ k.key_type.removeprefix('ssh-') }}</span>
|
|
||||||
{% if k.used_by %}<span class="badge ok">benutzt von {{ k.used_by|join(', ') }}</span>{% else %}<span class="badge">unbenutzt</span>{% endif %}</div>
|
|
||||||
<div class="row">
|
|
||||||
<details class="rename"><summary class="btn small">Umbenennen</summary>
|
|
||||||
<form class="row" method="post" action="/keys/{{ k.id }}/rename">{{ csrf_field(csrf) }}
|
|
||||||
<input type="text" name="name" value="{{ k.name }}" aria-label="Neuer Name" required maxlength="100">
|
|
||||||
<button class="small primary">Speichern</button></form></details>
|
|
||||||
<form class="inline" method="post" action="/keys/{{ k.id }}/delete" data-confirm="Schlüssel „{{ k.name }}“ löschen?">{{ csrf_field(csrf) }}
|
|
||||||
<button class="small danger"{% if k.used_by %} disabled title="Wird noch benutzt"{% endif %}>Löschen</button></form>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<div class="small muted">Fingerabdruck <span class="mono">{{ k.fingerprint }}</span> · angelegt {{ k.created_at|localdt }}</div>
|
|
||||||
<details class="small"><summary>Öffentlicher Schlüssel / authorized_keys</summary>
|
|
||||||
<div class="field"><label>Zeile für <code>~/.ssh/authorized_keys</code> (nur SFTP erlaubt, empfohlen)</label>
|
|
||||||
<pre class="mono card" id="auth-{{ k.id }}">{{ k.authorized_keys_line }}</pre>
|
|
||||||
<button class="small" type="button" data-copy="auth-{{ k.id }}">kopieren</button></div>
|
|
||||||
<div class="field"><label>Öffentlicher Schlüssel</label>
|
|
||||||
<pre class="mono card" id="pub-{{ k.id }}">{{ k.public_key }}</pre>
|
|
||||||
<button class="small" type="button" data-copy="pub-{{ k.id }}">kopieren</button></div>
|
|
||||||
</details>
|
|
||||||
</div>
|
|
||||||
{% endfor %}
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<h2>Neuen Schlüssel erzeugen</h2>
|
|
||||||
<form class="form card row" method="post" action="/keys/generate">{{ csrf_field(csrf) }}
|
|
||||||
<input class="w-auto" type="text" name="name" placeholder="Name, z. B. paperless-lxc" aria-label="Name" required maxlength="100">
|
|
||||||
<button class="primary">Ed25519-Schlüssel erzeugen</button>
|
|
||||||
</form>
|
|
||||||
|
|
||||||
<h2>Vorhandenen Schlüssel importieren</h2>
|
|
||||||
<form class="form card stack" method="post" action="/keys/import">{{ csrf_field(csrf) }}
|
|
||||||
<div class="fields">
|
|
||||||
<div class="field"><label for="k-name">Name</label><input id="k-name" type="text" name="name" required maxlength="100"></div>
|
|
||||||
<div class="field"><label for="k-pass">Passphrase (falls vorhanden)</label><input id="k-pass" type="password" name="passphrase" autocomplete="off"></div>
|
|
||||||
<div class="field wide"><label for="k-priv">Privater Schlüssel</label>
|
|
||||||
<textarea id="k-priv" name="private_key" class="mono" rows="8" autocomplete="off" spellcheck="false"
|
|
||||||
placeholder="-----BEGIN OPENSSH PRIVATE KEY----- … -----END OPENSSH PRIVATE KEY-----" required></textarea>
|
|
||||||
<span class="hint">Ed25519, ECDSA oder RSA (mind. 2048 Bit), OpenSSH- oder PEM-Format. Eine Passphrase wird nur zum Entschlüsseln benutzt.</span></div>
|
|
||||||
</div>
|
|
||||||
<button>Importieren</button>
|
|
||||||
</form>
|
|
||||||
{% endblock %}
|
|
||||||
@@ -1,15 +1,17 @@
|
|||||||
{% extends "base.html" %}
|
{% extends "base.html" %}
|
||||||
{% from "macros.html" import field, select, check %}
|
{% from "macros.html" import field, select, check %}
|
||||||
{% block title %}{{ 'Anbindung bearbeiten' if mount else 'Neue Anbindung' }}{% endblock %}
|
{% block title %}{{ 'Anbindung bearbeiten' if mount else ('Anbindung kopieren' if copy_from else 'Neue Anbindung') }}{% endblock %}
|
||||||
{% block content %}
|
{% block content %}
|
||||||
<h1>{{ 'Anbindung „' ~ mount.name ~ '“ bearbeiten' if mount else 'Neue Anbindung' }}</h1>
|
<h1>{% if mount %}Anbindung „{{ mount.name }}“ bearbeiten{% elif copy_from %}Kopie von „{{ copy_from.name }}“{% else %}Neue Anbindung{% endif %}</h1>
|
||||||
|
{% if copy_from %}<p class="msg info small">Alle Werte stammen aus „{{ copy_from.name }}“. Zugangsdaten werden übernommen, solange du das Passwort leer lässt – die Kopie bekommt ihr eigenes Exemplar.{% if copy_from.credential and copy_from.credential.public_key and not copy_from.credential.managed %} Der SSH-Schlüssel bleibt derselbe, ein neuer Eintrag in <code>authorized_keys</code> ist nicht nötig.{% endif %}</p>{% endif %}
|
||||||
{% if errors.get('_') %}<p class="msg error">{{ errors['_'] }}</p>{% endif %}
|
{% if errors.get('_') %}<p class="msg error">{{ errors['_'] }}</p>{% endif %}
|
||||||
<form class="form" method="post" data-dynamic>
|
<form class="form" method="post" data-dynamic>
|
||||||
<input type="hidden" name="csrf_token" value="{{ csrf }}">
|
<input type="hidden" name="csrf_token" value="{{ csrf }}">
|
||||||
{% if mount %}<input type="hidden" name="mount_id" value="{{ mount.id }}">{% endif %}
|
{% if mount %}<input type="hidden" name="mount_id" value="{{ mount.id }}">{% endif %}
|
||||||
|
{% if copy_from %}<input type="hidden" name="copy_from" value="{{ copy_from.id }}">{% endif %}
|
||||||
<fieldset><legend>Verbindung <a class="legend-help" href="/help#anbindung-felder" title="Hilfe">?</a></legend><div class="fields">
|
<fieldset><legend>Verbindung <a class="legend-help" href="/help#anbindung-felder" title="Hilfe">?</a></legend><div class="fields">
|
||||||
{{ field('name', 'Name', d.name, errors=errors, attrs='required maxlength="100"', hint=('Verzeichnis: ' ~ mount.mountpoint if mount and mount.mountpoint else '')) }}
|
{{ field('name', 'Name', d.name, errors=errors, attrs='required maxlength="100"', hint=('Verzeichnis: ' ~ mount.mountpoint if mount and mount.mountpoint else '')) }}
|
||||||
{{ select('type', 'Typ', [('cifs', 'SMB/CIFS'), ('nfs4', 'NFS 4'), ('nfs3', 'NFS 3'), ('sftp', 'SFTP (nur Ziel)')], d.type.value, errors=errors) }}
|
{{ select('type', 'Typ', [('cifs', 'SMB/CIFS'), ('nfs4', 'NFS 4'), ('nfs3', 'NFS 3'), ('sftp', 'SFTP')], d.type.value, errors=errors) }}
|
||||||
{{ field('server', 'Server', d.server, errors=errors, placeholder='192.168.1.10 oder nas.lan', attrs='required') }}
|
{{ field('server', 'Server', d.server, errors=errors, placeholder='192.168.1.10 oder nas.lan', attrs='required') }}
|
||||||
<div class="field{{ ' has-error' if errors.get('share') }}">
|
<div class="field{{ ' has-error' if errors.get('share') }}">
|
||||||
<label for="f-share">Freigabe / Pfad</label>
|
<label for="f-share">Freigabe / Pfad</label>
|
||||||
@@ -25,16 +27,21 @@
|
|||||||
<fieldset data-show-for="type:cifs sftp"><legend>Anmeldung <a class="legend-help" href="/help#anbindung-felder" title="Hilfe">?</a></legend><div class="fields">
|
<fieldset data-show-for="type:cifs sftp"><legend>Anmeldung <a class="legend-help" href="/help#anbindung-felder" title="Hilfe">?</a></legend><div class="fields">
|
||||||
<div data-show-for="type:sftp">{{ select('auth', 'Anmeldeart', [('ssh_key', 'SSH-Schlüssel'), ('password', 'Passwort')], d.auth.value, errors=errors) }}</div>
|
<div data-show-for="type:sftp">{{ select('auth', 'Anmeldeart', [('ssh_key', 'SSH-Schlüssel'), ('password', 'Passwort')], d.auth.value, errors=errors) }}</div>
|
||||||
<div data-show-for="type:sftp"><div data-show-for="auth:ssh_key">
|
<div data-show-for="type:sftp"><div data-show-for="auth:ssh_key">
|
||||||
{{ select('key_id', 'Schlüssel', key_options, d.key_id, errors=errors, hint='Gespeicherte Schlüssel verwalten unter „Schlüssel“.') }}
|
{{ select('key_id', 'Schlüssel', key_options, d.key_id, errors=errors, hint='Verwalten unter „Zugangsdaten“.') }}
|
||||||
</div></div>
|
</div></div>
|
||||||
{{ field('username', 'Benutzername', d.username, errors=errors, attrs='autocomplete="off"') }}
|
<div data-show-for="type:cifs|auth:password">
|
||||||
<div data-show-for="type:cifs">{{ field('domain', 'Domäne (optional)', d.domain, errors=errors) }}</div>
|
{{ select('cred_id', 'Zugangsdaten', password_options, d.cred_id, errors=errors, hint='Gespeicherte Passwörter verwalten unter „Zugangsdaten“.') }}
|
||||||
<div class="field{{ ' has-error' if errors.get('password') }}" id="pw-field" data-show-for="type:cifs|auth:password">
|
</div>
|
||||||
|
<div data-show-for="auth:ssh_key|cred_id:own">
|
||||||
|
{{ field('username', 'Benutzername', d.username, errors=errors, attrs='autocomplete="off"') }}
|
||||||
|
</div>
|
||||||
|
<div data-show-for="cred_id:own"><div data-show-for="type:cifs">{{ field('domain', 'Domäne (optional)', d.domain, errors=errors) }}</div></div>
|
||||||
|
<div data-show-for="cred_id:own"><div class="field{{ ' has-error' if errors.get('password') }}" id="pw-field" data-show-for="type:cifs|auth:password">
|
||||||
<label for="f-password">Passwort</label>
|
<label for="f-password">Passwort</label>
|
||||||
<input id="f-password" type="password" name="password" autocomplete="new-password" placeholder="{{ '••••••' if has_secret }}">
|
<input id="f-password" type="password" name="password" autocomplete="new-password" placeholder="{{ '••••••' if has_secret }}">
|
||||||
{% if errors.get('password') %}<span class="error">{{ errors['password'] }}</span>{% endif %}
|
{% if errors.get('password') %}<span class="error">{{ errors['password'] }}</span>{% endif %}
|
||||||
<span class="hint">{{ 'Leer lassen = unverändert. ' if has_secret }}<span data-show-for="type:cifs">Benutzer und Passwort leer = Gastzugang.</span></span>
|
<span class="hint">{{ 'Leer lassen = unverändert. ' if has_secret }}<span data-show-for="type:cifs">Benutzer und Passwort leer = Gastzugang.</span></span>
|
||||||
</div>
|
</div></div>
|
||||||
</div>
|
</div>
|
||||||
{% if public_key %}<div class="field wide"><label>Öffentlicher Schlüssel dieser Anbindung (in ~/.ssh/authorized_keys des Zielbenutzers eintragen)</label>
|
{% if public_key %}<div class="field wide"><label>Öffentlicher Schlüssel dieser Anbindung (in ~/.ssh/authorized_keys des Zielbenutzers eintragen)</label>
|
||||||
<pre class="mono card" id="pubkey">{{ public_key }}</pre><button class="small" type="button" data-copy="pubkey">kopieren</button></div>{% endif %}
|
<pre class="mono card" id="pubkey">{{ public_key }}</pre><button class="small" type="button" data-copy="pubkey">kopieren</button></div>{% endif %}
|
||||||
|
|||||||
@@ -22,19 +22,18 @@
|
|||||||
{% endif %}
|
{% endif %}
|
||||||
{% endif %}
|
{% endif %}
|
||||||
<a class="btn small" href="/mounts/{{ m.id }}/edit">Bearbeiten</a>
|
<a class="btn small" href="/mounts/{{ m.id }}/edit">Bearbeiten</a>
|
||||||
|
<a class="btn small" href="/mounts/{{ m.id }}/copy" title="Neue Anbindung mit denselben Werten und Zugangsdaten">Kopieren</a>
|
||||||
<form class="inline" method="post" action="/mounts/{{ m.id }}/delete" data-confirm="„{{ m.name }}“ wirklich löschen?">{{ csrf_field(csrf) }}<button class="small danger">Löschen</button></form>
|
<form class="inline" method="post" action="/mounts/{{ m.id }}/delete" data-confirm="„{{ m.name }}“ wirklich löschen?">{{ csrf_field(csrf) }}<button class="small danger">Löschen</button></form>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
<div class="muted small mono">{{ m.type.value }} · {% if m.type == MountType.CIFS %}//{{ m.server }}/{{ m.share }}{% elif m.type == MountType.SFTP %}{{ m.server }}:{{ m.port or 22 }}{{ m.share }}{% else %}{{ m.server }}:{{ m.share }}{% endif %}
|
<div class="muted small mono">{{ m.type.value }} · {% if m.type == MountType.CIFS %}//{{ m.server }}/{{ m.share }}{% elif m.type == MountType.SFTP %}{{ m.server }}:{{ m.port or 22 }}{{ m.share }}{% else %}{{ m.server }}:{{ m.share }}{% endif %}
|
||||||
{% if m.mountpoint %} → {{ m.mountpoint }}{% endif %}</div>
|
{% if m.mountpoint %} → {{ m.mountpoint }}{% endif %}</div>
|
||||||
{% if m.last_error %}<pre class="small text-err">{{ m.last_error }}</pre>{% endif %}
|
{% if m.last_error %}<pre class="small text-err">{{ m.last_error }}</pre>{% endif %}
|
||||||
|
{% if key_names.get(m.id) %}<div class="small">Anmeldung mit gespeicherten Zugangsdaten <a href="/credentials">„{{ key_names[m.id] }}“</a></div>{% endif %}
|
||||||
{% if m.type == MountType.SFTP %}
|
{% if m.type == MountType.SFTP %}
|
||||||
<div class="small">Hostschlüssel: {% if m.host_key %}<span class="mono">{{ m.host_key.split()[0] }}</span> festgehalten
|
<div class="small">Hostschlüssel: {% if m.host_key %}<span class="mono">{{ m.host_key.split()[0] }}</span> festgehalten
|
||||||
<form class="inline" method="post" action="/mounts/{{ m.id }}/reset-hostkey" data-confirm="Hostschlüssel zurücksetzen? Nur tun, wenn der Server neu aufgesetzt wurde.">{{ csrf_field(csrf) }}<button class="small">zurücksetzen</button></form>
|
<form class="inline" method="post" action="/mounts/{{ m.id }}/reset-hostkey" data-confirm="Hostschlüssel zurücksetzen? Nur tun, wenn der Server neu aufgesetzt wurde.">{{ csrf_field(csrf) }}<button class="small">zurücksetzen</button></form>
|
||||||
{% else %}<span class="text-warn">noch nicht bestätigt – bitte „Testen“</span>{% endif %}</div>
|
{% else %}<span class="text-warn">noch nicht bestätigt – bitte „Testen“</span>{% endif %}</div>
|
||||||
{% if key_names.get(m.id) %}
|
|
||||||
<div class="small">Anmeldung mit gespeichertem Schlüssel <a href="/keys">„{{ key_names[m.id] }}“</a></div>
|
|
||||||
{% endif %}
|
|
||||||
{% if pubkeys.get(m.id) %}
|
{% if pubkeys.get(m.id) %}
|
||||||
<details class="small"><summary>Öffentlicher Schlüssel für authorized_keys</summary>
|
<details class="small"><summary>Öffentlicher Schlüssel für authorized_keys</summary>
|
||||||
<pre class="mono" id="pub-{{ m.id }}">{{ pubkeys[m.id] }}</pre>
|
<pre class="mono" id="pub-{{ m.id }}">{{ pubkeys[m.id] }}</pre>
|
||||||
|
|||||||
@@ -1,10 +1,13 @@
|
|||||||
{% extends "base.html" %}
|
{% extends "base.html" %}
|
||||||
{% from "macros.html" import field, select, check %}
|
{% from "macros.html" import field, select, check %}
|
||||||
{% block title %}{{ 'Regel bearbeiten' if rule_id else 'Neue Regel' }}{% endblock %}
|
{% block title %}{{ 'Regel bearbeiten' if rule_id else ('Regel kopieren' if copy_of else 'Neue Regel') }}{% endblock %}
|
||||||
{% block content %}
|
{% block content %}
|
||||||
<h1>{{ 'Regel bearbeiten' if rule_id else 'Neue Regel' }}</h1>
|
<h1>{% if rule_id %}Regel bearbeiten{% elif copy_of %}Kopie von „{{ copy_of }}“{% else %}Neue Regel{% endif %}</h1>
|
||||||
|
{% if copy_of %}<p class="msg info small">Alle Einstellungen stammen aus „{{ copy_of }}“. Die Kopie ist zunächst
|
||||||
|
<strong>deaktiviert</strong>, damit sie nicht mit dem Original um dieselben Dateien konkurriert – Quelle bzw. Ziel
|
||||||
|
anpassen, speichern, Probelauf machen, dann aktivieren. Protokoll und Buchführung beginnen bei der Kopie neu.</p>{% endif %}
|
||||||
{% if errors.get('_') %}<p class="msg error">{{ errors['_'] }}</p>{% endif %}
|
{% if errors.get('_') %}<p class="msg error">{{ errors['_'] }}</p>{% endif %}
|
||||||
{% if not src_mounts %}<p class="msg info">Zuerst eine SMB- oder NFS-Anbindung als Quelle <a href="/mounts/new">anlegen</a>.</p>{% endif %}
|
{% if not src_mounts %}<p class="msg info">Zuerst eine Anbindung als Quelle <a href="/mounts/new">anlegen</a>.</p>{% endif %}
|
||||||
<form class="form" method="post" data-dynamic>
|
<form class="form" method="post" data-dynamic>
|
||||||
<input type="hidden" name="csrf_token" value="{{ csrf }}">
|
<input type="hidden" name="csrf_token" value="{{ csrf }}">
|
||||||
<fieldset><legend>Allgemein <a class="legend-help" href="/help#regeln" title="Hilfe">?</a></legend><div class="fields">
|
<fieldset><legend>Allgemein <a class="legend-help" href="/help#regeln" title="Hilfe">?</a></legend><div class="fields">
|
||||||
@@ -30,6 +33,13 @@
|
|||||||
{{ select('on_collision', 'Wenn Datei existiert', [('rename', 'umbenennen (scan-1.pdf)'), ('skip', 'überspringen'), ('overwrite', 'überschreiben')], d.on_collision.value, errors=errors) }}
|
{{ select('on_collision', 'Wenn Datei existiert', [('rename', 'umbenennen (scan-1.pdf)'), ('skip', 'überspringen'), ('overwrite', 'überschreiben')], d.on_collision.value, errors=errors) }}
|
||||||
</div></fieldset>
|
</div></fieldset>
|
||||||
|
|
||||||
|
<fieldset><legend>Archiv <a class="legend-help" href="/help#archiv" title="Hilfe">?</a></legend><div class="fields">
|
||||||
|
{{ select('archive_kind', 'Archiv', [('none', 'kein Archiv'), ('mount', 'Anbindung'), ('local', 'Lokaler Pfad')], d.archive_kind.value, errors=errors, hint='Zusätzliche Kopie. Die Quelle wird erst gelöscht, wenn Ziel UND Archiv sie haben.') }}
|
||||||
|
<div data-show-for="archive_kind:mount">{{ select('archive_mount_id', 'Archiv-Anbindung', dst_options, d.archive_mount_id or '', errors=errors) }}</div>
|
||||||
|
<div data-show-for="archive_kind:mount local">{{ field('archive_path', 'Archivordner', d.archive_path, errors=errors, hint='Anbindung: relativer Unterordner (wird angelegt). Lokal: absoluter Pfad, muss existieren.') }}</div>
|
||||||
|
<div data-show-for="archive_kind:mount local">{{ select('archive_layout', 'Ablage', [('flat', 'wie in der Quelle'), ('date', 'nach Datum (Jahr/Monat)')], d.archive_layout.value, errors=errors, hint='Im Archiv wird nie überschrieben – gleiche Namen werden umbenannt.') }}</div>
|
||||||
|
</div></fieldset>
|
||||||
|
|
||||||
<fieldset><legend>Sicherheit und Grenzen <a class="legend-help" href="/help#regel-sicherheit" title="Hilfe">?</a></legend><div class="fields">
|
<fieldset><legend>Sicherheit und Grenzen <a class="legend-help" href="/help#regel-sicherheit" title="Hilfe">?</a></legend><div class="fields">
|
||||||
{{ field('stability_seconds', 'Stabilitätsfenster (s)', d.stability_seconds, type='number', errors=errors, attrs='min="0"', hint='Größe und Zeit müssen so lange gleich bleiben') }}
|
{{ field('stability_seconds', 'Stabilitätsfenster (s)', d.stability_seconds, type='number', errors=errors, attrs='min="0"', hint='Größe und Zeit müssen so lange gleich bleiben') }}
|
||||||
{{ field('min_age_seconds', 'Mindestalter (s)', d.min_age_seconds, type='number', errors=errors, attrs='min="0"') }}
|
{{ field('min_age_seconds', 'Mindestalter (s)', d.min_age_seconds, type='number', errors=errors, attrs='min="0"') }}
|
||||||
|
|||||||
@@ -18,6 +18,7 @@
|
|||||||
<button class="small" hx-post="/rules/{{ r.id }}/dryrun" hx-target="#dry-{{ r.id }}" hx-swap="innerHTML">Probelauf <span class="htmx-indicator">…</span></button>
|
<button class="small" hx-post="/rules/{{ r.id }}/dryrun" hx-target="#dry-{{ r.id }}" hx-swap="innerHTML">Probelauf <span class="htmx-indicator">…</span></button>
|
||||||
<form class="inline" method="post" action="/rules/{{ r.id }}/toggle">{{ csrf_field(csrf) }}<button class="small">{{ 'Deaktivieren' if r.enabled else 'Aktivieren' }}</button></form>
|
<form class="inline" method="post" action="/rules/{{ r.id }}/toggle">{{ csrf_field(csrf) }}<button class="small">{{ 'Deaktivieren' if r.enabled else 'Aktivieren' }}</button></form>
|
||||||
<a class="btn small" href="/rules/{{ r.id }}/edit">Bearbeiten</a>
|
<a class="btn small" href="/rules/{{ r.id }}/edit">Bearbeiten</a>
|
||||||
|
<a class="btn small" href="/rules/{{ r.id }}/copy" title="Neue Regel mit denselben Einstellungen">Kopieren</a>
|
||||||
<form class="inline" method="post" action="/rules/{{ r.id }}/delete" data-confirm="Regel „{{ r.name }}“ und ihr Protokoll löschen?">{{ csrf_field(csrf) }}<button class="small danger">Löschen</button></form>
|
<form class="inline" method="post" action="/rules/{{ r.id }}/delete" data-confirm="Regel „{{ r.name }}“ und ihr Protokoll löschen?">{{ csrf_field(csrf) }}<button class="small danger">Löschen</button></form>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -65,6 +65,27 @@
|
|||||||
<button>Passwort ändern</button>
|
<button>Passwort ändern</button>
|
||||||
</form>
|
</form>
|
||||||
|
|
||||||
|
<h2 id="anmeldung">Anmeldung <a class="legend-help" href="/help#anmeldung" title="Hilfe">?</a></h2>
|
||||||
|
<form class="form card stack" method="post" action="/settings/auth">
|
||||||
|
{{ csrf_field(csrf) }}
|
||||||
|
<p class="small muted">Deine Adresse gerade: <code>{{ client_ip }}</code></p>
|
||||||
|
<div class="fields">
|
||||||
|
<div class="field wide"><label for="f-nets">Ohne Passwort aus diesen Netzen</label>
|
||||||
|
<input id="f-nets" type="text" name="trusted_networks" value="{{ cfg.trusted_networks }}" placeholder="z. B. 10.0.10.0/24, 10.1.5.0/24">
|
||||||
|
<span class="hint">Mit Komma getrennt; einzelne Adressen gehen auch. Leer = niemand ohne Passwort. <code>0.0.0.0/0, ::/0</code> = Anmeldung ganz aus.</span></div>
|
||||||
|
<div class="field wide"><label for="f-allowed">Mit Passwort aus diesen Netzen</label>
|
||||||
|
<input id="f-allowed" type="text" name="allowed_networks" value="{{ cfg.allowed_networks }}" placeholder="leer = alle übrigen Netze">
|
||||||
|
<span class="hint">Leer = von überall sonst mit Passwort. Ausgefüllt = <strong>alle anderen Netze werden abgewiesen</strong> (auch die Anmeldeseite). <code>/healthz</code> bleibt für die Überwachung erreichbar.</span></div>
|
||||||
|
<div class="field"><label for="f-cur-auth">Aktuelles Passwort (zur Bestätigung)</label>
|
||||||
|
<input id="f-cur-auth" type="password" name="current" autocomplete="current-password" required></div>
|
||||||
|
</div>
|
||||||
|
<p class="msg warn small">NetMover läuft als root und kennt die Zugangsdaten deiner Freigaben. Wer die Oberfläche ohne Passwort erreicht,
|
||||||
|
kann alles ändern – Netze „ohne Passwort“ daher so eng wie möglich wählen. Hinter einem Reverse Proxy <code>NETMOVER_FORWARDED_ALLOW_IPS</code> setzen,
|
||||||
|
sonst sieht jede Anfrage aus, als käme sie vom Proxy. Eine Einstellung, mit der du dich selbst aussperren würdest, wird nicht gespeichert.</p>
|
||||||
|
{% if loopback_trusted %}<p class="msg warn small">„Ohne Passwort“ enthält 127.0.0.1/::1. Läuft ein Reverse Proxy auf diesem Server, gilt dann <strong>jede</strong> Anfrage über den Proxy als vertrauenswürdig.</p>{% endif %}
|
||||||
|
<div class="row"><button class="primary">Anmeldung speichern</button></div>
|
||||||
|
</form>
|
||||||
|
|
||||||
<h2>Export / Import</h2>
|
<h2>Export / Import</h2>
|
||||||
<div class="card stack">
|
<div class="card stack">
|
||||||
<p class="small muted">Export enthält Anbindungen, Regeln und Einstellungen – <strong>ohne</strong> Zugangsdaten und Webhook-URL.
|
<p class="small muted">Export enthält Anbindungen, Regeln und Einstellungen – <strong>ohne</strong> Zugangsdaten und Webhook-URL.
|
||||||
|
|||||||
@@ -48,7 +48,11 @@ class _Server(paramiko.ServerInterface):
|
|||||||
|
|
||||||
|
|
||||||
class _Handle(SFTPHandle):
|
class _Handle(SFTPHandle):
|
||||||
pass
|
def stat(self) -> Any: # fstat wie bei OpenSSH
|
||||||
|
try:
|
||||||
|
return SFTPAttributes.from_stat(os.fstat(self.readfile.fileno()))
|
||||||
|
except OSError as e:
|
||||||
|
return SFTPServer.convert_errno(e.errno)
|
||||||
|
|
||||||
|
|
||||||
class _RootedSFTP(SFTPServerInterface):
|
class _RootedSFTP(SFTPServerInterface):
|
||||||
|
|||||||
@@ -0,0 +1,351 @@
|
|||||||
|
"""0.7.0: Archiv – Kopie ans Ziel UND ins Archiv, Quelle erst danach löschen."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
from collections.abc import Iterator
|
||||||
|
from datetime import datetime
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from sqlalchemy import select
|
||||||
|
from sqlalchemy.orm import Session, sessionmaker
|
||||||
|
|
||||||
|
from netmover.db import session_scope
|
||||||
|
from netmover.models import (
|
||||||
|
ArchivedFile,
|
||||||
|
ArchiveKind,
|
||||||
|
CollisionPolicy,
|
||||||
|
ItemStatus,
|
||||||
|
Rule,
|
||||||
|
RuleAction,
|
||||||
|
RunStatus,
|
||||||
|
)
|
||||||
|
from netmover.mounting.fake import FakeMountBackend
|
||||||
|
from netmover.mounting.manager import MountManager
|
||||||
|
from netmover.mover.destinations import LocalDestination, TransferError
|
||||||
|
from netmover.mover.engine import Engine, RuleSnapshot, RunResult
|
||||||
|
from netmover.mover.ledger import Ledger
|
||||||
|
from netmover.mover.runner import RuleRunner
|
||||||
|
from netmover.timeouts import CancelToken
|
||||||
|
from tests.conftest import MountFactory
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def rule_id(make_mount: MountFactory, session_factory: sessionmaker[Session]) -> int:
|
||||||
|
mid = make_mount()
|
||||||
|
with session_scope(session_factory) as s:
|
||||||
|
r = Rule(name="r", src_mount_id=mid, dst_path="/x")
|
||||||
|
s.add(r)
|
||||||
|
s.flush()
|
||||||
|
return r.id
|
||||||
|
|
||||||
|
|
||||||
|
FIXED = datetime(2026, 10, 12, 9, 30).timestamp()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def engine(session_factory: sessionmaker[Session]) -> Engine:
|
||||||
|
return Engine(Ledger(session_factory), sleep=lambda _: None, clock=lambda: FIXED)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def dirs(tmp_path: Path) -> tuple[Path, Path, Path]:
|
||||||
|
src, dst, arch = tmp_path / "src", tmp_path / "dst", tmp_path / "archiv"
|
||||||
|
for d in (src, dst, arch):
|
||||||
|
d.mkdir()
|
||||||
|
return src, dst, arch
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def cross_device(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
"""tmp_path liegt auf einem Gerät – realistischer Weg über Kopie erzwingen."""
|
||||||
|
monkeypatch.setattr(LocalDestination, "move_same_device", lambda *a, **k: None)
|
||||||
|
|
||||||
|
|
||||||
|
def snap(rule_id: int, **kw: object) -> RuleSnapshot:
|
||||||
|
return RuleSnapshot(id=rule_id, name="r", stability_seconds=0, **kw) # type: ignore[arg-type]
|
||||||
|
|
||||||
|
|
||||||
|
def run(
|
||||||
|
engine: Engine, rule: RuleSnapshot, src: Path, dst: Path, arch: Path | None, **kw: object
|
||||||
|
) -> RunResult:
|
||||||
|
archive = LocalDestination(arch) if arch is not None else None
|
||||||
|
return engine.execute(rule, src, LocalDestination(dst), CancelToken(), archive=archive, **kw) # type: ignore[arg-type]
|
||||||
|
|
||||||
|
|
||||||
|
def write(p: Path, data: bytes = b"inhalt") -> Path:
|
||||||
|
p.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
p.write_bytes(data)
|
||||||
|
return p
|
||||||
|
|
||||||
|
|
||||||
|
def names(d: Path) -> list[str]:
|
||||||
|
return sorted(str(p.relative_to(d)) for p in d.rglob("*") if p.is_file())
|
||||||
|
|
||||||
|
|
||||||
|
def test_move_delivers_to_target_and_archive_then_deletes_source(
|
||||||
|
engine: Engine, rule_id: int, dirs: tuple[Path, Path, Path]
|
||||||
|
) -> None:
|
||||||
|
src, dst, arch = dirs
|
||||||
|
data = os.urandom(200_000)
|
||||||
|
write(src / "scan001.pdf", data)
|
||||||
|
result = run(engine, snap(rule_id), src, dst, arch)
|
||||||
|
assert result.status is RunStatus.OK
|
||||||
|
item = result.items[0]
|
||||||
|
assert item.status is ItemStatus.MOVED and item.message and "archiviert:" in item.message
|
||||||
|
assert (dst / "scan001.pdf").read_bytes() == data == (arch / "scan001.pdf").read_bytes()
|
||||||
|
assert not (src / "scan001.pdf").exists()
|
||||||
|
assert names(dst) == ["scan001.pdf"] and names(arch) == ["scan001.pdf"] # keine .part-Reste
|
||||||
|
|
||||||
|
|
||||||
|
def test_date_layout(engine: Engine, rule_id: int, dirs: tuple[Path, Path, Path]) -> None:
|
||||||
|
src, dst, arch = dirs
|
||||||
|
write(src / "max" / "a.pdf")
|
||||||
|
run(engine, snap(rule_id, recursive=True, archive_layout="date"), src, dst, arch)
|
||||||
|
assert names(arch) == ["2026/10/max/a.pdf"] and names(dst) == ["max/a.pdf"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_archive_failure_delivers_nothing_and_keeps_source(
|
||||||
|
engine: Engine, rule_id: int, dirs: tuple[Path, Path, Path], monkeypatch: pytest.MonkeyPatch
|
||||||
|
) -> None:
|
||||||
|
src, dst, arch = dirs
|
||||||
|
write(src / "a.pdf")
|
||||||
|
real = LocalDestination._link_noreplace
|
||||||
|
|
||||||
|
def fail_in_archive(self: LocalDestination, tmp: str, final: str) -> None:
|
||||||
|
if self.root == arch:
|
||||||
|
raise TransferError("Archiv voll")
|
||||||
|
real(self, tmp, final)
|
||||||
|
|
||||||
|
monkeypatch.setattr(LocalDestination, "_link_noreplace", fail_in_archive)
|
||||||
|
result = run(engine, snap(rule_id), src, dst, arch)
|
||||||
|
item = result.items[0]
|
||||||
|
assert item.status is ItemStatus.FAILED and item.message == "Archiv: Archiv voll (Versuch 1)"
|
||||||
|
assert (src / "a.pdf").exists() and names(dst) == [] and names(arch) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_target_failure_after_archive_never_archives_twice(
|
||||||
|
engine: Engine,
|
||||||
|
rule_id: int,
|
||||||
|
dirs: tuple[Path, Path, Path],
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
session_factory: sessionmaker[Session],
|
||||||
|
) -> None:
|
||||||
|
src, dst, arch = dirs
|
||||||
|
write(src / "a.pdf")
|
||||||
|
real = LocalDestination._link_noreplace
|
||||||
|
|
||||||
|
def fail_in_target(self: LocalDestination, tmp: str, final: str) -> None:
|
||||||
|
if self.root == dst:
|
||||||
|
raise TransferError("Ziel gesperrt")
|
||||||
|
real(self, tmp, final)
|
||||||
|
|
||||||
|
monkeypatch.setattr(LocalDestination, "_link_noreplace", fail_in_target)
|
||||||
|
first = run(engine, snap(rule_id), src, dst, arch)
|
||||||
|
assert first.items[0].status is ItemStatus.FAILED
|
||||||
|
assert (src / "a.pdf").exists() and names(arch) == ["a.pdf"] and names(dst) == []
|
||||||
|
with session_factory() as s:
|
||||||
|
assert len(s.scalars(select(ArchivedFile)).all()) == 1
|
||||||
|
|
||||||
|
monkeypatch.setattr(LocalDestination, "_link_noreplace", real) # Ziel wieder frei
|
||||||
|
second = run(engine, snap(rule_id), src, dst, arch)
|
||||||
|
assert second.items[0].status is ItemStatus.MOVED
|
||||||
|
assert names(arch) == ["a.pdf"] # keine zweite Archivkopie (a-1.pdf)
|
||||||
|
assert names(dst) == ["a.pdf"] and not (src / "a.pdf").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_archive_never_overwrites(
|
||||||
|
engine: Engine, rule_id: int, dirs: tuple[Path, Path, Path]
|
||||||
|
) -> None:
|
||||||
|
src, dst, arch = dirs
|
||||||
|
write(arch / "scan.pdf", b"alt im Archiv")
|
||||||
|
write(src / "scan.pdf", b"neu")
|
||||||
|
run(engine, snap(rule_id, on_collision=CollisionPolicy.OVERWRITE), src, dst, arch)
|
||||||
|
assert (arch / "scan.pdf").read_bytes() == b"alt im Archiv"
|
||||||
|
assert (arch / "scan-1.pdf").read_bytes() == b"neu"
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_space_in_archive_writes_nothing(
|
||||||
|
engine: Engine, rule_id: int, dirs: tuple[Path, Path, Path], monkeypatch: pytest.MonkeyPatch
|
||||||
|
) -> None:
|
||||||
|
src, dst, arch = dirs
|
||||||
|
write(src / "a.pdf", b"x" * 1000)
|
||||||
|
real = LocalDestination.free_bytes
|
||||||
|
monkeypatch.setattr(
|
||||||
|
LocalDestination, "free_bytes", lambda self: 10 if self.root == arch else real(self)
|
||||||
|
)
|
||||||
|
result = run(engine, snap(rule_id), src, dst, arch)
|
||||||
|
assert "zu wenig Platz im Archiv" in (result.items[0].message or "")
|
||||||
|
assert names(dst) == [] and names(arch) == [] and (src / "a.pdf").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_source_changed_during_archive_copy(
|
||||||
|
engine: Engine, rule_id: int, dirs: tuple[Path, Path, Path], monkeypatch: pytest.MonkeyPatch
|
||||||
|
) -> None:
|
||||||
|
src, dst, arch = dirs
|
||||||
|
f = write(src / "a.pdf", b"original")
|
||||||
|
real = LocalDestination._apply_meta
|
||||||
|
|
||||||
|
def change(self: LocalDestination, rel: str, meta: object) -> list[str]:
|
||||||
|
if self.root == arch:
|
||||||
|
f.write_bytes(b"geaendert!!")
|
||||||
|
return real(self, rel, meta) # type: ignore[arg-type]
|
||||||
|
|
||||||
|
monkeypatch.setattr(LocalDestination, "_apply_meta", change)
|
||||||
|
result = run(engine, snap(rule_id), src, dst, arch)
|
||||||
|
assert result.items[0].status is ItemStatus.SKIPPED
|
||||||
|
assert names(dst) == [] and names(arch) == [] and f.exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_copy_mode_with_archive(
|
||||||
|
engine: Engine, rule_id: int, dirs: tuple[Path, Path, Path]
|
||||||
|
) -> None:
|
||||||
|
src, dst, arch = dirs
|
||||||
|
f = write(src / "a.pdf", b"v1")
|
||||||
|
rule = snap(rule_id, action=RuleAction.COPY)
|
||||||
|
for _ in range(3):
|
||||||
|
run(engine, rule, src, dst, arch)
|
||||||
|
assert f.exists() and names(dst) == ["a.pdf"] and names(arch) == ["a.pdf"]
|
||||||
|
f.write_bytes(b"version 2")
|
||||||
|
run(engine, rule, src, dst, arch)
|
||||||
|
assert names(dst) == ["a-1.pdf", "a.pdf"] and names(arch) == ["a-1.pdf", "a.pdf"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_dry_run_shows_archive_and_writes_nothing(
|
||||||
|
engine: Engine, rule_id: int, dirs: tuple[Path, Path, Path]
|
||||||
|
) -> None:
|
||||||
|
src, dst, arch = dirs
|
||||||
|
write(src / "a.pdf")
|
||||||
|
result = run(engine, snap(rule_id, archive_layout="date"), src, dst, arch, dry_run=True)
|
||||||
|
assert result.items[0].status is ItemStatus.PLANNED
|
||||||
|
assert f"Archiv: {arch / '2026/10/a.pdf'}" in (result.items[0].message or "")
|
||||||
|
assert names(dst) == [] and names(arch) == [] and (src / "a.pdf").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_without_archive_unchanged(
|
||||||
|
engine: Engine, rule_id: int, dirs: tuple[Path, Path, Path]
|
||||||
|
) -> None:
|
||||||
|
src, dst, arch = dirs
|
||||||
|
write(src / "a.pdf")
|
||||||
|
result = run(engine, snap(rule_id), src, dst, None)
|
||||||
|
assert result.items[0].message is None and names(arch) == []
|
||||||
|
|
||||||
|
|
||||||
|
# -- Regellauf ----------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def setup(
|
||||||
|
manager: MountManager,
|
||||||
|
fake_backend: FakeMountBackend,
|
||||||
|
make_mount: MountFactory,
|
||||||
|
session_factory: sessionmaker[Session],
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> Iterator[tuple[int, Path, Path, Path]]:
|
||||||
|
scans = fake_backend.export("nas", "scans")
|
||||||
|
arch_remote = fake_backend.export("nas", "archiv")
|
||||||
|
src_id = make_mount("scans", share="scans")
|
||||||
|
arch_id = make_mount("archiv", share="archiv")
|
||||||
|
manager.mount(src_id)
|
||||||
|
manager.mount(arch_id)
|
||||||
|
consume = tmp_path / "consume"
|
||||||
|
consume.mkdir()
|
||||||
|
with session_scope(session_factory) as s:
|
||||||
|
r = Rule(
|
||||||
|
name="Scans → Paperless + Archiv",
|
||||||
|
src_mount_id=src_id,
|
||||||
|
dst_path=str(consume),
|
||||||
|
stability_seconds=0,
|
||||||
|
archive_kind=ArchiveKind.MOUNT,
|
||||||
|
archive_mount_id=arch_id,
|
||||||
|
archive_path="scans",
|
||||||
|
)
|
||||||
|
s.add(r)
|
||||||
|
s.flush()
|
||||||
|
rid = r.id
|
||||||
|
yield rid, scans, consume, arch_remote
|
||||||
|
|
||||||
|
|
||||||
|
def test_runner_with_archive_mount(
|
||||||
|
setup: tuple[int, Path, Path, Path],
|
||||||
|
manager: MountManager,
|
||||||
|
session_factory: sessionmaker[Session],
|
||||||
|
) -> None:
|
||||||
|
rid, scans, consume, arch = setup
|
||||||
|
write(scans / "doc.pdf", b"%PDF")
|
||||||
|
result = RuleRunner(session_factory, manager, sleep=lambda _: None).run(rid)
|
||||||
|
assert result.status is RunStatus.OK, result.message
|
||||||
|
assert (
|
||||||
|
(consume / "doc.pdf").read_bytes() == b"%PDF" == (arch / "scans" / "doc.pdf").read_bytes()
|
||||||
|
)
|
||||||
|
assert not (scans / "doc.pdf").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_unavailable_archive_makes_rule_wait(
|
||||||
|
setup: tuple[int, Path, Path, Path],
|
||||||
|
manager: MountManager,
|
||||||
|
fake_backend: FakeMountBackend,
|
||||||
|
session_factory: sessionmaker[Session],
|
||||||
|
) -> None:
|
||||||
|
rid, scans, consume, _arch = setup
|
||||||
|
write(scans / "doc.pdf")
|
||||||
|
fake_backend.vanish(manager.mountpoint_for("archiv"))
|
||||||
|
result = RuleRunner(session_factory, manager, sleep=lambda _: None).run(rid)
|
||||||
|
assert result.unavailable and result.status is RunStatus.ERROR
|
||||||
|
assert (scans / "doc.pdf").exists() and list(consume.iterdir()) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_mount_used_as_archive_cannot_be_deleted(
|
||||||
|
setup: tuple[int, Path, Path, Path],
|
||||||
|
session_factory: sessionmaker[Session],
|
||||||
|
manager: MountManager,
|
||||||
|
) -> None:
|
||||||
|
from netmover.admin import AdminError, delete_mount
|
||||||
|
from netmover.models import Mount
|
||||||
|
|
||||||
|
with session_scope(session_factory) as s:
|
||||||
|
arch_id = s.scalar(select(Mount.id).where(Mount.name == "archiv"))
|
||||||
|
assert arch_id is not None
|
||||||
|
with pytest.raises(AdminError, match="Wird noch von Regeln benutzt"):
|
||||||
|
delete_mount(s, manager, arch_id)
|
||||||
|
|
||||||
|
|
||||||
|
def test_rule_form_archive_validation(
|
||||||
|
session_factory: sessionmaker[Session], make_mount: MountFactory
|
||||||
|
) -> None:
|
||||||
|
from netmover.admin import parse_rule_form, save_rule
|
||||||
|
|
||||||
|
mid = make_mount()
|
||||||
|
base = {
|
||||||
|
"name": "x",
|
||||||
|
"src_mount_id": str(mid),
|
||||||
|
"dst_kind": "local",
|
||||||
|
"dst_path": "/srv/consume",
|
||||||
|
"interval_seconds": "30",
|
||||||
|
"stability_seconds": "5",
|
||||||
|
"op_timeout_seconds": "120",
|
||||||
|
}
|
||||||
|
with session_scope(session_factory) as s:
|
||||||
|
_, err = parse_rule_form(
|
||||||
|
s, {**base, "archive_kind": "local", "archive_path": "/srv/consume"}
|
||||||
|
)
|
||||||
|
assert "derselbe Ordner" in err["archive_path"]
|
||||||
|
_, err = parse_rule_form(s, {**base, "archive_kind": "mount", "archive_path": "a"})
|
||||||
|
assert "archive_mount_id" in err
|
||||||
|
_, err = parse_rule_form(s, {**base, "archive_kind": "local", "archive_path": "relativ"})
|
||||||
|
assert "absoluten Pfad" in err["archive_path"]
|
||||||
|
d, err = parse_rule_form(
|
||||||
|
s,
|
||||||
|
{
|
||||||
|
**base,
|
||||||
|
"archive_kind": "mount",
|
||||||
|
"archive_mount_id": str(mid),
|
||||||
|
"archive_path": "archiv/scans",
|
||||||
|
"archive_layout": "date",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
assert err == {}
|
||||||
|
r, _ = save_rule(s, d)
|
||||||
|
assert r is not None and r.archive_kind is ArchiveKind.MOUNT
|
||||||
|
assert r.archive_path == "archiv/scans" and r.archive_layout.value == "date"
|
||||||
@@ -0,0 +1,167 @@
|
|||||||
|
"""Anmeldung je Netz: ohne Passwort / mit Passwort / abgewiesen."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from netmover.settings_store import load_settings, save_settings
|
||||||
|
from netmover.web.auth import (
|
||||||
|
ACCESS_DENY,
|
||||||
|
ACCESS_NOPW,
|
||||||
|
ACCESS_PW,
|
||||||
|
access_decision,
|
||||||
|
includes_loopback,
|
||||||
|
parse_networks,
|
||||||
|
)
|
||||||
|
from tests.webutil import PASSWORD, csrf_of
|
||||||
|
|
||||||
|
|
||||||
|
def _other(client: TestClient, host: str) -> TestClient:
|
||||||
|
"""Weiterer Client für dieselbe App (ohne eigenen Lebenszyklus) mit fester Adresse."""
|
||||||
|
return TestClient(client.app, follow_redirects=False, client=(host, 51000))
|
||||||
|
|
||||||
|
|
||||||
|
def _save(c: TestClient, trusted: str = "", allowed: str = "", current: str = PASSWORD) -> Any:
|
||||||
|
token = (
|
||||||
|
csrf_of(c.get("/settings").text)
|
||||||
|
if "X-Test-CSRF" not in c.headers
|
||||||
|
else c.headers["X-Test-CSRF"]
|
||||||
|
)
|
||||||
|
return c.post(
|
||||||
|
"/settings/auth",
|
||||||
|
data={
|
||||||
|
"csrf_token": token,
|
||||||
|
"trusted_networks": trusted,
|
||||||
|
"allowed_networks": allowed,
|
||||||
|
"current": current,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _set(state: Any, **kw: str) -> None:
|
||||||
|
cfg = load_settings(state.session_factory)
|
||||||
|
for k, v in kw.items():
|
||||||
|
setattr(cfg, k, v)
|
||||||
|
save_settings(state.session_factory, cfg)
|
||||||
|
|
||||||
|
|
||||||
|
# -- Entscheidung ------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("host", "trusted", "allowed", "expected"),
|
||||||
|
[
|
||||||
|
("10.0.10.50", "", "", ACCESS_PW), # Standard: überall mit Passwort
|
||||||
|
("10.0.10.50", "10.0.10.0/24", "", ACCESS_NOPW),
|
||||||
|
("10.1.5.9", "10.0.10.0/24", "", ACCESS_PW), # übrige Netze mit Passwort
|
||||||
|
("10.1.5.9", "10.0.10.0/24", "10.1.5.0/24", ACCESS_PW),
|
||||||
|
("192.168.5.5", "10.0.10.0/24", "10.1.5.0/24", ACCESS_DENY),
|
||||||
|
("10.0.10.50", "10.0.10.0/24", "10.1.5.0/24", ACCESS_NOPW), # ohne PW schlägt Liste
|
||||||
|
("2001:db8::5", "0.0.0.0/0, ::/0", "", ACCESS_NOPW), # Anmeldung ganz aus
|
||||||
|
("testclient", "", "10.0.0.0/8", ACCESS_DENY), # keine IP-Adresse
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_access_decision(host: str, trusted: str, allowed: str, expected: str) -> None:
|
||||||
|
assert access_decision(host, "always", trusted, allowed) == expected
|
||||||
|
|
||||||
|
|
||||||
|
def test_legacy_off_mode_still_works() -> None:
|
||||||
|
assert access_decision("192.168.5.5", "off", "", "10.1.5.0/24") == ACCESS_NOPW
|
||||||
|
|
||||||
|
|
||||||
|
# -- Oberfläche --------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_default_requires_login(client: TestClient) -> None:
|
||||||
|
assert _other(client, "10.0.10.50").get("/").status_code == 303
|
||||||
|
|
||||||
|
|
||||||
|
def test_change_needs_current_password(client: TestClient, state: Any) -> None:
|
||||||
|
_save(client, trusted="10.0.10.0/24", current="falsch")
|
||||||
|
assert load_settings(state.session_factory).trusted_networks == ""
|
||||||
|
assert "Aktuelles Passwort falsch" in client.get("/settings").text
|
||||||
|
|
||||||
|
|
||||||
|
def test_without_password_from_trusted_network(client: TestClient, state: Any) -> None:
|
||||||
|
_save(client, trusted="10.0.10.0/24")
|
||||||
|
lan = _other(client, "10.0.10.50")
|
||||||
|
r = lan.get("/")
|
||||||
|
assert r.status_code == 200 and "ohne Anmeldung (vertrautes Netz)" in r.text
|
||||||
|
assert "Abmelden" not in r.text and lan.get("/login").headers["location"] == "/"
|
||||||
|
assert lan.post("/maintenance").status_code == 403 # CSRF gilt weiterhin
|
||||||
|
assert (
|
||||||
|
lan.post("/maintenance", data={"csrf_token": csrf_of(lan.get("/").text)}).status_code == 303
|
||||||
|
)
|
||||||
|
assert _other(client, "10.1.5.9").get("/").status_code == 303 # übrige mit Passwort
|
||||||
|
|
||||||
|
|
||||||
|
def test_other_networks_are_rejected_when_password_list_is_set(
|
||||||
|
client: TestClient, state: Any
|
||||||
|
) -> None:
|
||||||
|
_save(client, trusted="0.0.0.0/0") # damit der Test-Client (keine IP) weiterkommt …
|
||||||
|
_set(state, trusted_networks="10.0.10.0/24", allowed_networks="10.1.5.0/24") # … dann scharf
|
||||||
|
assert _other(client, "10.0.10.50").get("/").status_code == 200
|
||||||
|
assert _other(client, "10.1.5.9").get("/").headers["location"].startswith("/login")
|
||||||
|
blocked = _other(client, "192.168.5.5")
|
||||||
|
for url in ("/", "/login", "/setup", "/mounts"):
|
||||||
|
assert blocked.get(url).status_code == 403, url
|
||||||
|
assert blocked.get("/healthz").status_code == 200 # Überwachung bleibt erreichbar
|
||||||
|
assert blocked.get("/static/app.css").status_code == 200
|
||||||
|
|
||||||
|
|
||||||
|
def test_cannot_lock_yourself_out(client: TestClient, state: Any) -> None:
|
||||||
|
lan = _other(client, "10.0.10.50")
|
||||||
|
_save(client, trusted="10.0.10.0/24")
|
||||||
|
# aus 10.0.10.50 eine Liste speichern, in der das eigene Netz fehlt
|
||||||
|
r = _save(lan, trusted="", allowed="10.1.5.0/24")
|
||||||
|
assert r.status_code == 303
|
||||||
|
assert "ausgesperrt" in lan.get("/settings").text
|
||||||
|
cfg = load_settings(state.session_factory)
|
||||||
|
assert cfg.trusted_networks == "10.0.10.0/24" and cfg.allowed_networks == ""
|
||||||
|
# eigenes Netz bei „mit Passwort“ – erlaubt
|
||||||
|
_save(lan, trusted="10.0.10.0/24", allowed="10.1.5.0/24")
|
||||||
|
assert load_settings(state.session_factory).allowed_networks == "10.1.5.0/24"
|
||||||
|
|
||||||
|
|
||||||
|
def test_everything_without_password(client: TestClient) -> None:
|
||||||
|
_save(client, trusted="0.0.0.0/0, ::/0")
|
||||||
|
r = _other(client, "192.168.77.1").get("/rules")
|
||||||
|
assert r.status_code == 200 and ">ohne Anmeldung<" in r.text
|
||||||
|
|
||||||
|
|
||||||
|
def test_setup_still_required_without_password(raw_client: TestClient, state: Any) -> None:
|
||||||
|
_set(state, trusted_networks="0.0.0.0/0")
|
||||||
|
assert raw_client.get("/").headers["location"] == "/setup"
|
||||||
|
|
||||||
|
|
||||||
|
def test_validation_and_loopback_hint(client: TestClient, state: Any) -> None:
|
||||||
|
_save(client, trusted="10.0.10.0/24, quatsch")
|
||||||
|
assert "kein gültiges Netz" in client.get("/settings").text
|
||||||
|
assert load_settings(state.session_factory).trusted_networks == ""
|
||||||
|
_save(client, trusted="127.0.0.1")
|
||||||
|
assert "jede</strong> Anfrage über den Proxy" in client.get("/settings").text
|
||||||
|
|
||||||
|
|
||||||
|
def test_network_parsing() -> None:
|
||||||
|
nets = parse_networks("10.0.10.0/24; 192.168.1.20 fe80::/10")
|
||||||
|
assert [str(n) for n in nets] == ["10.0.10.0/24", "192.168.1.20/32", "fe80::/10"]
|
||||||
|
assert includes_loopback("10.0.0.0/8, 127.0.0.0/8") and not includes_loopback("10.0.0.0/8")
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
parse_networks("10.0.0.0/33")
|
||||||
|
|
||||||
|
|
||||||
|
def test_reset_password_turns_login_back_on(
|
||||||
|
state: Any, settings: Any, monkeypatch: pytest.MonkeyPatch
|
||||||
|
) -> None:
|
||||||
|
from netmover import cli
|
||||||
|
from netmover.web.auth import set_password
|
||||||
|
|
||||||
|
set_password(state.session_factory, PASSWORD)
|
||||||
|
_set(state, auth_mode="off", trusted_networks="0.0.0.0/0", allowed_networks="10.1.5.0/24")
|
||||||
|
monkeypatch.setattr(cli, "get_settings", lambda: settings)
|
||||||
|
assert cli.main(["reset-password"]) == 0
|
||||||
|
cfg = load_settings(state.session_factory)
|
||||||
|
assert cfg.auth_mode == "always" and cfg.trusted_networks == "" and cfg.allowed_networks == ""
|
||||||
@@ -0,0 +1,194 @@
|
|||||||
|
"""0.8.0: Anbindungen kopieren – inklusive Zugangsdaten."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from sqlalchemy import select
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from netmover.admin import MountInput, copy_name, delete_mount, save_mount
|
||||||
|
from netmover.crypto import SecretBox
|
||||||
|
from netmover.models import Credential, CredentialKind, Mount, MountType
|
||||||
|
from netmover.mounting.fake import FakeMountBackend
|
||||||
|
from netmover.mounting.manager import MountManager
|
||||||
|
from tests.webutil import post
|
||||||
|
|
||||||
|
|
||||||
|
def _cifs(**kw: object) -> MountInput:
|
||||||
|
data: dict[str, object] = {
|
||||||
|
"name": "NAS",
|
||||||
|
"type": MountType.CIFS,
|
||||||
|
"server": "10.0.10.6",
|
||||||
|
"share": "Scans",
|
||||||
|
"username": "scanner",
|
||||||
|
"password": "Pa$$wort",
|
||||||
|
}
|
||||||
|
data.update(kw)
|
||||||
|
return MountInput(**data) # type: ignore[arg-type]
|
||||||
|
|
||||||
|
|
||||||
|
def test_copy_reuses_own_password_as_independent_copy(
|
||||||
|
session: Session, secretbox: SecretBox, manager: MountManager
|
||||||
|
) -> None:
|
||||||
|
src, _ = save_mount(session, secretbox, manager, _cifs())
|
||||||
|
assert src is not None and src.credential is not None
|
||||||
|
copy, err = save_mount(
|
||||||
|
session,
|
||||||
|
secretbox,
|
||||||
|
manager,
|
||||||
|
_cifs(name="Archiv", share="Archiv", password=""),
|
||||||
|
copy_from=src,
|
||||||
|
)
|
||||||
|
assert not err and copy is not None and copy.credential is not None
|
||||||
|
assert copy.credential.id != src.credential.id and not copy.credential.managed
|
||||||
|
assert secretbox.decrypt(copy.credential.secret_enc) == "Pa$$wort"
|
||||||
|
assert copy.credential.username == "scanner"
|
||||||
|
# Vorlage löschen – die Kopie behält ihre Zugangsdaten
|
||||||
|
delete_mount(session, manager, src.id)
|
||||||
|
session.flush()
|
||||||
|
assert secretbox.decrypt(copy.credential.secret_enc) == "Pa$$wort"
|
||||||
|
|
||||||
|
|
||||||
|
def test_copy_with_new_password_uses_new_one(
|
||||||
|
session: Session, secretbox: SecretBox, manager: MountManager
|
||||||
|
) -> None:
|
||||||
|
src, _ = save_mount(session, secretbox, manager, _cifs())
|
||||||
|
copy, _ = save_mount(
|
||||||
|
session, secretbox, manager, _cifs(name="B", share="B", password="anders"), copy_from=src
|
||||||
|
)
|
||||||
|
assert copy is not None and copy.credential is not None
|
||||||
|
assert secretbox.decrypt(copy.credential.secret_enc) == "anders"
|
||||||
|
|
||||||
|
|
||||||
|
def test_copy_with_stored_credentials_shares_them(
|
||||||
|
session: Session, secretbox: SecretBox, manager: MountManager
|
||||||
|
) -> None:
|
||||||
|
from netmover.credentials import create_password
|
||||||
|
|
||||||
|
cred = create_password(session, secretbox, "NAS-Scanner", "scanner", "x")
|
||||||
|
src, _ = save_mount(
|
||||||
|
session, secretbox, manager, _cifs(username="", password="", cred_id=str(cred.id))
|
||||||
|
)
|
||||||
|
copy, _ = save_mount(
|
||||||
|
session,
|
||||||
|
secretbox,
|
||||||
|
manager,
|
||||||
|
_cifs(name="B", share="B", username="", password="", cred_id=str(cred.id)),
|
||||||
|
copy_from=src,
|
||||||
|
)
|
||||||
|
assert (
|
||||||
|
copy is not None and copy.credential is cred and src is not None and src.credential is cred
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_copy_sftp_keeps_keypair_and_host_key_only_for_same_server(
|
||||||
|
session: Session, secretbox: SecretBox, manager: MountManager
|
||||||
|
) -> None:
|
||||||
|
sftp = {
|
||||||
|
"type": MountType.SFTP,
|
||||||
|
"server": "10.0.10.17",
|
||||||
|
"share": "/consume",
|
||||||
|
"port": 22,
|
||||||
|
"username": "paperless",
|
||||||
|
"auth": CredentialKind.SSH_KEY,
|
||||||
|
"password": "",
|
||||||
|
}
|
||||||
|
src, _ = save_mount(session, secretbox, manager, _cifs(name="Paperless", **sftp))
|
||||||
|
assert src is not None and src.credential is not None
|
||||||
|
src.host_key = "ssh-ed25519 AAAAhost"
|
||||||
|
same, _ = save_mount(
|
||||||
|
session,
|
||||||
|
secretbox,
|
||||||
|
manager,
|
||||||
|
_cifs(name="Paperless 2", **{**sftp, "share": "/consume2"}),
|
||||||
|
copy_from=src,
|
||||||
|
)
|
||||||
|
other, _ = save_mount(
|
||||||
|
session,
|
||||||
|
secretbox,
|
||||||
|
manager,
|
||||||
|
_cifs(name="Anderer", **{**sftp, "server": "10.0.10.99"}),
|
||||||
|
copy_from=src,
|
||||||
|
)
|
||||||
|
assert same is not None and other is not None
|
||||||
|
for m in (same, other):
|
||||||
|
assert m.credential is not None and m.credential.id != src.credential.id
|
||||||
|
assert m.credential.public_key == src.credential.public_key # gleiches Schlüsselpaar
|
||||||
|
assert m.credential.secret_enc == src.credential.secret_enc
|
||||||
|
assert same.host_key == "ssh-ed25519 AAAAhost" and other.host_key is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_copy_name(session: Session, secretbox: SecretBox, manager: MountManager) -> None:
|
||||||
|
save_mount(session, secretbox, manager, _cifs())
|
||||||
|
assert copy_name(session, "NAS") == "NAS (Kopie)"
|
||||||
|
save_mount(session, secretbox, manager, _cifs(name="NAS (Kopie)", share="x"))
|
||||||
|
assert copy_name(session, "NAS") == "NAS (Kopie 2)"
|
||||||
|
|
||||||
|
|
||||||
|
# -- Oberfläche --------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
MOUNT = {
|
||||||
|
"name": "NAS",
|
||||||
|
"type": "cifs",
|
||||||
|
"server": "nas",
|
||||||
|
"share": "scans",
|
||||||
|
"username": "scanner",
|
||||||
|
"password": "Geheim,123",
|
||||||
|
"auto_mount": "on",
|
||||||
|
"enabled": "on",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _mid(state: Any, name: str) -> int:
|
||||||
|
with state.session_factory() as s:
|
||||||
|
mid = s.scalar(select(Mount.id).where(Mount.name == name))
|
||||||
|
assert mid is not None
|
||||||
|
return mid
|
||||||
|
|
||||||
|
|
||||||
|
def test_copy_button_form_test_and_save(
|
||||||
|
client: TestClient, state: Any, backend: FakeMountBackend
|
||||||
|
) -> None:
|
||||||
|
backend.export("nas", "scans")
|
||||||
|
backend.export("nas", "archiv")
|
||||||
|
post(client, "/mounts/new", MOUNT)
|
||||||
|
src_id = _mid(state, "NAS")
|
||||||
|
assert f'href="/mounts/{src_id}/copy"' in client.get("/mounts").text
|
||||||
|
|
||||||
|
form = client.get(f"/mounts/{src_id}/copy").text
|
||||||
|
assert "Kopie von „NAS“" in form and 'value="NAS (Kopie)"' in form
|
||||||
|
assert 'value="scanner"' in form and "Geheim,123" not in form
|
||||||
|
assert f'name="copy_from" value="{src_id}"' in form
|
||||||
|
|
||||||
|
# Testen mit übernommenem Passwort (Feld leer)
|
||||||
|
copy_form = {
|
||||||
|
**MOUNT,
|
||||||
|
"name": "NAS (Kopie)",
|
||||||
|
"share": "archiv",
|
||||||
|
"password": "",
|
||||||
|
"copy_from": str(src_id),
|
||||||
|
}
|
||||||
|
r = post(client, "/mounts/test", copy_form)
|
||||||
|
assert "Einhängen erfolgreich" in r.text
|
||||||
|
spec = [c for c in backend.calls if c.action == "mount"][-1].spec
|
||||||
|
assert spec is not None and spec.credentials is not None
|
||||||
|
assert spec.credentials.password == "Geheim,123"
|
||||||
|
|
||||||
|
r = post(client, f"/mounts/{src_id}/copy", copy_form)
|
||||||
|
assert r.status_code == 303, r.text
|
||||||
|
with state.session_factory() as s:
|
||||||
|
copy = s.get(Mount, _mid(state, "NAS (Kopie)"))
|
||||||
|
assert copy is not None and copy.share == "archiv" and copy.slug == "nas-kopie"
|
||||||
|
assert copy.credential is not None
|
||||||
|
assert state.secrets.decrypt(copy.credential.secret_enc) == "Geheim,123"
|
||||||
|
assert len(s.scalars(select(Credential)).all()) == 2
|
||||||
|
|
||||||
|
|
||||||
|
def test_copy_validation_error_keeps_copy_context(client: TestClient, state: Any) -> None:
|
||||||
|
post(client, "/mounts/new", MOUNT)
|
||||||
|
src_id = _mid(state, "NAS")
|
||||||
|
r = post(client, f"/mounts/{src_id}/copy", {**MOUNT, "name": "NAS", "password": ""})
|
||||||
|
assert r.status_code == 400 and "bereits vergeben" in r.text and "Kopie von „NAS“" in r.text
|
||||||
@@ -0,0 +1,168 @@
|
|||||||
|
"""0.9.0: Regeln kopieren – vorausgefüllt, zunächst deaktiviert."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from sqlalchemy import select
|
||||||
|
|
||||||
|
from netmover.db import session_scope
|
||||||
|
from netmover.models import (
|
||||||
|
ArchiveKind,
|
||||||
|
ArchiveLayout,
|
||||||
|
CopiedFile,
|
||||||
|
Mount,
|
||||||
|
MountType,
|
||||||
|
Rule,
|
||||||
|
RuleAction,
|
||||||
|
Run,
|
||||||
|
)
|
||||||
|
from tests.webutil import post
|
||||||
|
|
||||||
|
|
||||||
|
def _setup(state: Any, tmp_path: Path) -> tuple[int, int]:
|
||||||
|
consume = tmp_path / "consume"
|
||||||
|
consume.mkdir()
|
||||||
|
with session_scope(state.session_factory) as s:
|
||||||
|
m = Mount(
|
||||||
|
name="NAS",
|
||||||
|
slug="nas",
|
||||||
|
type=MountType.CIFS,
|
||||||
|
server="nas",
|
||||||
|
share="scans",
|
||||||
|
mountpoint=str(tmp_path / "mnt" / "nas"),
|
||||||
|
)
|
||||||
|
a = Mount(
|
||||||
|
name="Archiv",
|
||||||
|
slug="archiv",
|
||||||
|
type=MountType.CIFS,
|
||||||
|
server="nas",
|
||||||
|
share="archiv",
|
||||||
|
mountpoint=str(tmp_path / "mnt" / "archiv"),
|
||||||
|
)
|
||||||
|
s.add_all([m, a])
|
||||||
|
s.flush()
|
||||||
|
r = Rule(
|
||||||
|
name="Scans → Paperless",
|
||||||
|
src_mount_id=m.id,
|
||||||
|
src_subpath="max",
|
||||||
|
dst_path=str(consume),
|
||||||
|
pattern="*.pdf;*.jpg",
|
||||||
|
recursive=True,
|
||||||
|
interval_seconds=45,
|
||||||
|
stability_seconds=7,
|
||||||
|
action=RuleAction.COPY,
|
||||||
|
set_mode="0640",
|
||||||
|
quarantine_path="_q",
|
||||||
|
archive_kind=ArchiveKind.MOUNT,
|
||||||
|
archive_mount_id=a.id,
|
||||||
|
archive_path="scans",
|
||||||
|
archive_layout=ArchiveLayout.DATE,
|
||||||
|
)
|
||||||
|
s.add(r)
|
||||||
|
s.flush()
|
||||||
|
s.add(Run(rule_id=r.id))
|
||||||
|
s.add(CopiedFile(rule_id=r.id, src_relpath="a.pdf", size=1, mtime_ns=1, dst_path="x"))
|
||||||
|
return r.id, m.id
|
||||||
|
|
||||||
|
|
||||||
|
def _rule(state: Any, name: str) -> Rule:
|
||||||
|
with state.session_factory() as s:
|
||||||
|
r = s.scalars(select(Rule).where(Rule.name == name)).one()
|
||||||
|
s.expunge(r)
|
||||||
|
return r
|
||||||
|
|
||||||
|
|
||||||
|
def _form_from_page(html: str) -> dict[str, str]:
|
||||||
|
"""Wie ein Browser: alle Felder des Formulars mit ihren vorausgefüllten Werten."""
|
||||||
|
import re
|
||||||
|
|
||||||
|
form: dict[str, str] = {}
|
||||||
|
for m in re.finditer(r'<input[^>]*name="([^"]+)"[^>]*>', html):
|
||||||
|
tag, name = m.group(0), m.group(1)
|
||||||
|
if 'type="checkbox"' in tag:
|
||||||
|
if " checked" in tag:
|
||||||
|
form[name] = "on"
|
||||||
|
else:
|
||||||
|
v = re.search(r'value="([^"]*)"', tag)
|
||||||
|
form[name] = v.group(1) if v else ""
|
||||||
|
for m in re.finditer(r'<select[^>]*name="([^"]+)"[^>]*>(.*?)</select>', html, re.S):
|
||||||
|
sel = re.search(r'<option value="([^"]*)" selected>', m.group(2))
|
||||||
|
form[m.group(1)] = sel.group(1) if sel else ""
|
||||||
|
return form
|
||||||
|
|
||||||
|
|
||||||
|
def test_copy_form_is_prefilled_and_disabled(
|
||||||
|
client: TestClient, state: Any, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
rid, _ = _setup(state, tmp_path)
|
||||||
|
assert f'href="/rules/{rid}/copy"' in client.get("/rules").text
|
||||||
|
html = client.get(f"/rules/{rid}/copy").text
|
||||||
|
assert "Kopie von „Scans → Paperless“" in html and "deaktiviert" in html
|
||||||
|
form = _form_from_page(html)
|
||||||
|
assert form["name"] == "Scans → Paperless (Kopie)"
|
||||||
|
assert "enabled" not in form # Häkchen „Aktiv“ nicht gesetzt
|
||||||
|
assert form["pattern"] == "*.pdf;*.jpg" and form["src_subpath"] == "max"
|
||||||
|
assert form["archive_kind"] == "mount" and form["archive_layout"] == "date"
|
||||||
|
|
||||||
|
|
||||||
|
def test_copy_saves_new_rule_with_all_settings(
|
||||||
|
client: TestClient, state: Any, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
rid, _ = _setup(state, tmp_path)
|
||||||
|
form = _form_from_page(client.get(f"/rules/{rid}/copy").text)
|
||||||
|
form["src_subpath"] = "erika" # typische Anpassung
|
||||||
|
r = post(client, f"/rules/{rid}/copy", form)
|
||||||
|
assert r.status_code == 303, r.text
|
||||||
|
assert "Die Kopie ist deaktiviert" in client.get("/rules").text
|
||||||
|
|
||||||
|
orig, copy = _rule(state, "Scans → Paperless"), _rule(state, "Scans → Paperless (Kopie)")
|
||||||
|
assert copy.id != orig.id and copy.enabled is False and orig.enabled is True
|
||||||
|
assert copy.src_subpath == "erika" and orig.src_subpath == "max"
|
||||||
|
for f in (
|
||||||
|
"src_mount_id",
|
||||||
|
"dst_path",
|
||||||
|
"pattern",
|
||||||
|
"recursive",
|
||||||
|
"interval_seconds",
|
||||||
|
"stability_seconds",
|
||||||
|
"action",
|
||||||
|
"set_mode",
|
||||||
|
"quarantine_path",
|
||||||
|
"archive_kind",
|
||||||
|
"archive_mount_id",
|
||||||
|
"archive_path",
|
||||||
|
"archive_layout",
|
||||||
|
):
|
||||||
|
assert getattr(copy, f) == getattr(orig, f), f
|
||||||
|
# Protokoll und Buchführung beginnen neu
|
||||||
|
with state.session_factory() as s:
|
||||||
|
assert s.scalars(select(Run).where(Run.rule_id == copy.id)).all() == []
|
||||||
|
assert s.scalars(select(CopiedFile).where(CopiedFile.rule_id == copy.id)).all() == []
|
||||||
|
assert state.scheduler.next_run(copy.id) is None # deaktiviert → kein Termin
|
||||||
|
|
||||||
|
|
||||||
|
def test_copy_names_count_up(client: TestClient, state: Any, tmp_path: Path) -> None:
|
||||||
|
rid, _ = _setup(state, tmp_path)
|
||||||
|
for expected in ("Scans → Paperless (Kopie)", "Scans → Paperless (Kopie 2)"):
|
||||||
|
form = _form_from_page(client.get(f"/rules/{rid}/copy").text)
|
||||||
|
assert form["name"] == expected
|
||||||
|
post(client, f"/rules/{rid}/copy", form)
|
||||||
|
|
||||||
|
|
||||||
|
def test_copy_validation_error_keeps_context(
|
||||||
|
client: TestClient, state: Any, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
rid, _ = _setup(state, tmp_path)
|
||||||
|
form = _form_from_page(client.get(f"/rules/{rid}/copy").text)
|
||||||
|
form["name"] = "Scans → Paperless" # schon vergeben
|
||||||
|
r = post(client, f"/rules/{rid}/copy", form)
|
||||||
|
assert r.status_code == 400 and "bereits vergeben" in r.text and "Kopie von" in r.text
|
||||||
|
with state.session_factory() as s:
|
||||||
|
assert len(s.scalars(select(Rule)).all()) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_copy_of_missing_rule_redirects(client: TestClient) -> None:
|
||||||
|
assert client.get("/rules/4711/copy").headers["location"] == "/rules"
|
||||||
@@ -13,9 +13,16 @@ from sqlalchemy import select
|
|||||||
from sqlalchemy.orm import Session, sessionmaker
|
from sqlalchemy.orm import Session, sessionmaker
|
||||||
|
|
||||||
from netmover.admin import AdminError, MountInput, delete_mount, save_mount
|
from netmover.admin import AdminError, MountInput, delete_mount, save_mount
|
||||||
|
from netmover.credentials import (
|
||||||
|
delete_key,
|
||||||
|
fingerprint,
|
||||||
|
generate_key,
|
||||||
|
import_key,
|
||||||
|
list_keys,
|
||||||
|
rename_key,
|
||||||
|
)
|
||||||
from netmover.crypto import SecretBox, generate_ssh_keypair
|
from netmover.crypto import SecretBox, generate_ssh_keypair
|
||||||
from netmover.db import session_scope
|
from netmover.db import session_scope
|
||||||
from netmover.keys import delete_key, fingerprint, generate_key, import_key, list_keys, rename_key
|
|
||||||
from netmover.models import Credential, CredentialKind, Mount, MountType
|
from netmover.models import Credential, CredentialKind, Mount, MountType
|
||||||
from netmover.mounting.manager import MountConfig, MountManager
|
from netmover.mounting.manager import MountConfig, MountManager
|
||||||
from tests.sftp_stub import StubSFTPServer
|
from tests.sftp_stub import StubSFTPServer
|
||||||
@@ -142,7 +149,7 @@ def test_import_errors(session: Session, secretbox: SecretBox) -> None:
|
|||||||
|
|
||||||
def test_names(session: Session, secretbox: SecretBox) -> None:
|
def test_names(session: Session, secretbox: SecretBox) -> None:
|
||||||
generate_key(session, secretbox, "eins")
|
generate_key(session, secretbox, "eins")
|
||||||
with pytest.raises(AdminError, match="existiert bereits"):
|
with pytest.raises(AdminError, match="existieren bereits"):
|
||||||
generate_key(session, secretbox, "eins")
|
generate_key(session, secretbox, "eins")
|
||||||
with pytest.raises(AdminError, match="reserviert"):
|
with pytest.raises(AdminError, match="reserviert"):
|
||||||
generate_key(session, secretbox, "mount:x")
|
generate_key(session, secretbox, "mount:x")
|
||||||
@@ -275,3 +282,102 @@ def test_list_keys_ignores_per_mount_credentials(
|
|||||||
save_mount(session, secretbox, manager, _sftp_input())
|
save_mount(session, secretbox, manager, _sftp_input())
|
||||||
assert list_keys(session) == []
|
assert list_keys(session) == []
|
||||||
assert session.scalars(select(Credential)).one().managed is False
|
assert session.scalars(select(Credential)).one().managed is False
|
||||||
|
|
||||||
|
|
||||||
|
# -- Gespeicherte Passwörter (0.5.0) ----------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _cifs(**kw: object) -> MountInput:
|
||||||
|
data: dict[str, object] = {
|
||||||
|
"name": "nas",
|
||||||
|
"type": MountType.CIFS,
|
||||||
|
"server": "10.0.10.6",
|
||||||
|
"share": "scans",
|
||||||
|
}
|
||||||
|
data.update(kw)
|
||||||
|
return MountInput(**data) # type: ignore[arg-type]
|
||||||
|
|
||||||
|
|
||||||
|
def test_stored_password_shared_by_mounts(
|
||||||
|
session: Session, secretbox: SecretBox, manager: MountManager
|
||||||
|
) -> None:
|
||||||
|
from netmover.credentials import create_password, list_passwords, update_password
|
||||||
|
|
||||||
|
cred = create_password(session, secretbox, "NAS-Scanner", "scanner", "Pa$$,wort", "WG")
|
||||||
|
a, err = save_mount(session, secretbox, manager, _cifs(cred_id=str(cred.id)))
|
||||||
|
assert not err and a is not None
|
||||||
|
b, err = save_mount(
|
||||||
|
session, secretbox, manager, _cifs(name="archiv", share="archiv", cred_id=str(cred.id))
|
||||||
|
)
|
||||||
|
assert not err and b is not None
|
||||||
|
assert a.credential is cred and b.credential is cred and a.username is None
|
||||||
|
assert list_passwords(session)[0].used_by == ["archiv", "nas"]
|
||||||
|
|
||||||
|
# ein neues Passwort gilt für beide
|
||||||
|
update_password(session, secretbox, cred.id, "scanner", "neu!", "WG")
|
||||||
|
from netmover.admin import credential_for_test, mount_input_from_model
|
||||||
|
|
||||||
|
for m in (a, b):
|
||||||
|
data = mount_input_from_model(m)
|
||||||
|
assert data.cred_id == str(cred.id) and data.username == "scanner"
|
||||||
|
c = credential_for_test(data, m, secretbox, session)
|
||||||
|
assert c is not None and c.secret == "neu!" and c.domain == "WG"
|
||||||
|
|
||||||
|
# Anbindung löschen lässt die gespeicherten Zugangsdaten stehen
|
||||||
|
delete_mount(session, manager, a.id)
|
||||||
|
from netmover.credentials import delete_credential
|
||||||
|
|
||||||
|
with pytest.raises(AdminError, match="noch benutzt"):
|
||||||
|
delete_credential(session, cred.id)
|
||||||
|
delete_mount(session, manager, b.id)
|
||||||
|
assert delete_credential(session, cred.id) == "NAS-Scanner"
|
||||||
|
|
||||||
|
|
||||||
|
def test_switch_from_own_to_stored_password_cleans_up(
|
||||||
|
session: Session, secretbox: SecretBox, manager: MountManager
|
||||||
|
) -> None:
|
||||||
|
from netmover.credentials import create_password
|
||||||
|
|
||||||
|
m, _ = save_mount(session, secretbox, manager, _cifs(username="alt", password="pw"))
|
||||||
|
assert m is not None and m.credential is not None
|
||||||
|
own_id = m.credential.id
|
||||||
|
cred = create_password(session, secretbox, "geteilt", "scanner", "x")
|
||||||
|
m, err = save_mount(session, secretbox, manager, _cifs(cred_id=str(cred.id)), m.id)
|
||||||
|
assert not err and m is not None and m.credential is cred
|
||||||
|
assert session.get(Credential, own_id) is None
|
||||||
|
# zurück auf eigene Angaben: gespeicherter Eintrag bleibt unverändert
|
||||||
|
m, err = save_mount(session, secretbox, manager, _cifs(username="eigen", password="pw2"), m.id)
|
||||||
|
assert not err and m is not None and not m.credential.managed # type: ignore[union-attr]
|
||||||
|
assert cred.username == "scanner" and secretbox.decrypt(cred.secret_enc) == "x"
|
||||||
|
|
||||||
|
|
||||||
|
def test_stored_password_validation(session: Session, secretbox: SecretBox) -> None:
|
||||||
|
from netmover.credentials import create_password, update_password
|
||||||
|
|
||||||
|
with pytest.raises(AdminError, match="Passwort fehlt"):
|
||||||
|
create_password(session, secretbox, "a", "u", "")
|
||||||
|
with pytest.raises(AdminError):
|
||||||
|
create_password(session, secretbox, "a", "u", "zeile\numbruch")
|
||||||
|
cred = create_password(session, secretbox, "a", "u", "pw")
|
||||||
|
update_password(session, secretbox, cred.id, "u2", "") # leer = unverändert
|
||||||
|
assert cred.username == "u2" and secretbox.decrypt(cred.secret_enc) == "pw"
|
||||||
|
|
||||||
|
|
||||||
|
def test_stored_password_mount_actually_mounts(
|
||||||
|
session_factory: sessionmaker[Session],
|
||||||
|
secretbox: SecretBox,
|
||||||
|
manager: MountManager,
|
||||||
|
fake_backend: object,
|
||||||
|
) -> None:
|
||||||
|
from netmover.credentials import create_password
|
||||||
|
|
||||||
|
fake_backend.export("10.0.10.6", "scans") # type: ignore[attr-defined]
|
||||||
|
with session_scope(session_factory) as s:
|
||||||
|
cred = create_password(s, secretbox, "NAS", "scanner", "Pa$$wort", "WG")
|
||||||
|
m, _ = save_mount(s, secretbox, manager, _cifs(cred_id=str(cred.id)))
|
||||||
|
assert m is not None
|
||||||
|
mid = m.id
|
||||||
|
manager.mount(mid)
|
||||||
|
spec = fake_backend.calls[-1].spec # type: ignore[attr-defined]
|
||||||
|
assert spec.credentials.username == "scanner" and spec.credentials.password == "Pa$$wort"
|
||||||
|
assert spec.credentials.domain == "WG"
|
||||||
@@ -251,7 +251,7 @@ def test_source_not_deletable_keeps_target_and_prevents_duplicate(
|
|||||||
raise PermissionError(errno.EACCES, "Permission denied", str(path))
|
raise PermissionError(errno.EACCES, "Permission denied", str(path))
|
||||||
real_unlink(path) # type: ignore[arg-type]
|
real_unlink(path) # type: ignore[arg-type]
|
||||||
|
|
||||||
monkeypatch.setattr("netmover.mover.engine.os.unlink", deny)
|
monkeypatch.setattr("netmover.mover.sources.os.unlink", deny)
|
||||||
first = run(engine, snap(rule_id), src, dst)
|
first = run(engine, snap(rule_id), src, dst)
|
||||||
assert first.items[0].status is ItemStatus.FAILED
|
assert first.items[0].status is ItemStatus.FAILED
|
||||||
assert "nicht löschbar" in (first.items[0].message or "")
|
assert "nicht löschbar" in (first.items[0].message or "")
|
||||||
|
|||||||
@@ -0,0 +1,398 @@
|
|||||||
|
"""0.5.0: SFTP als Quelle – dieselben Garantien wie bei eingehängten Freigaben."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import errno
|
||||||
|
import hashlib
|
||||||
|
import os
|
||||||
|
from collections.abc import Callable, Iterator
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from sqlalchemy import select
|
||||||
|
from sqlalchemy.orm import Session, sessionmaker
|
||||||
|
|
||||||
|
from netmover.crypto import SecretBox
|
||||||
|
from netmover.db import session_scope
|
||||||
|
from netmover.models import (
|
||||||
|
CopiedFile,
|
||||||
|
Credential,
|
||||||
|
CredentialKind,
|
||||||
|
ItemStatus,
|
||||||
|
Mount,
|
||||||
|
MountType,
|
||||||
|
PatternType,
|
||||||
|
Rule,
|
||||||
|
RuleAction,
|
||||||
|
RunStatus,
|
||||||
|
)
|
||||||
|
from netmover.mounting.manager import MountManager
|
||||||
|
from netmover.mounting.sftp import SftpConfig, connect
|
||||||
|
from netmover.mover.destinations import LocalDestination, SftpDestination
|
||||||
|
from netmover.mover.engine import Engine, RuleSnapshot, RunResult
|
||||||
|
from netmover.mover.ledger import Ledger
|
||||||
|
from netmover.mover.runner import RuleRunner
|
||||||
|
from netmover.mover.scan import Matcher
|
||||||
|
from netmover.mover.sources import SftpSource
|
||||||
|
from netmover.timeouts import CancelToken
|
||||||
|
from tests.conftest import MountFactory
|
||||||
|
from tests.sftp_stub import StubSFTPServer
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def server(tmp_path: Path) -> Iterator[StubSFTPServer]:
|
||||||
|
root = tmp_path / "sftp-root"
|
||||||
|
for d in ("eingang", "ziel", "consume"):
|
||||||
|
(root / d).mkdir(parents=True)
|
||||||
|
srv = StubSFTPServer(root).start()
|
||||||
|
yield srv
|
||||||
|
srv.stop()
|
||||||
|
|
||||||
|
|
||||||
|
def _cfg(server: StubSFTPServer) -> SftpConfig:
|
||||||
|
return SftpConfig(
|
||||||
|
"127.0.0.1",
|
||||||
|
server.port,
|
||||||
|
server.username,
|
||||||
|
"/",
|
||||||
|
private_key=server.client_private_key,
|
||||||
|
host_key=server.host_key_string,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def source(server: StubSFTPServer) -> Iterator[SftpSource]:
|
||||||
|
s = SftpSource(connect(_cfg(server)), "/", "eingang", label="nas2")
|
||||||
|
yield s
|
||||||
|
s.close()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def rule_id(make_mount: MountFactory, session_factory: sessionmaker[Session]) -> int:
|
||||||
|
mid = make_mount()
|
||||||
|
with session_scope(session_factory) as s:
|
||||||
|
r = Rule(name="r", src_mount_id=mid, dst_path="/x")
|
||||||
|
s.add(r)
|
||||||
|
s.flush()
|
||||||
|
return r.id
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def hooks() -> list[Callable[[], None]]:
|
||||||
|
return []
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def engine(session_factory: sessionmaker[Session], hooks: list[Callable[[], None]]) -> Engine:
|
||||||
|
def sleep(_: float) -> None:
|
||||||
|
for h in hooks:
|
||||||
|
h()
|
||||||
|
|
||||||
|
return Engine(Ledger(session_factory), sleep=sleep)
|
||||||
|
|
||||||
|
|
||||||
|
def snap(rule_id: int, **kw: object) -> RuleSnapshot:
|
||||||
|
return RuleSnapshot(id=rule_id, name="r", stability_seconds=2, **kw) # type: ignore[arg-type]
|
||||||
|
|
||||||
|
|
||||||
|
def write(path: Path, data: bytes = b"inhalt") -> Path:
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
path.write_bytes(data)
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
def run(
|
||||||
|
engine: Engine, rule: RuleSnapshot, src: SftpSource, dest: object, **kw: object
|
||||||
|
) -> RunResult:
|
||||||
|
return engine.execute(rule, src, dest, CancelToken(), **kw) # type: ignore[arg-type]
|
||||||
|
|
||||||
|
|
||||||
|
def sha(p: Path) -> str:
|
||||||
|
return hashlib.sha256(p.read_bytes()).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
# -- Scan ----------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_scan_filters_like_local_sources(source: SftpSource, server: StubSFTPServer) -> None:
|
||||||
|
root = server.root / "eingang"
|
||||||
|
for name in ("a.pdf", "b.PDF", "notiz.txt", ".versteckt.pdf", "halb.pdf.part", "x.tmp"):
|
||||||
|
write(root / name)
|
||||||
|
write(root / "max" / "c.pdf")
|
||||||
|
write(root / "@eaDir" / "a.pdf" / "thumb.pdf")
|
||||||
|
write(root / "quarantaene" / "d.pdf")
|
||||||
|
m = Matcher("*.pdf", PatternType.GLOB)
|
||||||
|
assert [c.relpath for c in source.scan(m, recursive=False)] == ["a.pdf", "b.PDF"]
|
||||||
|
rec = [
|
||||||
|
c.relpath
|
||||||
|
for c in source.scan(m, recursive=True, exclude=frozenset({"eingang/quarantaene"}))
|
||||||
|
]
|
||||||
|
assert rec == ["a.pdf", "b.PDF", "max/c.pdf"]
|
||||||
|
c = source.scan(m, recursive=False)[0]
|
||||||
|
assert c.path is None and c.state.size == len(b"inhalt")
|
||||||
|
|
||||||
|
|
||||||
|
def test_missing_source_dir_aborts(
|
||||||
|
engine: Engine, rule_id: int, server: StubSFTPServer, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
src = SftpSource(connect(_cfg(server)), "/", "gibtsnicht")
|
||||||
|
try:
|
||||||
|
result = run(
|
||||||
|
engine, snap(rule_id), src, LocalDestination(tmp_path / "dst", create_root=True)
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
src.close()
|
||||||
|
assert result.status is RunStatus.ERROR and "existiert nicht" in (result.message or "")
|
||||||
|
|
||||||
|
|
||||||
|
# -- Verschieben / Kopieren ----------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_move_sftp_to_local(
|
||||||
|
engine: Engine, rule_id: int, source: SftpSource, server: StubSFTPServer, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
f = write(server.root / "eingang" / "scan001.pdf", os.urandom(300_000))
|
||||||
|
digest = sha(f)
|
||||||
|
dst = tmp_path / "dst"
|
||||||
|
dst.mkdir()
|
||||||
|
write(dst / "scan001.pdf", b"alt")
|
||||||
|
result = run(engine, snap(rule_id), source, LocalDestination(dst))
|
||||||
|
assert result.status is RunStatus.OK, result.message
|
||||||
|
assert [(i.src, i.status) for i in result.items] == [("scan001.pdf", ItemStatus.MOVED)]
|
||||||
|
assert sha(dst / "scan001-1.pdf") == digest and (dst / "scan001.pdf").read_bytes() == b"alt"
|
||||||
|
assert not f.exists()
|
||||||
|
assert sorted(p.name for p in dst.iterdir()) == ["scan001-1.pdf", "scan001.pdf"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_move_sftp_to_sftp(
|
||||||
|
engine: Engine, rule_id: int, source: SftpSource, server: StubSFTPServer
|
||||||
|
) -> None:
|
||||||
|
f = write(server.root / "eingang" / "a.pdf", os.urandom(50_000))
|
||||||
|
digest = sha(f)
|
||||||
|
dest = SftpDestination(connect(_cfg(server)), "/ziel", label="nas3")
|
||||||
|
try:
|
||||||
|
result = run(engine, snap(rule_id), source, dest)
|
||||||
|
finally:
|
||||||
|
dest.close()
|
||||||
|
assert result.items[0].status is ItemStatus.MOVED and result.items[0].dst == "nas3:/ziel/a.pdf"
|
||||||
|
assert sha(server.root / "ziel" / "a.pdf") == digest and not f.exists()
|
||||||
|
assert list((server.root / "ziel").glob(".netmover-*")) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_copy_mode_remembers_versions(
|
||||||
|
engine: Engine,
|
||||||
|
rule_id: int,
|
||||||
|
source: SftpSource,
|
||||||
|
server: StubSFTPServer,
|
||||||
|
tmp_path: Path,
|
||||||
|
session_factory: sessionmaker[Session],
|
||||||
|
) -> None:
|
||||||
|
f = write(server.root / "eingang" / "a.pdf", b"v1")
|
||||||
|
dst = tmp_path / "dst"
|
||||||
|
dst.mkdir()
|
||||||
|
rule = snap(rule_id, action=RuleAction.COPY)
|
||||||
|
for _ in range(3):
|
||||||
|
run(engine, rule, source, LocalDestination(dst))
|
||||||
|
assert sorted(p.name for p in dst.iterdir()) == ["a.pdf"] and f.exists()
|
||||||
|
f.write_bytes(b"version 2") # andere Größe → neue Version
|
||||||
|
run(engine, rule, source, LocalDestination(dst))
|
||||||
|
assert sorted(p.name for p in dst.iterdir()) == ["a-1.pdf", "a.pdf"]
|
||||||
|
with session_factory() as s:
|
||||||
|
assert len(s.scalars(select(CopiedFile)).all()) == 2
|
||||||
|
|
||||||
|
|
||||||
|
def test_mtime_is_carried_over(
|
||||||
|
engine: Engine, rule_id: int, source: SftpSource, server: StubSFTPServer, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
f = write(server.root / "eingang" / "a.pdf")
|
||||||
|
os.utime(f, (1_600_000_000, 1_600_000_123))
|
||||||
|
dst = tmp_path / "dst"
|
||||||
|
dst.mkdir()
|
||||||
|
run(engine, snap(rule_id, action=RuleAction.COPY), source, LocalDestination(dst))
|
||||||
|
assert int((dst / "a.pdf").stat().st_mtime) == 1_600_000_123
|
||||||
|
|
||||||
|
|
||||||
|
# -- Fallstricke ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_growing_file_is_not_touched(
|
||||||
|
engine: Engine,
|
||||||
|
rule_id: int,
|
||||||
|
source: SftpSource,
|
||||||
|
server: StubSFTPServer,
|
||||||
|
tmp_path: Path,
|
||||||
|
hooks: list[Callable[[], None]],
|
||||||
|
) -> None:
|
||||||
|
growing = write(server.root / "eingang" / "wachsend.pdf", b"x" * 10)
|
||||||
|
hooks.append(lambda: growing.write_bytes(b"x" * 20))
|
||||||
|
dst = tmp_path / "dst"
|
||||||
|
dst.mkdir()
|
||||||
|
result = run(engine, snap(rule_id), source, LocalDestination(dst))
|
||||||
|
assert result.items[0].status is ItemStatus.SKIPPED and growing.exists()
|
||||||
|
assert list(dst.iterdir()) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_source_changed_during_transfer_keeps_source(
|
||||||
|
engine: Engine,
|
||||||
|
rule_id: int,
|
||||||
|
source: SftpSource,
|
||||||
|
server: StubSFTPServer,
|
||||||
|
tmp_path: Path,
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
) -> None:
|
||||||
|
f = write(server.root / "eingang" / "a.pdf", b"original")
|
||||||
|
real = LocalDestination._apply_meta
|
||||||
|
|
||||||
|
def change(self: LocalDestination, rel: str, meta: object) -> list[str]:
|
||||||
|
f.write_bytes(b"geaendert!!")
|
||||||
|
return real(self, rel, meta) # type: ignore[arg-type]
|
||||||
|
|
||||||
|
monkeypatch.setattr(LocalDestination, "_apply_meta", change)
|
||||||
|
dst = tmp_path / "dst"
|
||||||
|
dst.mkdir()
|
||||||
|
result = run(engine, snap(rule_id), source, LocalDestination(dst))
|
||||||
|
assert result.items[0].status is ItemStatus.SKIPPED and f.exists()
|
||||||
|
assert list(dst.iterdir()) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_source_not_deletable_prevents_duplicates(
|
||||||
|
engine: Engine,
|
||||||
|
rule_id: int,
|
||||||
|
source: SftpSource,
|
||||||
|
server: StubSFTPServer,
|
||||||
|
tmp_path: Path,
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
) -> None:
|
||||||
|
write(server.root / "eingang" / "a.pdf")
|
||||||
|
|
||||||
|
def deny(*a: object, **kw: object) -> None:
|
||||||
|
raise PermissionError(errno.EACCES, "Permission denied")
|
||||||
|
|
||||||
|
monkeypatch.setattr(source.client, "remove", deny)
|
||||||
|
dst = tmp_path / "dst"
|
||||||
|
dst.mkdir()
|
||||||
|
first = run(engine, snap(rule_id), source, LocalDestination(dst))
|
||||||
|
assert first.items[0].status is ItemStatus.FAILED and "nicht löschbar" in (
|
||||||
|
first.items[0].message or ""
|
||||||
|
)
|
||||||
|
second = run(engine, snap(rule_id), source, LocalDestination(dst))
|
||||||
|
assert second.items == [] and sorted(p.name for p in dst.iterdir()) == ["a.pdf"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_quarantine_on_sftp_source(
|
||||||
|
engine: Engine,
|
||||||
|
rule_id: int,
|
||||||
|
source: SftpSource,
|
||||||
|
server: StubSFTPServer,
|
||||||
|
tmp_path: Path,
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
) -> None:
|
||||||
|
from netmover.mover.destinations import TransferError
|
||||||
|
|
||||||
|
def broken(*a: object, **kw: object) -> None:
|
||||||
|
raise TransferError("kaputt")
|
||||||
|
|
||||||
|
monkeypatch.setattr(LocalDestination, "_link_noreplace", broken)
|
||||||
|
write(server.root / "eingang" / "defekt.pdf")
|
||||||
|
dst = tmp_path / "dst"
|
||||||
|
dst.mkdir()
|
||||||
|
rule = snap(rule_id, quarantine_after=2)
|
||||||
|
for _ in range(2):
|
||||||
|
run(engine, rule, source, LocalDestination(dst), quarantine_rel="eingang/_q")
|
||||||
|
r = run(engine, rule, source, LocalDestination(dst), quarantine_rel="eingang/_q")
|
||||||
|
assert r.items[0].status is ItemStatus.QUARANTINED
|
||||||
|
assert r.items[0].dst == "nas2:/eingang/_q/defekt.pdf"
|
||||||
|
assert (server.root / "eingang" / "_q" / "defekt.pdf").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_connection_loss_aborts_run(
|
||||||
|
engine: Engine,
|
||||||
|
rule_id: int,
|
||||||
|
source: SftpSource,
|
||||||
|
server: StubSFTPServer,
|
||||||
|
tmp_path: Path,
|
||||||
|
hooks: list[Callable[[], None]],
|
||||||
|
) -> None:
|
||||||
|
write(server.root / "eingang" / "a.pdf")
|
||||||
|
write(server.root / "eingang" / "b.pdf")
|
||||||
|
hooks.append(lambda: source.session.transport.close()) # Verbindung bricht weg
|
||||||
|
dst = tmp_path / "dst"
|
||||||
|
dst.mkdir()
|
||||||
|
result = run(engine, snap(rule_id), source, LocalDestination(dst))
|
||||||
|
assert result.status is RunStatus.ERROR
|
||||||
|
assert (server.root / "eingang" / "a.pdf").exists() and list(dst.iterdir()) == []
|
||||||
|
|
||||||
|
|
||||||
|
# -- Regellauf mit SFTP-Quelle ------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_runner_with_sftp_source(
|
||||||
|
server: StubSFTPServer,
|
||||||
|
manager: MountManager,
|
||||||
|
session_factory: sessionmaker[Session],
|
||||||
|
secretbox: SecretBox,
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
write(server.root / "eingang" / "scan.pdf", b"%PDF")
|
||||||
|
consume = tmp_path / "consume"
|
||||||
|
consume.mkdir()
|
||||||
|
with session_scope(session_factory) as s:
|
||||||
|
c = Credential(
|
||||||
|
name="k",
|
||||||
|
kind=CredentialKind.SSH_KEY,
|
||||||
|
username=server.username,
|
||||||
|
secret_enc=secretbox.encrypt(server.client_private_key),
|
||||||
|
)
|
||||||
|
s.add(c)
|
||||||
|
s.flush()
|
||||||
|
m = Mount(
|
||||||
|
name="Scanner-Server",
|
||||||
|
slug="scanner",
|
||||||
|
type=MountType.SFTP,
|
||||||
|
server="127.0.0.1",
|
||||||
|
share="/",
|
||||||
|
port=server.port,
|
||||||
|
cred_id=c.id,
|
||||||
|
host_key=server.host_key_string,
|
||||||
|
)
|
||||||
|
s.add(m)
|
||||||
|
s.flush()
|
||||||
|
r = Rule(
|
||||||
|
name="SFTP → lokal",
|
||||||
|
src_mount_id=m.id,
|
||||||
|
src_subpath="eingang",
|
||||||
|
dst_path=str(consume),
|
||||||
|
stability_seconds=2,
|
||||||
|
)
|
||||||
|
s.add(r)
|
||||||
|
s.flush()
|
||||||
|
rid = r.id
|
||||||
|
result = RuleRunner(session_factory, manager, sleep=lambda _: None).run(rid)
|
||||||
|
assert result.status is RunStatus.OK, result.message
|
||||||
|
assert result.items[0].dst == str(consume / "scan.pdf")
|
||||||
|
assert (consume / "scan.pdf").read_bytes() == b"%PDF"
|
||||||
|
assert not (server.root / "eingang" / "scan.pdf").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_rule_form_accepts_sftp_source_with_min_stability(
|
||||||
|
session_factory: sessionmaker[Session],
|
||||||
|
) -> None:
|
||||||
|
from netmover.admin import parse_rule_form
|
||||||
|
|
||||||
|
with session_scope(session_factory) as s:
|
||||||
|
m = Mount(name="sftp", slug="sftp", type=MountType.SFTP, server="10.0.0.1", share="/in")
|
||||||
|
s.add(m)
|
||||||
|
s.flush()
|
||||||
|
form = {
|
||||||
|
"name": "x",
|
||||||
|
"src_mount_id": str(m.id),
|
||||||
|
"dst_kind": "local",
|
||||||
|
"dst_path": "/tmp",
|
||||||
|
"interval_seconds": "30",
|
||||||
|
"stability_seconds": "1",
|
||||||
|
"op_timeout_seconds": "120",
|
||||||
|
}
|
||||||
|
_, errors = parse_rule_form(s, form)
|
||||||
|
assert "src_mount_id" not in errors and "mindestens 2 s" in errors["stability_seconds"]
|
||||||
|
_, errors = parse_rule_form(s, {**form, "stability_seconds": "3"})
|
||||||
|
assert errors == {}
|
||||||
@@ -512,12 +512,16 @@ def test_export_excludes_secrets_and_import_roundtrip(
|
|||||||
def test_keys_page_generate_import_select_delete(client: TestClient, state: NetMover) -> None:
|
def test_keys_page_generate_import_select_delete(client: TestClient, state: NetMover) -> None:
|
||||||
from netmover.crypto import generate_ssh_keypair
|
from netmover.crypto import generate_ssh_keypair
|
||||||
|
|
||||||
r = post(client, "/keys/generate", {"name": "paperless-lxc"})
|
r = post(client, "/credentials/keys/generate", {"name": "paperless-lxc"})
|
||||||
assert r.status_code == 303
|
assert r.status_code == 303
|
||||||
private, public = generate_ssh_keypair()
|
private, public = generate_ssh_keypair()
|
||||||
r = post(client, "/keys/import", {"name": "alt", "private_key": private, "passphrase": ""})
|
r = post(
|
||||||
|
client,
|
||||||
|
"/credentials/keys/import",
|
||||||
|
{"name": "alt", "private_key": private, "passphrase": ""},
|
||||||
|
)
|
||||||
assert r.status_code == 303
|
assert r.status_code == 303
|
||||||
page = client.get("/keys").text
|
page = client.get("/credentials").text
|
||||||
assert "paperless-lxc" in page and "alt" in page
|
assert "paperless-lxc" in page and "alt" in page
|
||||||
assert public.split()[1] in page # öffentlicher Teil sichtbar
|
assert public.split()[1] in page # öffentlicher Teil sichtbar
|
||||||
for line in private.splitlines()[1:-1]: # keine Zeile des privaten Schlüssels
|
for line in private.splitlines()[1:-1]: # keine Zeile des privaten Schlüssels
|
||||||
@@ -546,27 +550,31 @@ def test_keys_page_generate_import_select_delete(client: TestClient, state: NetM
|
|||||||
)
|
)
|
||||||
assert r.status_code == 303, r.text
|
assert r.status_code == 303, r.text
|
||||||
mounts = client.get("/mounts").text
|
mounts = client.get("/mounts").text
|
||||||
assert "gespeichertem Schlüssel" in mounts and "„alt“" in mounts
|
assert "gespeicherten Zugangsdaten" in mounts and "„alt“" in mounts
|
||||||
edit = client.get(f"/mounts/{_mount_id(state, 'paperless')}/edit").text
|
edit = client.get(f"/mounts/{_mount_id(state, 'paperless')}/edit").text
|
||||||
assert f'<option value="{key_id}" selected>' in edit and 'value="root"' in edit
|
assert f'<option value="{key_id}" selected>' in edit and 'value="root"' in edit
|
||||||
|
|
||||||
post(client, f"/keys/{key_id}/delete")
|
post(client, f"/credentials/{key_id}/delete")
|
||||||
assert "noch benutzt" in client.get("/keys").text
|
assert "noch benutzt" in client.get("/credentials").text
|
||||||
post(client, f"/mounts/{_mount_id(state, 'paperless')}/delete")
|
post(client, f"/mounts/{_mount_id(state, 'paperless')}/delete")
|
||||||
post(client, f"/keys/{key_id}/delete")
|
post(client, f"/credentials/{key_id}/delete")
|
||||||
page = client.get("/keys").text
|
page = client.get("/credentials").text
|
||||||
assert "gelöscht" in page and ">alt<" not in page
|
assert "gelöscht" in page and ">alt<" not in page
|
||||||
|
|
||||||
|
|
||||||
def test_key_import_error_is_shown(client: TestClient) -> None:
|
def test_key_import_error_is_shown(client: TestClient) -> None:
|
||||||
post(client, "/keys/import", {"name": "x", "private_key": "kein schlüssel", "passphrase": ""})
|
post(
|
||||||
assert "Kein lesbarer privater Schlüssel" in client.get("/keys").text
|
client,
|
||||||
|
"/credentials/keys/import",
|
||||||
|
{"name": "x", "private_key": "kein schlüssel", "passphrase": ""},
|
||||||
|
)
|
||||||
|
assert "Kein lesbarer privater Schlüssel" in client.get("/credentials").text
|
||||||
|
|
||||||
|
|
||||||
def test_no_escaped_attributes_in_pages(client: TestClient, state: NetMover) -> None:
|
def test_no_escaped_attributes_in_pages(client: TestClient, state: NetMover) -> None:
|
||||||
"""Attribute in {{ … }} werden von Jinja maskiert – dann landet " im HTML."""
|
"""Attribute in {{ … }} werden von Jinja maskiert – dann landet " im HTML."""
|
||||||
|
|
||||||
post(client, "/keys/generate", {"name": "k"})
|
post(client, "/credentials/keys/generate", {"name": "k"})
|
||||||
for url in (
|
for url in (
|
||||||
"/",
|
"/",
|
||||||
"/mounts",
|
"/mounts",
|
||||||
@@ -581,3 +589,35 @@ def test_no_escaped_attributes_in_pages(client: TestClient, state: NetMover) ->
|
|||||||
html = client.get(url).text
|
html = client.get(url).text
|
||||||
broken = re.findall(r"<[a-z][^<>]*="[^<>]*>", html) # nur in Tags, nicht im Text
|
broken = re.findall(r"<[a-z][^<>]*="[^<>]*>", html) # nur in Tags, nicht im Text
|
||||||
assert not broken, (url, broken[:2])
|
assert not broken, (url, broken[:2])
|
||||||
|
|
||||||
|
|
||||||
|
def test_credentials_page_passwords(
|
||||||
|
client: TestClient, state: NetMover, backend: FakeMountBackend
|
||||||
|
) -> None:
|
||||||
|
r = post(
|
||||||
|
client,
|
||||||
|
"/credentials/passwords",
|
||||||
|
{"name": "NAS-Scanner", "username": "scanner", "password": "Geheim,123", "domain": ""},
|
||||||
|
)
|
||||||
|
assert r.status_code == 303
|
||||||
|
page = client.get("/credentials").text
|
||||||
|
assert "NAS-Scanner" in page and "scanner" in page and "Geheim,123" not in page
|
||||||
|
with state.session_factory() as s:
|
||||||
|
cid = s.scalar(select(Credential.id).where(Credential.name == "NAS-Scanner"))
|
||||||
|
form = client.get("/mounts/new").text
|
||||||
|
assert f'<option value="{cid}"' in form and "eigene Angaben für diese Anbindung" in form
|
||||||
|
backend.export("nas", "scans")
|
||||||
|
r = post(
|
||||||
|
client, "/mounts/new", {**MOUNT_FORM, "username": "", "password": "", "cred_id": str(cid)}
|
||||||
|
)
|
||||||
|
assert r.status_code == 303, r.text
|
||||||
|
assert "gespeicherten Zugangsdaten" in client.get("/mounts").text
|
||||||
|
mid = _mount_id(state, "Synology Scans")
|
||||||
|
assert "Quelle mit Löschen" in post(client, f"/mounts/{mid}/test").text
|
||||||
|
post(
|
||||||
|
client,
|
||||||
|
f"/credentials/{cid}/password",
|
||||||
|
{"username": "scanner2", "domain": "", "password": ""},
|
||||||
|
)
|
||||||
|
assert "gilt ab sofort für alle" in client.get("/credentials").text
|
||||||
|
assert client.get("/keys").headers["location"] == "/credentials"
|
||||||
|
|||||||