kstyagi/website-downloader
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update retention policy to a maximum of 2 days for completed archives and adjust related configurations

Browse Source
This commit is contained in:
kstyagi23
2026-09-05 06:37:35 +05:30
parent 48402f7d61
commit 9c1a301c1a
10 changed files with 477 additions and 92 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+4 -1
View File
@@ -60,6 +60,9 @@ python -m uvicorn app.main:app --host 127.0.0.1 --port 8787 --workers 2
The default bind address is loopback. Do not expose SiteHarbor directly to the public Internet without authentication, a network egress policy, rate limits, and operational controls.
Completed ZIP archives and their reports are retained for at most two days after a capture finishes.
The maintenance worker removes expired files and marks the capture as expired.
## Configuration
All configuration uses environment variables prefixed with `SITEHARBOR_`.
@@ -75,7 +78,7 @@ All configuration uses environment variables prefixed with `SITEHARBOR_`.
| `SITEHARBOR_ALLOW_PRIVATE_NETWORKS` | `false` | Development-only override that permits loopback and private hosts. |
| `SITEHARBOR_ALLOW_NONSTANDARD_PORTS` | `false` | Development-only override for ports other than 80 and 443. |
| `SITEHARBOR_RESPECT_ROBOTS` | `true` | Respect robots rules during capture. |
| `SITEHARBOR_DEFAULT_RETENTION_DAYS` | `7` | Default retention period for completed artifacts. |
| `SITEHARBOR_DEFAULT_RETENTION_DAYS` | `2` | Default retention period for completed artifacts; values cannot exceed two days. |
| `SITEHARBOR_PROXY_URL` | unset | Operator-approved HTTP(S) proxy route. Visitors can only opt into this configured route; they cannot supply proxy endpoints. |
For a local fixture site on `localhost:8000`, use both development overrides: