From 2d05524dfd9ff9fd5571ca3459ff086eb059bfb7 Mon Sep 17 00:00:00 2001 From: txoof-bot <337660340+txoof-bot@users.noreply.github.com> Date: Mon, 9 Nov 2026 07:51:16 +0100 Subject: [PATCH 1/4] Specification: screens and drivers (#346) Co-Authored-By: Claude Opus 5.5 --- docs/specs/screens.md | 114 ++++++++++++++++++++++++++++++++++++++++++ mkdocs.yml | 1 + 2 files changed, 115 insertions(+) create mode 100644 docs/specs/screens.md diff --git a/docs/specs/screens.md b/docs/specs/screens.md new file mode 100644 index 0000000..d030719 --- /dev/null +++ b/docs/specs/screens.md @@ -0,0 +1,114 @@ +# Screens and drivers + +A screen driver is the code that sends a picture to one screen model. The drivers are in epdlib, the separate library that also draws the layouts. PaperPi runs every screen write in a small helper process with a time limit, so a screen that stops answering can never freeze PaperPi. Writes are fast where they can be, full when another plugin comes on the screen, and the screen is cleaned once an hour. + +Which screens work, and how to set them up, is on [Supported screens](../screens.md). This page describes how PaperPi talks to them. + +## What every driver does + +Every driver offers the same parts, so PaperPi works the same on any screen: + +| Part | What it does | +| ------------- | -------------------------------------------------------------------------------------------------- | +| description | the model, its size in pixels, what it shows (black and white, grays, colors), whether it has fast writes, and whether it was tested | +| `init` | starts the screen and checks that it answers (the IT8951 board must report the right size) | +| `write` | shows a picture, fast or full | +| `clear` | makes the screen blank | +| `sleep` | puts the screen into low power; the next `write` or `clear` wakes it | +| `close` | releases the screen's data line (SPI) and pins (GPIO), also after an error | + +- **Every part has a time limit.** A driver never waits forever for the screen; when the time is up, it stops with a timeout error. +- **The drivers use only `gpiod` and `spidev`,** the standard Linux libraries for pins and SPI. They work on every Pi from the Pi 3 on. +- **Waveshare's small screens** use Waveshare's own program file for each model, unchanged. Only their shared helper file is replaced: it adds the time limits and claims only the pins the screen needs. + +## One write + +1. PaperPi sends the picture to the screen's helper process and waits, at most the time limit. +2. The driver wakes the screen, draws the picture and waits until the screen's busy signal (a wire that stays on while the screen redraws) switches off. +3. The screen goes to sleep until the next write. + +Note: Waveshare warns that a screen left powered for a long time can be damaged, so the screen sleeps after every write. + +**A write that hangs is not waited for:** the helper process is stopped, which makes the operating system release the screen's data line and pins. The next write starts a new helper process, and `init` resets the screen. + +## Fast, full and cleaning writes + +| Write | When | On the IT8951 screens | +| -------------- | ---------------------------------------------------------------------- | ------------------------------------------------------- | +| **fast** | a new picture from the plugin already on the screen (a clock tick, the next song) | no flash; only the changed part is sent | +| **full** | another plugin comes on the screen, or after `max_refresh` fast writes in a row | a flash, then a clean picture in 16 grays | +| **cleaning** | every `clean_every` seconds, at the next write, and at the first write after a start | a longer flash to white, then the picture in full | + +- **Plugins don't choose:** PaperPi decides from what changed. +- **Screens without fast writes** always do a full write, which leaves no leftovers, so PaperPi doesn't clean them. +- **Fast writes leave faint leftovers** of earlier pictures. A full write removes most of them; only a cleaning write removes all of them. + +## Colors and grays + +- The driver says what the screen can show. Each block of a layout is drawn directly in that form. +- **Text is never dithered** (dithering uses dot patterns to fake in-between shades), so it stays sharp. Pictures are dithered on their own block. +- A block marked for color uses color only on color screens; on other screens it is drawn in gray or black and white. +- `color = false` in `[display]` draws in gray on a color screen. + +## The virtual screen + +The virtual screen saves each picture as a PNG file instead of showing it. It is used for the tests, the pictures in this manual, `paperpi render` and the previews in the web interface. It can be any size and any kind of screen (black and white, 16 grays, 7 colors), so a plugin can be checked for any screen without the hardware. + +## Start and stop + +- **At start**, PaperPi starts the screen in its own thread and checks that it answers. If it doesn't, PaperPi keeps running and the web interface shows the error, for example "screen not answering: check the cable and the selected model". It does not crash and restart over and over. +- **On a stop that was asked for** (Ctrl+C, `systemctl stop`, a shutdown or restart of the Pi), the screen is cleared, or keeps its picture with `on_exit = "keep"`. +- **After a crash** the picture stays: the restart draws it again soon, and a clear would only add a flash. + +Note: the clear at a stop is skipped when a write is still running after 2 seconds, when writes are paused, or when the screen was never started, so stopping stays quick. A second stop signal ends it at once. + +## A screen that keeps failing + +The screen watchdog is a set of rules for a screen that keeps failing: + +1. **After 3 failed writes in a row,** the helper process is restarted, which resets the screen. +2. **When the write after 3 such resets fails too,** writes pause. The next try is after 10 minutes, and each failed try doubles the wait, up to 6 hours. PaperPi keeps running, and the web interface shows the problem. +3. **A config reload, or Try again now,** tries the screen again at once. +4. **When a helper process can't be stopped** (it is stuck inside the operating system), only a restart of PaperPi can help: PaperPi exits and systemd starts it again. This happens at most once an hour, so it can't become a restart loop. + +Note: a wrong setting or a switched-off SPI is not fixed by trying again, so trying less often keeps the log short. + +## Numbers + +| What | Value | Setting | +| -------------------------------------------- | --------------------------------------------------------- | --------------- | +| fast writes before a full one | 4 | `max_refresh` | +| time between cleaning writes | 1 hour (0: never; else at least 10 minutes) | `clean_every` | +| what happens at a stop | clear the screen | `on_exit` | +| longest first write, before the redraw time is measured | 2 minutes (color screens: 5 minutes) | none | +| longest write after that | 3 times the last redraw, at least 30 seconds and at most 5 minutes | none | +| failed writes before a reset | 3 in a row | none | +| resets before writes pause | 3 | none | +| wait while writes pause | 10 minutes, doubled after each failed try, at most 6 hours | none | +| wait for a running write at a stop | 2 seconds | none | +| longest clear at a stop | 30 seconds | none | +| PaperPi exits for a stuck helper process | at most once an hour | none | +| IT8951: one wait for the busy signal | 2 seconds (5 seconds after the reset in `init`) | none | +| IT8951: one whole `init`, `write` or `clear` | 15 seconds | none | + +## Not built yet + +- **Red and yellow on three-color screens:** they show only black and white for now. +- **A screen and a HiFiBerry on the e-Paper HAT:** both need GPIO 18, so they can't be used on the same Pi. +- **A memory limit for PaperPi** in its systemd service, as a safety net if it ever used too much memory. + +## Why it works this way + +- **A helper process for every write:** one stuck call deep inside a library can't always be stopped from inside Python, but a separate process can always be stopped. PaperPi v1 waited for the screen without a time limit, which is the likely cause of its freezes. +- **Waveshare's own files, unchanged:** about 60 models work without writing a driver for each, and only the shared helper file needed the time limits. +- **On the IT8951, fast writes use the DU mode:** the faster A2 mode left lines, stray pixels and negative shadows in the screen tests. A changed part with grays (such as the soft edges of text) uses GL16 instead. +- **Cleaning once an hour:** one full write did not remove all leftovers in the screen tests; the longer flash to white did. +- **The full story** is in the old design notes [`display-driver-interface.md`](https://github.com/txoof/PaperPi/blob/main/docs/history/decisions/display-driver-interface.md) and [`it8951-driver.md`](https://github.com/txoof/PaperPi/blob/main/docs/history/decisions/it8951-driver.md), and the screen tests in [`it8951-test-report.md`](https://github.com/txoof/PaperPi/blob/main/docs/it8951-test-report.md). + +## In the code + +- `src/paperpi/screen.py`: the helper process, the time limits and the screen watchdog. +- `src/paperpi/scheduler.py`: choosing fast, full and cleaning writes. +- `src/paperpi/config.py`: the `[display]` settings. +- `src/paperpi/limits.py`: the numbers above that have no setting. +- epdlib (a separate repository, [txoof/epdlib](https://github.com/txoof/epdlib)): the drivers, the layouts and the virtual screen. diff --git a/mkdocs.yml b/mkdocs.yml index 8d4fb7a..8d8f1dc 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -96,3 +96,4 @@ nav: - The config file: specs/config.md - Which plugin is shown: specs/scheduling.md - Plugins and setting helpers: specs/plugins.md + - Screens and drivers: specs/screens.md From 349dbb2ad453a2777231508e51a5fbf9f2c814eb Mon Sep 17 00:00:00 2001 From: txoof-bot <337660340+txoof-bot@users.noreply.github.com> Date: Mon, 9 Nov 2026 07:51:59 +0100 Subject: [PATCH 2/4] Specification: errors, time limits and freezes (#346) Co-Authored-By: Claude Opus 5.5 --- docs/specs/errors.md | 123 +++++++++++++++++++++++++++++++++++++++++++ mkdocs.yml | 1 + 2 files changed, 124 insertions(+) create mode 100644 docs/specs/errors.md diff --git a/docs/specs/errors.md b/docs/specs/errors.md new file mode 100644 index 0000000..845eada --- /dev/null +++ b/docs/specs/errors.md @@ -0,0 +1,123 @@ +# Errors, time limits and freezes + +PaperPi must keep running for months without anyone looking after it. PaperPi v1 stopped updating the screen after 3 to 6 months without crashing, so nothing restarted it. v2 follows a few strict rules to prevent that: every wait has a time limit, plugins and screen writes run in separate processes, logs and saved files have size limits, and a watchdog restarts PaperPi when it stops reporting that it is still running. + +What to do when something goes wrong is on [Troubleshooting](../troubleshooting.md). This page describes the rules inside PaperPi. + +## The rules + +1. **Every wait has a time limit:** for the screen, the network, a plugin and a file. +2. **All durations use the monotonic clock,** a clock that only counts forward and does not jump when the time of day is corrected. +3. **The screen's data line and pins are opened once and always closed,** also after an error. +4. **An error is handled, with a limit on retries, or it ends PaperPi.** No loop logs the same error forever. +5. **Plugins run in their own process** with a time limit they can't get around ([Plugins](plugins.md)). +6. **Logs and saved files have size limits.** +7. **A watchdog restarts PaperPi** when it stops reporting "still running". +8. **PaperPi records** the time since the last screen write, its memory use, its open files and the free disk space. + +## The watchdog + +- **The scheduler's loop reports "still running" every 30 seconds.** Only the loop reports, so if the loop gets stuck, the reports stop. +- **If no report arrives for 2 minutes,** systemd restarts PaperPi. +- **"Still running" does not need a recent screen write.** A broken or unplugged screen would otherwise make PaperPi restart over and over, and the web interface, which shows the error, would keep disappearing. A screen that keeps failing has its own rules ([A screen that keeps failing](screens.md#a-screen-that-keeps-failing)). + +Each report also replaces the file `/run/paperpi/health` (in memory, emptied at every start of the Pi) with one line: the time, the time since the last screen write, the screen's state (`ok`, `failing` or `paused`), memory use, open files and free disk. `paperpi health` fails when the report is missing or older than 2 minutes. The same values go to the log once an hour, so slow growth can be seen over weeks. + +Note: the systemd service needs `Type=notify`, `WatchdogSec=120` and `RuntimeDirectory=paperpi` for this; the installer sets them up ([How the service works](../install-systemd.md)). + +## When things fail + +| What fails | What PaperPi does | +| --------------------------------------------- | ------------------------------------------------------------------------------------------ | +| a plugin update | skips it and tries again at the next refresh; after 3 failures in a row, leaves the plugin out for 30 minutes ([Which plugin is shown](scheduling.md#failures)) | +| a web request of a plugin | tries once more after 5 seconds; then the plugin's update fails | +| a screen write | tries again, resets the screen, then pauses writes ([A screen that keeps failing](screens.md#a-screen-that-keeps-failing)) | +| a screen helper process that can't be stopped | exits, so systemd restarts PaperPi; at most once an hour | +| the config file | runs on the last good copy ([The config file](config.md#checking-the-file)) | +| an error nobody catches | logs it with its details, then exits; systemd restarts PaperPi | + +## Time limits + +| What | Time limit | +| ---------------------------- | ---------------------------------------------------------------------------------------------- | +| one plugin update | 60 seconds (at most 600), set per plugin with `time_limit` | +| one web request of a plugin | 10 seconds to connect, 30 seconds in all, including one retry after 5 seconds | +| one screen write | 3 times the last redraw, at least 30 seconds, at most 5 minutes; before the first one: 2 minutes (color screens: 5) | +| reading the config file on a reload | 10 seconds; a file that takes longer is not applied | + +Note: only `time_limit` is a setting. The other limits are set in one place in the code, `src/paperpi/limits.py`. + +A web answer may be at most 5 MB after unpacking; a plugin that downloads pictures can ask for more. All plugins use one helper for web requests, `paperpi.webrequest`, so the same limits apply to all of them. + +## Logs + +- **The system log** (`journalctl -u paperpi`) gets warnings and errors, the start and stop lines and the hourly health line; everything with `--verbose`. Raspberry Pi OS keeps the system log in memory only, so it is gone after a restart of the Pi. +- **PaperPi's own log file**, `paperpi.log` in `/var/lib/paperpi`, keeps the warnings, errors, starts, stops, crashes and the health line, so they survive a restart of the Pi. At 1 MB it becomes `paperpi.log.1` and a new file starts, so both together stay under about 2 MB. +- **The web interface** shows the last 5,000 lines (INFO and up) on its Full log page, and what needs attention on its Warnings page. +- **Plugins' log lines** are sent to PaperPi with their result, at most 100 per update, with secret settings replaced by `****`. +- **An error that repeats** is logged once; after that, a summary line every 10 minutes, e.g. "same error repeated 40 times". + +## Saved files + +Each plugin has its own storage folder in `/var/lib/paperpi/plugins/`. + +| Limit | Default | What happens | +| -------------- | -------- | ------------------------------------------------------------------------------ | +| `storage_mb` | 500 MB | over it, the files changed longest ago are removed until the folder fits | +| `storage_days` | 30 days | files not changed for this long are removed; `0` keeps them | + +- A plugin may suggest its own values (a photo album may need much more); the setting in the config file always wins. `paperpi list` shows the values used. +- PaperPi cleans the folders at start and after a plugin's update (at most once every 5 minutes per plugin). +- **Nothing outside the plugin's folder is ever removed.** Links are not followed, and a drive mounted inside the folder is skipped. +- Folders that no `[[plugin]]` block uses any more (for example after an `id` was changed) are not cleaned, because they may hold photos. They are named in the log at start, to be removed by hand. +- **PaperPi keeps at least 2 GB free** on the disk. Below that, plugins are told the disk is low, so they don't save more, and the log says so. PaperPi removes nothing else, because the space may be used by something outside PaperPi. + +Note: a file that is only read keeps getting older. A plugin that must keep a file (for example a login token) writes it again within `storage_days`, or suggests `storage_days = 0`. + +## Files on the SD card + +A frozen PaperPi v1 was not fixed by restarting PaperPi or the Pi, only by wiping the SD card and installing everything again. So v2 keeps everything it stores small and replaceable: + +- the config file has a last good copy; +- files are replaced in one step (first written to a temporary file), so a power cut can't leave half a file; +- logs and plugin folders have size limits, and PaperPi keeps 2 GB free; +- the health file is in `/run/paperpi`, in memory, so it causes no SD card writes. + +## Numbers + +| What | Value | +| -------------------------------------------- | --------------------------- | +| "still running" report | every 30 seconds | +| watchdog restarts PaperPi after | 2 minutes without a report | +| health values in the log | once an hour | +| log lines kept for the web interface | 5,000 | +| log file size before a new one starts | 1 MB (two files: about 2 MB) | +| longest log line | 10,000 characters | +| log lines per plugin update | 100 | +| summary of a repeated error | every 10 minutes | +| free disk PaperPi keeps | 2 GB | +| longest clean-up of one storage folder | 10 seconds | +| most files looked at in one clean-up | 100,000 | + +## Not built yet + +- **A memory limit for PaperPi** in its systemd service (`MemoryMax=`), as a safety net if it ever used too much memory. +- **The screen helper process's own log lines** (epdlib's) are not sent to PaperPi's log yet. +- **Changing the web request limits** in the config file. + +## Why it works this way + +- **"Still running" means the loop runs, not that the screen works:** otherwise a broken screen would cause a restart loop, and the web interface that shows the error would keep going away. +- **A screen that keeps failing pauses instead of exiting:** with a broken screen, exiting would also become a restart loop. +- **Size limits per plugin, not for all plugins together:** SD cards are large, and a photo plugin needs much more than a weather plugin. The 2 GB of free space protects the rest of the system. +- **The full story** is in the old design notes [`errors-and-time-limits.md`](https://github.com/txoof/PaperPi/blob/main/docs/history/decisions/errors-and-time-limits.md) and [`freeze-prevention.md`](https://github.com/txoof/PaperPi/blob/main/docs/history/decisions/freeze-prevention.md). + +## In the code + +- `src/paperpi/limits.py`: every time limit and size limit. +- `src/paperpi/health.py`: the "still running" reports to systemd and the health file. +- `src/paperpi/logbook.py`: the log file, the lines for the web interface and repeated errors. +- `src/paperpi/storage.py`: the storage folders, their limits and the free disk check. +- `src/paperpi/webrequest.py`: web requests with their time and size limits. +- `src/paperpi/screen.py`: the screen's time limits and watchdog ([Screens and drivers](screens.md)). +- `src/paperpi/files.py`: writing a file in one step. diff --git a/mkdocs.yml b/mkdocs.yml index 8d8f1dc..8f8bd6d 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -97,3 +97,4 @@ nav: - Which plugin is shown: specs/scheduling.md - Plugins and setting helpers: specs/plugins.md - Screens and drivers: specs/screens.md + - Errors and time limits: specs/errors.md From 420375a3a51446fb49aac5ef39e8671188061a53 Mon Sep 17 00:00:00 2001 From: txoof-bot <337660340+txoof-bot@users.noreply.github.com> Date: Mon, 9 Nov 2026 07:52:14 +0100 Subject: [PATCH 3/4] errors spec: the repeated-error summary is not built yet (#346) Co-Authored-By: Claude Opus 5.5 --- docs/specs/errors.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/specs/errors.md b/docs/specs/errors.md index 845eada..3a86157 100644 --- a/docs/specs/errors.md +++ b/docs/specs/errors.md @@ -55,7 +55,6 @@ A web answer may be at most 5 MB after unpacking; a plugin that downloads pictur - **PaperPi's own log file**, `paperpi.log` in `/var/lib/paperpi`, keeps the warnings, errors, starts, stops, crashes and the health line, so they survive a restart of the Pi. At 1 MB it becomes `paperpi.log.1` and a new file starts, so both together stay under about 2 MB. - **The web interface** shows the last 5,000 lines (INFO and up) on its Full log page, and what needs attention on its Warnings page. - **Plugins' log lines** are sent to PaperPi with their result, at most 100 per update, with secret settings replaced by `****`. -- **An error that repeats** is logged once; after that, a summary line every 10 minutes, e.g. "same error repeated 40 times". ## Saved files @@ -94,7 +93,6 @@ A frozen PaperPi v1 was not fixed by restarting PaperPi or the Pi, only by wipin | log file size before a new one starts | 1 MB (two files: about 2 MB) | | longest log line | 10,000 characters | | log lines per plugin update | 100 | -| summary of a repeated error | every 10 minutes | | free disk PaperPi keeps | 2 GB | | longest clean-up of one storage folder | 10 seconds | | most files looked at in one clean-up | 100,000 | @@ -104,6 +102,7 @@ A frozen PaperPi v1 was not fixed by restarting PaperPi or the Pi, only by wipin - **A memory limit for PaperPi** in its systemd service (`MemoryMax=`), as a safety net if it ever used too much memory. - **The screen helper process's own log lines** (epdlib's) are not sent to PaperPi's log yet. - **Changing the web request limits** in the config file. +- **A summary for an error that repeats:** log it once, then a line every 10 minutes such as "same error repeated 40 times". Some parts already do this on their own (the screen watchdog, the low-disk warning). ## Why it works this way From 8420b860c45e7d8f3dd699419c8381a19e1ca959 Mon Sep 17 00:00:00 2001 From: txoof-bot <337660340+txoof-bot@users.noreply.github.com> Date: Mon, 9 Nov 2026 07:53:17 +0100 Subject: [PATCH 4/4] Specification: the web interface; code links point to the new specifications (#346) Co-Authored-By: Claude Opus 5.5 --- docs/specs/web.md | 91 +++++++++++++++++++ install/paperpi.service | 2 +- mkdocs.yml | 1 + src/paperpi/health.py | 2 +- src/paperpi/limits.py | 2 +- src/paperpi/logbook.py | 4 +- src/paperpi/plugins/splash_screen/__init__.py | 2 +- src/paperpi/screen.py | 2 +- src/paperpi/storage.py | 2 +- src/paperpi/web/__init__.py | 2 +- src/paperpi/web/password.py | 2 +- src/paperpi/web/pictures.py | 2 +- src/paperpi/web/preview.py | 2 +- src/paperpi/webrequest.py | 2 +- 14 files changed, 105 insertions(+), 13 deletions(-) create mode 100644 docs/specs/web.md diff --git a/docs/specs/web.md b/docs/specs/web.md new file mode 100644 index 0000000..2be3335 --- /dev/null +++ b/docs/specs/web.md @@ -0,0 +1,91 @@ +# The web interface + +The web interface is the main way to set up and change PaperPi: plugins, the screen and the password. It runs inside PaperPi itself, on the Pi, and is light enough for a Pi 3. The Pi builds every page and sends it ready to show; a small script, htmx, updates parts of a page without reloading it. It is for the home network only. + +What each page does is on [Web Interface](../web-interface.md). This page describes how the web interface is built and kept safe. + +## How it is built + +- **Inside `paperpi run`,** in its own thread, not as a separate program. It can tell the scheduler directly to apply a change, and it needs no extra Python process (about 40 MB of memory). If it can't start (for example the port is taken), the log says why and the screen keeps running. +- **Pages are built on the Pi** from templates, with FastAPI (a Python web framework built on pydantic, the package that also checks the config file) and jinja2 (page templates). No build step and no second programming language. +- **htmx** (a small script of about 15 KB) updates one part of a page without reloading it, such as the picture on the screen now or the list of warnings. +- **A page that needs dragging** (the plugin list on the home page) uses a small library loaded as a plain file. Buttons do the same job, so everything works on a phone too. + +## Plugin settings pages + +- **One page handles every plugin.** It reads the plugin's settings description and builds the form from it: number fields, lists to choose from, password fields for secrets. +- **The page has two sections:** General (the settings every plugin has, with "Time limit, alerts, storage" folded inside) and the plugin's own settings, in the groups the plugin names. +- **On Save,** the values are checked with the same description, and errors are shown next to their field. Only settings that changed are written to the config file, and a saved change applies at once ([The config file](config.md#applying-changes)). +- **An empty field means the default.** A value changed back to the default is taken out of the file again. +- **A secret field is always empty on the page.** Leaving it empty keeps the saved secret. +- **Setting helpers** add a search box to the page ([Plugins and setting helpers](plugins.md#setting-helpers)). + +Plugins added later show up without changes to the web code. + +## Pictures + +| Where | What it shows | +| ------------------------ | ------------------------------------------------------------------------------------------------------- | +| home page | a copy of what is on the screen now, with the plugin's name and when it was drawn; it updates by itself | +| **Preview** on a settings page | the plugin drawn with the values in the form, before saving; the real screen is not touched | +| Plugin Library | an example picture of every layout of every plugin, drawn from its sample data | + +- **A preview** draws the block as **Save** would write it, at the screen's size and for the screen's kind. It runs in its own process with a time limit, in a new, empty storage folder that is removed afterwards, so it can't change the plugin's saved files. Real data may take at most 45 seconds; when it fails, takes longer or has nothing to show, the preview uses the sample data and says why. One preview is drawn at a time. +- **Preview pictures are sent inside the page,** never saved as files. +- **Library pictures are drawn ahead** and saved in `library-pictures/` in `/var/lib/paperpi`, so the page opens at once. They wait 60 seconds after PaperPi starts, so the first screen update goes first. A picture is drawn again only when the plugin, the screen or PaperPi changes. + +Note: the Library pictures are drawn in 256 grays so they are easy to see on a phone, also for a black-and-white screen; a color screen keeps its colors. + +## Logging in + +- **The first person to open the web interface sets the password.** No step is needed during the install. +- **After logging in,** the browser stays logged in for 1 year, with a cookie (a small token the browser keeps). There is a **Log out** button. A new password logs out every browser. +- **The password is stored as a scrypt hash:** a scrambled form that can be checked but not turned back into the password. One check takes about 0.1 seconds on a Pi. +- **A forgotten password:** `sudo paperpi reset-password` removes it from the config file; after a reload, the next visitor sets a new one. The log-in page shows these steps. +- **Every password field has a Show button,** so what was typed can be checked before sending. +- **Without logging in,** anyone on the home network sees only the picture on the screen now (on the log-in page), never the other plugins or the settings. + +Note: `login = false` in `[web]` switches logging in off. Then anyone on the home network can change the settings; the config check gives a hint and the home page says so. + +## Safety + +- **For the home network only.** The web interface uses plain HTTP, not HTTPS (an encrypted connection), because HTTPS needs a certificate, and without a public one browsers show a warning page. Do not open it to the internet. +- **Forms are only accepted from the web interface's own pages,** and its pages can't be shown inside another website. Without this, a web page on the internet could make the visitor's browser send a form to PaperPi. +- **It only answers when opened by an IP address** (e.g. `http://192.168.1.20:8080`) or `localhost`. A web page could otherwise point a name of its own at the Pi and set the first password itself. +- **The pages load only PaperPi's own files,** plus pictures sent inside the page. A plugin can't add its own script to a page. +- **Secrets are never shown again** once saved, and never written to the log. + +## Numbers + +| What | Value | +| ---------------------------------------- | ----------------------------------------------- | +| port | 8080 (`[web] port`) | +| staying logged in | 1 year | +| longest real-data preview | 45 seconds (or the plugin's own time limit, if shorter) | +| longest sample-data preview or Library picture | 10 seconds | +| Library pictures wait after a start | 60 seconds | +| wait for **Apply changes** | 3 seconds, then the page says it is still busy | +| **Keep these settings** after a new address or port | 5 minutes | + +## Not built yet + +- **A dashboard editor:** a page to place and resize several plugins on one screen. +- **HTTPS:** a manual page on putting your own web server in front of PaperPi, for anyone who wants it. +- **Names such as `paperpi.local`** instead of the IP address. + +## Why it works this way + +- **Pages built on the Pi with htmx:** a JavaScript app (React, Vue) would need a build step with Node.js, a second programming language, and more memory and processor time on a Pi 3. FastAPI is built on pydantic, so the forms come straight from the settings descriptions. +- **Inside PaperPi, not a separate program:** one program to install, one process fewer, and a change reaches the scheduler directly. +- **No limit on wrong password tries:** it is a household device on a home network, and each check is slow on purpose. +- **The full story** is in the old design note [`web-interface.md`](https://github.com/txoof/PaperPi/blob/main/docs/history/decisions/web-interface.md). + +## In the code + +- `src/paperpi/web/app.py`: the web server, its pages and the safety rules. +- `src/paperpi/web/auth.py` and `src/paperpi/web/password.py`: logging in and the password hash. +- `src/paperpi/web/forms.py`: the plugin settings pages. +- `src/paperpi/web/preview.py` and `src/paperpi/web/pictures.py`: the previews and the Library pictures. +- `src/paperpi/web/config_file.py` and `src/paperpi/web/changes.py`: writing the config file, and the Review changes page. +- `src/paperpi/web/keep.py`: **Keep these settings**. +- `src/paperpi/web/templates/`: the page templates. diff --git a/install/paperpi.service b/install/paperpi.service index 72ebc8c..1ac58a0 100644 --- a/install/paperpi.service +++ b/install/paperpi.service @@ -1,5 +1,5 @@ # PaperPi as a systemd service; install/paperpi-install puts it in /etc/systemd/system. -# Why each setting is here: docs/install-systemd.md and docs/history/decisions/errors-and-time-limits.md. +# Why each setting is here: docs/install-systemd.md and docs/specs/errors.md. [Unit] Description=PaperPi e-paper display Documentation=https://github.com/txoof/PaperPi diff --git a/mkdocs.yml b/mkdocs.yml index 8f8bd6d..2695777 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -98,3 +98,4 @@ nav: - Plugins and setting helpers: specs/plugins.md - Screens and drivers: specs/screens.md - Errors and time limits: specs/errors.md + - The web interface: specs/web.md diff --git a/src/paperpi/health.py b/src/paperpi/health.py index b80a82e..ea85e4f 100644 --- a/src/paperpi/health.py +++ b/src/paperpi/health.py @@ -1,6 +1,6 @@ """The health report: PaperPi says "still running" to systemd and to a small file. -The reasons are in ``docs/history/decisions/errors-and-time-limits.md``. In short: +The reasons are in ``docs/specs/errors.md``. In short: - The scheduler loop calls :meth:`Health.report` every :data:`~paperpi.limits.HEALTH_REPORT` seconds. When the loop gets stuck, the reports stop, and after diff --git a/src/paperpi/limits.py b/src/paperpi/limits.py index 49354df..98dc56d 100644 --- a/src/paperpi/limits.py +++ b/src/paperpi/limits.py @@ -1,6 +1,6 @@ """Default time and size limits, all in one place. -The numbers come from ``docs/history/decisions/errors-and-time-limits.md``. All durations are in +The numbers come from ``docs/specs/errors.md``. All durations are in seconds and are measured with the monotonic clock (``time.monotonic``), which does not jump when the system time is corrected. """ diff --git a/src/paperpi/logbook.py b/src/paperpi/logbook.py index 615526c..80b8e2b 100644 --- a/src/paperpi/logbook.py +++ b/src/paperpi/logbook.py @@ -1,7 +1,7 @@ """PaperPi's own log: the last lines in memory, and the important ones in a small file. -The reasons are in ``docs/history/decisions/web-interface.md`` ("Warnings page") and -``docs/history/decisions/errors-and-time-limits.md`` ("Logs"). In short: +The reasons are in ``docs/specs/web.md`` ("Warnings page") and +``docs/specs/errors.md`` ("Logs"). In short: - ``paperpi run`` calls :func:`start`. From then on, every log line of level INFO and up (DEBUG too with ``--verbose``) is kept in memory: the last :data:`~paperpi.limits.LOG_LINES`. diff --git a/src/paperpi/plugins/splash_screen/__init__.py b/src/paperpi/plugins/splash_screen/__init__.py index b905cab..15757aa 100644 --- a/src/paperpi/plugins/splash_screen/__init__.py +++ b/src/paperpi/plugins/splash_screen/__init__.py @@ -9,7 +9,7 @@ It shows the address of PaperPi's web interface by IP address, and a QR code with that address, so a phone can open the web interface at once. (Not by host name: the web -interface only accepts IP addresses, see docs/history/decisions/web-interface.md.) +interface only accepts IP addresses, see docs/specs/web.md.) """ import socket diff --git a/src/paperpi/screen.py b/src/paperpi/screen.py index c6dcb4d..b15d266 100644 --- a/src/paperpi/screen.py +++ b/src/paperpi/screen.py @@ -1,6 +1,6 @@ """The screen: writes run in a separate helper process, watched by a time limit. -See ``docs/history/decisions/display-driver-interface.md`` and ``errors-and-time-limits.md``. +See ``docs/specs/screens.md``. The screen driver (from epdlib) is made and used only in a small helper process. PaperPi sends it each image and waits for the answer, at most the time limit. A write that hangs is diff --git a/src/paperpi/storage.py b/src/paperpi/storage.py index c8d6910..0515eca 100644 --- a/src/paperpi/storage.py +++ b/src/paperpi/storage.py @@ -1,6 +1,6 @@ """Size and age limits for the plugins' storage folders, and the low-disk check. -See "Saved files" in ``docs/history/decisions/errors-and-time-limits.md``. Each plugin has its own +See "Saved files" in ``docs/specs/errors.md``. Each plugin has its own folder (``/plugins//``) with two limits, set per plugin (``storage_mb``, ``storage_days``) or suggested by the plugin: diff --git a/src/paperpi/web/__init__.py b/src/paperpi/web/__init__.py index c517af5..479f51f 100644 --- a/src/paperpi/web/__init__.py +++ b/src/paperpi/web/__init__.py @@ -3,5 +3,5 @@ It runs inside ``paperpi run``, in its own thread (:mod:`paperpi.web.server`). The pages are built on the Pi (:mod:`paperpi.web.app`). The password is scrambled, checked and saved by :mod:`paperpi.web.password`; the log-in cookie is in :mod:`paperpi.web.auth`. See -``docs/history/decisions/web-interface.md``. +``docs/specs/web.md``. """ diff --git a/src/paperpi/web/password.py b/src/paperpi/web/password.py index 5497803..4686190 100644 --- a/src/paperpi/web/password.py +++ b/src/paperpi/web/password.py @@ -5,7 +5,7 @@ - :func:`save_password_hash` writes the hash into ``[web]`` of the config file (``None`` removes it, for ``paperpi reset-password``), keeping the rest of the file as it is. -See ``docs/history/decisions/web-interface.md``. +See ``docs/specs/web.md``. """ from __future__ import annotations diff --git a/src/paperpi/web/pictures.py b/src/paperpi/web/pictures.py index 33a7c7b..419da41 100644 --- a/src/paperpi/web/pictures.py +++ b/src/paperpi/web/pictures.py @@ -1,7 +1,7 @@ """Example pictures for the Plugin Library: each layout of each plugin drawn with its sample data. -See ``docs/history/decisions/web-interface.md``. The pictures are drawn ahead, in the background, +See ``docs/specs/web.md``. The pictures are drawn ahead, in the background, at the screen's size (in 256 shades of gray, or in color for a color screen), and saved in ``library-pictures/`` in the state folder, so the Library page opens at once (agreed with txoof, M5 part 3b-2). diff --git a/src/paperpi/web/preview.py b/src/paperpi/web/preview.py index 9677eb5..5ddfaac 100644 --- a/src/paperpi/web/preview.py +++ b/src/paperpi/web/preview.py @@ -1,6 +1,6 @@ """Previews: draw a plugin with the settings in its form, before they are saved. -See ``docs/history/decisions/web-interface.md``. A preview runs one update in its own process with a +See ``docs/specs/web.md``. A preview runs one update in its own process with a time limit, as the scheduler does (:func:`paperpi.runner.run_update`), at the screen's size and for the screen's type. The real screen is not touched. Each preview gets a new, empty storage folder that is deleted afterwards, so a preview can't change the plugin's saved diff --git a/src/paperpi/webrequest.py b/src/paperpi/webrequest.py index cd2793d..bb3370d 100644 --- a/src/paperpi/webrequest.py +++ b/src/paperpi/webrequest.py @@ -1,6 +1,6 @@ """The one helper every plugin uses for web requests. -The limits come from ``docs/history/decisions/errors-and-time-limits.md`` and :mod:`paperpi.limits`: +The limits come from ``docs/specs/errors.md`` and :mod:`paperpi.limits`: - 10 seconds to connect, - 30 seconds in total, including one retry after 5 seconds,