From 70b726dfce70d2860ec84df04606f1ae63d26f2c Mon Sep 17 00:00:00 2001 From: Mattias Hemmingsson Date: Sat, 7 Nov 2026 10:21:45 +0100 Subject: [PATCH] Connect to ollebo.com with the map key and register maps The Settings check called /api/v1 with the key, which ollebo.com never accepted, so it always showed Disconnected. It now calls dw's new /api/selfhosted API (Ollebo/dw branch mapserver-link) and shows which mapserver and Space the key belongs to. Finished maps are sent to that Space. Saving Settings or rescanning also sends the public URL and every map already built, which covers maps made before a key was set and updates tile URLs after the public URL changes. ollebo.com being down or the key being wrong never stops a map from being built locally. The API default drops the /v1 that dw never served. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01AwkBcZCmo27zHnnRnAcKiy --- README.md | 20 +++--- code/config.py | 21 +----- code/mapmaker/makingGeotiff.py | 80 +++++++++++++++++++--- code/ollebo.py | 71 +++++++++++++++++++ code/start.py | 11 ++- code/templates/settings.html | 8 ++- docker-compose.yaml | 2 +- install/docker-install.sh | 2 +- install/helm/mapserver/templates/NOTES.txt | 5 +- install/helm/mapserver/values.yaml | 6 +- 10 files changed, 175 insertions(+), 51 deletions(-) create mode 100644 code/ollebo.py diff --git a/README.md b/README.md index 93506ba..cf690bd 100644 --- a/README.md +++ b/README.md @@ -35,13 +35,12 @@ ollebo.com only ever sees URLs that point at your mapserver. The raster pixels s 1. Sign in at [ollebo.com](https://ollebo.com). 2. Create a mapserver entry in the space you want to expose maps to. ollebo.com generates a unique key. -3. Copy that key and set it as the `OLLEBO_KEY` environment variable on this server (the install scripts below prompt you for it). -4. Set `OLLEBO_SPACE_ID` to the space ID ollebo.com shows you. This becomes the first key in the terracotta `[space, map, variant]` schema, so it scopes every tile URL to your space. +3. Paste that key into this server's **Settings** page, or set it as `OLLEBO_KEY` (the install scripts below prompt you for it). The key decides which space your maps appear in. +4. Set the public tile URL. Maps that are already built are sent to ollebo.com when you save Settings, and new maps are sent when they finish. -Everyone in that space on ollebo.com can then see your maps the same way you do. +Everyone in that space on ollebo.com can then see your maps, as long as their browser can reach your public tile URL. If you delete a map on ollebo.com, it comes back the next time this server rescans or you save Settings. -> **Status — pairing isn't built yet.** -> ollebo.com issues the key, but nothing on ollebo.com accepts it yet, so the Settings page shows Disconnected even with a valid key. The mapserver does not call back to ollebo.com to register itself. Until then, maps are visible through the local UI at `http://localhost:8888` but won't appear on ollebo.com automatically. +> **Maps are private on ollebo.com, but the tile server has no login.** Only your space lists them, yet anyone who has a tile URL can fetch the tiles. ## External reachability @@ -116,7 +115,7 @@ Key `values.yaml` knobs: | Value | What it controls | |---|---| | `image.repository`, `image.tag` | Image to deploy. Defaults to the chart's `appVersion`. Set `latest` for the newest build of `main`, or `sha-` for a specific commit. | -| `ollebo.key` | Stored in a Secret. Pairing with ollebo.com isn't built yet. | +| `ollebo.key` | Your map key from ollebo.com, stored in a Secret. | | `ollebo.spaceId`, `ollebo.apiUrl` | Non-sensitive config, stored in a ConfigMap. | | `terracotta.publicUrl` | Public URL for tile fetches — must match your Ingress host. | | `ingress.enabled`, `ingress.host`, `ingress.className`, `ingress.tlsSecret` | Ingress wiring. | @@ -132,14 +131,14 @@ After install, `helm get notes mapserver` prints the resolved tile URL and a `ku ## Configuration reference -These are the environment variables actually consumed by code (and one that's reserved for future use): +These are the environment variables consumed by code: | Variable | Default | Notes | |---|---|---| -| `OLLEBO_KEY` | _(empty)_ | **Reserved** — paired with an ollebo.com space. Not yet read by code. Set it now anyway. | -| `OLLEBO_SPACE_ID` | `local` | Tenant scope. First key in the terracotta `[space, map, variant]` schema. | +| `OLLEBO_KEY` | _(empty)_ | Your map key from ollebo.com. A key saved on the Settings page takes priority. | +| `OLLEBO_SPACE_ID` | `local` | First key in the terracotta `[space, map, variant]` schema. It doesn't need to match your ollebo.com space. | | `TERRACOTTA_PUBLIC_URL` | `http://localhost:5001` | Browser-facing tile URL. Must be reachable from any viewer. | -| `API` | `https://www.ollebo.com/api/v1` | ollebo.com API base. Used by the dormant `updateMap()` registration call. | +| `API` | `https://www.ollebo.com/api` | ollebo.com API base. Where this server checks its key and registers maps. | | `TERRACOTTA_DB_PATH` | `/data/db/terracotta.sqlite` | SQLite location inside the container. | | `TERRACOTTA_PROVIDER` | `sqlite` | terracotta driver. Only `sqlite` is exercised today. | @@ -165,7 +164,6 @@ Pull requests build the image for verification but don't push. The Docker instal ## Known rough edges - **No auth on the tile server.** See "External reachability." -- **Pairing with ollebo.com isn't built.** See the pairing section above. - **No login on the web UI.** It can upload files and show the map key, so keep it private. - **Single-replica only on Kubernetes** because of SQLite + RWO PVCs. - **Container build is fragile.** `Dockerfile_build` uses `--break-system-packages` on Ubuntu 24.04 with pinned `terracotta==0.10.1`. A move to a venv-based image is on the roadmap; for now, rebuild from the pinned base if pip breakage occurs. diff --git a/code/config.py b/code/config.py index c27f67f..5d76367 100644 --- a/code/config.py +++ b/code/config.py @@ -1,8 +1,6 @@ import os import sqlite3 -import requests - DB_PATH = os.environ.get("TERRACOTTA_DB_PATH", "/data/db/terracotta.sqlite") @@ -64,21 +62,4 @@ def terracotta_public_url(): def api_url(): - return os.environ.get("API", "https://www.ollebo.com/api/v1") - - -def check_connection(timeout=5): - key = ollebo_key() - if not key: - return False, "No map key configured" - try: - resp = requests.get( - api_url(), - headers={"Authorization": "Bearer " + key}, - timeout=timeout, - ) - if 200 <= resp.status_code < 400: - return True, "HTTP {0}".format(resp.status_code) - return False, "HTTP {0}".format(resp.status_code) - except requests.RequestException as e: - return False, str(e) + return os.environ.get("API", "https://www.ollebo.com/api") diff --git a/code/mapmaker/makingGeotiff.py b/code/mapmaker/makingGeotiff.py index f623dad..1a71588 100644 --- a/code/mapmaker/makingGeotiff.py +++ b/code/mapmaker/makingGeotiff.py @@ -2,23 +2,18 @@ import rasterio from rasterio.warp import transform_bounds -import requests import config +import ollebo from mapmaker.maps import terracotta_client from mapmaker.maps.detect import detect_input_type from mapmaker.maps.variants import VARIANTS -API_URL = os.environ.get("API", "https://www.ollebo.com/api/v1") SOURCE_DIR = "/data/maps" COG_DIR = "/data/cogs" -def updateMap(map): - return requests.post("{0}/maps/".format(API_URL), json=map) - - def getGeoTiffData(tiff_file): with rasterio.open(tiff_file) as src: src_crs = src.crs @@ -141,13 +136,82 @@ def makingMap(map): elif len(successful) < len(variants_meta): map["action"] = "partial" - # ollebo.com sync — left dormant; will be wired up in a follow-up. - # updateMap(map) + if successful: + ollebo.register_map(map_id, _registration( + map_id, map.get("tags"), map["tilesURL"], variants_meta, map["mapData"] + )) print(map) return map +def _registration(map_id, tags, tiles_url, variants, map_data): + return { + "mapData": map_data, + "name": map_id, + "tags": tags or "", + "tilesURL": tiles_url, + "variants": variants, + } + + +def _registered_variants(space_id, map_id): + """The VARIANTS names this map has datasets for. RGB is stored as rgb__r/g/b.""" + driver = terracotta_client._driver() + with driver.connect(): + if not driver.key_names: + return set() + datasets = driver.get_datasets(where={"space": space_id, "map": map_id}) + names = set() + for key_tuple in datasets.keys(): + keys = dict(zip(driver.key_names, key_tuple)) + names.add(keys["variant"].split("__")[0]) + return names + + +def sync_with_ollebo(): + """Tell ollebo.com the public URL and every map already built here. + + For maps built before a key was set, and for updating tile URLs after the + public URL changes. Returns (registered, failed). + """ + registered, failed = 0, 0 + if not ollebo.save_public_url(): + return registered, failed + + space_id = os.environ.get("OLLEBO_SPACE_ID", "local") + map_ids = sorted(_registered_map_ids(space_id)) + for map_id in map_ids: + try: + source_path = os.path.join(SOURCE_DIR, map_id + ".tif") + input_type, _ = detect_input_type(source_path) + built = _registered_variants(space_id, map_id) + variants = [ + { + "variant": spec.name, + "tilesURL": _tile_url(space_id, map_id, spec), + "colormap": spec.colormap, + "singleband": spec.singleband, + "status": "ok", + } + for spec in VARIANTS[input_type] + if spec.name in built + ] + sent = bool(variants) and ollebo.register_map(map_id, _registration( + map_id, "", variants[0]["tilesURL"], variants, getGeoTiffData(source_path) + )) + except Exception as e: + print("sync: {0} failed: {1}".format(map_id, e)) + sent = False + if sent: + registered += 1 + else: + failed += 1 + + print("sync: registered={0} failed={1}".format(registered, failed)) + return registered, failed + + def _registered_map_ids(space_id): try: driver = terracotta_client._driver() diff --git a/code/ollebo.py b/code/ollebo.py new file mode 100644 index 0000000..1b2a69c --- /dev/null +++ b/code/ollebo.py @@ -0,0 +1,71 @@ +"""Client for dw's /api/selfhosted API, authenticated by the mapserver key. + +A failure here must never stop the server working locally, so callers get a +result or a False back, and nothing raises. +""" +import requests + +import config + + +TIMEOUT = 10 + + +def _request(method, path, **kwargs): + return requests.request( + method, + "{0}/selfhosted{1}".format(config.api_url().rstrip("/"), path), + headers={"Authorization": "Bearer " + config.ollebo_key()}, + timeout=TIMEOUT, + **kwargs + ) + + +def check_connection(): + """Return (connected, detail), where detail is for showing to the user.""" + if not config.ollebo_key(): + return False, "No map key configured" + try: + resp = _request("GET", "/self") + except requests.RequestException as e: + return False, "Can't reach ollebo.com: {0}".format(e.__class__.__name__) + if resp.status_code == 401: + return False, "Map key not accepted" + if resp.status_code != 200: + return False, "HTTP {0}".format(resp.status_code) + try: + body = resp.json() + return True, "{0} in {1}".format(body["mapserver"]["name"], body["space"]["slug"]) + except (ValueError, KeyError, TypeError): + return False, "Unexpected response from ollebo.com" + + +def _send(method, path, what, **kwargs): + if not config.ollebo_key(): + return False + try: + resp = _request(method, path, **kwargs) + except requests.RequestException as e: + print("ollebo: {0} failed: {1}".format(what, e.__class__.__name__)) + return False + if not resp.ok: + print("ollebo: {0} failed: HTTP {1}".format(what, resp.status_code)) + return resp.ok + + +def save_public_url(): + return _send( + "PATCH", + "/self", + "saving public URL", + json={"url": config.terracotta_public_url()}, + ) + + +def register_map(map_id, payload): + return _send( + "PUT", + "/maps/{0}".format(requests.utils.quote(map_id, safe="")), + "registering {0}".format(map_id), + json=payload, + ) diff --git a/code/start.py b/code/start.py index a08736d..23818c6 100755 --- a/code/start.py +++ b/code/start.py @@ -8,8 +8,9 @@ from werkzeug.utils import secure_filename import config +import ollebo from listMaps import listFiles, variants_for_map -from mapmaker.makingGeotiff import makingMap, scan_and_ingest +from mapmaker.makingGeotiff import makingMap, scan_and_ingest, sync_with_ollebo UPLOAD_FOLDER = "/data/maps" @@ -53,6 +54,7 @@ def map(): @app.route("/rescan", methods=["POST"]) def rescan(): scan_and_ingest() + sync_with_ollebo() return redirect("/") @@ -82,9 +84,10 @@ def settings(): "terracotta_public_url", request.form.get("terracotta_public_url", "").strip(), ) - return redirect("/settings") + registered, failed = sync_with_ollebo() + return redirect("/settings?registered={0}&failed={1}".format(registered, failed)) - connected, status_detail = config.check_connection() + connected, status_detail = ollebo.check_connection() return render_template( "settings.html", title="Settings", @@ -92,6 +95,8 @@ def settings(): terracotta_public_url=config.terracotta_public_url(), connected=connected, status_detail=status_detail, + registered=request.args.get("registered", type=int), + failed=request.args.get("failed", type=int), ) diff --git a/code/templates/settings.html b/code/templates/settings.html index b93978d..87b3289 100644 --- a/code/templates/settings.html +++ b/code/templates/settings.html @@ -6,7 +6,7 @@

Settings

{% if connected %}
- Connected to ollebo.com + Connected to ollebo.com{% if status_detail %} — {{ status_detail }}{% endif %}
{% else %}
@@ -15,6 +15,12 @@

Settings

{% endif %} + {% if registered or failed %} +

+ Sent {{ registered or 0 }} map(s) to ollebo.com{% if failed %}. {{ failed }} failed; see the server log{% endif %}. +

+ {% endif %} +
diff --git a/docker-compose.yaml b/docker-compose.yaml index 1720087..58aab8e 100755 --- a/docker-compose.yaml +++ b/docker-compose.yaml @@ -11,7 +11,7 @@ services: ports: - 8888:8080 environment: - - API=${API:-https://www.ollebo.com/api/v1} + - API=${API:-https://www.ollebo.com/api} - OLLEBO_KEY=${OLLEBO_KEY:-} - OLLEBO_SPACE_ID=${OLLEBO_SPACE_ID:-local} - TERRACOTTA_DB_PATH=${TERRACOTTA_DB_PATH:-/data/db/terracotta.sqlite} diff --git a/install/docker-install.sh b/install/docker-install.sh index 009a1be..0c5ceb4 100755 --- a/install/docker-install.sh +++ b/install/docker-install.sh @@ -65,7 +65,7 @@ if [[ "$skip_prompts" -eq 0 ]]; then prompt TERRACOTTA_PUBLIC_URL \ "TERRACOTTA_PUBLIC_URL (browser-facing tile URL; use your ngrok/cloudflared https URL if exposing externally)" \ "http://localhost:5001" - prompt API "API (ollebo.com API base)" "https://www.ollebo.com/api/v1" + prompt API "API (ollebo.com API base)" "https://www.ollebo.com/api" cat > "$env_file" <