Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 9 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,13 +35,12 @@ ollebo.com only ever sees URLs that point at your mapserver. The raster pixels s

1. Sign in at [ollebo.com](https://ollebo.com).
2. Create a mapserver entry in the space you want to expose maps to. ollebo.com generates a unique key.
3. Copy that key and set it as the `OLLEBO_KEY` environment variable on this server (the install scripts below prompt you for it).
4. Set `OLLEBO_SPACE_ID` to the space ID ollebo.com shows you. This becomes the first key in the terracotta `[space, map, variant]` schema, so it scopes every tile URL to your space.
3. Paste that key into this server's **Settings** page, or set it as `OLLEBO_KEY` (the install scripts below prompt you for it). The key decides which space your maps appear in.
4. Set the public tile URL. Maps that are already built are sent to ollebo.com when you save Settings, and new maps are sent when they finish.

Everyone in that space on ollebo.com can then see your maps the same way you do.
Everyone in that space on ollebo.com can then see your maps, as long as their browser can reach your public tile URL. If you delete a map on ollebo.com, it comes back the next time this server rescans or you save Settings.

> **Status — pairing isn't built yet.**
> ollebo.com issues the key, but nothing on ollebo.com accepts it yet, so the Settings page shows Disconnected even with a valid key. The mapserver does not call back to ollebo.com to register itself. Until then, maps are visible through the local UI at `http://localhost:8888` but won't appear on ollebo.com automatically.
> **Maps are private on ollebo.com, but the tile server has no login.** Only your space lists them, yet anyone who has a tile URL can fetch the tiles.

## External reachability

Expand Down Expand Up @@ -116,7 +115,7 @@ Key `values.yaml` knobs:
| Value | What it controls |
|---|---|
| `image.repository`, `image.tag` | Image to deploy. Defaults to the chart's `appVersion`. Set `latest` for the newest build of `main`, or `sha-<short>` for a specific commit. |
| `ollebo.key` | Stored in a Secret. Pairing with ollebo.com isn't built yet. |
| `ollebo.key` | Your map key from ollebo.com, stored in a Secret. |
| `ollebo.spaceId`, `ollebo.apiUrl` | Non-sensitive config, stored in a ConfigMap. |
| `terracotta.publicUrl` | Public URL for tile fetches — must match your Ingress host. |
| `ingress.enabled`, `ingress.host`, `ingress.className`, `ingress.tlsSecret` | Ingress wiring. |
Expand All @@ -132,14 +131,14 @@ After install, `helm get notes mapserver` prints the resolved tile URL and a `ku

## Configuration reference

These are the environment variables actually consumed by code (and one that's reserved for future use):
These are the environment variables consumed by code:

| Variable | Default | Notes |
|---|---|---|
| `OLLEBO_KEY` | _(empty)_ | **Reserved** — paired with an ollebo.com space. Not yet read by code. Set it now anyway. |
| `OLLEBO_SPACE_ID` | `local` | Tenant scope. First key in the terracotta `[space, map, variant]` schema. |
| `OLLEBO_KEY` | _(empty)_ | Your map key from ollebo.com. A key saved on the Settings page takes priority. |
| `OLLEBO_SPACE_ID` | `local` | First key in the terracotta `[space, map, variant]` schema. It doesn't need to match your ollebo.com space. |
| `TERRACOTTA_PUBLIC_URL` | `http://localhost:5001` | Browser-facing tile URL. Must be reachable from any viewer. |
| `API` | `https://www.ollebo.com/api/v1` | ollebo.com API base. Used by the dormant `updateMap()` registration call. |
| `API` | `https://www.ollebo.com/api` | ollebo.com API base. Where this server checks its key and registers maps. |
| `TERRACOTTA_DB_PATH` | `/data/db/terracotta.sqlite` | SQLite location inside the container. |
| `TERRACOTTA_PROVIDER` | `sqlite` | terracotta driver. Only `sqlite` is exercised today. |

Expand All @@ -165,7 +164,6 @@ Pull requests build the image for verification but don't push. The Docker instal
## Known rough edges

- **No auth on the tile server.** See "External reachability."
- **Pairing with ollebo.com isn't built.** See the pairing section above.
- **No login on the web UI.** It can upload files and show the map key, so keep it private.
- **Single-replica only on Kubernetes** because of SQLite + RWO PVCs.
- **Container build is fragile.** `Dockerfile_build` uses `--break-system-packages` on Ubuntu 24.04 with pinned `terracotta==0.10.1`. A move to a venv-based image is on the roadmap; for now, rebuild from the pinned base if pip breakage occurs.
Expand Down
21 changes: 1 addition & 20 deletions code/config.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
import os
import sqlite3

import requests


DB_PATH = os.environ.get("TERRACOTTA_DB_PATH", "/data/db/terracotta.sqlite")

Expand Down Expand Up @@ -64,21 +62,4 @@ def terracotta_public_url():


def api_url():
return os.environ.get("API", "https://www.ollebo.com/api/v1")


def check_connection(timeout=5):
key = ollebo_key()
if not key:
return False, "No map key configured"
try:
resp = requests.get(
api_url(),
headers={"Authorization": "Bearer " + key},
timeout=timeout,
)
if 200 <= resp.status_code < 400:
return True, "HTTP {0}".format(resp.status_code)
return False, "HTTP {0}".format(resp.status_code)
except requests.RequestException as e:
return False, str(e)
return os.environ.get("API", "https://www.ollebo.com/api")
80 changes: 72 additions & 8 deletions code/mapmaker/makingGeotiff.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,23 +2,18 @@

import rasterio
from rasterio.warp import transform_bounds
import requests

import config
import ollebo
from mapmaker.maps import terracotta_client
from mapmaker.maps.detect import detect_input_type
from mapmaker.maps.variants import VARIANTS


API_URL = os.environ.get("API", "https://www.ollebo.com/api/v1")
SOURCE_DIR = "/data/maps"
COG_DIR = "/data/cogs"


def updateMap(map):
return requests.post("{0}/maps/".format(API_URL), json=map)


def getGeoTiffData(tiff_file):
with rasterio.open(tiff_file) as src:
src_crs = src.crs
Expand Down Expand Up @@ -141,13 +136,82 @@ def makingMap(map):
elif len(successful) < len(variants_meta):
map["action"] = "partial"

# ollebo.com sync — left dormant; will be wired up in a follow-up.
# updateMap(map)
if successful:
ollebo.register_map(map_id, _registration(
map_id, map.get("tags"), map["tilesURL"], variants_meta, map["mapData"]
))

print(map)
return map


def _registration(map_id, tags, tiles_url, variants, map_data):
return {
"mapData": map_data,
"name": map_id,
"tags": tags or "",
"tilesURL": tiles_url,
"variants": variants,
}


def _registered_variants(space_id, map_id):
"""The VARIANTS names this map has datasets for. RGB is stored as rgb__r/g/b."""
driver = terracotta_client._driver()
with driver.connect():
if not driver.key_names:
return set()
datasets = driver.get_datasets(where={"space": space_id, "map": map_id})
names = set()
for key_tuple in datasets.keys():
keys = dict(zip(driver.key_names, key_tuple))
names.add(keys["variant"].split("__")[0])
return names


def sync_with_ollebo():
"""Tell ollebo.com the public URL and every map already built here.

For maps built before a key was set, and for updating tile URLs after the
public URL changes. Returns (registered, failed).
"""
registered, failed = 0, 0
if not ollebo.save_public_url():
return registered, failed

space_id = os.environ.get("OLLEBO_SPACE_ID", "local")
map_ids = sorted(_registered_map_ids(space_id))
for map_id in map_ids:
try:
source_path = os.path.join(SOURCE_DIR, map_id + ".tif")
input_type, _ = detect_input_type(source_path)
built = _registered_variants(space_id, map_id)
variants = [
{
"variant": spec.name,
"tilesURL": _tile_url(space_id, map_id, spec),
"colormap": spec.colormap,
"singleband": spec.singleband,
"status": "ok",
}
for spec in VARIANTS[input_type]
if spec.name in built
]
sent = bool(variants) and ollebo.register_map(map_id, _registration(
map_id, "", variants[0]["tilesURL"], variants, getGeoTiffData(source_path)
))
except Exception as e:
print("sync: {0} failed: {1}".format(map_id, e))
sent = False
if sent:
registered += 1
else:
failed += 1

print("sync: registered={0} failed={1}".format(registered, failed))
return registered, failed


def _registered_map_ids(space_id):
try:
driver = terracotta_client._driver()
Expand Down
71 changes: 71 additions & 0 deletions code/ollebo.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
"""Client for dw's /api/selfhosted API, authenticated by the mapserver key.

A failure here must never stop the server working locally, so callers get a
result or a False back, and nothing raises.
"""
import requests

import config


TIMEOUT = 10


def _request(method, path, **kwargs):
return requests.request(
method,
"{0}/selfhosted{1}".format(config.api_url().rstrip("/"), path),
headers={"Authorization": "Bearer " + config.ollebo_key()},
timeout=TIMEOUT,
**kwargs
)


def check_connection():
"""Return (connected, detail), where detail is for showing to the user."""
if not config.ollebo_key():
return False, "No map key configured"
try:
resp = _request("GET", "/self")
except requests.RequestException as e:
return False, "Can't reach ollebo.com: {0}".format(e.__class__.__name__)
if resp.status_code == 401:
return False, "Map key not accepted"
if resp.status_code != 200:
return False, "HTTP {0}".format(resp.status_code)
try:
body = resp.json()
return True, "{0} in {1}".format(body["mapserver"]["name"], body["space"]["slug"])
except (ValueError, KeyError, TypeError):
return False, "Unexpected response from ollebo.com"


def _send(method, path, what, **kwargs):
if not config.ollebo_key():
return False
try:
resp = _request(method, path, **kwargs)
except requests.RequestException as e:
print("ollebo: {0} failed: {1}".format(what, e.__class__.__name__))
return False
if not resp.ok:
print("ollebo: {0} failed: HTTP {1}".format(what, resp.status_code))
return resp.ok


def save_public_url():
return _send(
"PATCH",
"/self",
"saving public URL",
json={"url": config.terracotta_public_url()},
)


def register_map(map_id, payload):
return _send(
"PUT",
"/maps/{0}".format(requests.utils.quote(map_id, safe="")),
"registering {0}".format(map_id),
json=payload,
)
11 changes: 8 additions & 3 deletions code/start.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,9 @@
from werkzeug.utils import secure_filename

import config
import ollebo
from listMaps import listFiles, variants_for_map
from mapmaker.makingGeotiff import makingMap, scan_and_ingest
from mapmaker.makingGeotiff import makingMap, scan_and_ingest, sync_with_ollebo


UPLOAD_FOLDER = "/data/maps"
Expand Down Expand Up @@ -53,6 +54,7 @@ def map():
@app.route("/rescan", methods=["POST"])
def rescan():
scan_and_ingest()
sync_with_ollebo()
return redirect("/")


Expand Down Expand Up @@ -82,16 +84,19 @@ def settings():
"terracotta_public_url",
request.form.get("terracotta_public_url", "").strip(),
)
return redirect("/settings")
registered, failed = sync_with_ollebo()
return redirect("/settings?registered={0}&failed={1}".format(registered, failed))

connected, status_detail = config.check_connection()
connected, status_detail = ollebo.check_connection()
return render_template(
"settings.html",
title="Settings",
ollebo_key=config.ollebo_key(),
terracotta_public_url=config.terracotta_public_url(),
connected=connected,
status_detail=status_detail,
registered=request.args.get("registered", type=int),
failed=request.args.get("failed", type=int),
)


Expand Down
8 changes: 7 additions & 1 deletion code/templates/settings.html
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ <h1 class="text-2xl font-semibold mb-4">Settings</h1>
{% if connected %}
<div class="mb-6 inline-flex items-center px-3 py-1.5 rounded-full bg-green-100 text-green-800 text-sm font-medium">
<span class="w-2 h-2 mr-2 rounded-full bg-green-500"></span>
Connected to ollebo.com
Connected to ollebo.com{% if status_detail %} — {{ status_detail }}{% endif %}
</div>
{% else %}
<div class="mb-6 inline-flex items-center px-3 py-1.5 rounded-full bg-red-100 text-red-800 text-sm font-medium">
Expand All @@ -15,6 +15,12 @@ <h1 class="text-2xl font-semibold mb-4">Settings</h1>
</div>
{% endif %}

{% if registered or failed %}
<p class="mb-6 text-sm text-gray-700">
Sent {{ registered or 0 }} map(s) to ollebo.com{% if failed %}. {{ failed }} failed; see the server log{% endif %}.
</p>
{% endif %}

<form method="post" action="/settings" class="space-y-6">
<div>
<label for="ollebo_key" class="block text-sm font-medium text-gray-900 mb-1">Map key</label>
Expand Down
2 changes: 1 addition & 1 deletion docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ services:
ports:
- 8888:8080
environment:
- API=${API:-https://www.ollebo.com/api/v1}
- API=${API:-https://www.ollebo.com/api}
- OLLEBO_KEY=${OLLEBO_KEY:-}
- OLLEBO_SPACE_ID=${OLLEBO_SPACE_ID:-local}
- TERRACOTTA_DB_PATH=${TERRACOTTA_DB_PATH:-/data/db/terracotta.sqlite}
Expand Down
2 changes: 1 addition & 1 deletion install/docker-install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ if [[ "$skip_prompts" -eq 0 ]]; then
prompt TERRACOTTA_PUBLIC_URL \
"TERRACOTTA_PUBLIC_URL (browser-facing tile URL; use your ngrok/cloudflared https URL if exposing externally)" \
"http://localhost:5001"
prompt API "API (ollebo.com API base)" "https://www.ollebo.com/api/v1"
prompt API "API (ollebo.com API base)" "https://www.ollebo.com/api"

cat > "$env_file" <<EOF
# Written by install/docker-install.sh
Expand Down
5 changes: 2 additions & 3 deletions install/helm/mapserver/templates/NOTES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@ terracotta.publicUrl to a publicly resolvable URL.

{{- if not .Values.ollebo.key }}

NOTE: ollebo.key is empty. The mapserver runs fine without it. Pairing with
ollebo.com isn't built yet, so setting a key does not link the two, and the
Settings page will show Disconnected either way.
NOTE: ollebo.key is empty. The mapserver runs fine, but your maps won't appear
on ollebo.com until you add a key with --set ollebo.key or on the Settings page.
{{- end }}
6 changes: 3 additions & 3 deletions install/helm/mapserver/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@ image:
# Pairing with an ollebo.com space. `key` is stored in a Secret.
# `spaceId` and `apiUrl` live in a ConfigMap.
ollebo:
# Reserved for the future ollebo.com registration handshake. Safe to set now;
# not yet consumed by code.
# The key from the mapserver you created at ollebo.com. It decides which Space
# your maps appear in.
key: ""
spaceId: local
apiUrl: https://www.ollebo.com/api/v1
apiUrl: https://www.ollebo.com/api

terracotta:
# Browser-facing tile URL. Must be reachable from any viewer (including
Expand Down