feat!: descriptor graph-object binding as the default

.gitignore

@@ -190,7 +190,7 @@ cython_debug/
 #  that can be found at https://github.com/github/gitignore/blob/main/Global/VisualStudioCode.gitignore
 #  and can be added to the global gitignore or merged into this file. However, if you prefer,
 #  you could uncomment the following to ignore the entire vscode folder
-# .vscode/
+.vscode/
 
 # Ruff stuff:
 .ruff_cache/
@@ -223,7 +223,11 @@ benchmark_comparison.txt
 *.pwd.yaml
 */osw_files/*
 
+# Node (src/oold/ui/vue)
+node_modules/
+
 # Local
 CLAUDE.md
 AGENTS.md
+.claude/
 .ign

README.md

@@ -120,7 +120,24 @@ f = model.Foo(
 
 Thanks to the resolver mechanism, these IRIs turn into fully-fledged objects as soon as you need them.
 
-More details see [example code](./tests/test_oold.py)
+Writing a model by hand, declare links with `Link[T]` / `LinkList[T]`. This is the recommended notation: it is the only one a type checker reads correctly in both directions - the resolved object you get back, and the object, IRI or JSON object you may assign:
+
+```python
+from oold.model import Link, LinkedBaseModel, LinkList, OoldField
+
+class Person(LinkedBaseModel):
+    id: str
+    name: str | None = None
+    employer: Link["Organization | None"] = OoldField()
+    knows: LinkList["Person"] = OoldField()
+
+alice = Person(id="ex:alice", knows=["ex:bob", {"id": "ex:carol"}])
+alice.knows[0].name    # a Person, resolved on access
+```
+
+Existing declarations keep working unchanged, including the `range` form code generation emits.
+
+More details see [example code](./tests/test_oold.py) and [Object Graph Mapping](./docs/how-to/object-graph-mapping.md).
 
 ### RDF-Export
 

docs/design/downstream-migration.md

@@ -0,0 +1,291 @@
+# Downstream API surface and which patterns become obsolete
+
+Companion to [graph-object-binding.md](graph-object-binding.md), tracked in
+[oold-python#107].
+
+Downstream inherits its API from `oold.model.LinkedBaseModel` through
+`opensemantic.OswBaseModel`:
+
+```
+opensemantic.OswBaseModel  ->  oold.model.LinkedBaseModel  ->  pydantic.BaseModel
+opensemantic.v1.OswBaseModel -> oold.model.v1.LinkedBaseModel -> pydantic.v1.BaseModel
+```
+
+`to_json`, `to_jsonld`, `from_json` and `from_jsonld` are **inherited from
+`LinkedBaseModel`**, not defined by `OswBaseModel`, so replacing the binding
+changes them for every consumer.
+
+## Measured usage
+
+Scanned: the generated `opensemantic.*-python` packages plus several
+applications built on them (vendored `.venv`, `.tox` and `site-packages` copies
+excluded). Application code is referred to generically below; the counts are
+what matters for the compatibility decision.
+
+| member | sites | verdict |
+|---|---:|---|
+| `json_schema_extra` | 2499 | v2 declaration, already supported unchanged |
+| `OswBaseModel` | 246 | subclass of `LinkedBaseModel` |
+| `range=` | 187 | **v1 declaration style** (extras land in `field_info.extra`) |
+| `get_cls_iri` | 42 | unchanged |
+| `get_iri_ref` | 24 | **keep** (see pattern E) |
+| `LinkedBaseModel` | 16 | direct base-class references |
+| `__iris__` | 15 | **keep, read and write** (pattern C) |
+| `from pydantic import` / `.v1 import` | 25 / 14 | **both versions in active use** |
+| `to_json` / `from_json` | 9 / 9 | portable |
+| `to_jsonld` / `from_jsonld` | 4 / 1 | portable |
+| `cast` / `cast_none_to_default` | 3 / 2 | portable |
+| `get_raw` | 2 | obsolete (pattern A) |
+| `LinkedBaseModelList` | 0 | no downstream use |
+| `store_jsonld` | 0 | no downstream use |
+
+## Why these patterns exist
+
+Most of them are **workarounds for the shipped binding's hidden I/O**: plain
+attribute access may perform a synchronous, un-batchable backend call inside
+`__getattribute__`. Callers who cannot afford that, or cannot tell whether a
+value is resolved, route around the getter. Under the descriptor binding -
+where reads are batched, cached and return real objects - the reason for most
+of these disappears.
+
+### A. The resolved-or-IRI dance - **obsolete**
+
+Application helpers repeat a shape equivalent to:
+
+```python
+def _load_first_relation(field_name):
+    raw_value = (entity.get_raw(field_name)
+                 if callable(getattr(entity, "get_raw", None))
+                 else getattr(entity, field_name, None))
+    if raw_value:
+        return raw_value[0] if isinstance(raw_value, list) else raw_value
+    relation_ids = (entity.get_iri_ref(field_name)
+                    if callable(getattr(entity, "get_iri_ref", None))
+                    else None)
+    ...
+```
+
+It exists because the caller cannot ask "is this resolved?" without risking
+resolution, and must therefore handle both representations. Under the new
+binding the whole helper collapses to:
+
+```python
+values = entity.field          # real objects, batched and cached
+return values[0] if values else None
+```
+
+Retires `get_raw` (2 sites) entirely.
+
+### B. `try`/`except` around an attribute read - **obsolete**
+
+Seen inside a published `opensemantic.*` package:
+
+```python
+try:
+    char_iri = getattr(channel, "characteristic", None)
+except (ValueError, ImportError):
+    return None
+```
+
+Catching **`ImportError` from an attribute read** is the clearest symptom of the
+problem: the getter resolves, which constructs a type, which may import. With
+resolution moved out of the read path this guard has no reason to exist.
+
+### C. Writing `__iris__` to fabricate a link - **obsolete, but must keep working**
+
+Also inside a published `opensemantic.*` package:
+
+```python
+self.__iris__ = {"characteristic": characteristic_class.get_cls_iri()}
+```
+
+A stub object is given a link by writing the internal side-dict, because there
+was no clean way to set a link to an IRI. Under the new binding that is simply:
+
+```python
+self.characteristic = characteristic_class.get_cls_iri()   # coerced to a link
+```
+
+Because this is **written**, not just read, a read-only `__iris__` shim would
+silently drop the assignment. The compatibility layer therefore implements
+`__iris__` as a read/write property.
+
+### D. Hedging both representations - **simplifies**
+
+```python
+target = self.load_typed(obj.get_iri_ref("some_type") or obj.some_type, SomeType)
+```
+
+`... or ...` hedges: use the IRI if present, else whatever the attribute holds.
+With one predictable representation this becomes a single expression.
+
+### E. Comparing identity without fetching - **legitimate, keep**
+
+```python
+if module_iri in (subprocess.get_iri_ref("tool") or []):
+    ...
+```
+
+Here the caller genuinely wants the IRI, not the object: resolving every related
+entity only to compare identity would be wasteful even when batched. This is a
+real primitive, not a workaround, so **`get_iri_ref` stays** with its current
+name and return shape (`str | list[str] | None`).
+
+### F. Capability probing - **obsolete**
+
+```python
+entity.get_raw(f) if callable(getattr(entity, "get_raw", None)) else ...
+```
+
+Defensive checks for whether the API exists at all. A stable base class removes
+the need.
+
+## The metaclass identity must move with the base class
+
+Found by running a downstream test suite against a swapped binding rather than
+by reading code. `opensemantic.characteristics.quantitative._static` does:
+
+```python
+from oold.model import LinkedBaseModelMetaClass as ModelMetaclass   # aliased!
+
+class QuantityValueMetaclass(ModelMetaclass): ...
+class QuantityValue(OswBaseModel, metaclass=QuantityValueMetaclass): ...
+```
+
+Downstream **imports and subclasses oold's metaclass**, aliased to a name that
+makes it look like pydantic's. Replacing only `LinkedBaseModel` leaves
+`LinkedBaseModelMetaClass` pointing at the old class, so `QuantityValue`'s
+metaclass is no longer a subclass of its base's and the import dies with:
+
+```text
+TypeError: metaclass conflict: the metaclass of a derived class must be a
+(non-strict) subclass of the metaclasses of all its bases
+```
+
+The whole package tree fails to import - not a subtle behavioural drift but a
+hard failure at collection time. So `LinkedBaseModelMetaClass` is **part of the
+public API** and its identity has to be carried over together with the base
+class, either by keeping the name bound to the new metaclass or by having the
+new metaclass inherit from it.
+
+Verified: after also rebinding the metaclass, the same suite imports cleanly and
+passes.
+
+## The type registry must be the same object
+
+The same class of problem, found by enumerating every symbol downstream imports
+from `oold`:
+
+| imported from `oold.model` | sites |
+|---|---:|
+| `_types` | **7** |
+| `LinkedBaseModel` | 2 |
+| `LinkedBaseModelMetaClass` (aliased) | 1 |
+| `BaseController` | 1 |
+
+`_types` - the *private* registry - is imported more often than the base class
+itself, and it is **written to**:
+
+```python
+from oold.model import _types
+_types[SomeClass.get_cls_iri()] = SomeClass
+```
+
+both in shipped package code and in examples. A replacement that keeps its own
+registry dict does not see those entries, so polymorphic resolution silently
+falls back to the declared target. Unlike the metaclass conflict this fails
+**quietly**, which makes it the more dangerous of the two.
+
+Fix: share the object, do not copy it - `use_type_registry(oold.model._types)`.
+
+## How to solve both blockers
+
+The rule the two findings share: **downstream imports `oold.model` internals by
+name and mutates them, so the replacement must preserve names and object
+identity, not merely behaviour.** Concretely:
+
+1. **Name the new metaclass `LinkedBaseModelMetaClass`.** Downstream subclasses
+   whatever that name resolves to, so pointing it at the new metaclass makes
+   `class Derived(Base, metaclass=CustomMeta)` consistent by construction.
+   Inheriting the *old* metaclass from the new one does not work - the derived
+   metaclass must be a subclass of the base's, not the other way round.
+2. **Bind the new registry to the existing `_types` dict** rather than creating
+   one, so entries written through either name are visible to both.
+3. **Keep `LinkedBaseModel` and `BaseController` as the exported names** for the
+   new implementations.
+4. Everything under `oold.backend.*` is untouched by the swap - the remaining
+   downstream imports (`interface`, `document_store`, `auth`) need no action.
+
+Both fixes are verified: with the metaclass rebound the previously failing suite
+imports and passes, and with the registry shared a downstream registration
+resolves through the new binding.
+
+## Consequences for the replacement
+
+**Must be preserved** (compatibility layer, `oold/experimental/compat.py`):
+
+- `get_iri_ref(field)` -> `str | list[str] | None`, unchanged name and shape
+- `__iris__`, readable **and assignable**
+- `to_json`, `to_jsonld`, `from_json`, `from_jsonld`
+- `cast`, `cast_none_to_default`, `get_cls_iri`, `export_schema`, `full_dict`
+- both declaration styles: `json_schema_extra={"range": ...}` and bare `range=`
+- **`LinkedBaseModelMetaClass`** - downstream subclasses it (see above)
+- **`_types`** - the same dict object, downstream writes into it
+- **pydantic v1 and v2**
+
+**May be deprecated once downstream is updated**: `get_raw`, and the defensive
+idioms in patterns A, B, C, F. They can be kept as thin shims and removed on a
+later major version - none of them needs to survive in the new design on its
+own merits.
+
+**Not needed**: `LinkedBaseModelList` and `store_jsonld` have no downstream
+callers, so the rich list operations and the store helper carry no
+compatibility obligation (they remain available, but need not constrain the
+design).
+
+## Verification plan
+
+Parity is asserted only when these pass unchanged against the new base:
+
+1. the `oold-python` suite,
+2. the test suites of the applications that call `get_iri_ref` directly and
+   assert on its return shape,
+3. a regenerated `opensemantic.core` diffed against the released package.
+
+Status: step 2 has been run with the real switch (not a shim) against three application suites, each compared to a baseline taken
+on the same machine and the same backend state:
+
+| suite | baseline | with the switch |
+| --- | --- | --- |
+| live-backend controller suite calling `get_iri_ref` | 2 passed | 2 passed |
+| dashboard suite | 20 passed | 20 passed |
+| utilities suite | 194 passed, 9 failed | 194 passed, 9 failed (same set) |
+
+The utilities suite fails identically with and without the switch; those
+failures predate it.
+
+On the strength of that, the descriptor binding is now the **default**:
+`oold.model.LinkedBaseModel` is the descriptor model, and the legacy binding is
+reachable with `OOLD_DESCRIPTOR_BINDING=0`.
+
+### The shim was not sufficient verification
+
+Swapping the base class through a `sitecustomize` shim passed; the real switch
+failed at import on the first generated package. Four binding defects surfaced
+that fixtures had not provoked - a subclass redeclaring an inherited link field,
+a link field default that can only raise, `__setattr__(..., internal=True)`, and
+`to_json()` leaving `UUID` objects in the v1 path. All are fixed, with
+regression tests in `tests/test_downstream_shapes.py`.
+
+The lesson for the remaining migration steps: verify with the switch, against a
+generated package, on a live backend. Two of the four defects were invisible to
+every unit test.
+
+## The metaclass identity is a migration constraint
+
+Downstream subclasses `LinkedBaseModelMetaClass`, so the name has to keep
+resolving to whatever metaclass `LinkedBaseModel` uses. The query DSL built on
+that metaclass is carried over unchanged, typed subscription overloads included;
+see `graph-object-binding.md`.
+
+[oold-python#107]: https://github.com/OO-LD/oold-python/issues/107