Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -171,11 +171,11 @@ Discovery rows never carry a value, so they can be forwarded as they stand. `Mas
and a maintainer can replay it through the same loader, adapter and fidelity harness the vendored captures use. It can carry values because it is safe by construction: `SchemaOneAdapter.export_masked_tree` runs the tree through
`span_panel_api_schema_1.tree_export`, which masks by rule, so a consumer forwards the result whole and decides nothing about what is sensitive.

The masks match the site properties (`utility-meter-serial-number`, `wifi-ssid`, `postal-code`, `address-lines`, `locality`, `region`, `latitude`, `longitude`) and `serial-number` by property id on any node, and the panel's `info/name` and its
`$description` `name`; `name` on any other node or device is kept. Device ids become placeholders that start `masked-`, and a final sweep replaces any device id, and any serial that reads as an identifier (six characters or more with a digit; a placeholder
such as `0` or `N/A` is masked at its own property only), still present in any wire string. It decodes JSON string literals to do so, to a bound of four nested levels, and in the plain text between them a JSON-style escape written out (`\n`, `\u00b0`) is a
boundary before an identifier, as a backslash alone is. A `$description` is read as parsed to a looser bound of thirty-two containers, after which a subtree is masked whole only if something in it would be swept. The module and `_Sweep` docstrings in
`tree_export.py` hold the rules; change them there.
The masks match the site properties (`utility-meter-serial-number`, `wifi-ssid`, `postal-code`, `address-lines`, `locality`, `region`, `latitude`, `longitude`, `time-zone` as `UTC`, `country-code` as `ZZ`) and `serial-number` by property id on any node,
and the panel's `info/name` and its `$description` `name`; `name` on any other node or device is kept. Device ids become placeholders that start `masked-`, and a final sweep replaces any device id, and any serial that reads as an identifier (six characters
or more with a digit; a placeholder such as `0` or `N/A` is masked at its own property only), still present in any wire string. It decodes JSON string literals to do so, to a bound of four nested levels, and in the plain text between them a JSON-style
escape written out (`\n`, `\u00b0`) is a boundary before an identifier, as a backslash alone is. A `$description` is read as parsed to a looser bound of thirty-two containers, after which a subtree is masked whole only if something in it would be swept.
The module and `_Sweep` docstrings in `tree_export.py` hold the rules; change them there.

It is optional, like the discovery rows: `TreeExportingAdapter` is a separate `runtime_checkable` protocol, not a `SchemaAdapter` member, so no `ADAPTER_CONTRACT` bump and no adapter distribution is rejected. `SpanMqttClient.masked_tree()` returns `None`
for an adapter that does not offer it, which includes the flat adapter, and before the root device has described itself.
Expand Down
5 changes: 4 additions & 1 deletion packages/schema-1/src/span_panel_api_schema_1/tree_export.py
Original file line number Diff line number Diff line change
Expand Up @@ -93,13 +93,16 @@
"region": "Masked Region",
"latitude": "0.0",
"longitude": "0.0",
"time-zone": "UTC",
"country-code": "ZZ",
}
)
"""Site values replaced whole, by property id on whichever node publishes them.

The panel declares them under ``info`` and ``status``, but nothing in the
specification ties a site property to a node, which is why ``SITE_PROPERTIES``
(``const.py``) is read the same way. ``country-code`` and ``time-zone`` are kept.
(``const.py``) is read the same way. ``time-zone`` and ``country-code`` are masked
to ``UTC`` and ``ZZ``, valid values a consumer that parses the export still reads.
The serial number is matched by property id too; the site's name is not, since
``name`` is every device's label and only the panel's ``info/name`` is masked.
"""
Expand Down
22 changes: 21 additions & 1 deletion tests/reference_payloads/reidentify.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,8 @@
"latitude": "47.6",
"longitude": "-122.3",
"utility-meter-serial-number": "SEEDED-METER-77",
"time-zone": "Etc/GMT+9",
"country-code": "QZ",
}
"""The planted value of each masked site property, by property id on whichever node publishes it."""
_SEEDED_SITE_NAME: Final = "Seeded Residence"
Expand Down Expand Up @@ -105,7 +107,8 @@ def reidentify(tree: RetainedTopicTree, root_id: str, seed: int) -> Reidentified
becomes a distinct vendor-style serial, except a serial that is a device's
id, as the panel's is: it becomes that device's new id, so the new root id
is also the panel serial embedded in its children's ids. Every masked site
property takes its value in `_SEEDED_SITE`, and the panel's `info/name` and
property takes its value in `_SEEDED_SITE`, and one a device declares but leaves
unvalued or valued `""` is planted too (a capture may only declare it), and the panel's `info/name` and
`$description` `name` the site's name. A description `name` that is a
device's id (as the captures' are) follows the id.

Expand Down Expand Up @@ -169,6 +172,10 @@ def fresh(make: Callable[[], str]) -> str:
planted.add(values[topic])
if _DESCRIPTION in topics:
description = _description_object(topics[_DESCRIPTION])
for topic in _declared_site_topics(description):
if not values.get(topic):
values[topic] = _SEEDED_SITE[topic.partition("/")[2]]
planted.add(values[topic])
name = description.get("name")
if device_id == new_root and isinstance(name, str) and name:
description["name"] = _SEEDED_SITE_NAME
Expand All @@ -180,6 +187,19 @@ def fresh(make: Callable[[], str]) -> str:
return Reidentified(seeded, new_root, frozenset(planted))


def _declared_site_topics(description: Mapping[str, object]) -> list[str]:
"""The `<node>/<property>` topics a description declares for a masked site property."""
declared = description.get("nodes")
if not isinstance(declared, dict):
return []
found: list[str] = []
for node_id, node in declared.items():
node_properties = node.get("properties") if isinstance(node, dict) else None
if isinstance(node_properties, dict):
found.extend(f"{node_id}/{property_id}" for property_id in node_properties if property_id in _SEEDED_SITE)
return found


def _description_object(text: str) -> dict[str, object]:
description: object = json.loads(text)
if not isinstance(description, dict):
Expand Down
8 changes: 3 additions & 5 deletions tests/test_schema_one_tree_export.py
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,8 @@ def test_ids_tied_in_natural_order_are_named_the_same_in_any_input_order() -> No
"info/region": "Masked Region",
"info/latitude": "0.0",
"info/longitude": "0.0",
"status/time-zone": "UTC",
"info/country-code": "ZZ",
}
_FEEDS = "connection/feeds-device-id"
_FED_BY = "connection/fed-by-device-id"
Expand All @@ -274,7 +276,6 @@ def test_ids_tied_in_natural_order_are_named_the_same_in_any_input_order() -> No
"info/country-code": ("QZ", "string"),
"status/time-zone": ("Etc/GMT+9", "string"),
}
_KEPT_SITE = ("info/country-code", "status/time-zone")
_SEEDED_LABEL = "Seeded Lugs Label"


Expand Down Expand Up @@ -745,9 +746,6 @@ def test_kept_values_are_published(case: Case) -> None:
topics, labelled = _seeded(case)
export, ids = _exported(topics, case.root_id)
panel_values = _valued(export["devices"]["masked-panel"])
assert {topic: panel_values[topic] for topic in _KEPT_SITE} == {
topic: topics[case.root_id][topic] for topic in _KEPT_SITE
}
label = export["devices"][ids[labelled]]
assert _valued(label)[_NAME] == _SEEDED_LABEL
assert label["description"]["name"] == _label(_description_of(topics[labelled])["name"], ids)
Expand Down Expand Up @@ -873,7 +871,7 @@ def test_site_and_serial_values_are_masked_on_any_node(node: str) -> None:
_declare(description, topic, _SEEDED_BY_PROPERTY[_property_id(topic)][1])
topics[_REFERENCE_ROOT][topic] = value
topics[_REFERENCE_ROOT]["$description"] = json.dumps(description)
kept = {f"{node}/name": "Seeded Site Label", f"{node}/country-code": "QZ", f"{node}/time-zone": "Etc/GMT+9"}
kept = {f"{node}/name": "Seeded Site Label"}
topics[_REFERENCE_ROOT].update(kept)
topics[circuit][f"{node}/serial-number"] = "SEEDSN00002"
export, ids = _exported(topics, _REFERENCE_ROOT)
Expand Down
Loading