diff --git a/CHANGELOG.md b/CHANGELOG.md index 631f7c3f..a8fb319e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,7 @@ it releases that version ([RELEASING.md](https://github.com/energy-models/mathsp ## Upcoming version +- feat(language): a dimension's members are read by name ([#8](https://github.com/fluxopt/mathspec/pull/8)) - feat(language)!: a dimension lists its members, strictly ([#9](https://github.com/fluxopt/mathspec/pull/9)) - Breaks a file: `relations:` is gone. A set of combinations of labels is a dimension with `over:`, read through its members with `at()` and tested by its bare name in a `where:`. A dimension twice in a relation is an alias member. - feat(language): a dimension lists its members ([#1017](https://github.com/energy-models/mathspec/pull/1017)) diff --git a/docs/about/limits.md b/docs/about/limits.md index f01d4490..9f09db7e 100644 --- a/docs/about/limits.md +++ b/docs/about/limits.md @@ -116,10 +116,10 @@ same thing. | A vocabulary for tracked metrics: `impacts:`, `effects:`, a `costs` dimension | A tracked metric is already a [named expression](../reference/language/named.md#expressions). | An `impact` dimension and one named expression. A constraint can cap the expression, the objective can weight it, and a tool reads it back after the solver runs. | | Helpers for one domain, such as `reduce_carrier_dim` | A macro already states a helper. | A [component library](../examples/library/index.md) of macros. Its macros use only the operators that exist. | | A Python application programming interface (API) for building specs | The file is already the API, and the file is what you review and diff. | YAML, or a `dict` with the same keys. A tool can combine dicts before it calls `to_spec`. | -| Access to a member of a dimension that lists its members, by a dot or by aligning `flow <= capacity` | [Each member is already a mapping](../reference/language/dimensions.md#over) from each tuple to its label. | A lookup or a sum through that mapping: `at(capacity, by=connection[unit])`. | +| Access to a member of a dimension that lists its members, by a dot, as in `connection.unit` | [Each member is already a mapping](../reference/language/dimensions.md#over) from each tuple to its label. | A lookup or a sum through that mapping: `at(capacity, by=connection[unit])`. | | A `relations:` section for a set of combinations of labels | A [dimension with members](../reference/language/dimensions.md#over) already lists the combinations, and its mappings already give each one its labels. | `dimensions: { connection: { over: [generator, bus] } }`, tested by its bare name in a `where:` on an entry over every member. | -| Two dimensions in one `dims:` that share a member, or a dimension beside its own member, read as their cross product | A tool may store each dimension as the columns of its members, so the shared member would index the entry twice. | One of the two in `dims:`, and a `where:` that reads a relation through the shared member, or an alias of the member: `site: { over: [from_unit, bus] }`. | -| An order along a dimension that lists its members, for `shift`, `sum_back`, `position`, `piecewise:` or `sos:` of `type: 2` | A tuple of labels has no order, and each member can already declare one. | An entry over the members, with `where:` the bare name of the dimension. The construct then steps along an ordered member. | +| Two dimensions in one `dims:` that share a member, read as their cross product | A tool may store each dimension as the columns of its members, so the shared member would index the entry twice. | The entry over the members, at the rows both list: mathspec [reads `dims: [connection, active]` so](../reference/language/dimensions.md#members-by-name). An alias of the member keeps the two apart. | +| An order along a dimension that lists its members, for `shift`, `sum_back`, `position`, `piecewise:` or `sos:` of `type: 2` | A tuple of labels has no order, and each member can already declare one. | A step along an ordered member, which [reads the entry over the members](../reference/language/dimensions.md#members-by-name). | | A `join` operator apart from `sum` and `at`, such as `sum(join(p, generator[bus]), over=generator)` | [`sum(by=)` and `at(by=)` already state a join](relations-as-linear-maps.md#why-the-language-has-no-join-operator). | A sum or a lookup with `by=`, as in the first row. | ### Data preparation diff --git a/docs/examples/connection.md b/docs/examples/connection.md index 4742dd61..313d7136 100644 --- a/docs/examples/connection.md +++ b/docs/examples/connection.md @@ -18,7 +18,8 @@ the bus, so two ports of one unit may attach to the same bus. [`where:`](../reference/language/parameters-variables-constraints.md#where) keeps only the output ports, and `missing: 1` applies only there. At an input port, `injection` takes its case, and elsewhere `efficiency` times the flow of -the unit, read with `at`. The sum in `balance` adds each connection to the bus +the unit, read at each connection +[by member name](../reference/language/dimensions.md#members-by-name). The sum in `balance` adds each connection to the bus it attaches to. @@ -64,7 +65,7 @@ expressions: dims: [connection, snapshot] cases: draws: { when: "connection[role] == 'input'", expression: "-at(flow, by=connection[unit])" } - otherwise: efficiency * at(flow, by=connection[unit]) + otherwise: efficiency * flow constraints: balance: diff --git a/docs/examples/sparse.md b/docs/examples/sparse.md index be4c2ec3..1457c6ec 100644 --- a/docs/examples/sparse.md +++ b/docs/examples/sparse.md @@ -19,10 +19,12 @@ the sets in one way: `connection` is a dimension whose labels are the pairs of a unit and a port. `efficiency` and `flow` are over it, so `missing: 1` fills only the ports it -lists, and a data row for a port it does not list is refused. `active` is a -dimension over `unit` and `period`. `output` is over both members, and -`where: active` keeps the pairs that `active` lists, so `ramp_up` can step -along the ordered member `period`. +lists, and a data row for a port it does not list is refused. `port_limit` +reads `capacity` at the unit of each connection, and `split` sums over `port`, +both [by member name](../reference/language/dimensions.md#members-by-name). +`active` is a dimension over `unit` and `period`, and `output` is over it. +`ramp_up` steps `output` along the ordered member `period`, so mathspec reads +`output` over `unit` and `period`, confined to the pairs that `active` lists. [#1019](https://github.com/energy-models/mathspec/issues/1019) compares how three designs for sparse sets write this spec. @@ -51,18 +53,18 @@ parameters: load: { dims: [bus, period], description: demand to be met } variables: - output: { dims: [unit, period], where: active, bounds: { lower: 0 }, description: what a unit produces } + output: { dims: [active], bounds: { lower: 0 }, description: what a unit produces } flow: { dims: [connection, period], bounds: { lower: 0 }, description: what a port takes of it } constraints: port_limit: description: a port takes no more than the unit's capacity dims: [connection, period] - expression: flow <= at(capacity, by=connection[unit]) + expression: flow <= capacity split: description: the ports of a unit take what it produces dims: [unit, period] - expression: sum(flow, over=connection, by=connection[unit]) == output + expression: sum(flow, over=port) == output ramp_up: description: output rises by at most the ramp from one active period to the next dims: [unit, period] diff --git a/docs/examples/times/capacity.md b/docs/examples/times/capacity.md index 1c84d2ef..a97a615b 100644 --- a/docs/examples/times/capacity.md +++ b/docs/examples/times/capacity.md @@ -18,9 +18,10 @@ without storage, with the upper bound only. The GAMS conditions map to the spec as follows: - `RTP_VINTYR` and `PRC_TS` are dimensions with - [members](../../reference/language/dimensions.md#over). `act` is over every - member of both, and `where: "prc_ts AND rtp_vintyr"` keeps the coordinates - that both list. + [members](../../reference/language/dimensions.md#over). `act` is over both, + which share `region` and `process`, so mathspec + [reads it over their members](../../reference/language/dimensions.md#members-by-name), + at the coordinates that both list. - `TS_MAP(R,S,TS)` is the dimension `ts_map`. Its upper timeslice is the alias `upper_timeslice`, so the two timeslices are two members. `eq_capact_up` reads the activity at each tuple of `ts_map`, sums it to each @@ -69,8 +70,7 @@ variables: cap: { dims: [region, period, process], bounds: { lower: 0 }, description: installed capacity (VAR_CAP) } act: description: activity of a vintage of a process in a timeslice (VAR_ACT) - dims: [region, vintage, period, process, timeslice] - where: "prc_ts AND rtp_vintyr" + dims: [prc_ts, rtp_vintyr] bounds: { lower: 0 } constraints: diff --git a/docs/examples/times/trade.md b/docs/examples/times/trade.md index 992c4d72..07d28397 100644 --- a/docs/examples/times/trade.md +++ b/docs/examples/times/trade.md @@ -19,8 +19,10 @@ as TIMES' `TOP_IRE` does, each a tuple of its five members, and `ire_flo`, exist is refused. The data of `ire_flo` has the five columns that the TIMES data already has, and no label is invented. -`balance` adds the imports of each region and commodity through the members -`region` and `commodity` of each link. It sums the exports to each +`balance` adds the imports of each region and commodity: it sums over the +other three members of each link, which +[groups by the two it leaves](../../reference/language/dimensions.md#members-by-name). +It sums the exports to each `from_region` and `from_commodity`, and reads those back onto `region` and `commodity` through the mapping of each alias. @@ -66,9 +68,9 @@ constraints: dims: [commodity, region] expression: >- supply - + sum(ire_flo * export, over=top_ire, by=top_ire[commodity, region]) - - sum(sum(sum(export, over=top_ire, by=top_ire[from_commodity, from_region]), - over=from_region, by=from_region[region]), over=from_commodity, by=from_commodity[commodity]) + + sum(ire_flo * export, over=[from_commodity, from_region, process]) + - sum(sum(sum(export, over=[commodity, process, region]), over=from_region, by=from_region[region]), + over=from_commodity, by=from_commodity[commodity]) >= demand objective: diff --git a/docs/reference/language/dimensions.md b/docs/reference/language/dimensions.md index 58712207..e23a4f05 100644 --- a/docs/reference/language/dimensions.md +++ b/docs/reference/language/dimensions.md @@ -242,54 +242,50 @@ mathspec refuses a declared mapping with the name of a member. With mappings: 'connection': mapping 'unit' has the name of a member of 'connection', which already gives each tuple its label of 'unit'. Read the member as connection[unit], or rename the mapping. ``` -### One dimension in every operator - -A dimension with members is one dimension wherever a `dims:` list or an -operator names it. mathspec does not align an entry over a member with an -entry over the dimension. With `flow <= capacity` in `limit` above, mathspec -prints: - -```text -Constraint 'limit': an operand over ['unit'] meets one over 'connection'. 'unit' is a member of 'connection', which is one dimension here. Read the operand at each tuple: at(capacity, by=connection[unit]), or sum flow to the member: sum(flow, over=connection, by=connection[unit]). -``` - -mathspec suggests the lookup through the mapping of the member, or the sum -of `flow` to the member. The same refusal applies wherever the two operands -meet: in a product, in a sum, and in the objective, as in -`sum(flow * capacity)`, in an [assumption](assumptions.md), and in the -`where:` of a `piecewise:` entry. - -### Each member once in `dims:` - -A tool may store a dimension with members as the columns of its members. So a -`dims:` list reaches each member once. mathspec refuses two dimensions in one -`dims:` that share a member, and a dimension beside one of its own members. -With `site: { over: [unit, bus] }` and `dims: [connection, site, snapshot]`, -mathspec prints: - -```text -Variable 'flow': 'connection' and 'site' in dims: share the member 'unit', so the entry would be indexed by 'unit' twice. Keep 'connection' in dims:, list ['bus'] beside it, and confine the entry with where: "at(site, by=connection[unit])". Or declare an alias of 'unit', such as from_unit: {alias: unit}, and list it in 'site' in place of 'unit', so the members differ. -``` - -With `dims: [connection, unit, snapshot]`, mathspec prints: - -```text -Variable 'flow': 'connection' and 'unit' in dims: share the member 'unit', so the entry would be indexed by 'unit' twice. Delete 'unit' from dims: and read it as connection[unit], or declare an alias of 'unit', such as from_unit: {alias: unit}, and list it in place of 'unit'. -``` - -The refusal applies to every `dims:` list: a parameter, a variable, a -constraint, a named expression, a given entry and a `piecewise:` entry. A -different unit at each connection is an alias of `unit`, which is a member of -its own. - -A sum over a member is refused the same way. With `sum(flow, over=unit)`, -mathspec prints: - -```text -Constraint 'balance': sum(over=unit) but the expression has only the dims ['connection', 'snapshot']. 'unit' is a member of 'connection', which is one dimension here. Write sum(..., over=connection, by=connection[port]). -``` - -mathspec suggests the sum over the dimension, grouped by the other member. +### Members by name + +A member is a dimension of its own, so an entry may name it where it means the +dimension that lists it. mathspec reads the name as the call through the +dimension that it stands for, and builds the same program. With `flow` over +`[connection, snapshot]`, `capacity` over `[unit]` and `active` over +`[unit, period]`: + +| Written | Read as | +| ------------------------------------------------------------------------- | ---------------------------------------------------------------------- | +| `flow <= capacity` | `flow <= at(capacity, by=connection[unit])` | +| `sum(flow, over=port)` | `sum(flow, over=connection, by=connection[unit])` | +| `sum(flow, over=[port, unit])` | `sum(flow, over=connection)` | +| `dims: [active, connection, snapshot]` | `dims: [period, port, snapshot, unit]`, `where: active AND connection` | +| `output: { dims: [active] }`, and `shift(output, along=period, offset=1)` | `output: { dims: [period, unit], where: active }` | + +The first three rows read one operand through the dimension: + +- An operand over a member meets an operand over the dimension in a product, + a sum, a comparison, the objective, a condition or an + [assumption](assumptions.md), and mathspec reads it at each tuple. +- A `sum` over members of a dimension that its operand is over sums over the + dimension, grouped by the members that `over=` leaves. + +The last two rows read the entry over the members: + +- Two dimensions in one `dims:` that share a member, or a dimension beside one + of its own members, make the entry over the members, each once. The entry is + confined to the tuples that every dimension lists, by a `where:` that names + them. +- A `shift`, `sum_back` or `position` along a member makes the entry it steps + in, and the entry it steps, over the members, with `where:` the dimension. + The dimension has no order, and the member can have one. + +An entry over the members reads an operand over the dimension at the members, +as `sum(flow, over=connection, by=connection[port, unit])` does. An entry over +the dimension reads an entry over the members at each tuple, as +`at(output, by=active[period, unit])` does. + +A parameter over two dimensions that share a member differs in one way from +its spelling with `where:`. A tool refuses a data row outside the tuples that +both list, where it drops a row outside a `where:`. The program holds those +tuples as the +[`domain`](../program.md#mathspec.program.Parameter.domain) of the parameter. A `dims:` list is the domain of an entry, so a tool refuses a data row at a tuple that the dimension does not list. A @@ -297,14 +293,14 @@ tuple that the dimension does not list. A At load, mathspec also refuses each of these, and prints the rewrite: -| Refused | What you write instead | -| ---------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ | -| a member that has members of its own | the members themselves, in one list | -| a dimension twice, as in `over: [region, region]` | an [alias](#alias) of the dimension as one of the members | -| `ordered: true`, `dtype:` or `one_of:` beside `over:` | the key on a member | -| `shift`, `sum_back`, `position`, `piecewise:` or `sos:` of `type: 2` along the dimension | an entry over the members, with `where:` the bare name of the dimension | -| a comparison of the dimension with a label, as in `connection == 'a'` | a comparison of a member, as in `connection[unit] == 'u1'` | -| two dimensions in one `dims:` that share a member, or a dimension beside its own member | [one of the two, and a `where:`, or an alias](#each-member-once-in-dims) | +| Refused | What you write instead | +| ---------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- | +| a member that has members of its own | the members themselves, in one list | +| a dimension twice, as in `over: [region, region]` | an [alias](#alias) of the dimension as one of the members | +| `ordered: true`, `dtype:` or `one_of:` beside `over:` | the key on a member | +| `shift`, `sum_back`, `position`, `piecewise:` or `sos:` of `type: 2` along the dimension | a step along a member, which [reads the entry over the members](#members-by-name) | +| a comparison of the dimension with a label, as in `connection == 'a'` | a comparison of a member, as in `connection[unit] == 'u1'` | +| two dimensions in the `dims:` of a `piecewise:` entry that share a member | the members themselves in `dims:`, and a `where:` that names the dimensions | ### The bare name in a condition diff --git a/examples/connection.yaml b/examples/connection.yaml index db4b7ff1..e1c8f8d7 100644 --- a/examples/connection.yaml +++ b/examples/connection.yaml @@ -43,7 +43,7 @@ expressions: dims: [connection, snapshot] cases: draws: { when: "connection[role] == 'input'", expression: "-at(flow, by=connection[unit])" } - otherwise: efficiency * at(flow, by=connection[unit]) + otherwise: efficiency * flow constraints: balance: diff --git a/examples/sparse.yaml b/examples/sparse.yaml index 152d5774..1d3d6c6a 100644 --- a/examples/sparse.yaml +++ b/examples/sparse.yaml @@ -25,18 +25,18 @@ parameters: load: { dims: [bus, period], description: demand to be met } variables: - output: { dims: [unit, period], where: active, bounds: { lower: 0 }, description: what a unit produces } + output: { dims: [active], bounds: { lower: 0 }, description: what a unit produces } flow: { dims: [connection, period], bounds: { lower: 0 }, description: what a port takes of it } constraints: port_limit: description: a port takes no more than the unit's capacity dims: [connection, period] - expression: flow <= at(capacity, by=connection[unit]) + expression: flow <= capacity split: description: the ports of a unit take what it produces dims: [unit, period] - expression: sum(flow, over=connection, by=connection[unit]) == output + expression: sum(flow, over=port) == output ramp_up: description: output rises by at most the ramp from one active period to the next dims: [unit, period] diff --git a/examples/times/capacity.yaml b/examples/times/capacity.yaml index f98e6941..2a71aa91 100644 --- a/examples/times/capacity.yaml +++ b/examples/times/capacity.yaml @@ -37,8 +37,7 @@ variables: cap: { dims: [region, period, process], bounds: { lower: 0 }, description: installed capacity (VAR_CAP) } act: description: activity of a vintage of a process in a timeslice (VAR_ACT) - dims: [region, vintage, period, process, timeslice] - where: "prc_ts AND rtp_vintyr" + dims: [prc_ts, rtp_vintyr] bounds: { lower: 0 } constraints: diff --git a/examples/times/trade.yaml b/examples/times/trade.yaml index c2196875..e9d87d97 100644 --- a/examples/times/trade.yaml +++ b/examples/times/trade.yaml @@ -40,9 +40,9 @@ constraints: dims: [commodity, region] expression: >- supply - + sum(ire_flo * export, over=top_ire, by=top_ire[commodity, region]) - - sum(sum(sum(export, over=top_ire, by=top_ire[from_commodity, from_region]), - over=from_region, by=from_region[region]), over=from_commodity, by=from_commodity[commodity]) + + sum(ire_flo * export, over=[from_commodity, from_region, process]) + - sum(sum(sum(export, over=[commodity, process, region]), over=from_region, by=from_region[region]), + over=from_commodity, by=from_commodity[commodity]) >= demand objective: diff --git a/src/mathspec/_condition_resolver.py b/src/mathspec/_condition_resolver.py index e39ca2b9..37aa449a 100644 --- a/src/mathspec/_condition_resolver.py +++ b/src/mathspec/_condition_resolver.py @@ -413,9 +413,10 @@ def _expression_comparison(self, node: UnresolvedComparisonNode) -> ExpressionCo self.errors.extend(zero_divisor_refusals(side, context, ns.spec.parameters, {})) if len(self.errors) > found: return node - left, right = sides - if self._expressions.met(left, right): + if (pair := self._expressions.aligned(*sides)) is None: return node + left, right = pair + dims = set(sorted_dims(left.dims, right.dims)) if all(_is_number(side) for side in sides): self.errors.append( f"{context}: '{node.left} {node.op} {node.right}' compares two numbers. Name the parameter one " diff --git a/src/mathspec/_expression_resolver.py b/src/mathspec/_expression_resolver.py index 42b63843..5764b9df 100644 --- a/src/mathspec/_expression_resolver.py +++ b/src/mathspec/_expression_resolver.py @@ -13,7 +13,7 @@ from __future__ import annotations from dataclasses import dataclass, replace -from typing import TYPE_CHECKING, assert_never, cast +from typing import TYPE_CHECKING, Literal, assert_never, cast from mathspec._dimensions import check_meeting, check_node, joined, operand_over, sorted_dims from mathspec._expression_parser import ( @@ -158,8 +158,9 @@ def arith(self, node: ArithmeticNode) -> Expression | None: def _binary(self, node: BinaryOperatorNode) -> Expression | None: """A subtraction is an addition of the negation, so a program has one additive node.""" left, right = self.arith(node.left), self.arith(node.right) - if left is None or right is None or self.met(left, right): + if left is None or right is None or (pair := self.aligned(left, right)) is None: return None + left, right = pair dims = sorted_dims(left.dims, right.dims) match node.op: case '+': @@ -175,6 +176,77 @@ def _binary(self, node: BinaryOperatorNode) -> Expression | None: case _: assert_never(node.op) + def aligned(self, left: Expression, right: Expression) -> tuple[Expression, Expression] | None: + """*left* and *right*, an operand over members of a dimension the other carries read at each of its tuples. + + A member operand meets the dimension by name: ``flow <= capacity`` reads + ``capacity`` at the unit of each connection, as + ``at(capacity, by=connection[unit])`` does, and builds the same node. + Where the entry is over the members and not the dimension, the operand + over the dimension is read at the members instead, as + [`framed`][mathspec._expression_resolver.ExpressionResolver.framed] reads it. + + Returns: + The two operands, or ``None`` with the refusal appended where a + read fails or the two still meet a member against its dimension. + """ + while (read := self._read_at_tuples(left, right)) is not None: + if read is False: + return None + left, right = read + return None if self.met(left, right) else (left, right) + + def _read_at_tuples( + self, left: Expression, right: Expression + ) -> tuple[Expression, Expression] | Literal[False] | None: + """The pair with one operand read through the first dimension the other carries members of, or ``None`` where none is.""" + spec = self.ns.spec + for one, other, flipped in ((left, right, False), (right, left, True)): + for name in sorted(set(one.dims) - set(other.dims)): + members = {dim for _, dim in spec.dimensions[name].members} + if not (stray := tuple(sorted((members & set(other.dims)) - set(one.dims)))): + continue + if name not in self.frame and members <= self.frame: + read = self._sum(one, (name,), ColumnsNode(name, tuple(sorted(members)))) + if read is None: + return False + return (other, read) if flipped else (read, other) + join = self.lookup(ColumnsNode(name, stray), frozenset(other.dims)) + if join is None or (read := self._joined(other, join)) is None: + return False + return (read, one) if flipped else (one, read) + return None + + def framed(self, node: Expression) -> Expression | None: + """*node* read at the entry's own dimensions, where it meets one of them by its members. + + Two readings meet the frame by name. A *node* over members of a + dimension the frame lists is read at each of its tuples, as + ``at(node, by=dimension[members])`` reads it. A *node* over a dimension + whose members the frame lists is read at those members, as + ``sum(node, over=dimension, by=dimension[members])`` reads it. The frame + lists each tuple once, so each group holds one term. + + Returns: + The node, or ``None`` with the refusal appended where a read fails. + """ + spec, frame = self.ns.spec, self.frame + for name in sorted(frame - set(node.dims)): + members = {dim for _, dim in spec.dimensions[name].members} + if stray := tuple(sorted((members & set(node.dims)) - frame)): + join = self.lookup(ColumnsNode(name, stray), frozenset(node.dims)) + if join is None or (read := self._joined(node, join)) is None: + return None + node = read + for name in sorted(set(node.dims) - frame): + members = {dim for _, dim in spec.dimensions[name].members} + if members and members <= frame: + summed = self._sum(node, (name,), ColumnsNode(name, tuple(sorted(members)))) + if summed is None: + return None + node = summed + return node + def met(self, left: Expression, right: Expression) -> bool: """Whether *left* and *right* meet a member against the dimension that lists it, the refusal appended.""" try: @@ -293,6 +365,8 @@ def _sum(self, operand: Expression, over: tuple[str, ...] | None, columns: Colum call, so no axis is ever left open. A name in *over* that is no column of the relation is a dim, summed away after the group-by. """ + if columns is None and over is not None and (meant := self._over_members(operand, over)) is not None: + over, columns = meant if columns is None: if over is None: return self._bare_sum(operand) @@ -307,6 +381,27 @@ def _sum(self, operand: Expression, over: tuple[str, ...] | None, columns: Colum grouped = self._summed(through, join.axes) return self._summed(grouped, tuple(Axis(d) for d in plain)) if plain and grouped is not None else grouped + def _over_members( + self, operand: Expression, over: tuple[str, ...] + ) -> tuple[tuple[str, ...], ColumnsNode | None] | None: + """``over=`` naming members of a dimension *operand* carries, as the sum through that dimension it means. + + ``sum(flow, over=port)`` on ``flow`` over ``connection`` sums over + ``connection`` grouped by its other members, as + ``sum(flow, over=connection, by=connection[unit])`` does. ``None`` + where *over* names no such member, or members of more than one + dimension, which the plain sum then refuses. + """ + if not (stray := {d for d in over if d not in operand.dims}): + return None + for name in sorted(operand.dims): + members = {dim for _, dim in self.ns.spec.dimensions[name].members} + if members and stray <= members: + rest = tuple(sorted(members - set(over))) + summed = tuple(sorted({*(d for d in over if d not in members), name})) + return summed, ColumnsNode(name, rest) if rest else None + return None + def _summed(self, operand: Expression, over: tuple[Axis, ...]) -> Sum | None: """A [`Sum`][mathspec.program.Sum] over *over*, which removes the dims its plain axes name; a join's axes are no dims.""" closed = {axis.dimension for axis in over if axis.column is None} diff --git a/src/mathspec/_flat.py b/src/mathspec/_flat.py new file mode 100644 index 00000000..cdd15dfa --- /dev/null +++ b/src/mathspec/_flat.py @@ -0,0 +1,167 @@ +# SPDX-FileCopyrightText: mathspec Contributors +# +# SPDX-License-Identifier: MIT + +"""The spec an entry over dimensions with members lowers as, where it needs their members by name. + +Two cases read an entry over its members rather than over its tuples: two of +its dimensions share a member, as in ``dims: [connection, active]``, or a +step runs along a member, as ``shift(output, along=period)`` does on +``output`` over ``active``. Each such entry lowers as the same entry over the +members, with ``where:`` the dimensions it listed, which is the spelling a +dimension without an order and without a shared member must write. Every +other entry keeps its dimensions, and meets these by name. +""" + +from __future__ import annotations + +import re +from typing import TYPE_CHECKING, cast + +from mathspec._expression_parser import FunctionCallNode, NameNode, nodes, parse_expression +from mathspec.errors import LanguageError +from mathspec.program import And, Mask, RelationDefined + +if TYPE_CHECKING: + from collections.abc import Iterable, Iterator, Mapping + + from mathspec.program import Condition + from mathspec.spec import ConstraintSpec, ExpressionSpec, ParameterSpec, Spec, VariableSpec + + type Framed = ParameterSpec | VariableSpec | ConstraintSpec | ExpressionSpec + +_SECTIONS = ('parameters', 'variables', 'constraints', 'expressions') +_STEP = re.compile(r'\balong\s*=\s*([A-Za-z_]\w*)|\bposition\(\s*([A-Za-z_]\w*)') + +type Entry = tuple[str, str] + + +def flat_spec(spec: Spec) -> tuple[Spec, dict[str, Mask]]: + """*spec* with each entry that needs its members by name declared over them, and the domain of each such parameter. + + Returns *spec* itself, and no domain, where no entry needs it. A + parameter's domain is the tuples its ``dims:`` listed: a tool refuses its + data row outside them, where the ``where:`` of the flat spelling drops it. + """ + from mathspec.spec import Spec + + tuples = {name: frozenset(dim for _, dim in d.members) for name, d in spec.dimensions.items() if d.members} + entries: dict[Entry, Framed] = { + (section, name): entry + for section in _SECTIONS + for name, entry in cast('Mapping[str, Framed]', getattr(spec, section)).items() + if set(entry.dims or ()) & set(tuples) + } + flat = _flattened(entries, tuples, spec) + if not flat: + return spec, {} + written = spec.to_dict() + for (section, name), over in flat.items(): + declared = cast('dict[str, dict[str, object]]', written[section]) + declared[name] = _over_members(section, declared[name], entries[section, name], over, tuples) + domains = { + name: _domain(entries[section, name].dims or (), tuples) for section, name in flat if section == 'parameters' + } + return Spec.model_validate(written), domains + + +def _flattened( + entries: Mapping[Entry, Framed], tuples: Mapping[str, frozenset[str]], spec: Spec +) -> dict[Entry, frozenset[str]]: + """Each entry to read over its members, with the dimensions it lists that it reads so. + + An entry whose dimensions share a member, an entry that steps along a + member of one of its dimensions, and an entry that a step reads so. + """ + stepped = _stepped(spec, tuples, entries) + flat = { + key + for key, entry in entries.items() + if _shares(entry.dims or (), tuples) or _steps(entry, tuples) or key[1] in stepped + } + return {key: frozenset(d for d in entries[key].dims or () if d in tuples) for key in flat} + + +def _shares(dims: Iterable[str], tuples: Mapping[str, frozenset[str]]) -> bool: + """Whether two of *dims* reach one member: two dimensions with members, or one beside a member of its own.""" + reached = [member for d in dims for member in tuples.get(d, {d})] + return len(reached) != len(set(reached)) + + +def _members(entry: Framed, tuples: Mapping[str, frozenset[str]]) -> set[str]: + """The members of the dimensions *entry* is over, less those it is over as dimensions of their own.""" + dims = set(entry.dims or ()) + return {m for d in dims if d in tuples for m in tuples[d]} - dims + + +def _steps(entry: Framed, tuples: Mapping[str, frozenset[str]]) -> bool: + """Whether the entry steps along a member of one of its dimensions, which a tuple has no order to step along.""" + members = _members(entry, tuples) + return any(set(found) & members for found in _STEP.findall(' '.join(_texts(entry)))) + + +def _stepped(spec: Spec, tuples: Mapping[str, frozenset[str]], entries: Mapping[Entry, Framed]) -> set[str]: + """The entries a ``shift`` or ``sum_back`` steps along a member of one of their dimensions, anywhere in the file.""" + members_of = {name: _members(entry, tuples) for (_, name), entry in entries.items()} + texts = [text for entry in _every_entry(spec) for text in _texts(entry)] + if spec.objective is not None: + texts.append(spec.objective.expression) + stepped: set[str] = set() + for text in texts: + try: + tree = parse_expression(text) + except LanguageError: + continue + for call in nodes(tree): + if not (isinstance(call, FunctionCallNode) and call.name in ('shift', 'sum_back')): + continue + if not isinstance(along := call.kwargs.get('along'), NameNode): + continue + read = {n.name for n in nodes(*call.args) if isinstance(n, NameNode)} + stepped |= {name for name in read & set(members_of) if along.name in members_of[name]} + return stepped + + +def _every_entry(spec: Spec) -> Iterator[Framed]: + """Every entry of the four sections that declare ``dims:``.""" + for section in _SECTIONS: + yield from cast('Mapping[str, Framed]', getattr(spec, section)).values() + + +def _texts(entry: Framed) -> Iterator[str]: + """Every expression and condition *entry* writes.""" + for key in ('where', 'expression', 'otherwise'): + if isinstance(text := getattr(entry, key, None), str): + yield text + if (bounds := getattr(entry, 'bounds', None)) is not None: + yield from (side for side in (bounds.lower, bounds.upper) if isinstance(side, str)) + for case in getattr(entry, 'cases', {}).values(): + yield case.when + if isinstance(case.expression, str): + yield case.expression + + +def _over_members( + section: str, written: dict[str, object], entry: Framed, over: frozenset[str], tuples: Mapping[str, frozenset[str]] +) -> dict[str, object]: + """The written *entry* over the members of the dimensions *over* names, confined to them by ``where:``. + + A named expression takes no ``where:``, so it is over the members alone, + and reads absent wherever the entries in its body are. + """ + flat = dict(written) + flat['dims'] = sorted({m for d in entry.dims or () for m in (tuples[d] if d in over else {d})}) + if section != 'expressions': + listed = ' AND '.join(sorted(over)) + own = getattr(entry, 'where', None) + flat['where'] = listed if own is None else f'({listed}) AND ({own})' + return flat + + +def _domain(dims: Iterable[str], tuples: Mapping[str, frozenset[str]]) -> Mask: + """The tuples a parameter's ``dims:`` listed, as the condition that holds at its domain.""" + listed: list[Condition] = [RelationDefined(d, tuple(sorted(tuples[d]))) for d in sorted(dims) if d in tuples] + root = listed[0] + for atom in listed[1:]: + root = And(root, atom) + return Mask(root) diff --git a/src/mathspec/_resolution.py b/src/mathspec/_resolution.py index dfdd576b..d643e18f 100644 --- a/src/mathspec/_resolution.py +++ b/src/mathspec/_resolution.py @@ -345,9 +345,9 @@ def resolve_expression( ) -> Expression | None: """Build the program tree *node* stands for, checking every name and operator call shape on the way. - *frame* is the ``dims:`` of the entry *node* belongs to: where it declares - a dimension with members and a member, an operand over the member meets - one over the dimension as their cross product. + *frame* is the ``dims:`` of the entry *node* belongs to. A tree that + meets it by the members of a dimension is read at it + ([`ExpressionResolver.framed`][mathspec._expression_resolver.ExpressionResolver.framed]). Returns: The tree, or ``None`` once anything failed — appending to *errors* @@ -359,7 +359,10 @@ def resolve_expression( """ before = len(errors) - resolved = ExpressionResolver(ns, context, errors, formals=formals, frame=frozenset(frame)).build(node) + resolver = ExpressionResolver(ns, context, errors, formals=formals, frame=frozenset(frame)) + resolved = resolver.build(node) + if resolved is not None and frame: + resolved = resolver.framed(resolved) return None if len(errors) > before else resolved @@ -456,7 +459,10 @@ def resolve_constraint_text( found = len(errors) resolver = ExpressionResolver(ns, context, errors, frame=frozenset(frame)) left, right = resolver.build(ast.left), resolver.build(ast.right) - if len(errors) > found or left is None or right is None or resolver.met(left, right): + if len(errors) > found or left is None or right is None or (pair := resolver.aligned(left, right)) is None: + return None + left, right = resolver.framed(pair[0]), resolver.framed(pair[1]) + if left is None or right is None: return None if any(_over_the_ceiling(side, context, errors, ceiling=2) for side in (left, right)): return None diff --git a/src/mathspec/_validation.py b/src/mathspec/_validation.py index 0199eff6..ec9ef5f7 100644 --- a/src/mathspec/_validation.py +++ b/src/mathspec/_validation.py @@ -187,7 +187,6 @@ def _frame_dimensions(spec: Spec) -> Iterator[str]: for d, count in Counter(dims).items() if count > 1 ) - yield from _shared_members(kind, name, dims, spec) def _shared_members(kind: str, name: str, dims: Iterable[str], spec: Spec) -> Iterator[str]: diff --git a/src/mathspec/program.py b/src/mathspec/program.py index d784fa92..dda99a06 100644 --- a/src/mathspec/program.py +++ b/src/mathspec/program.py @@ -753,6 +753,10 @@ class Parameter: ``dims``. Where the condition is false, the parameter is absent and a tool drops its rows of data there. ``missing`` applies where the condition is true. """ + domain: Mask | None = None + """The tuples of the dimensions with members that the file lists in ``dims:``, where the program reads the parameter + over their members, or ``None``. A tool refuses a row of data outside them, where it drops a row outside ``where``. + """ @dataclass(frozen=True) diff --git a/src/mathspec/spec.py b/src/mathspec/spec.py index c97479d6..ee8881ad 100644 --- a/src/mathspec/spec.py +++ b/src/mathspec/spec.py @@ -1238,9 +1238,17 @@ def program(self) -> Program: so a consumer building rows reads ``spec.expand(...).program`` and refuses an entry it does not take. """ + from dataclasses import replace + + from mathspec._flat import flat_spec from mathspec._lowering import lower - return lower(self) + flat, domains = flat_spec(self) + if flat is self: + return lower(self) + built = flat.program + parameters = {n: replace(p, domain=domains[n]) if n in domains else p for n, p in built.parameters.items()} + return replace(built, parameters=Frozen(parameters)) @classmethod @override diff --git a/tests/fixtures/strict/connection.yaml b/tests/fixtures/strict/connection.yaml new file mode 100644 index 00000000..db4b7ff1 --- /dev/null +++ b/tests/fixtures/strict/connection.yaml @@ -0,0 +1,55 @@ +# SPDX-FileCopyrightText: mathspec Contributors +# +# SPDX-License-Identifier: MIT + +description: Units that draw power from one bus and deliver it to others, through numbered ports. + +dimensions: + snapshot: { dtype: int, description: dispatch periods } + bus: { description: nodes of the network } + unit: { description: "converters, such as an electrolyser or a heat pump" } + port: { dtype: int, description: "an attachment of a unit, numbered as PyPSA's bus0, bus1, ..." } + role: { description: input or output } + connection: + description: the bus each port of a unit attaches to, and whether the port draws or delivers + over: [unit, port] + +mappings: + connection: { bus: { dim: bus }, role: { dim: role } } + +parameters: + capacity: { dims: [unit], description: the most a unit can draw at its input } + efficiency: + description: share of the drawn power that an output port delivers + dims: [connection, snapshot] + where: "connection[role] == 'output'" + missing: 1 + load: { dims: [bus, snapshot], description: demand to be met } + cost: { dims: [bus], description: price of power bought at a bus } + +variables: + flow: + description: power a unit draws at its input + dims: [unit, snapshot] + bounds: { lower: 0, upper: capacity } + supply: + description: power bought at a bus + dims: [bus, snapshot] + bounds: { lower: 0 } + +expressions: + injection: + description: what a port puts into its bus + dims: [connection, snapshot] + cases: + draws: { when: "connection[role] == 'input'", expression: "-at(flow, by=connection[unit])" } + otherwise: efficiency * at(flow, by=connection[unit]) + +constraints: + balance: + dims: [bus, snapshot] + expression: supply + sum(injection, over=connection, by=connection[bus]) == load + +objective: + sense: minimize + expression: sum(cost * supply) diff --git a/tests/fixtures/strict/sparse.yaml b/tests/fixtures/strict/sparse.yaml new file mode 100644 index 00000000..152d5774 --- /dev/null +++ b/tests/fixtures/strict/sparse.yaml @@ -0,0 +1,51 @@ +# SPDX-FileCopyrightText: mathspec Contributors +# +# SPDX-License-Identifier: MIT + +description: >- + Units that feed buses through the ports they have, and run only in the + periods they are active: two sparse sets, read four ways. + +dimensions: + unit: { description: a converter } + port: { description: an output of a unit } + bus: { description: a node of the network } + period: { dtype: int, ordered: true, description: a step of the horizon } + connection: { over: [unit, port], description: the ports a unit has } + active: { over: [unit, period], description: the periods a unit runs in } + +mappings: + connection: + bus: { dim: bus, description: the bus a port feeds } + +parameters: + efficiency: { dims: [connection, period], missing: 1, description: share of the output a port delivers } + capacity: { dims: [unit], description: the most a unit produces in a period } + ramp: { dims: [unit], description: the most a unit's output rises from one period to the next } + load: { dims: [bus, period], description: demand to be met } + +variables: + output: { dims: [unit, period], where: active, bounds: { lower: 0 }, description: what a unit produces } + flow: { dims: [connection, period], bounds: { lower: 0 }, description: what a port takes of it } + +constraints: + port_limit: + description: a port takes no more than the unit's capacity + dims: [connection, period] + expression: flow <= at(capacity, by=connection[unit]) + split: + description: the ports of a unit take what it produces + dims: [unit, period] + expression: sum(flow, over=connection, by=connection[unit]) == output + ramp_up: + description: output rises by at most the ramp from one active period to the next + dims: [unit, period] + expression: output - shift(output, along=period, offset=1) <= ramp + balance: + description: what the ports deliver to a bus meets its load + dims: [bus, period] + expression: sum(efficiency * flow, over=connection, by=connection[bus]) >= load + +objective: + sense: minimize + expression: sum(output) diff --git a/tests/fixtures/strict/times/capacity.yaml b/tests/fixtures/strict/times/capacity.yaml new file mode 100644 index 00000000..f98e6941 --- /dev/null +++ b/tests/fixtures/strict/times/capacity.yaml @@ -0,0 +1,62 @@ +# SPDX-FileCopyrightText: mathspec Contributors +# +# SPDX-License-Identifier: MIT + +description: >- + Capacity transfer and capacity–activity from TIMES: EQ_CPT and the upper + form of EQ_CAPACT, for a process without vintages and without storage. + +dimensions: + region: { description: a region of the model } + period: { dtype: int, ordered: true, description: a milestone year } + vintage: { dtype: int, description: the period a capacity was built in } + process: { description: a technology } + timeslice: { description: "a slice of the year, at any level of the timeslice tree" } + upper_timeslice: { alias: timeslice, description: a timeslice that another is at or below } + rtp_vintyr: + description: the vintages of a process that exist in a period (RTP_VINTYR) + over: [region, vintage, period, process] + prc_ts: + description: the timeslices a process operates at (PRC_TS) + over: [region, process, timeslice] + ts_map: + description: each timeslice and the timeslices at or below it (TS_MAP) + over: [region, upper_timeslice, timeslice] + +parameters: + coef_cpt: { dims: [region, vintage, period, process], description: the share of a vintage that still exists in a period } + ncap_pasti: { dims: [region, vintage, process], missing: 0, description: capacity built before the first period } + coef_af_up: { dims: [region, period, process, timeslice], missing: absent, description: the upper availability factor } + prc_capact: { dims: [region, process], description: the activity a unit of capacity can produce in a year } + g_yrfr: { dims: [region, timeslice], description: the share of the year a timeslice covers } + obj_icost: { dims: [region, vintage, process], description: "the cost of a unit of new capacity, discounted" } + obj_acost: { dims: [region, period, process], description: "the cost of a unit of activity, discounted" } + +variables: + ncap: { dims: [region, vintage, process], bounds: { lower: 0 }, description: new capacity (VAR_NCAP) } + cap: { dims: [region, period, process], bounds: { lower: 0 }, description: installed capacity (VAR_CAP) } + act: + description: activity of a vintage of a process in a timeslice (VAR_ACT) + dims: [region, vintage, period, process, timeslice] + where: "prc_ts AND rtp_vintyr" + bounds: { lower: 0 } + +constraints: + eq_cpt: + description: the installed capacity of each period, from the new capacity of each vintage (EQ_CPT) + dims: [region, period, process] + expression: cap == sum(coef_cpt * (ncap + ncap_pasti), over=vintage) + eq_capact_up: + description: activity summed over the timeslices at or below each one, up to the available capacity (EQ_CAPACT, UP) + dims: [region, period, process, timeslice] + where: coef_af_up + expression: >- + sum(sum(at(sum(act, over=vintage), by=ts_map[region, timeslice]), over=ts_map, by=ts_map[region, upper_timeslice]), + over=upper_timeslice, by=upper_timeslice[timeslice]) + <= coef_af_up * prc_capact * g_yrfr * cap + +objective: + sense: minimize + expression: >- + sum(obj_icost * ncap) + + sum(obj_acost * sum(act, over=[vintage, timeslice])) diff --git a/tests/fixtures/strict/times/trade.yaml b/tests/fixtures/strict/times/trade.yaml new file mode 100644 index 00000000..c2196875 --- /dev/null +++ b/tests/fixtures/strict/times/trade.yaml @@ -0,0 +1,50 @@ +# SPDX-FileCopyrightText: mathspec Contributors +# +# SPDX-License-Identifier: MIT + +description: >- + Trade between regions from TIMES: IRE_FLO, the efficiency of a flow from one + region and commodity to another through a process, over each region and + commodity twice. + +dimensions: + region: { description: a region of the model } + commodity: { description: an energy carrier or a material } + process: { description: a technology that carries trade } + from_region: { alias: region, description: the region an export leaves } + from_commodity: { alias: commodity, description: the commodity an export leaves as } + top_ire: + description: the processes that carry each link from a region and commodity to another (TOP_IRE) + over: [from_region, from_commodity, region, commodity, process] + +parameters: + ire_flo: + description: the share of an export that arrives as an import (IRE_FLO) + dims: [top_ire] + demand: { dims: [commodity, region], description: what a region needs of a commodity } + supply_cost: { dims: [commodity, region], description: the cost of a unit produced in a region } + trade_cost: + description: the cost of a unit exported along a link + dims: [top_ire] + +variables: + supply: { dims: [commodity, region], bounds: { lower: 0 }, description: what a region produces of a commodity } + export: + description: what leaves along a link through a process (VAR_IRE) + dims: [top_ire] + bounds: { lower: 0 } + +constraints: + balance: + description: production and imports less exports meet the demand of each region and commodity + dims: [commodity, region] + expression: >- + supply + + sum(ire_flo * export, over=top_ire, by=top_ire[commodity, region]) + - sum(sum(sum(export, over=top_ire, by=top_ire[from_commodity, from_region]), + over=from_region, by=from_region[region]), over=from_commodity, by=from_commodity[commodity]) + >= demand + +objective: + sense: minimize + expression: sum(supply_cost * supply) + sum(trade_cost * export) diff --git a/tests/test_dimensions.py b/tests/test_dimensions.py index b2c4ee78..403946f7 100644 --- a/tests/test_dimensions.py +++ b/tests/test_dimensions.py @@ -673,13 +673,17 @@ def test_a_row_over_more_dims_than_the_entry_it_reads_is_refused(): def _derived(node: Expression, spec: Spec) -> frozenset[str]: - """The dims of *node* by the rules of the language reference, derived from the spec without reading ``dims``.""" + """The dims of *node* by the rules of the language reference, derived from the entries without reading ``dims``. + + An entry over members its file reads by name is over them in the program, so a leaf takes the program entry's dims. + """ if isinstance(node, Constant): return frozenset() if isinstance(node, ParameterReference): - return frozenset({**spec.parameters, **spec.given.parameters}[node.name].dims) + return frozenset({**spec.given.parameters, **spec.program.parameters}[node.name].dims) if isinstance(node, VariableReference): - return frozenset({**spec.variables, **spec.given.variables, **spec.given.expressions}[node.name].dims or ()) + declared = {**spec.given.variables, **spec.given.expressions, **spec.program.variables} + return frozenset(declared[node.name].dims or ()) if isinstance(node, Dual): return frozenset({**spec.constraints, **spec.given.constraints}[node.constraint].dims) if isinstance(node, ExpressionReference): diff --git a/tests/test_tuple_dimensions.py b/tests/test_tuple_dimensions.py index c2afeb29..d1cf4a1b 100644 --- a/tests/test_tuple_dimensions.py +++ b/tests/test_tuple_dimensions.py @@ -125,83 +125,6 @@ def test_a_member_is_read_as_a_map_of_the_dimension(expression, dims): assert sorted({*constraint.lhs.dims, *constraint.rhs.dims}) == sorted(dims) -@pytest.mark.parametrize( - 'patch', - [ - pytest.param(_constraint('flow <= capacity', ['connection', 'unit', 'snapshot']), id='a-constraint'), - pytest.param( - {'expressions.e': {'dims': ['connection', 'unit', 'snapshot'], 'expression': 'flow * capacity'}}, - id='a-named-expression', - ), - pytest.param( - _constraint('sum(flow * capacity, over=snapshot) <= 1', ['connection', 'unit']), - id='under-a-sum-that-keeps-both', - ), - pytest.param( - { - 'variables.both': { - 'dims': ['connection', 'unit'], - 'where': 'at(rating, by=connection[unit, port]) > capacity', - } - }, - id='a-condition', - ), - pytest.param( - { - 'constraints.probe': { - 'dims': ['connection', 'unit'], - 'expression': 'at(rating, by=connection[unit, port]) <= capacity', - 'where': 'at(rating, by=connection[unit, port]) > capacity', - } - }, - id='a-constraint-where', - ), - pytest.param( - { - 'parameters.p': { - 'dims': ['connection', 'unit'], - 'where': 'at(rating, by=connection[unit, port]) > capacity', - } - }, - id='a-parameter-where', - ), - pytest.param( - { - 'expressions.e': { - 'dims': ['connection', 'unit', 'snapshot'], - 'cases': { - 'a': { - 'when': 'at(rating, by=connection[unit, port]) > capacity', - 'expression': 'flow * capacity', - } - }, - 'otherwise': 'flow * capacity', - } - }, - id='a-case', - ), - pytest.param( - { - 'parameters.xs': {'dims': ['period']}, - 'parameters.ys': {'dims': ['period']}, - 'variables.w': {'dims': ['connection']}, - 'variables.v': {'dims': ['unit']}, - 'variables.y': {'dims': ['connection', 'unit']}, - 'piecewise.curve': { - 'dims': ['connection', 'unit'], - 'along': 'period', - 'links': {'x': ['w + v', 'xs'], 'y': ['y', 'ys']}, - }, - }, - id='a-piecewise-link', - ), - ], -) -def test_a_frame_over_the_dimension_and_its_member_is_refused(patch): - """Rule 11 of #1011 read these as the cross product; a later reading by member name would read them as a join.""" - assert "'connection' and 'unit' in dims: share the member 'unit'" in _refusal(**patch) - - _SPAN = {'dimensions.span': {'over': ['unit', 'period']}} @@ -275,27 +198,6 @@ def test_a_tuple_dimension_tests_membership_only_where_the_entry_has_every_membe assert 'outside the declared dims' in message -_STEPPED = {'dimensions.span': {'over': ['unit', 'period']}, 'variables.out': {'dims': ['span']}} - - -@pytest.mark.parametrize( - 'patch', - [ - pytest.param(_constraint('out - shift(out, along=period, offset=1) <= 1', ['span']), id='shift'), - pytest.param(_constraint('sum_back(out, along=period, window=2) <= 1', ['span']), id='sum-back'), - pytest.param( - {'constraints.probe': {'dims': ['span'], 'where': 'position(period) > 0', 'expression': 'out <= 1'}}, - id='position', - ), - ], -) -def test_a_step_along_a_member_of_an_entry_over_the_tuple_names_the_flat_entry(patch): - """The entry is over the tuple, which has no order, so the rewrite is the entry over the members, confined by the tuple.""" - message = _refusal(**_STEPPED, **patch) - assert "'period' is a member of 'span'" in message, message - assert "dims: ['period', 'unit'], with where: span" in message, message - - def test_a_condition_compares_a_member_label(): spec = to_spec(varied(MODEL, **{'variables.flow.where': "connection[unit] == 'gas'"})) (atom,) = spec.program.variables['flow'].where.atoms @@ -401,23 +303,6 @@ def test_a_sparse_set_with_an_ordered_member_is_read_at_a_tuple(): ("Sos 'pick': type: 2 along connection reads the order of 'connection'", "over ['port', 'unit']"), id='sos-of-type-2', ), - pytest.param( - {'variables.both': {'dims': ['connection', 'unit']}} - | _constraint('at(both, by=connection[unit]) >= 0', ['connection']), - ( - "Variable 'both': 'connection' and 'unit' in dims: share the member 'unit'", - 'read it as connection[unit]', - ), - id='an-operand-with-the-tuple-and-a-member', - ), - pytest.param( - _constraint('sum(flow, over=unit) >= 0', ['connection', 'snapshot']), - ( - "'unit' is a member of 'connection'", - 'sum(..., over=connection, by=connection[port])', - ), - id='over-names-a-member', - ), pytest.param( _constraint('sum(flow, over=unit, by=connection[port]) >= 0', ['port', 'snapshot']), ("over=unit names a member of 'connection'", 'sum(..., over=connection, by=connection[port])'), @@ -428,110 +313,11 @@ def test_a_sparse_set_with_an_ordered_member_is_read_at_a_tuple(): ("over=unit names a member of 'connection'", 'sum(..., over=connection, by=connection[bus])'), id='over-names-a-member-beside-a-declared-mapping', ), - pytest.param( - _constraint('sum(flow, over=[unit, port]) >= 0', ['snapshot']), - ("'unit' is a member of 'connection'", 'Write sum(..., over=connection).'), - id='over-names-every-member', - ), - pytest.param( - _constraint('sum(shipped, over=region) >= 0', ['trade']), - ("'region' is a member of 'trade'", 'Write sum(..., over=trade, by=trade[from_region]).'), - id='over-names-a-member-beside-its-alias', - ), - pytest.param( - _constraint('flow <= capacity', ['connection', 'snapshot']), - ( - "an operand over ['unit'] meets one over 'connection'", - 'at(capacity, by=connection[unit])', - 'sum(flow, over=connection, by=connection[unit])', - ), - id='a-member-operand-meets-the-tuple', - ), - pytest.param( - _constraint('flow <= capacity', ['unit', 'snapshot']), - ("meets one over 'connection'", 'sum(flow, over=connection, by=connection[unit])'), - id='a-member-operand-meets-the-tuple-in-a-frame-over-the-member', - ), - pytest.param( - {'objective': {'sense': 'minimize', 'expression': 'sum(flow * capacity)'}}, - ( - "The objective: an operand over ['unit'] meets one over 'connection'", - 'at(capacity, by=connection[unit])', - ), - id='a-member-operand-meets-the-tuple-in-a-product', - ), - pytest.param( - _constraint('sum(flow * capacity, over=unit) <= 1', ['connection', 'snapshot']), - ("meets one over 'connection'", 'at(capacity, by=connection[unit])'), - id='a-member-operand-meets-the-tuple-under-a-sum', - ), - pytest.param( - _constraint('sum(flow * capacity, over=unit) <= 1', ['connection', 'unit', 'snapshot']), - ("Constraint 'probe': 'connection' and 'unit' in dims: share the member 'unit'",), - id='a-member-operand-meets-the-tuple-under-a-sum-over-the-member-the-frame-declares', - ), - pytest.param( - _constraint('sum(flow * capacity) <= 1', ['connection', 'snapshot']), - ("meets one over 'connection'", 'at(capacity, by=connection[unit])'), - id='a-member-operand-meets-the-tuple-under-a-sum-over-everything', - ), - pytest.param( - {'assumptions.a': 'at(rating, by=connection[unit, port]) > capacity'}, - ("Assumption 'a': an operand over ['unit'] meets one over 'connection'",), - id='a-member-operand-meets-the-tuple-in-an-assumption', - ), - pytest.param( - { - 'parameters.xs': {'dims': ['period']}, - 'parameters.ys': {'dims': ['period']}, - 'variables.y': {'dims': ['connection']}, - 'piecewise.curve': { - 'dims': ['connection'], - 'along': 'period', - 'links': {'x': ['y', 'xs'], 'y': ['y', 'ys']}, - 'where': 'at(rating, by=connection[unit, port]) > capacity', - }, - }, - ("piecewise 'curve' where: an operand over ['unit'] meets one over 'connection'",), - id='a-member-operand-meets-the-tuple-in-a-piecewise-where-its-assumptions-repeat', - ), - pytest.param( - {'variables.flow.where': 'at(rating, by=connection[unit, port]) > capacity'}, - ("Variable 'flow': an operand over ['unit'] meets one over 'connection'",), - id='a-member-operand-meets-the-tuple-in-a-condition', - ), - pytest.param( - {'expressions.e': {'dims': ['unit'], 'expression': '2 * capacity'}} - | _constraint('flow <= e', ['connection', 'snapshot']), - ('at(e, by=connection[unit])',), - id='a-named-expression-over-a-member-meets-the-tuple', - ), - pytest.param( - {'expressions.e': {'dims': ['connection', 'snapshot'], 'expression': 'capacity'}}, - ("the body carries dims ['unit']", 'at(capacity, by=connection[unit])'), - id='a-named-expression-body-over-a-member', - ), - pytest.param( - { - 'expressions.e': { - 'dims': ['connection', 'snapshot'], - 'cases': {'a': {'when': "connection[unit] == 'u'", 'expression': 'capacity'}}, - 'otherwise': 'flow', - } - }, - ("the value carries dims ['unit']", 'at(capacity, by=connection[unit])'), - id='a-case-value-over-a-member', - ), pytest.param( {'variables.u': {'dims': ['unit']}} | _constraint('sum(u, over=unit, by=connection[port]) >= 0', ['port']), ('sum(at(u, by=connection[unit]), over=connection, by=connection[port])',), id='over-names-a-member-of-an-operand-without-the-dimension', ), - pytest.param( - {'variables.u': {'dims': ['unit']}} | _constraint('u <= 1', ['connection']), - ("carries dims ['unit'] that are not in its dims", 'at(u, by=connection[unit])'), - id='a-constraint-over-a-member-its-dims-lack', - ), pytest.param( {'variables.flow.bounds.upper': 'capacity'}, ( diff --git a/tests/test_tuple_sugar.py b/tests/test_tuple_sugar.py new file mode 100644 index 00000000..61caaba8 --- /dev/null +++ b/tests/test_tuple_sugar.py @@ -0,0 +1,228 @@ +# SPDX-FileCopyrightText: mathspec Contributors +# +# SPDX-License-Identifier: MIT + +"""A dimension's members read by name: each shorter spelling builds the program of the spelling it stands for. + +A spelling that names a member where the tuple is meant loads as the call +through the tuple that it means, so the program cannot tell the two apart. A +parameter over two tuples is the one exception: its data is refused outside +the rows that both list. +""" + +from __future__ import annotations + +from pathlib import Path +from typing import Any + +import pytest + +from mathspec import to_spec +from mathspec.program import And, Mask, RelationDefined +from tests.fixtures import EXAMPLES, varied + +#: Units on the ports they have and in the periods they run, and the sites of +#: each unit: three tuples, two of which share `unit` with `connection`. +MODEL: dict[str, Any] = { + 'dimensions': { + 'unit': {}, + 'port': {}, + 'bus': {}, + 'snapshot': {'dtype': 'int'}, + 'period': {'dtype': 'int', 'ordered': True}, + 'connection': {'over': ['unit', 'port']}, + 'active': {'over': ['unit', 'period']}, + 'site': {'over': ['unit', 'bus']}, + }, + 'parameters': { + 'capacity': {'dims': ['unit']}, + 'ramp': {'dims': ['unit']}, + 'rating': {'dims': ['bus']}, + }, + 'variables': {'flow': {'dims': ['connection', 'snapshot']}}, +} + + +def _program(**patch: Any): + return to_spec(varied(MODEL, **patch)).program + + +def _same(sugar: dict[str, Any], spelled: dict[str, Any], section: str, name: str) -> None: + built, meant = _program(**sugar), _program(**spelled) + assert getattr(built, section)[name] == getattr(meant, section)[name], 'the sugar builds what it stands for' + + +@pytest.mark.parametrize( + ('sugar', 'spelled'), + [ + pytest.param( + 'flow <= capacity', 'flow <= at(capacity, by=connection[unit])', id='a-constraint-reads-the-member' + ), + pytest.param( + 'flow * capacity <= 1', 'flow * at(capacity, by=connection[unit]) <= 1', id='a-product-reads-the-member' + ), + pytest.param('capacity >= flow', 'at(capacity, by=connection[unit]) >= flow', id='the-member-on-the-left'), + ], +) +def test_an_operand_over_a_member_meets_the_tuple_by_name(sugar, spelled): + """Was: refused, with the at() rewrite. A member operand is read at each tuple, as at(…, by=tuple[member]) reads it.""" + frame = ['connection', 'snapshot'] + _same( + {'constraints.c': {'dims': frame, 'expression': sugar}}, + {'constraints.c': {'dims': frame, 'expression': spelled}}, + 'constraints', + 'c', + ) + + +def test_a_member_meets_the_tuple_in_the_objective(): + built = _program(objective={'expression': 'sum(flow * capacity)'}).objective + meant = _program(objective={'expression': 'sum(flow * at(capacity, by=connection[unit]))'}).objective + assert built == meant, 'the objective reads the member at each tuple too' + + +@pytest.mark.parametrize( + ('dims', 'sugar', 'spelled'), + [ + pytest.param( + ['snapshot', 'unit'], + 'sum(flow, over=port)', + 'sum(flow, over=connection, by=connection[unit])', + id='one-member-summed-away', + ), + pytest.param( + ['snapshot'], 'sum(flow, over=[port, unit])', 'sum(flow, over=connection)', id='every-member-summed-away' + ), + pytest.param( + ['unit'], + 'sum(flow, over=[port, snapshot])', + 'sum(flow, over=[connection, snapshot], by=connection[unit])', + id='a-member-beside-a-plain-dimension', + ), + ], +) +def test_a_sum_over_a_member_sums_the_tuple_grouped_by_the_rest(dims, sugar, spelled): + """Was: refused, with the sum through the tuple as the rewrite.""" + _same( + {'constraints.c': {'dims': dims, 'expression': f'{sugar} <= 1'}}, + {'constraints.c': {'dims': dims, 'expression': f'{spelled} <= 1'}}, + 'constraints', + 'c', + ) + + +@pytest.mark.parametrize( + ('section', 'sugar', 'spelled'), + [ + pytest.param( + 'variables', + {'dims': ['active', 'connection']}, + {'dims': ['period', 'port', 'unit'], 'where': 'active AND connection'}, + id='a-variable', + ), + pytest.param( + 'variables', + {'dims': ['active', 'site'], 'where': 'rating > 0'}, + {'dims': ['bus', 'period', 'unit'], 'where': '(active AND site) AND (rating > 0)'}, + id='a-variable-with-its-own-where', + ), + pytest.param( + 'variables', + {'dims': ['connection', 'unit']}, + {'dims': ['port', 'unit'], 'where': 'connection'}, + id='a-tuple-beside-its-own-member', + ), + pytest.param( + 'parameters', + {'dims': ['active', 'connection']}, + {'dims': ['period', 'port', 'unit'], 'where': 'active AND connection'}, + id='a-parameter', + ), + ], +) +def test_two_tuples_that_share_a_member_are_joined_on_it(section, sugar, spelled): + """Was: refused, as an entry indexed by the shared member twice. The entry is over the members, at the rows both list.""" + built, meant = _program(**{f'{section}.e': sugar}), _program(**{f'{section}.e': spelled}) + entry, flat = getattr(built, section)['e'], getattr(meant, section)['e'] + assert entry.dims == flat.dims, 'the entry is over the members, each once' + assert entry.where == flat.where, 'and confined to the rows both tuples list' + + +def test_a_parameter_over_two_tuples_refuses_a_row_outside_both(): + """The one change no flat spelling states: a tool refuses its data row outside the joined rows, where a `where:` drops it.""" + joined = _program(**{'parameters.e': {'dims': ['active', 'connection']}}).parameters['e'] + flat = _program(**{'parameters.e': {'dims': ['period', 'port', 'unit'], 'where': 'active AND connection'}}) + listed = And(RelationDefined('active', ('period', 'unit')), RelationDefined('connection', ('port', 'unit'))) + assert joined.domain == Mask(listed), 'the joined rows are the domain' + assert flat.parameters['e'].domain is None, 'a where: is no domain: a tool drops a row outside it' + + +#: `output` over the tuples of `active`, and a ramp that steps along its ordered member. +_STEPPED = { + 'variables.output': {'dims': ['active']}, + 'constraints.ramp_up': { + 'dims': ['active'], + 'expression': 'output - shift(output, along=period, offset=1) <= ramp', + }, +} + +#: The same, over the members, as a dimension without an order must spell it. +_SPELLED = { + 'variables.output': {'dims': ['period', 'unit'], 'where': 'active'}, + 'constraints.ramp_up': { + 'dims': ['period', 'unit'], + 'where': 'active', + 'expression': 'output - shift(output, along=period, offset=1) <= ramp', + }, +} + + +@pytest.mark.parametrize(('section', 'name'), [('variables', 'output'), ('constraints', 'ramp_up')]) +def test_a_step_along_a_member_reads_the_entry_over_its_members(section, name): + """Was: refused, as a step along a member of an entry over the tuple, with the flat entry as the rewrite.""" + _same(_STEPPED, _SPELLED, section, name) + + +def test_an_entry_over_the_tuple_reads_a_stepped_one_at_each_tuple(): + """`peak` stays over `active`, and reads `output`, which a step reads over its members, at each tuple.""" + peak = {'constraints.peak': {'dims': ['active'], 'expression': 'output <= ramp'}} + spelled = 'at(output, by=active[period, unit]) <= at(ramp, by=active[unit])' + flat = {'constraints.peak': {'dims': ['active'], 'expression': spelled}} + _same(_STEPPED | peak, _SPELLED | flat, 'constraints', 'peak') + + +def test_an_entry_over_the_members_reads_a_tuple_at_them(): + """An entry over two tuples is over their members, and reads `flow`, over `connection`, at each pair of them.""" + limit = {'parameters.limit': {'dims': ['period', 'unit']}} + sugar = {'dims': ['active', 'connection', 'snapshot'], 'expression': 'flow <= limit'} + read = 'sum(flow, over=connection, by=connection[port, unit])' + spelled = { + 'dims': ['period', 'port', 'snapshot', 'unit'], + 'where': 'active AND connection', + 'expression': f'{read} <= limit', + } + _same(limit | {'constraints.c': sugar}, limit | {'constraints.c': spelled}, 'constraints', 'c') + + +#: Each example this branch shortens, beside the spelling `compare/tuple-strict` writes. +STRICT = Path(__file__).parent / 'fixtures' / 'strict' +SHORTENED = sorted(path.relative_to(STRICT) for path in STRICT.rglob('*.yaml')) + + +@pytest.mark.parametrize('path', SHORTENED, ids=str) +def test_a_shortened_example_builds_the_program_of_its_strict_spelling(path): + """Every #9 spelling keeps its meaning: the shorter file and the strict one build one program.""" + built, meant = to_spec(EXAMPLES / path).program, to_spec(STRICT / path).program + domains = {n: p.domain for n, p in built.parameters.items() if p.domain is not None} + assert not domains or set(domains) <= set(built.parameters), 'only a parameter carries a domain' + stripped = {n: _without_domain(p) for n, p in built.parameters.items()} + assert stripped == dict(meant.parameters), 'the parameters differ in their domain alone' + for section in ('variables', 'constraints', 'expressions'): + assert dict(getattr(built, section)) == dict(getattr(meant, section)), f'{section} are the strict ones' + assert built.objective == meant.objective + + +def _without_domain(parameter): + from dataclasses import replace + + return replace(parameter, domain=None) diff --git a/tests/test_validation.py b/tests/test_validation.py index 1c200505..10382ab2 100644 --- a/tests/test_validation.py +++ b/tests/test_validation.py @@ -3310,28 +3310,6 @@ def test_a_divisor_a_shift_fills_with_another_value_loads(divisor): @pytest.mark.parametrize( ('patch', 'fragments'), [ - pytest.param( - {'variables.pair': {'dims': ['connection', 'site']}}, - ( - "Variable 'pair': 'connection' and 'site' in dims: share the member 'unit'", - 'where: "at(site, by=connection[unit])"', - "declare an alias of 'unit'", - ), - id='two-tuple-dimensions-in-a-variable', - ), - pytest.param( - {'constraints.pair': {'dims': ['connection', 'site'], 'expression': 'flow >= 0'}}, - ("Constraint 'pair': 'connection' and 'site' in dims: share the member 'unit'",), - id='two-tuple-dimensions-in-a-constraint', - ), - pytest.param( - {'parameters.rating': {'dims': ['connection', 'unit']}}, - ( - "Parameter 'rating': 'connection' and 'unit' in dims: share the member 'unit'", - 'read it as connection[unit]', - ), - id='a-tuple-dimension-beside-its-own-member', - ), pytest.param( { 'dimensions.period': {'dtype': 'int', 'ordered': True},