From 58d96e25ff820c734a4c4c2f3349ea602174eeef Mon Sep 17 00:00:00 2001 From: Nicholas Jakobsen Date: Tue, 27 Oct 2026 11:37:19 -0700 Subject: [PATCH 1/3] docs: add a language guide for the words LaForge's docs and code use Issue: LaForge had no record of which word names each concept, so the README and code mixed synonyms, such as a source that "reports" or "supplies" a value. `.ai/language.md` lists the term to use for each concept, the words to avoid, the example domain, and the writing rules. `CLAUDE.md` points to it. Co-Authored-By: Claude Opus 5.5 --- .ai/language.md | 55 +++++++++++++++++++++++++++++++++++++++++++++++++ CLAUDE.md | 3 +++ 2 files changed, 58 insertions(+) create mode 100644 .ai/language.md create mode 100644 CLAUDE.md diff --git a/.ai/language.md b/.ai/language.md new file mode 100644 index 0000000..8f091b3 --- /dev/null +++ b/.ai/language.md @@ -0,0 +1,55 @@ +# Language guide + +How LaForge's README, code comments, spec descriptions, error messages, commit messages and PR bodies use words. Use the terms below, and only these, for the concepts they name, so a reader never wonders whether two words mean two things. + +## Vocabulary + +### Sources and values + +| Say | Meaning | Don't say | +|---|---|---| +| a source **provides** a value | A data source has a value for an attribute. Nil is a value too, so a source that provides nil provides the attribute, and one that leaves the attribute out doesn't. The application records what a source provides, so a source never records or updates. | reports, supplies, sends, records, updates, has an opinion | +| **record** a value | Store values from a source. Importers, editors, forms and forges are application code, so they record. One call that records values from a source is **a recording** from the source. | report, import, save, the source records, the source's recording | +| **data entry**, or **entry** | One source's recorded value for one attribute of one record. | report, datum | +| **forge** | Combine a record's entries into its attributes and its tracked associations' membership. | build, sync, resolve | +| **precedence** | How sources rank against each other. A higher precedence wins. | priority, weight, source of truth | +| **wins**, **outranks** | A value wins when forging takes it. A source outranks another when its precedence is higher. | overrides, beats | +| **precedence rule** | A source's precedence at one level. | override, setting | +| **level** | One attribute of one record, one attribute of a model, a whole model, or global. | scope, tier | +| **precedence order** | The sources in order at one level. | ranking, priority list | +| **ruleset**, **base rules** | Defined, with the rest of the ruleset vocabulary, in the comment on `class << self` in `lib/laforge/rulesets.rb`. | profile, domain | +| **inactive source** | A source whose entries are kept but left out of forging. | disabled, paused | + +### Associations + +| Say | Meaning | Don't say | +|---|---|---| +| **tracked association** | A `has_many` or `has_one` whose membership LaForge forges from sources. | managed association | +| **owner**, **member** | The record a tracked association belongs to, which Rails calls its `owner`, and one of the association's records, which Rails calls an associated record. | parent, child, associated record | +| **member list** | What one source provides for a tracked association on one owner. A **complete list** covers every member, and a **partial list** only the members it names. An **edit** changes how a source's member list covers the members it names, and is never a kind of member list. | report, a bare "list" | +| a member list **covers** a member | It includes or excludes the member. Its **coverage** is which members it includes, which it excludes, and which it doesn't cover. A partial list no longer covers a member once its inclusion or exclusion expires, while a complete list still excludes a member whose inclusion expired. | has an opinion, stance, view, verdict | +| **includes**, **excludes** | How a member list covers a member. | adds, removes or keeps, which say what a forge or membership does to a member | +| **membership** | Which members an association ends up with, decided member by member. | final state | +| a member list **withdraws** a member | It stops including a member it included. A partial list then no longer covers the member, so other sources decide it, while a complete list excludes it. | drops, abandons | +| **withdraw a source** from a record | Take back everything the source provided for the record, its entries and its member lists' coverage alike. | remove, forget | +| **pending** | Values a member list provides for an included member, waiting to be recorded on it. | queued, staged | +| **tombstone** | What a partial list keeps of a member it excluded, once the member is removed. | archive, ghost | + +## Examples + +Examples use a map system, never a real client, product or field: + +- **Sources:** `cadastre` (an importer), `business_directory` (outranks it), `business_registry` (outranks both where `precedence:` is passed with its values), `map_editor` (manual edits, the highest), `geocoder` (provides values only). +- **Models:** `Place`, `Category` through `PlaceCategory`, `ParcelLink` (with `Parcel` and `ParcelLinkType`), `Tenant` (polymorphic `occupiable`, `registration_number`, status `open` or `closed`), `OpeningHour`. + +Run a new concept past the maintainers before adding it to the example domain. + +## Writing + +- **American spelling:** behavior, recognize, summarize. +- **People:** they/them for anyone whose pronouns you don't know. +- **Colons:** only to introduce a list, never inside a sentence. +- **Errors:** say what the developer has to declare or pass to fix it, naming the model, association or option. +- **README sections:** open with the common case. Put an unusual situation in a subsection named for the situation. +- **Code comments:** a comment explains what a clearer name or design can't. Before writing one, try renaming the method or variable, or restructuring the code, so that it needs no explanation. A class comment says what the class is for, as "Used to …" or a noun phrase defining the concept. A class or module comment that is the first thing inside `module LaForge`, or inside a reopened class, opens with its subject's name in backticks, as in "`Forging` is used to …". A method comment starts with what the method does or returns. Define working vocabulary once, at the module that owns it. Never narrate a change. +- **No self-narration:** say the thing, not that you're about to say it. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..a89b45b --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,3 @@ +# LaForge + +Follow `.ai/language.md` for the words used in the README, code comments, spec descriptions, error messages, commit messages and PR bodies. From d218da25c250014bd3b2f648a0c5f703daf84121 Mon Sep 17 00:00:00 2001 From: Nicholas Jakobsen Date: Tue, 27 Oct 2026 11:45:24 -0700 Subject: [PATCH 2/3] refactor: call a source's association data its member list, as the language guide does Issue: `.ai/language.md` names what one source provides for a tracked association its member list (a complete list, a partial list or edits) and says a source provides values, but `lib/` and `spec/` still called it the source's "report", said sources "report" or "supply" values, and said a member list has an "opinion" on a member where the guide says it covers the member, and used British spellings such as "recognise", "behaviour" and "cancelled". Identifiers, comments, error messages and spec descriptions now use the guide's words and American spelling. `TrackedAssociation#report`/`#reports` become `#member_list`/`#member_lists`, `Forging#association_reports` becomes `#association_member_lists`, `MemberList#reported_keys` becomes `#listed_keys`, `Merge#combined_report` becomes `#combined_member_list`, `Recording#record_reported` becomes `#record_pending`, `TrackedAssociation#reported_columns` becomes `#member_columns`, `#recognise_tombstones` becomes `#recognize_tombstones`, and the hash `Merge#run` returns is built as `result`. Comments that said a member list has an opinion on a member now say it covers the member, and the C16 spec says an omitted attribute is not provided. The spec domain's `reported_parcel_links` scope becomes `provided_parcel_links`. Stored JSON keys are unchanged. Co-Authored-By: Claude Opus 5.5 --- lib/laforge/activerecord.rb | 4 +- lib/laforge/association_config.rb | 31 +- lib/laforge/attribute_groups.rb | 10 +- lib/laforge/data_entry.rb | 2 + lib/laforge/data_source.rb | 1 + lib/laforge/expiring.rb | 25 +- lib/laforge/forging.rb | 17 +- lib/laforge/inspection.rb | 6 +- lib/laforge/member_list.rb | 144 +++++---- lib/laforge/member_list/complete.rb | 18 +- lib/laforge/member_list/partial.rb | 29 +- lib/laforge/merge.rb | 79 ++--- lib/laforge/payload.rb | 5 +- lib/laforge/pending_writes.rb | 8 +- lib/laforge/precedence.rb | 7 +- lib/laforge/precedence_cache.rb | 6 +- lib/laforge/precedence_rule.rb | 2 +- lib/laforge/record.rb | 8 +- lib/laforge/recording.rb | 69 ++-- lib/laforge/recording_settings.rb | 29 +- lib/laforge/tombstone.rb | 12 +- lib/laforge/tracked_association.rb | 306 +++++++++--------- spec/lib/activerecord_spec.rb | 2 +- spec/lib/association_capabilities_spec.rb | 292 ++++++++--------- .../association_importer_scenarios_spec.rb | 52 +-- spec/lib/usage_spec.rb | 26 +- spec/support/association_capabilities.rb | 8 +- 27 files changed, 611 insertions(+), 587 deletions(-) diff --git a/lib/laforge/activerecord.rb b/lib/laforge/activerecord.rb index 470fa95..17d5f0a 100644 --- a/lib/laforge/activerecord.rb +++ b/lib/laforge/activerecord.rb @@ -1,4 +1,5 @@ module LaForge + # `LaForge::ActiveRecord` is used to declare a model laforged, with `laforged` module ActiveRecord extend ActiveSupport::Concern @@ -74,7 +75,8 @@ def tombstones(name) proxy_association.owner.laforge.tombstones(name) end - # Returns the expiries in the reports of the tracked association `name` (see Inspection#membership_expiries) + # Returns the expiries in the member lists of the tracked association `name` (see + # Inspection#membership_expiries) def membership_expiries(name) proxy_association.owner.laforge.membership_expiries(name) end diff --git a/lib/laforge/association_config.rb b/lib/laforge/association_config.rb index b460f39..e071349 100644 --- a/lib/laforge/association_config.rb +++ b/lib/laforge/association_config.rb @@ -1,20 +1,21 @@ module LaForge - # The configuration of a tracked association, derived from its reflection wherever it can be. A has_one is tracked as - # a list of at most one member. + # `AssociationConfig` is the configuration of a tracked association, derived from its reflection wherever it can be. A + # has_one is tracked as a member list of at most one member. # # A member is named by its key, which is the first of these that applies: # - the columns given with `key:` # - the foreign keys of the records a join row references, with the type column of a polymorphic one # - the member's primary key, for a has_many - # A has_one with none of these has no key, so every source reports the same single member. + # A has_one with none of these has no key, so every source lists the same single member. # - # Where members are keyed by primary key, a member a source names without one is a new member. Reports name it by a - # placeholder key, `{ 'new' => token }`, until the owner is saved after the forge that builds it, and by its id after. + # Where members are keyed by primary key, a member a source names without one is a new member. Member lists name it by + # a placeholder key, `{ 'new' => token }`, until the owner is saved after the forge that builds it, and by its id + # after. # - # A member keyed by primary key that a partial report excludes is removed like any other, and every report then names - # it by a tombstone key, `{ 'tombstone' => token }` (see Tombstone). `tombstone_matcher:` is a callable given a member - # a source reports and the tombstones' records, which returns the record that member is, or nil. Only a has_many keyed - # by primary key takes one. + # A member keyed by primary key that a partial list excludes is removed like any other, and every member list then + # names it by a tombstone key, `{ 'tombstone' => token }` (see Tombstone). `tombstone_matcher:` is a callable given a + # member a source lists and the tombstones' records, which returns the record that member is, or nil. Only a has_many + # keyed by primary key takes one. # # Member order is written to `order_column:` when given, or else to the integer column the association scope orders # by, unless it identifies the member or its owner. `order_column: false` writes none. @@ -57,7 +58,7 @@ def members_of(owner) end # Returns the owner's members by key, without the rows missing a key value, which LaForge leaves alone. Raises - # `DuplicateMemberKey` when several rows share a key, since a report naming that key could mean any of them. + # `DuplicateMemberKey` when several rows share a key, since a member list naming that key could mean any of them. def identified_members_of(owner) members = members_of(owner).reject { |record| blank_key?(record_key(record)) }.group_by { |record| record_key(record) } duplicated = members.select { |_, records| records.size > 1 } @@ -144,7 +145,7 @@ def build_attributes(member) end # Raises when `members` include a member keyed by primary key and the association has no `tombstone_matcher:`. An - # excluded member is removed, and only the matcher can recognise it when a source reports it again. + # excluded member is removed, and only the matcher can recognize it when a source lists it again. def assert_excludable!(owner, members) return unless primary_keyed? && !tombstone_matcher @@ -152,8 +153,8 @@ def assert_excludable!(owner, members) return if ids.empty? raise ArgumentError, "Cannot exclude #{klass.name} #{ids.join(', ')} from #{owner_class.name} #{owner.id.inspect}'s #{name}: excluding it removes it per `dependent:`, " \ - "and without a tombstone_matcher LaForge could not recognise it when a source reports it again, so it would come back. " \ - "Declare tombstone_matcher: on #{owner_class.name}'s #{name}, a callable given the reported #{klass.name} and the excluded ones " \ + "and without a tombstone_matcher LaForge could not recognize it when a source lists it again, so it would come back. " \ + "Declare tombstone_matcher: on #{owner_class.name}'s #{name}, a callable given the listed #{klass.name} and the excluded ones " \ "that returns the one it is, or nil, such as " \ "`laforged associations: { #{name}: { tombstone_matcher: ->(#{klass.model_name.element}, tombstones) { tombstones.find { |tombstone| ... } } } }`" end @@ -187,7 +188,7 @@ def normalize_members(members, attributes: true, edit: false) # Returns [[action, member], ...] for a source's edits, given as a list of members to include or as # { include: [...], exclude: [...], forget: [...] }. Raises for a has_one. def edits(value) - raise ArgumentError, "#{owner_class.name}##{name} is a has_one, which takes a complete report" unless collection? + raise ArgumentError, "#{owner_class.name}##{name} is a has_one, which takes a complete list" unless collection? lists = value.is_a?(Hash) ? value.to_h.stringify_keys : { 'include' => value } unknown = lists.keys - %w[include exclude forget] @@ -356,7 +357,7 @@ def assert_tombstone_matcher_usable! return if collection? && primary_keyed? reason = - if !collection? then 'it is a has_one, which takes complete reports, so nothing is explicitly excluded from it' + if !collection? then 'it is a has_one, which takes complete lists, so nothing is explicitly excluded from it' else "its members are keyed by #{key.join(', ')}, so an exclusion names something that outlasts the row" end raise ArgumentError, "#{owner_class.name}##{name} does not use a tombstone_matcher: #{reason}. Remove tombstone_matcher:" diff --git a/lib/laforge/attribute_groups.rb b/lib/laforge/attribute_groups.rb index c5be388..4f650a2 100644 --- a/lib/laforge/attribute_groups.rb +++ b/lib/laforge/attribute_groups.rb @@ -1,8 +1,8 @@ module LaForge - # Used to read and check a laforged model's attribute groups. An attribute group is a set of attributes that come from - # one source together, such as the lines of a postal address, declared with `laforged attribute_groups:`. Sources - # record a group's attributes together, precedence rules are set on the group, and forging takes every attribute of - # the group from the highest-ranked source with an entry for any of them. + # `AttributeGroups` is used to read and check a laforged model's attribute groups. An attribute group is a set of + # attributes that come from one source together, such as the lines of a postal address, declared with + # `laforged attribute_groups:`. Sources provide a group's attributes together, precedence rules are set on the group, + # and forging takes every attribute of the group from the highest-ranked source with an entry for any of them. module AttributeGroups extend ActiveSupport::Concern @@ -30,7 +30,7 @@ def assert_whole_groups!(attribute_names, where: model.name) given = attributes & attribute_names next if given.empty? || given.size == attributes.size - raise ArgumentError, "#{where}: #{attributes.join(', ')} are attribute group #{group}, which a source records together (missing: #{(attributes - given).join(', ')}). " \ + raise ArgumentError, "#{where}: #{attributes.join(', ')} are attribute group #{group}, which a source provides together (missing: #{(attributes - given).join(', ')}). " \ 'Pass nil for a value the source does not have' end end diff --git a/lib/laforge/data_entry.rb b/lib/laforge/data_entry.rb index 94447ff..fd81296 100644 --- a/lib/laforge/data_entry.rb +++ b/lib/laforge/data_entry.rb @@ -1,4 +1,6 @@ module LaForge + # `DataEntry` is a data entry, one source's recorded value for one attribute of one record. A source's entry for a + # tracked association holds its member list (see `MemberList`). class DataEntry < ActiveRecord::Base self.table_name = "laforge_data_entries" include Expiring diff --git a/lib/laforge/data_source.rb b/lib/laforge/data_source.rb index d52fd02..0642331 100644 --- a/lib/laforge/data_source.rb +++ b/lib/laforge/data_source.rb @@ -1,4 +1,5 @@ module LaForge + # `DataSource` is a source of values, such as an importer or an editor, ranked against the others by precedence class DataSource < ActiveRecord::Base self.table_name = "laforge_data_sources" diff --git a/lib/laforge/expiring.rb b/lib/laforge/expiring.rb index ef2eba3..27ef3f9 100644 --- a/lib/laforge/expiring.rb +++ b/lib/laforge/expiring.rb @@ -1,12 +1,13 @@ module LaForge - # Used to let a change to one record expire. A change is a data entry, or a precedence rule on one record. A - # recording makes the change when it creates the row, and makes it again when it changes the row's value, precedence - # or expiry. A save that changes none of them, such as a forge's bookkeeping, does not make the change again. + # `Expiring` is used to let a change to one record expire. A change is a data entry, or a precedence rule on one + # record. A recording makes the change when it creates the row, and makes it again when it changes the row's value, + # precedence or expiry. A save that changes none of them, such as a forge's bookkeeping, does not make the change + # again. # # A change expires at a time, its `expires_at`, or on a source, its `expires_on_source`. An expiry on a source fires - # the first time that source records the same attribute, attribute group or association for the record after the - # change was made (see `changed_at`), whatever it reports. It sets `expires_at` to when the source recorded, and keeps - # the source, so the row still names what ended the change. + # the first time a value from that source is recorded for the same attribute, attribute group or association of the + # record after the change was made (see `changed_at`), whatever the source provides. It sets `expires_at` to when that + # value was recorded, and keeps the source, so the row still names what ended the change. # # Forging leaves an expired row out, as it leaves out an inactive source's, and the row is kept, so it can still be # read. An expiry takes effect when the record is next forged. Forging records on time, and removing expired rows, @@ -31,8 +32,8 @@ module Expiring scope :expired, ->(at = Time.current) { where(arel_table[:expires_at].lteq(at)) } scope :unexpired, ->(at = Time.current) { where(expires_at: nil).or(where(arel_table[:expires_at].gt(at))) } # `expiring` returns the rows whose `expires_at` is before `before`, those already expired included, such as to - # forge their records on time. A row that expires on a source has no `expires_at` until the source records, so it - # is left out until then. + # forge their records on time. A row that expires on a source has no `expires_at` until a value from the source is + # recorded, so it is left out until then. scope :expiring, ->(before:) { where(arel_table[:expires_at].lt(before)) } end @@ -42,8 +43,8 @@ module Expiring # - `removing` is given. It describes what was passed that removes the change, such as "a nil precedence", which # leaves nothing to expire. # - `expires_at` is not a time - # - `expires_on` is `source`, the source recording the change, whose next recording replaces the change and its - # expiry anyway + # - `expires_on` is `source`, the source the change comes from. The next values recorded from it replace the change + # and its expiry anyway. def self.columns(expires_at, expires_on, where:, source: nil, removing: nil) return NONE if expires_at.nil? && expires_on.nil? @@ -52,8 +53,8 @@ def self.columns(expires_at, expires_on, where:, source: nil, removing: nil) expires_on_source_id = expires_on && DataSource.normalize_source_id(expires_on) if expires_on_source_id && source && expires_on_source_id == DataSource.normalize_source_id(source) - raise ArgumentError, "#{where}: expires_on: names the source recording, whose next recording replaces this one and its expiry anyway. " \ - 'Pass the source whose next report should end the change' + raise ArgumentError, "#{where}: expires_on: names the source these values come from, and the next values recorded from it replace these and their expiry anyway. " \ + 'Pass another source, whose next values should end the change' end { expires_at: expires_at, expires_on_source_id: expires_on_source_id } diff --git a/lib/laforge/forging.rb b/lib/laforge/forging.rb index 3132655..2541b34 100644 --- a/lib/laforge/forging.rb +++ b/lib/laforge/forging.rb @@ -1,5 +1,5 @@ module LaForge - # Used to forge a laforged record's attributes and tracked associations from its data entries + # `Forging` is used to forge a laforged record's attributes and tracked associations from its data entries module Forging extend ActiveSupport::Concern @@ -48,7 +48,8 @@ def forge_diff(**forge_options) # entries for the whole group. # Entries marked for destruction are left out, so each of their attributes takes the value of the next entry, and an # attribute with no other entry is left out of the hash. - # An attribute of a group is nil when the source that supplies the group has no entry for it. + # A group's attributes all come from one source, so an attribute that source has no entry for is nil rather than + # falling back to another source's value. def forge_attributes(attributes: nil, sources: nil, except_sources: nil) # Each forge selects the ruleset and reads the model-wide and global rules again (see PrecedenceCache) reset_resolver_unless_held @@ -74,15 +75,15 @@ def forge_attributes(attributes: nil, sources: nil, except_sources: nil) end # @api private - # Returns `forge_diff` as it would be once `reported` is recorded on the record, then restores the record's entries - # and rules. `reported` is what its owner's reports say about it, as + # Returns `forge_diff` as it would be once `pending` is recorded on the record, then restores the record's entries + # and rules. `pending` is what its owner's member lists keep pending for it, as # [[source id, attributes, recorded_at, edit, settings], ...]. - def forge_diff_with(reported) + def forge_diff_with(pending) entries = record.data_entries.to_a values = entries.to_h { |entry| [entry, [entry.attributes.slice('value', 'recorded_at', 'expires_at', 'expires_on_source_id'), entry.remade_state, entry.marked_for_destruction?]] } rules = record.precedence_rules.to_a # recording builds, changes and expires rules, which the owner's next save writes settings = rules.to_h { |rule| [rule, [rule.attributes.slice('precedence', 'recorded_at', 'expires_at', 'expires_on_source_id'), rule.remade_state, rule.marked_for_destruction?]] } - reported.each { |source_id, attributes, recorded_at, edit, item_settings| record_reported(attributes, source_id, recorded_at, edit: edit, settings: item_settings) } + pending.each { |source_id, attributes, recorded_at, edit, item_settings| record_pending(attributes, source_id, recorded_at, edit: edit, settings: item_settings) } forge_diff ensure record.data_entries.target.replace(entries) @@ -105,9 +106,9 @@ def pending_writes end # @api private - # Returns the association's entries in precedence order, each paired with its report (see MemberList). Inactive + # Returns the association's entries in precedence order, each paired with its member list (see MemberList). Inactive # sources' entries and expired entries are left out unless `inactive` is true. - def association_reports(config, inactive: false) + def association_member_lists(config, inactive: false) entries = inactive ? filter_data_entries_in_memory(attributes: config.name) : forging_entries(attributes: config.name) by_precedence(entries).map { |data_entry| [data_entry, MemberList.new(data_entry.value, marked: data_entry.marked_for_destruction?)] } end diff --git a/lib/laforge/inspection.rb b/lib/laforge/inspection.rb index 473bd6f..207cafe 100644 --- a/lib/laforge/inspection.rb +++ b/lib/laforge/inspection.rb @@ -1,5 +1,5 @@ module LaForge - # Used to find where a laforged record's values and tracked associations' members came from + # `Inspection` is used to find where a laforged record's values and tracked associations' members came from module Inspection extend ActiveSupport::Concern @@ -34,8 +34,8 @@ def tombstones(name) TrackedAssociation.new(record, self.class.association(name)).tombstones end - # Returns the expiries in the reports of the tracked association `name` (see MemberList). The `expiring` scope (see - # Expiring) finds only rows, so it misses these. + # Returns the expiries in the member lists of the tracked association `name` (see MemberList). The `expiring` scope + # (see Expiring) finds only rows, so it misses these. def membership_expiries(name) TrackedAssociation.new(record, self.class.association(name)).membership_expiries end diff --git a/lib/laforge/member_list.rb b/lib/laforge/member_list.rb index 4ad8077..b1003d5 100644 --- a/lib/laforge/member_list.rb +++ b/lib/laforge/member_list.rb @@ -1,61 +1,52 @@ module LaForge - # Used to read and build the value of a source's `DataEntry` for a tracked association, which is the source's report - # of the association's members. Members are named by key (see AssociationConfig). + # `MemberList` is one source's member list for a tracked association on one owner, read from and written to the + # value of the source's `DataEntry` for the association. Members are named by key hash (see `AssociationConfig`). # - # A report is one of: - # - complete: has an opinion on every member. It includes the members it lists and excludes every other member some - # report names. - # - partial: has an opinion only on the members it includes or excludes, and leaves the rest to other sources. + # It comes in two kinds, each a subclass, which implement the `mode:` option of `record_data_entries`: + # - `MemberList::Complete`, recorded with `mode: :complete`, covers every member. It includes the members + # in its include list, and excludes every other member some member list names. + # - `MemberList::Partial`, recorded with `mode: :partial`, covers only the members it includes or + # excludes, and leaves the rest to other sources. Only a partial list has `forgetting`. + # `mode: :edit` changes the source's member list, of either kind, for the members it names, and starts a partial list + # when the source has none (see `TrackedAssociation#edit_member`). # - # A report holds these lists: - # - include: the members it includes. - # - exclude: the members a partial report excludes. A complete report has none, since it excludes a member by leaving + # A member list names a member when its include, exclude or withdrawn list holds the member (see `listed_keys`). It + # holds: + # - include: the members it includes + # - exclude: the members a partial list excludes. A complete list holds none, since it excludes a member by leaving # it out. - # - withdrawn: the members it included before and no longer names. They stay named until the next forge, which - # removes each one no report includes and drops the source's entries on each one it keeps, then empties the list. - # A row no report names is left alone. - # - pending: what the source said about included members besides their keys, until a forge of the owner includes the - # member and records it there. A member can have several items, recorded oldest first. Each item holds: - # - 'key': the member's key - # - 'attributes': what the source said about the member - # - 'recorded_at': when the source said it - # - 'edit': true for an item from an edit, whose nested lists are then recorded as edits - # - 'precedence', 'expires_at' and 'expires_on_source_id': when present, the settings it was recorded with (see - # RecordingSettings), given to every attribute the item records as recording them directly with those settings - # would. A nil 'precedence' removes the source's precedence for them. - # - tombstones: what each member a partial report excluded was, once a forge has removed its row (see Tombstone). - # Each item is { 'key' => { 'tombstone' => token }, 'attributes' => {...}, 'excluded_at' => time }, and every report - # names the member by that tombstone key from then on. A tombstone is kept until its source forgets the member, - # records a complete report, or withdraws its whole report. - # - expiries: the inclusions and exclusions given an expiry of their own, by an edit or by `LaForge.with_settings`. - # Each item is { 'key' => {...}, 'recorded_at' => time, 'expires_at' => time, 'expires_on_source_id' => id }, and - # 'recorded_at' is when the inclusion or exclusion was made (see Expiring). Once an item has expired the report has - # no opinion on that member, except that a complete report excludes it like any member it does not include, and the - # member's tombstone stops counting (see `counting_tombstones`). An expired item is kept, and so is the member in - # the report's lists. + # - withdrawn: the members it included before and now neither includes nor excludes, and the members an edit + # excluded from a complete list. The next forge withdraws the source from each one membership keeps and removes + # the rest, then empties the withdrawn list. + # - pending: what the source said about each included member besides its key (see `pending_item`), kept until a + # forge includes the member and records it on the member + # - tombstones: what a partial list keeps of each member it excluded, once a forge has removed the member's row (see + # `Tombstone` and `tombstone_item`). Every member list that has not expired names the member by the tombstone's + # key from then on. A tombstone is kept until its source forgets the member, provides a complete list, or is + # withdrawn from the owner (see `withdrawing_all`). + # - expiries: the inclusions and exclusions given an expiry of their own, by an edit or by `LaForge.with_settings` + # (see `expiry_item` and `Expiring`). Once one has expired, a partial list no longer covers that member, and the + # member's pending items and tombstone stop counting (see `counting_pending` and `counting_tombstones`). A complete + # list still excludes the member, as it excludes every member it does not include. The expired item is kept, and + # so is the member in the include, exclude or withdrawn list that holds it. # - # The stored value is: - # - complete: the array of included members, or { 'include' => [...] } with 'withdrawn', 'pending' and 'expiries' when - # any has items - # - partial: { 'partial' => true, 'include' => [...], 'exclude' => [...], 'withdrawn' => [...] }, with 'pending', - # 'tombstones' and 'expiries' when any has items - # - # Each kind is a subclass, `MemberList::Complete` or `MemberList::Partial`, with the same interface. A method that - # changes a report returns the stored value of the changed report, for the caller to write. + # The stored value is built by `Complete.value` and `Partial.value`. A method named for a change, such as `including` + # or `withdrawing_all`, returns the stored value of the changed member list for the caller to write, and leaves this + # member list as it is. class MemberList attr_reader :include, :exclude, :withdrawn, :pending, :tombstones, :expiries - # Reads a report from its stored value, as a `Complete` or a `Partial`. `marked: true` reads the report of an entry - # marked for destruction as a partial report with no opinion on any member, which withdraws every member it included - # or withdrew, so another source's report decides them. + # Reads a member list from its stored value, as a `Complete` or a `Partial`. `marked: true` reads the member list of + # an entry marked for destruction as a partial list that covers no member, which withdraws every member it + # included or withdrew, so another source's member list decides them. def self.new(value, marked: false) return super(value) unless equal?(MemberList) - report = (value.is_a?(Hash) && value['partial'] ? Partial : Complete).new(value) - marked ? Partial.new(Partial.value(withdrawn: report.include + report.withdrawn)) : report + list = (value.is_a?(Hash) && value['partial'] ? Partial : Complete).new(value) + marked ? Partial.new(Partial.value(withdrawn: list.include + list.withdrawn)) : list end - # Returns the report of a source that has none, which is a partial report with no opinion on any member + # Returns the member list of a source that has none, which is a partial list that covers no member def self.none Partial.new(Partial.value) end @@ -65,20 +56,29 @@ def self.expiry_item(key_hash, settings, recorded_at) { 'key' => key_hash, 'recorded_at' => recorded_at.utc.iso8601(6) }.merge(settings.expiry_item) end + # Returns a tombstones item for the member a partial list excluded, whose tombstone key is `key_hash`, holding the + # removed row's `attributes` and `excluded_at` def self.tombstone_item(key_hash, attributes, excluded_at) { 'key' => key_hash, 'attributes' => attributes, 'excluded_at' => excluded_at.utc.iso8601(6) } end - # Returns a pending item for the member with the given key, holding `attributes` as `RecordingSettings.storable` - # stores them and `settings` as `RecordingSettings#to_item` stores them + # Returns a pending item for the member with the key hash `key_hash`, holding: + # - 'key': `key_hash` + # - 'attributes': `attributes`, as `RecordingSettings.storable` stores them + # - 'recorded_at': `recorded_at` + # - 'edit': true for an item from an edit, whose nested member lists are then recorded as edits, and left out + # otherwise + # - 'precedence', 'expires_at' and 'expires_on_source_id': `settings`, as `RecordingSettings#to_item` stores them. + # Every attribute the item records takes them, as recording it directly with those settings would. A nil + # 'precedence' removes the source's precedence for them. def self.pending_item(key_hash, attributes, recorded_at, edit: false, settings: RecordingSettings::DEFAULT) item = { 'key' => key_hash, 'attributes' => RecordingSettings.storable(attributes), 'recorded_at' => recorded_at.utc.iso8601(6) } item['edit'] = true if edit item.merge(settings.to_item) end - # Returns the stored value for this report with each member whose key is in `names` named by the key hash `names` - # maps that key to, in every list, pending item and expiries item. For example, + # Returns the stored value for this member list with each member whose key is in `names` named by the key hash + # `names` maps that key to, in every list, pending item and expiries item. For example, # `{ ['new', '9f2c4e1a7b3d5f60'] => { 'id' => 42 } }` names a new member by the id it was saved with. Members that # end up with the same key are listed once. A tombstone whose key is renamed is dropped, since its member is named # by a row again. @@ -91,20 +91,20 @@ def renamed(config, names) with(include: unique.(include), exclude: unique.(exclude), withdrawn: unique.(withdrawn), pending: pending, tombstones: tombstones, expiries: expiries) end - # Returns the stored value for this report including the member named by the key hash `member`, with `item` as its - # newest pending item. The inclusion takes `expiry`, an expiries item, in place of any the member had, and has no - # expiry of its own when it is nil. + # Returns the stored value for this member list including the member named by the key hash `member`, with `item` as + # its newest pending item. The inclusion takes `expiry`, an expiries item, in place of any the member had, and has + # no expiry of its own when it is nil. def including(config, member, item, expiry: nil) key = config.key_of(member) with(include: others(config, include, key) + [member], exclude: others(config, exclude, key), withdrawn: others(config, withdrawn, key), pending: pending + [item], expiries: expiries_with(config, key, expiry)) end - # Returns the stored value for this report with each expiries item that expires on `source_id` and was made before - # `recorded_at` expiring at `recorded_at`, or nil when no item changes. An item that already expires by then is left - # as it is. An item not among `saved`, the items as the report was last saved, was made now. An item whose key a - # forge has renamed since, as to a tombstone, is still the saved one: it matches a saved item whose key the report - # no longer names in everything but the key. + # Returns the stored value for this member list with each expiries item that expires on `source_id` and was made + # before `recorded_at` expiring at `recorded_at`, or nil when no item changes. An item that already expires by then + # is left as it is. An item not among `saved`, the items as the member list was last saved, was made now. An item + # whose key a forge has renamed since, as to a tombstone, is still the saved one, since it matches a saved item + # whose key the member list no longer names in everything but the key. def expired_on(source_id, recorded_at, saved: []) keys = self.expiries.map { |item| item['key'] } renamed = saved.reject { |item| keys.include?(item['key']) }.map { |item| item.except('key') } @@ -120,17 +120,18 @@ def expired_on(source_id, recorded_at, saved: []) with(expiries: expiries) if changed end - # Returns the stored value for a partial report that withdraws every member this one names, so its source has no - # opinion on any. New members and tombstones are left out, and so are the pending items and tombstones. + # Returns the stored value for a partial list that withdraws every member this one names except new members and + # tombstones, so its source covers none. It holds nothing but the withdrawn list, so pending items, + # tombstones and expiries go. def withdrawing_all(config) - Partial.value(withdrawn: reported_keys(config).reject { |member| config.unsaved?(member) }) + Partial.value(withdrawn: listed_keys(config).reject { |member| config.unsaved?(member) }) end - # Returns the key hashes of the members this report included or withdrew that are not among `members`, except new - # members and tombstones. A report of `members` replacing this one withdraws them. + # Returns the key hashes of the members this member list included or withdrew that are not among `members`, except + # new members and tombstones. A member list of `members` replacing this one withdraws them. def left_out_by(config, members) - reported = members.map { |member| config.key_of(member) } - (include + withdrawn).map { |member| config.key_hash(member) }.uniq.reject { |member| reported.include?(config.key_of(member)) || config.unsaved?(member) } + listed = members.map { |member| config.key_of(member) } + (include + withdrawn).map { |member| config.key_hash(member) }.uniq.reject { |member| listed.include?(config.key_of(member)) || config.unsaved?(member) } end def includes?(config, key) @@ -147,8 +148,8 @@ def counting_tombstones(config) tombstones.reject { |item| expired?(config, config.key_of(item['key'])) } end - # Whether this report's inclusion or exclusion of the member with the given key has an expiry of its own that has - # passed by `at` + # Whether this member list's inclusion or exclusion of the member with the given key has an expiry of its own that + # has passed by `at` def expired?(config, key, at = Time.current) item = expiries.find { |candidate| config.key_of(candidate['key']) == key } item.present? && item['expires_at'].present? && Time.iso8601(item['expires_at']) <= at @@ -158,12 +159,12 @@ def withdrew?(config, key) withdrawn.any? { |member| config.key_of(member) == key } end - def reported_keys(config) + def listed_keys(config) (include + exclude + withdrawn).map { |member| config.key_hash(member) }.uniq end - # Returns the stored value for a report of this kind with the given lists in place of this report's own (see - # `value` on each kind) + # Returns the stored value for a member list of this kind with the given lists in place of this member list's own + # (see `value` on each kind) def with(include: self.include, exclude: self.exclude, withdrawn: self.withdrawn, pending: self.pending, tombstones: self.tombstones, expiries: self.expiries) self.class.value(include: include, exclude: exclude, withdrawn: withdrawn, pending: pending, tombstones: tombstones, expiries: expiries) end @@ -171,13 +172,14 @@ def with(include: self.include, exclude: self.exclude, withdrawn: self.withdrawn private # Returns the withdrawn list without the member named by the key hash `member`, then with it at the end when this - # report includes it by an inclusion that has not expired, or withdrew it + # member list includes it by an inclusion that has not expired, or withdrew it def withdrawn_with(config, member) key = config.key_of(member) others(config, withdrawn, key) + (decision(config, key) == :include || withdrew?(config, key) ? [config.key_hash(member)] : []) end - # Returns the expiries without the member's with the given key, then with `expiry` at the end unless it is nil + # Returns the expiries without the item of the member with the given key, then with `expiry` at the end unless it is + # nil def expiries_with(config, key, expiry) items_without(config, expiries, key) + [expiry].compact end diff --git a/lib/laforge/member_list/complete.rb b/lib/laforge/member_list/complete.rb index da6a146..40c392e 100644 --- a/lib/laforge/member_list/complete.rb +++ b/lib/laforge/member_list/complete.rb @@ -1,8 +1,8 @@ module LaForge class MemberList - # A complete report (see MemberList) + # `Complete` is a complete list (see `MemberList`) class Complete < MemberList - # Returns the value stored for a complete report with the given lists. It has no exclusions or tombstones, so + # Returns the value stored for a complete list with the given lists. It has no exclusions or tombstones, so # those given are left out, and so are pending items whose attributes are empty. def self.value(include: [], exclude: [], withdrawn: [], pending: [], tombstones: [], expiries: []) pending = pending.reject { |item| item['attributes'].empty? } @@ -15,7 +15,7 @@ def self.value(include: [], exclude: [], withdrawn: [], pending: [], tombstones: value end - # Returns the stored value for a complete report of `include` replacing `previous`, withdrawing what `previous` + # Returns the stored value for a complete list of `include` replacing `previous`, withdrawing what `previous` # included or withdrew and this one leaves out (see `MemberList#left_out_by`) def self.replacing(previous, config, include:, pending:, expiries:) value(include: include, withdrawn: previous.left_out_by(config, include), pending: pending, expiries: expiries) @@ -34,20 +34,20 @@ def partial? false end - # Returns :include for a member this report includes by an inclusion that has not expired, and :exclude for every - # other member + # Returns :include for a member this member list includes by an inclusion that has not expired, and :exclude for + # every other member def decision(config, key) includes?(config, key) && !expired?(config, key) ? :include : :exclude end - # Returns false, since an empty complete report still excludes every member + # Returns false, since an empty complete list still excludes every member def vacant? false end - # Returns the stored value for this report leaving out the member named by the key hash `member`, without its - # pending items, and withdrawing it. It is withdrawn even when this report did not include it, so a row no report - # named is removed too. The member's expiry is replaced as `including` replaces it. + # Returns the stored value for this member list leaving out the member named by the key hash `member`, without its + # pending items, and withdrawing it. It is withdrawn even when this member list did not include it, so a row no + # member list named is removed too. The member's expiry is replaced as `including` replaces it. def excluding(config, member, expiry: nil) key = config.key_of(member) with(include: others(config, include, key), withdrawn: others(config, withdrawn, key) + [config.key_hash(member)], pending: items_without(config, pending, key), diff --git a/lib/laforge/member_list/partial.rb b/lib/laforge/member_list/partial.rb index 16c5f40..8402b2d 100644 --- a/lib/laforge/member_list/partial.rb +++ b/lib/laforge/member_list/partial.rb @@ -1,8 +1,8 @@ module LaForge class MemberList - # A partial report (see MemberList) + # `Partial` is a partial list (see `MemberList`) class Partial < MemberList - # Returns the value stored for a partial report with the given lists. Pending items whose attributes are empty are + # Returns the value stored for a partial list with the given lists. Pending items whose attributes are empty are # left out. def self.value(include: [], exclude: [], withdrawn: [], pending: [], tombstones: [], expiries: []) pending = pending.reject { |item| item['attributes'].empty? } @@ -13,12 +13,12 @@ def self.value(include: [], exclude: [], withdrawn: [], pending: [], tombstones: value end - # Returns the stored value for a partial report of `include` and `exclude` replacing `previous`, withdrawing what + # Returns the stored value for a partial list of `include` and `exclude` replacing `previous`, withdrawing what # `previous` included or withdrew and this one does not name (see `MemberList#left_out_by`). It keeps the # tombstones of `previous`, and its exclusions of tombstones this one does not name, with their expiries. def self.replacing(previous, config, include:, exclude:, pending:, expiries:) - reported = (include + exclude).map { |member| config.key_of(member) } - kept = previous.exclude.select { |member| config.tombstone?(member) && !reported.include?(config.key_of(member)) } + listed = (include + exclude).map { |member| config.key_of(member) } + kept = previous.exclude.select { |member| config.tombstone?(member) && !listed.include?(config.key_of(member)) } kept_keys = kept.map { |member| config.key_of(member) } kept_expiries = previous.expiries.select { |item| kept_keys.include?(config.key_of(item['key'])) } value(include: include, exclude: exclude + kept, withdrawn: previous.left_out_by(config, include + exclude), pending: pending, tombstones: previous.tombstones, @@ -38,8 +38,8 @@ def partial? true end - # Returns :include or :exclude for the member with the given key, or nil when this report has no opinion on it. An - # expired inclusion or exclusion is no opinion. + # Returns :include or :exclude for the member with the given key, or nil when this member list doesn't cover it. + # An expired inclusion or exclusion no longer covers it. def decision(config, key) return if expired?(config, key) return :include if includes?(config, key) @@ -48,23 +48,24 @@ def decision(config, key) nil end - # Whether this report includes, excludes and has pending items for no member, so it says nothing once its - # withdrawn list is applied + # Whether this member list includes, excludes and has pending items for no member, so it says nothing once its + # withdrawn list is applied. Tombstones and expiries do not count, since each belongs to an inclusion or + # exclusion. def vacant? include.empty? && exclude.empty? && pending.empty? end - # Returns the stored value for this report with the member named by the key hash `member` added to its exclusions, - # without its pending items. The member's expiry is replaced as `including` replaces it. + # Returns the stored value for this member list with the member named by the key hash `member` added to its + # exclusions, without its pending items. The member's expiry is replaced as `including` replaces it. def excluding(config, member, expiry: nil) key = config.key_of(member) with(include: others(config, include, key), exclude: others(config, exclude, key) + [config.key_hash(member)], withdrawn: others(config, withdrawn, key), pending: items_without(config, pending, key), expiries: expiries_with(config, key, expiry)) end - # Returns the stored value for this report with no opinion on the member named by the key hash `member`. It drops - # the member's pending items, its expiry, and its tombstone when the member is one, and withdraws the member when - # this report included it by an inclusion that has not expired, or withdrew it. + # Returns the stored value for this member list no longer covering the member named by the key hash `member`. It + # drops the member's pending items, its expiry, and its tombstone when the member is one, and withdraws the member + # when this member list included it by an inclusion that has not expired, or withdrew it. def forgetting(config, member) key = config.key_of(member) with(include: others(config, include, key), exclude: others(config, exclude, key), withdrawn: withdrawn_with(config, member), diff --git a/lib/laforge/merge.rb b/lib/laforge/merge.rb index fa5bb9e..076b07b 100644 --- a/lib/laforge/merge.rb +++ b/lib/laforge/merge.rb @@ -1,6 +1,6 @@ module LaForge - # Used to merge what LaForge holds for two records of the same model into the survivor, as when a duplicate record is - # found: + # `Merge` is used to merge what LaForge holds for two records of the same model into the survivor, as when a duplicate + # record is found: # # - The duplicate's attribute entries and per-record precedence rules move to the survivor. When both records have one # for the same attribute and source, the winner's is kept and the other's destroyed, unless the winner's has expired @@ -8,11 +8,12 @@ module LaForge # - In each tracked association, the duplicate's members move to the survivor, keeping their ids and everything # recorded on them. Where both records have a member with the same key, as join rows naming the same record do, the # survivor's row is kept. The duplicate's row is merged into it the same way, then destroyed. Where both have a - # has_one member and their keys differ, or a source's has_one reports on both name members whose keys differ, the - # merge raises and nothing moves. - # - Where a source has a report on both records, the two are combined into the survivor's (see `combined_report`), - # with the expiry of the report that would be kept as an attribute entry is. An expired report adds nothing to one - # that has not expired. + # has_one member and their keys differ, or a source's member lists for the has_one on the two records name members + # whose keys differ, the merge raises and nothing moves. + # - Where a source has a member list on both records, the two are combined into the survivor's (see + # `combined_member_list`). The combined member list takes the expiry of the winner's, unless the winner's has + # expired and the other's has not (see `kept_row`). An expired member list adds nothing to one that has not + # expired. # # The survivor is not forged, so forge it afterwards. A member merged into one of the survivor's is forged and saved # by the merge. The duplicate is left for the application to archive or destroy, with no LaForge data, and @@ -22,37 +23,37 @@ module Merge def run(survivor, duplicate, winner:) assert_mergeable!(survivor, duplicate) - report = { entries_moved: 0, entries_conflicting: [], precedence_rules_moved: 0, precedence_rules_conflicting: [], members_moved: {}, members_merged: {} } + result = { entries_moved: 0, entries_conflicting: [], precedence_rules_moved: 0, precedence_rules_conflicting: [], members_moved: {}, members_merged: {} } - ::ActiveRecord::Base.transaction { merge_records(survivor, duplicate, winner, report) } + ::ActiveRecord::Base.transaction { merge_records(survivor, duplicate, winner, result) } [survivor, duplicate].each(&:reload) - report + result end private - def merge_records(survivor, duplicate, winner, report) + def merge_records(survivor, duplicate, winner, result) assert_has_ones_mergeable!(survivor, duplicate) - merge_entries(survivor, duplicate, winner, report) - merge_precedence_rules(survivor, duplicate, winner, report) - survivor.class.laforge.associations.each_value { |config| merge_members(survivor, duplicate, config, winner, report) } + merge_entries(survivor, duplicate, winner, result) + merge_precedence_rules(survivor, duplicate, winner, result) + survivor.class.laforge.associations.each_value { |config| merge_members(survivor, duplicate, config, winner, result) } end # Moves the duplicate's entries to the survivor. Where both have one for the same attribute and source, keeps the - # winner's attribute entry, or combines the two reports into the survivor's entry. - def merge_entries(survivor, duplicate, winner, report) + # winner's attribute entry, or combines the two member lists into the survivor's entry. + def merge_entries(survivor, duplicate, winner, result) kept = DataEntry.where(record: survivor).index_by { |entry| [entry.attribute_name, entry.source_id] } DataEntry.where(record: duplicate).find_each do |entry| other = kept[[entry.attribute_name, entry.source_id]] if other.nil? entry.update_columns(record_id: survivor.id) - report[:entries_moved] += 1 + result[:entries_moved] += 1 elsif (config = survivor.class.laforge.associations[entry.attribute_name]) winning = kept_row(other, entry, winner) losing = winning.equal?(other) ? entry : other - losing_report = losing.expired? && !winning.expired? ? MemberList.new([]) : MemberList.new(losing.value) - combined = combined_report(config, MemberList.new(winning.value), losing_report) + losing_list = losing.expired? && !winning.expired? ? MemberList.new([]) : MemberList.new(losing.value) + combined = combined_member_list(config, MemberList.new(winning.value), losing_list) assert_one_member!(survivor, duplicate, config, entry.source, combined) unless config.collection? other.update_columns(value: combined, recorded_at: [other.recorded_at, entry.recorded_at].max, expires_at: winning.expires_at, expires_on_source_id: winning.expires_on_source_id) @@ -60,7 +61,7 @@ def merge_entries(survivor, duplicate, winner, report) else keep_survivors = kept_row(other, entry, winner).equal?(other) conflict = { attribute: entry.attribute_name, source: entry.source, kept: keep_survivors ? :survivor : :duplicate } - report[:entries_conflicting] << conflict unless other.value == entry.value + result[:entries_conflicting] << conflict unless other.value == entry.value if keep_survivors entry.destroy! else @@ -71,7 +72,7 @@ def merge_entries(survivor, duplicate, winner, report) end end - def merge_precedence_rules(survivor, duplicate, winner, report) + def merge_precedence_rules(survivor, duplicate, winner, result) type = PrecedenceRule.model_type(survivor.class) kept = PrecedenceRule.where(record_type: type, record_id: survivor.id).index_by { |rule| [rule.attribute_name, rule.source_id] } @@ -79,10 +80,10 @@ def merge_precedence_rules(survivor, duplicate, winner, report) other = kept[[rule.attribute_name, rule.source_id]] if other.nil? rule.update_columns(record_id: survivor.id) - report[:precedence_rules_moved] += 1 + result[:precedence_rules_moved] += 1 else keep_survivors = kept_row(other, rule, winner).equal?(other) - report[:precedence_rules_conflicting] << { attribute: rule.attribute_name, source: rule.source, kept: keep_survivors ? :survivor : :duplicate } unless other.precedence == rule.precedence + result[:precedence_rules_conflicting] << { attribute: rule.attribute_name, source: rule.source, kept: keep_survivors ? :survivor : :duplicate } unless other.precedence == rule.precedence if keep_survivors rule.destroy! else @@ -111,16 +112,16 @@ def assert_has_ones_mergeable!(survivor, duplicate) end end - # Raises when a source's has_one reports on the two records, combined, include more than one member, as when neither - # has been forged yet. Raising rolls back what the merge has moved. + # Raises when a source's member lists for a has_one on the two records, combined, include more than one member, as + # when neither has been forged yet. Raising rolls back what the merge has moved. def assert_one_member!(survivor, duplicate, config, source, combined) keys = MemberList.new(combined).include.map { |member| config.key_of(member) } return unless keys.many? model = survivor.class.name - raise ArgumentError, "Cannot merge #{model} #{duplicate.id} into #{model} #{survivor.id}: #{source.name}'s reports of their #{config.name} name " \ + raise ArgumentError, "Cannot merge #{model} #{duplicate.id} into #{model} #{survivor.id}: #{source.name}'s member lists for their #{config.name} name " \ "#{keys.map { |key| config.describe_key(key) }.join(' and ')}, and #{model}##{config.name} holds one. " \ - 'Remove one of the reports, or give the members the same key so the merge combines them, then merge again' + 'Remove one of the member lists, or give the members the same key so the merge combines them, then merge again' end # Returns the one of the survivor's and the duplicate's rows for the same attribute and source that the merge keeps, @@ -132,31 +133,33 @@ def kept_row(survivors, duplicates, winner) # Moves the duplicate's members of the association to the survivor. A member with the same key as one of the # survivor's is merged into it, then destroyed. Members keyed by primary key always move. - def merge_members(survivor, duplicate, config, winner, report) + def merge_members(survivor, duplicate, config, winner, result) existing = config.primary_keyed? ? {} : config.identified_members_of(survivor) config.members_of(duplicate.reload).each do |member| kept = existing[config.record_key(member)] if kept - merge_records(kept, member, winner, report) + merge_records(kept, member, winner, result) member.reload.destroy! kept.reload.forge! # no forge of the survivor reaches a member it has nothing pending for - (report[:members_merged][config.name] ||= []) << { kept: kept.id, removed: member.id } + (result[:members_merged][config.name] ||= []) << { kept: kept.id, removed: member.id } else member[config.owner_foreign_key] = config.owner_key(survivor) member[config.owner_type_column] = survivor.class.polymorphic_name if config.owner_type_column member.save!(validate: false) - (report[:members_moved][config.name] ||= []) << member.id + (result[:members_moved][config.name] ||= []) << member.id end end end - # Returns one source's two reports for an association combined into one stored value, which is complete when the - # winner's report is. It includes every member either report includes, the winner's first, except a member the - # winner's partial report excludes. A partial result also excludes every member either report excludes that it does - # not include. Withdrawn members, pending items, tombstones and expiries come from both reports, and a member the - # result includes is not withdrawn. A member the winner's report names takes only the winner's expiries item. - def combined_report(config, winning, losing) + # Returns one source's two member lists for an association combined into one stored value, of the winner's kind: + # - include: every member either includes, the winner's first, except one the winner's partial list excludes + # - exclude, for a partial result: every member either excludes that the result does not include + # - withdrawn: both member lists' withdrawn members that the result does not include + # - pending: both member lists' items + # - tombstones, for a partial result: both member lists' tombstones + # - expiries: the winner's, and the other's for members the winner neither includes nor excludes + def combined_member_list(config, winning, losing) by_key = ->(list) { list.uniq { |member| config.key_of(member) } } included = by_key.(winning.include + losing.include.reject { |member| winning.partial? && winning.decision(config, config.key_of(member)) == :exclude }) included_keys = included.map { |member| config.key_of(member) } @@ -191,7 +194,7 @@ def assert_mergeable!(survivor, duplicate) # Merges the duplicate's LaForge data into the survivor (see Merge). The duplicate is the winner, so where both have # an attribute entry or rule for the same attribute and source, the duplicate's wins, as the argument does in - # `Hash#merge`. Returns a report of what moved and what conflicted, a merged member's own entries, rules and members + # `Hash#merge`. Returns a hash of what moved and what conflicted, a merged member's own entries, rules and members # included: # - entries_moved: how many entries moved to the survivor # - entries_conflicting: [{ attribute:, source:, kept: }] for each attribute entry both had with different values diff --git a/lib/laforge/payload.rb b/lib/laforge/payload.rb index 194a548..2410c3c 100644 --- a/lib/laforge/payload.rb +++ b/lib/laforge/payload.rb @@ -1,6 +1,7 @@ module LaForge - # Used to translate the shapes a Rails form submits into the edits `record_data_entries(..., mode: :edit)` takes, at - # every level, so a controller can record its permitted params as they are: + # `Payload` is used to translate the shapes a Rails form submits into the edits + # `record_data_entries(..., mode: :edit)` takes, at every level, so a controller can record its permitted params as + # they are: # # - `tenants_attributes`, an array or a hash keyed by index of member hashes, as `accepts_nested_attributes_for` # takes, becomes `tenants: { include: [...], exclude: [...] }`. A hash with a truthy `_destroy` excludes the member diff --git a/lib/laforge/pending_writes.rb b/lib/laforge/pending_writes.rb index 36e3899..7ad3a9e 100644 --- a/lib/laforge/pending_writes.rb +++ b/lib/laforge/pending_writes.rb @@ -1,9 +1,9 @@ module LaForge - # Used to hold a laforged record's pending writes, and to make them once the record is saved. Pending writes are the - # writes forges of the record have decided on but left for its next save to make: + # `PendingWrites` is used to hold a laforged record's pending writes, and to make them once the record is saved. + # Pending writes are the writes forges of the record have decided on but left for its next save to make: # - removing the members they no longer include, per the association's dependent option # - destroying the entries they withdrew from the record's members - # - naming the members they created by their new ids in the reports + # - naming the members they created by their new ids in the member lists # A member a forge removes stays in the pending writes until the save, so a later forge can include it again. class PendingWrites def initialize(record) @@ -59,7 +59,7 @@ def created_members(config) end # Makes the pending writes, once the record is saved: - # - names the members forges created by their new ids in the reports + # - names the members forges created by their new ids in the member lists # - deletes the entries they withdrew from members, publishing `DataEntry::CHANGED` for each # - removes the members they replaced or dropped, per the association's dependent option, skipping any never saved def write! diff --git a/lib/laforge/precedence.rb b/lib/laforge/precedence.rb index 5d6f2d3..1f8cf6d 100644 --- a/lib/laforge/precedence.rb +++ b/lib/laforge/precedence.rb @@ -1,5 +1,6 @@ module LaForge - # Used to set and read the precedence rules of a laforged record and, in the class methods, of its model + # `Precedence` is used to set and read the precedence rules of a laforged record and, in the class methods, of its + # model module Precedence extend ActiveSupport::Concern @@ -7,8 +8,8 @@ module Precedence CLASS_API = %i[update_attribute_source_precedence update_source_precedence update_precedence_order precedence_order source_precedences].freeze # Set the precedence of a source for one attribute, or one attribute group, of this record, whether or not the - # source has recorded it yet. Saved immediately on a persisted record, otherwise with the record. Pass nil to remove - # it. Raises for an attribute in a group, whose precedence is set on the group, and for a `ruleset:`, since a + # source has an entry for it yet. Saved immediately on a persisted record, otherwise with the record. Pass nil to + # remove it. Raises for an attribute in a group, whose precedence is set on the group, and for a `ruleset:`, since a # record's own rules apply whatever its ruleset. # `expires_at:` and `expires_on:` expire the precedence (see Expiring). Setting it again replaces the expiry, # leaving them out clears it, and passing either with a nil precedence raises. diff --git a/lib/laforge/precedence_cache.rb b/lib/laforge/precedence_cache.rb index ce95b19..3313101 100644 --- a/lib/laforge/precedence_cache.rb +++ b/lib/laforge/precedence_cache.rb @@ -1,7 +1,7 @@ module LaForge - # Used to cache the global and model-wide precedence rules, of the base rules and of each ruleset, which rulesets are - # in use, and which sources are inactive, for the duration of a block, such as an import run, so each forge doesn't - # read them again. A record's own rules are not cached, as they load with the record. + # `PrecedenceCache` is used to cache the global and model-wide precedence rules, of the base rules and of each + # ruleset, which rulesets are in use, and which sources are inactive, for the duration of a block, such as an import + # run, so each forge doesn't read them again. A record's own rules are not cached, as they load with the record. # # Changing a rule, or whether a source is active, through LaForge inside the block clears the cache, so later forges # see the change. A change made outside LaForge, by SQL or by another process, is not seen until the outermost block diff --git a/lib/laforge/precedence_rule.rb b/lib/laforge/precedence_rule.rb index eb17cfa..294c05b 100644 --- a/lib/laforge/precedence_rule.rb +++ b/lib/laforge/precedence_rule.rb @@ -1,5 +1,5 @@ module LaForge - # One rule ranking a source's values at one of four levels, from most to least specific: + # `PrecedenceRule` is one rule ranking a source's values at one of four levels, from most to least specific: # # record_type record_id attribute_name applies to # 'Tenant' 1 'name' one attribute of one record diff --git a/lib/laforge/record.rb b/lib/laforge/record.rb index 3e40fcc..465a3bb 100644 --- a/lib/laforge/record.rb +++ b/lib/laforge/record.rb @@ -1,8 +1,8 @@ module LaForge - # Used to record, forge and rank one laforged record, reached through `record.laforge`. Each laforged model has its - # own subclass, reached through `Model.laforge`, whose class methods act on the model. The model delegates `API` and - # `CLASS_API` to them, so `place.forge!` is `place.laforge.forge!`. The methods marked `@api private` are the - # low-level ones LaForge's own objects call. + # `Record` is used to record, forge and rank one laforged record, reached through `record.laforge`. Each laforged + # model has its own subclass, reached through `Model.laforge`, whose class methods act on the model. The model + # delegates `API` and `CLASS_API` to them, so `place.forge!` is `place.laforge.forge!`. The methods marked + # `@api private` are the low-level ones LaForge's own objects call. class Record include AttributeGroups include Recording diff --git a/lib/laforge/recording.rb b/lib/laforge/recording.rb index 4489ad9..b745165 100644 --- a/lib/laforge/recording.rb +++ b/lib/laforge/recording.rb @@ -1,5 +1,6 @@ module LaForge - # Used to record what sources say about a laforged record, as its data entries and its tracked associations' reports + # `Recording` is used to record what sources say about a laforged record, as its data entries and its tracked + # associations' member lists module Recording extend ActiveSupport::Concern @@ -13,7 +14,7 @@ module Recording # Omitting it keeps an existing precedence, and passing nil removes it. # Pass a record or nil for a belongs_to association, which is recorded as its foreign key, and its type when # polymorphic. - # A tracked association's value is a complete report, and the shapes a Rails form submits raise, as in the default + # A tracked association's value is a complete list, and the shapes a Rails form submits raise, as in the default # mode of `record_data_entries`. # `expires_at:` and `expires_on:` expire the entry, and a precedence passed with it, as for `record_data_entries`. A # value wrapped with `LaForge.with_settings` takes the wrapper's settings. @@ -24,22 +25,24 @@ def record_data_entry(attribute_name, value, source, precedence: NOT_GIVEN, expi end # Record several pieces of information from the same source. - # `mode:` says what kind of report a tracked association's value is (see MemberList): - # - :complete (the default): a complete report, given as the list of members it includes - # - :partial: a partial report, given as the list of members it includes or as { include: [...], exclude: [...] } - # - :edit: changes to the source's report for the members named, given as a list of members to include or as - # { include: [...], exclude: [...], forget: [...] }. The rest of the report is kept, and the members' nested lists - # are edits too. The shapes a Rails form submits, `*_attributes` for a tracked association and `*_ids`, are - # accepted (see Payload). - # A complete or partial report replaces the source's previous one. + # `mode:` says what a tracked association's value does to the source's member list (see MemberList): + # - :complete (the default): replaces it with a complete list, given as the list of members it includes + # - :partial: replaces it with a partial list, given as the list of members it includes or as + # { include: [...], exclude: [...] } + # - :edit: changes it for the members named, given as a list of members to include or as + # { include: [...], exclude: [...], forget: [...] }, and starts a partial list when the source has none. The rest + # of the member list is kept, and the members' nested member lists are edited the same way. A has_one's edit is + # its single member, recorded as a complete list. The shapes a Rails form submits, `*_attributes` for a tracked + # association and `*_ids`, are accepted (see Payload). # A member is a hash of its key and what the source says about it (see TrackedAssociation), or a saved record. # Where members are keyed by primary key, a hash without an id is a new member. # `precedence:` covers everything the call records, members' attributes included (see `record_data_entry`). # The attributes of an attribute group are recorded together, on the record and on members alike. # `expires_at:` and `expires_on:` expire what the call records, and a precedence passed with it (see Expiring). - # Recording again replaces the expiry, and leaving them out clears it. A complete or partial report expires as a - # whole, and what it says about its members is recorded on them with the same expiry. An edit's expiry goes to each - # member it includes or excludes and to a precedence passed with it, and not to the report (see MemberList). + # Recording again replaces the expiry, and leaving them out clears it. With `:complete` or `:partial`, the member + # list expires as a whole, and what it says about its members is recorded on them with the same expiry. With + # `:edit`, the expiry goes to each member the call includes or excludes and to a precedence passed with it, and not + # to the member list (see MemberList). # A part of the payload wrapped with `LaForge.with_settings` takes the wrapper's settings in place of the call's. def record_data_entries(attributes_hash, source, precedence: NOT_GIVEN, mode: :complete, expires_at: nil, expires_on: nil) raise ArgumentError, "Unknown mode #{mode.inspect}; pass :complete, :partial or :edit" unless %i[complete partial edit].include?(mode) @@ -74,8 +77,8 @@ def remove_data_entries(attribute_names: nil, sources: nil) end end - # Destroys the source's attribute entries on this record and withdraws every member its reports in tracked - # associations named, as when the source no longer reports the record. Saved immediately, without forging. + # Destroys the source's attribute entries on this record and withdraws every member its member lists in tracked + # associations named, as when the source no longer provides the record. Saved immediately, without forging. # Precedence rules for the source are kept. def withdraw_source(source) record.transaction do @@ -109,12 +112,13 @@ def record_attribute(attribute_name, value, source, settings: RecordingSettings: end # @api private - # Records what the source's report on the record's owner said about it at `recorded_at`, which the owner validated - # when recording the report. An attribute the record has a newer entry for from the same source is left out, so an - # older report does not undo it. With `edit`, nested lists are recorded as edits. `settings` cover everything - # recorded, nested lists included, except a part wrapped with settings of its own. What this records is made at - # `recorded_at` (see Expiring). Returns whether anything was recorded. - def record_reported(attributes, source, recorded_at, edit: false, settings: RecordingSettings::DEFAULT) + # Records `attributes`, what the source's member list on the record's owner keeps pending for the record, as made at + # `recorded_at`. The owner validated them when recording the member list. An attribute the record has a newer entry + # for from the same source is left out, so an older member list does not undo it. With `edit`, nested member lists + # are recorded as edits. `settings` cover everything recorded, nested member lists included, except a part wrapped + # with settings of its own. What this records is made at `recorded_at` (see Expiring). Returns whether anything was + # recorded. + def record_pending(attributes, source, recorded_at, edit: false, settings: RecordingSettings::DEFAULT) newer = filter_data_entries_in_memory(sources: source).select { |entry| entry.recorded_at && entry.recorded_at > recorded_at }.map(&:attribute_name) attributes = attributes.except(*newer) @@ -136,9 +140,9 @@ def record_reported(attributes, source, recorded_at, edit: false, settings: Reco # @api private # Withdraws the source from the record, a member of `owner`, in memory, as `owner`'s forge does when the source's - # report there excludes or has withdrawn it. The source's attribute entries are dropped, to be destroyed when - # `owner` is saved, and each of its reports in the record's tracked associations withdraws every member it named. - # Returns whether the source had any entry on the record not marked for destruction. + # member list there excludes or has withdrawn it. The source's attribute entries are dropped, to be destroyed when + # `owner` is saved, and each of its member lists in the record's tracked associations withdraws every member it + # named. Returns whether the source had any entry on the record not marked for destruction. def withdraw(source_id, owner) entries = filter_data_entries_in_memory(sources: source_id).reject(&:marked_for_destruction?) entries.each do |data_entry| @@ -173,10 +177,11 @@ def write_data_entry(attribute_name, value, source, settings: RecordingSettings: end # @api private - # Expires `rows`, an entry and a rule a replay of a report made at `recorded_at` has just written, on a recording - # their expiry's source made between the report and the replay. That recording found nothing waiting, since the - # rows did not exist yet. The source's entries for the attribute, attribute group or association show when it last - # recorded them, and the earliest after `recorded_at` counts. A recording that changed nothing leaves no trace. + # Expires `rows`, the entry and rule just written by replaying pending items made at `recorded_at`, on a recording + # from their expiry's source made between the pending items and the replay. That recording found nothing waiting, + # since the rows did not exist yet. The source's entries for the attribute, attribute group or association show + # when they were last recorded, and the earliest after `recorded_at` counts. A recording that changed nothing leaves + # no trace. def expire_replayed(rule_name, recorded_at, rows) attribute_names = attribute_groups.fetch(rule_name) { [rule_name] } rows.compact.filter_map(&:expires_on_source_id).uniq.each do |trigger_id| @@ -209,7 +214,7 @@ def filter_data_entries_in_memory(attributes: nil, sources: nil) # Sets the entry's `recorded_at` and notes the change made again (see Expiring#remade) when this recording creates # the entry or changes its value or expiry, from what is saved or from `before`, the columns as they were in memory. # An expiry on a source that fired in memory and is set back as saved is a change too. A recording replayed from a - # pending report makes it at `recorded_at`. + # member list's pending items makes it at `recorded_at`. def note_recorded(data_entry, recorded_at, before) changed = before != data_entry.attributes.slice(*RECORDED_COLUMNS) || RECORDED_COLUMNS.any? { |column| data_entry.will_save_change_to_attribute?(column) } return unless data_entry.new_record? || changed @@ -222,9 +227,9 @@ def note_recorded(data_entry, recorded_at, before) # the source and was made before `recorded_at`, among: # - the record's entries for it # - the record's own rules for it - # - for a tracked association, the inclusions and exclusions in other sources' reports - # Any recording counts, whether or not it changes a value. One from a report kept pending counts at the time it was - # reported. + # - for a tracked association, the inclusions and exclusions in other sources' member lists + # Any recording counts, whether or not it changes a value. One kept pending in a member list counts at the time it + # was recorded. def expire_on_recording(rule_name, source_id, recorded_at) attribute_names = attribute_groups.fetch(rule_name) { [rule_name] } waiting = filter_data_entries_in_memory(attributes: attribute_names) + record.precedence_rules.select { |rule| rule.attribute_name == rule_name } diff --git a/lib/laforge/recording_settings.rb b/lib/laforge/recording_settings.rb index 0be9678..b806d9f 100644 --- a/lib/laforge/recording_settings.rb +++ b/lib/laforge/recording_settings.rb @@ -4,10 +4,10 @@ module LaForge NOT_GIVEN = Object.new.freeze # Used to carry a recording's settings, the precedence and expiry it gives everything it records, however its payload - # is split. That covers attributes, attribute groups, belongs_to columns, association reports, nested member lists, - # edits, and the pending items reports keep for members until their owner is forged. The settings are built once - # where recording starts, passed down as they are, and applied where an entry is written and where the record's - # precedence rule for it is set. + # is split. That covers attributes, attribute groups, belongs_to columns, member lists at every level, edits, and the + # pending items member lists keep for members until their owner is forged. The settings are + # built once where recording starts, passed down as they are, and applied where an entry is written and where the + # record's precedence rule for it is set. # # A wrapper is a part of a payload given settings of its own with `LaForge.with_settings`. It takes them in place of # the call's, for everything recorded from it (see `for`). A stored wrapper is a wrapper kept in a pending item, as a @@ -40,8 +40,8 @@ def self.from_item(item) # Returns pending `items` (see MemberList) without each 'expires_on_source_id' that names a source deleted since, in # an item and in the stored wrappers in its attributes, so restoring the settings gives no expiry on a source, and - # 'expires_at' is kept. Deleting a source nulls the columns that name it, but not the ids in a report's value, which - # would then fail the foreign key where they are recorded. Looks the ids up in one query, when there are any. + # 'expires_at' is kept. Deleting a source nulls the columns that name it, but not the ids in a member list's value, + # which would then fail the foreign key where they are recorded. Looks the ids up in one query, when there are any. def self.without_deleted_sources(items) ids = items.flat_map { |item| [item['expires_on_source_id'], *stored_source_ids(item['attributes'])] }.compact.uniq return items if ids.empty? @@ -75,19 +75,20 @@ def self.with_stored_settings(value, &block) end # Raises ArgumentError, naming `where`, when anywhere in `value`: - # - a wrapper expires on `source_id`, the source recording, whose next recording replaces what it wraps anyway + # - a wrapper expires on `source_id`, the source of the values. The next values recorded from it replace what it + # wraps anyway. # - a hash has the STORED key, which would read back as a stored wrapper def self.assert_wrappers_valid!(value, source_id, where:) case value when WithSettings if value.settings.expiry && value.settings.expiry[:expires_on_source_id] == source_id - raise ArgumentError, "#{where}: LaForge.with_settings(expires_on:) names the source recording, whose next recording replaces this one and its expiry anyway. " \ - 'Pass the source whose next report should end the change' + raise ArgumentError, "#{where}: LaForge.with_settings(expires_on:) names the source this value comes from, and the next values recorded from it replace it and its expiry anyway. " \ + 'Pass another source, whose next values should end the change' end assert_wrappers_valid!(value.value, source_id, where: where) when Hash if value.key?(STORED) || value.key?(STORED.to_sym) - raise ArgumentError, "#{where}: #{STORED} is a key LaForge reserves for the settings it keeps with a value in a pending report, so a recorded hash cannot use it. " \ + raise ArgumentError, "#{where}: #{STORED} is a key LaForge reserves for the settings it keeps with a value in a member list's pending items, so a recorded hash cannot use it. " \ 'Rename the key, or give the value settings with LaForge.with_settings' end value.each_value { |item| assert_wrappers_valid!(item, source_id, where: where) } @@ -107,8 +108,8 @@ def self.assert_none_in!(value, where:) end # Raises ArgumentError, naming `where`, unless the attributes of each of `model`'s attribute groups in `values`, - # `{ attribute name => value }`, are all wrapped with equal settings or all unwrapped, since a source records a - # group together + # `{ attribute name => value }`, are all wrapped with equal settings or all unwrapped, since a group's values from a + # source are recorded together def self.assert_groups_alike!(model, values, where:) return unless model.respond_to?(:laforge) @@ -117,13 +118,13 @@ def self.assert_groups_alike!(model, values, where:) settings = attributes.select { |attribute| values.key?(attribute) }.map { |attribute| settings_of(values[attribute]) }.uniq next if settings.size < 2 - raise ArgumentError, "#{where}: #{attributes.join(', ')} are attribute group #{group}, which a source records together with the same settings; " \ + raise ArgumentError, "#{where}: #{attributes.join(', ')} are attribute group #{group}, whose values from a source are recorded together with the same settings; " \ 'wrap all of them with the same LaForge.with_settings, or none' end end # Returns `value` with each wrapper and stored wrapper replaced by the value it wraps, at every level. For example, - # `{ 'name' => LaForge.with_settings('Harbour Café', expires_at: time) }` becomes `{ 'name' => 'Harbour Café' }`. + # `{ 'name' => LaForge.with_settings('Harbor Café', expires_at: time) }` becomes `{ 'name' => 'Harbor Café' }`. def self.strip(value) case value when WithSettings then strip(value.value) diff --git a/lib/laforge/tombstone.rb b/lib/laforge/tombstone.rb index d91adeb..869ce7c 100644 --- a/lib/laforge/tombstone.rb +++ b/lib/laforge/tombstone.rb @@ -1,10 +1,10 @@ module LaForge - # A tombstone is what a partial report keeps of a member it excluded, once a forge has removed the member's row. Every - # report names the member by the tombstone's key from then on (see AssociationConfig). The association's tombstone - # matcher compares each new member a source reports with the tombstones, so membership decides a member reported again - # as the excluded member it was. + # `Tombstone` is a tombstone, what a partial list keeps of a member it excluded once a forge has removed the member's + # row. Every member list that has not expired names the member by the tombstone's key from then on (see + # AssociationConfig). The association's tombstone matcher compares each new member a source lists with the tombstones, + # so membership decides a member listed again as the excluded member it was. # - # - `source_id`: the source whose report excluded the member + # - `source_id`: the source whose member list excluded the member # - `record`: an unsaved member built from what the member was, which is what the tombstone matcher is given # - `to_h`: the tombstone's key. Pass it, or the tombstone, to a `forget:` edit from its source to drop the tombstone, # so it no longer keeps the member out. @@ -39,7 +39,7 @@ def to_h # Returns the tombstone that keeps the unsaved member `record` out, or nil. That is the first tombstone, in a tracked # association `record` would join, that the association's tombstone matcher matches and whose source outranks `source` - # there. Code that creates members itself, such as an importer that saves a Tenant rather than reporting it on its + # there. Code that creates members itself, such as an importer that saves a Tenant rather than listing it on its # Place, calls it before saving one. Raises when `record` is saved, or when none of its owners tracks it. def self.tombstone_blocking(record, source) TrackedAssociation.tombstone_blocking(record, source) diff --git a/lib/laforge/tracked_association.rb b/lib/laforge/tracked_association.rb index 9cd549f..8a093dc 100644 --- a/lib/laforge/tracked_association.rb +++ b/lib/laforge/tracked_association.rb @@ -1,23 +1,24 @@ module LaForge - # Used to record, forge and inspect one owner's has_many or has_one association declared with - # `laforged associations:`, described by an AssociationConfig, which also defines how members are named by key. The + # `TrackedAssociation` is used to record, forge and inspect one owner's has_many or has_one association declared with + # `laforged associations:`, described by an `AssociationConfig`, which also defines how members are named by key. The # class methods act on the association across owners. # - # Each source's entry for the association on the owner is its report of the members (see MemberList). Membership is - # whether a member belongs to the association, decided per member by the highest-precedence report with an opinion on - # it. A row no report names is left alone. + # Each source's entry for the association on the owner is its member list (see MemberList). Membership is + # whether a member belongs to the association, decided per member by the highest-precedence member list that covers + # it. A row no member list names is left alone. # # A forge of the owner applies membership. A member it removes goes when the owner is saved, per the association's - # `dependent:` option. What a report says about a member besides its key waits there as pending until a forge includes - # the member, and is then recorded on the member as the source's entries, like attributes recorded on the member - # directly. So nothing is created for a member membership excludes, and a member's attributes are resolved on the - # member alone. The owner's forge forges the members it records on, and any other member is forged on its own. + # `dependent:` option. What a member list says about a member besides its key waits there as pending until a forge + # includes the member, and is then recorded on the member as the source's entries, like attributes recorded on the + # member directly. So nothing is created for a member membership excludes, and a member's attributes are resolved on + # the member alone. The owner's forge forges the members it records on or withdraws a source from, and any other + # member is forged on its own. # - # Withdrawing a source from a member drops the source's entries on the member, and makes each of its reports for the - # member's own tracked associations withdraw every member it named. A forge withdraws each source whose report - # excludes or withdrew a member the forge keeps. + # Withdrawing a source from a member drops the source's entries on the member, and makes each of its member lists for + # the member's own tracked associations withdraw every member it named. A forge withdraws each source whose member + # list excludes or withdrew a member the forge keeps. # - # An instance reads the owner's reports and members when it first needs them, so each operation uses a new one. + # An instance reads the owner's member lists and members when it first needs them, so each operation uses a new one. class TrackedAssociation class << self # Makes the member model laforged if it is not, and autosaves the association so the owner's save writes what a @@ -32,8 +33,8 @@ def declare(owner_class, config) end # Returns { owner id => { source => [member hashes] } } for `records`, each member hash as - # `with_member_attributes` builds it. A source gives the members its report includes, then the other rows it has - # entries on. An owner no source has anything on is left out. + # `with_member_attributes` builds it. A source gives the members its member list includes, then the other rows it + # has entries on. An owner no source has anything on is left out. def source_values(owner_class, config, records) owner_keys = records.to_h { |record| [record.id, config.owner_key(record)] } # members point at the owner key, entries at the id entries = DataEntry.where(record_type: owner_class.base_class.name, record_id: owner_keys.keys, attribute_name: config.name).preload(:source).group_by(&:record_id) @@ -44,8 +45,8 @@ def source_values(owner_class, config, records) rows_by_owner = rows.group_by { |record| record[config.owner_foreign_key] } owner_keys.each_with_object({}) do |(owner_id, owner_key), result| - reports = (entries[owner_id] || []).to_h { |entry| [entry.source, MemberList.new(entry.value)] } - values = values_by_source(config, reports, rows_by_owner[owner_key] || [], member_entries) + member_lists = (entries[owner_id] || []).to_h { |entry| [entry.source, MemberList.new(entry.value)] } + values = values_by_source(config, member_lists, rows_by_owner[owner_key] || [], member_entries) result[owner_id] = values if values.any? end end @@ -74,22 +75,22 @@ def member_rows(owner_class, config, owner_keys) scope.to_a end - # Returns { source => [member hashes] } for one owner, given its `reports` by source, its member `rows`, and + # Returns { source => [member hashes] } for one owner, given its `member_lists` by source, its member `rows`, and # `member_entries` by member id - def values_by_source(config, reports, rows, member_entries) + def values_by_source(config, member_lists, rows, member_entries) rows = rows.reject { |record| config.blank_key?(config.record_key(record)) }.index_by { |record| config.record_key(record) } - sources = (reports.keys + rows.values.flat_map { |record| (member_entries[record.id] || []).map(&:source) }).uniq + sources = (member_lists.keys + rows.values.flat_map { |record| (member_entries[record.id] || []).map(&:source) }).uniq sources.to_h do |source| - listed = listed_members(config, reports[source], rows, member_entries, source) + listed = listed_members(config, member_lists[source], rows, member_entries, source) [source, listed + unlisted_members(config, listed, rows, member_entries, source)] end end - # Returns the member hash of each member `report` includes, or none when the source has no report - def listed_members(config, report, rows, member_entries, source) - (report&.include || []).map do |member| - pending = report.pending.select { |item| config.key_of(item['key']) == config.key_of(member) }.sort_by { |item| item['recorded_at'] } + # Returns the member hash of each member `list` includes, or none when the source has no member list + def listed_members(config, list, rows, member_entries, source) + (list&.include || []).map do |member| + pending = list.pending.select { |item| config.key_of(item['key']) == config.key_of(member) }.sort_by { |item| item['recorded_at'] } with_member_attributes(config, member, rows[config.key_of(member)], member_entries, source, pending) end end @@ -143,11 +144,12 @@ def initialize(owner, config) @config = config end - # Applies a source's edits to its report with `edit_member`, one member at a time, and leaves the rest of the report - # as it is. `value` is what `AssociationConfig#edits` takes. The settings' precedence is set for the source on the - # association, with their expiry, even when there is nothing to edit. Each member edited takes the settings, or its - # wrapper's. A has_one's edit is its single member, recorded as a complete report. `recorded_at` is given for an - # edit replayed from a pending report, whose rule is then made at that time rather than now (see Expiring). + # Applies a source's edits to its member list with `edit_member`, one member at a time, and leaves the rest of the + # member list as it is. `value` is what `AssociationConfig#edits` takes. The settings' precedence is set for the + # source on the association, with their expiry, even when there is nothing to edit. Each member edited takes the + # settings, or its wrapper's. A has_one's edit is its single member, recorded as a complete list. `recorded_at`, + # when given, is when the edit was made, and its precedence rule is made at that time rather than now (see + # Expiring). def edit(value, source, settings: RecordingSettings::DEFAULT, recorded_at: nil) at = recorded_at || Time.current return record(value, source, settings: settings, recorded_at: at) unless config.collection? @@ -159,18 +161,16 @@ def edit(value, source, settings: RecordingSettings::DEFAULT, recorded_at: nil) owner.laforge.expire_replayed(config.name, recorded_at, [rule]) if recorded_at end - # Writes the source's report of `members`, replacing its previous one. `members` is a list of members, or for - # `partial: true` either a list to include or { include: [...], exclude: [...] }. `settings` are the report's. Their - # precedence is set for the source on the association, their expiry is the report entry's and that rule's, and both - # are given to every attribute the report's pending items record. A wrapped member or exclusion takes its wrapper's - # settings instead, and an expiry of its own other than the report's is kept among the report's expiries (see - # MemberList). - # The members the previous report included or withdrew and this one does not name are withdrawn, except new members - # and tombstones. A partial report keeps the previous one's tombstones, and its exclusions of tombstones this one - # does not name, with their expiries. Raises for a partial report of a has_one, and for members AssociationConfig - # rejects. + # Writes the source's member list of `members`, replacing its previous one. `members` is a list of members, or for + # `partial: true` either a list to include or { include: [...], exclude: [...] }. `settings` are the member list's. + # Their precedence is set for the source on the association, their expiry is the member list's entry's and that + # rule's, and both are given to every attribute the member list's pending items record. A wrapped member or + # exclusion takes its wrapper's settings instead, and an expiry of its own other than the member list's is kept + # among the member list's expiries (see MemberList). It replaces the previous member list as + # `MemberList::Complete.replacing` and `MemberList::Partial.replacing` do. Raises for a partial list of a has_one, + # and for members AssociationConfig rejects. def record(members, source, settings: RecordingSettings::DEFAULT, partial: false, recorded_at: Time.current) - raise ArgumentError, "#{owner.class.name}##{config.name} is a has_one, which takes a complete report" if partial && !config.collection? + raise ArgumentError, "#{owner.class.name}##{config.name} is a has_one, which takes a complete list. Record it without mode: :partial" if partial && !config.collection? included, excluded = partial ? partial_members(members, settings) : [normalize_with_settings(members, settings), []] keys = included.map(&:first) @@ -179,32 +179,32 @@ def record(members, source, settings: RecordingSettings::DEFAULT, partial: false expiries = member_expiries(included + excluded, settings, recorded_at) value = - if partial then MemberList::Partial.replacing(report(source), config, include: keys, exclude: excluded.map(&:first), pending: pending, expiries: expiries) - else MemberList::Complete.replacing(report(source), config, include: keys, pending: pending, expiries: expiries) + if partial then MemberList::Partial.replacing(member_list(source), config, include: keys, exclude: excluded.map(&:first), pending: pending, expiries: expiries) + else MemberList::Complete.replacing(member_list(source), config, include: keys, pending: pending, expiries: expiries) end owner.laforge.write_data_entry(config.name, value, source, settings: settings, recorded_at: recorded_at) end - # Changes the source's report for one member. A source with no report, or an expired one, gets a partial one. - # `action` is one of: + # Changes the source's member list for one member. A source with no member list, or an expired one, gets a partial + # one. `action` is one of: # - :include: includes the member, and adds what the edit says about it as a pending item after the earlier ones, so # each edit changes only the attributes it names. Its nested lists are edits too. - # - :exclude: leaves the member out of a complete report, or adds it to a partial report's exclusions - # - :forget: drops the member from a partial report, and the report's tombstone when the member is one. Raises for a - # complete report, which has an opinion on every member. - # Excluding or forgetting a member drops its pending items. Excluding a member from a complete report withdraws it, - # and forgetting one withdraws it when the report included it by an inclusion that has not expired, or withdrew it. - # The member's inclusion or exclusion takes the settings' expiry as its own, in place of any an earlier edit gave - # it, and forgetting the member drops its expiry. The report entry keeps its own expiry, unless it has expired, when - # the new report has none. Raises for an expiry on an exclusion from a complete report, which cannot expire, and for - # a wrapper on a member to forget. + # - :exclude: leaves the member out of a complete list, or adds it to a partial list's exclusions + # - :forget: drops the member from a partial list, and the member list's tombstone when the member is one. Raises + # for a complete list, which covers every member. + # Excluding or forgetting a member drops its pending items, and may withdraw it (see + # `MemberList::Complete#excluding` and `MemberList::Partial#forgetting`). The member's inclusion or exclusion takes + # the settings' expiry as its own, in place of any an earlier edit gave it, and forgetting the member drops its + # expiry. The member list's entry keeps its own expiry, unless it has expired, when the new member list has none. + # Raises for an expiry on an exclusion from a complete list, which cannot expire, for a wrapper on a member to + # forget, and as `AssociationConfig#assert_references_exist!` and `AssociationConfig#assert_excludable!` do. def edit_member(member, source, action, recorded_at: Time.current, settings: RecordingSettings::DEFAULT) raise ArgumentError, "#{owner.class.name}##{config.name}: forgetting a member takes no settings; remove LaForge.with_settings around it" if action == :forget && RecordingSettings.wrapped?(member) (member, attributes, member_settings), = normalize_with_settings([member], settings, attributes: action == :include, edit: true) config.assert_references_exist!(owner, [member]) if action == :include config.assert_excludable!(owner, [member]) if action == :exclude - previous = report(source) + previous = member_list(source) assert_editable!(previous, member, action, member_settings) expiry = MemberList.expiry_item(config.key_hash(member), member_settings, recorded_at) if member_settings.expiring? @@ -217,7 +217,7 @@ def edit_member(member, source, action, recorded_at: Time.current, settings: Rec owner.laforge.write_data_entry(config.name, value, source, settings: edit_entry_settings(source, settings), recorded_at: recorded_at) end - # Expires at `recorded_at` the inclusions and exclusions in other sources' reports on the owner that expire on + # Expires at `recorded_at` the inclusions and exclusions in other sources' member lists on the owner that expire on # `source_id` and were made before `recorded_at` (see MemberList#expired_on) def expire_members_on(source_id, recorded_at) owner.laforge.filter_data_entries_in_memory(attributes: config.name).each do |entry| @@ -229,9 +229,9 @@ def expire_members_on(source_id, recorded_at) end end - # Returns the expiries in every one of the owner's reports for the association, as - # [{ source_id:, key:, expires_at:, expires_on_source_id: }]. Expired items are included, and so are the reports of - # inactive sources, expired reports and reports marked for destruction. + # Returns the expiries in every one of the owner's member lists for the association, as + # [{ source_id:, key:, expires_at:, expires_on_source_id: }]. Expired items are included, and so are the member + # lists of inactive sources, expired member lists and member lists marked for destruction. def membership_expiries owner.laforge.filter_data_entries_in_memory(attributes: config.name).flat_map do |entry| MemberList.new(entry.value).expiries.map do |item| @@ -241,12 +241,12 @@ def membership_expiries end # Applies membership to the owner's members in memory, for the owner's save to write. Does nothing when no source - # has reported the association. + # has a member list for the association. def forge - return if reports.empty? + return if member_lists.empty? include_removed_members - recognise_tombstones(save: true) + recognize_tombstones(save: true) discard_created records = included_records + undecided.map { |key| existing.delete(key) } # one an earlier forge removed stays in the owner's pending writes forge_members(records) @@ -257,17 +257,17 @@ def forge remove_unclaimed(records) end - # Returns what a forge would do to the members, without changing anything, or nil when no source has reported the - # association: + # Returns what a forge would do to the members, without changing anything, or nil when no source has a member + # list for the association: # - added: each member it would build, as its key, or as what the sources say about it for a new member # - removed: the key of each member it would remove # - changed: the key of each member whose pending items would change it, merged with those changes # - skipped: present only when it has members, the key of each one it would not build, because it is named by an id # that is not one of the owner's, or because no source described it and it is invalid with its key alone def diff - return if reports.empty? + return if member_lists.empty? - recognise_tombstones(save: false) + recognize_tombstones(save: false) changed = changed_members added, skipped = included.reject { |member| existing.key?(config.key_of(member)) }.partition { |member| buildable?(member) } @@ -278,9 +278,9 @@ def diff }.tap { |result| result[:skipped] = skipped.map { |member| config.key_hash(member) } if skipped.any? } end - # Names each saved member of `created` by its id in every report of the association that has not expired, in place - # of its placeholder key. Each changed report is saved at once and keeps its recorded_at, since its source said - # nothing new. + # Names each saved member of `created` by its id in every member list of the association that has not expired, in + # place of its placeholder key. Each changed member list is saved at once and keeps its recorded_at, since its + # source said nothing new. def name_created(created) names = created.select { |_, record| record.persisted? }.transform_values { |record| config.key_hash(record) } return if names.empty? @@ -302,18 +302,19 @@ def tombstone_blocking(record, source_id) match_tombstone(record, outranking) end - # Returns the tombstones in the reports a forge reads whose exclusions have not expired, the highest-precedence - # source's first and each token once, so inactive sources' tombstones are left out + # Returns the tombstones whose exclusions have not expired in the member lists a forge reads, which leaves out those + # of inactive sources, expired entries and entries marked for destruction. The highest-precedence source's come + # first, and each token once. def tombstones - reports.reject { |entry, _| entry.marked_for_destruction? } - .flat_map { |entry, report| report.counting_tombstones(config).map { |item| Tombstone.new(owner, config, entry.source_id, item) } } + member_lists.reject { |entry, _| entry.marked_for_destruction? } + .flat_map { |entry, list| list.counting_tombstones(config).map { |item| Tombstone.new(owner, config, entry.source_id, item) } } .uniq(&:token) end # Returns an unsaved member pointing at the owner, with the association scope's attributes and the member model's # columns among `attributes`, except its primary key. def unsaved_member(attributes) - new_member(reported_columns(RecordingSettings.strip(attributes))) + new_member(member_columns(RecordingSettings.strip(attributes))) end def member_sources(member) @@ -322,26 +323,26 @@ def member_sources(member) candidates = record ? record.laforge.forging_entries.reject { |entry| entry.marked_for_destruction? || config.nested_config(entry.attribute_name) } : [] record&.laforge&.reset_resolver_unless_held # a loaded member's selector value may have changed since it last ranked - deciding = reports.find { |_, report| report.decision(config, key) } + deciding = member_lists.find { |_, list| list.decision(config, key) } { membership: (deciding[0].source if deciding && deciding[1].decision(config, key) == :include), - listed_by: reports.select { |_, report| report.decision(config, key) == :include }.map { |entry, _| entry.source }, - excluded_by: reports.select { |_, report| report.partial? && report.decision(config, key) == :exclude }.map { |entry, _| entry.source }, + listed_by: member_lists.select { |_, list| list.decision(config, key) == :include }.map { |entry, _| entry.source }, + excluded_by: member_lists.select { |_, list| list.partial? && list.decision(config, key) == :exclude }.map { |entry, _| entry.source }, attributes: record ? record.laforge.winning_entries(candidates).transform_values { |winner, _| winner.source } : {} } end private - # Returns [included, excluded] for a partial report's `members`, a list to include or + # Returns [included, excluded] for a partial list's `members`, a list to include or # { include: [...], exclude: [...] }, both as `normalize_with_settings` returns them. Raises for other keys in the # hash, and for an exclusion `AssociationConfig#assert_excludable!` rejects. def partial_members(members, settings) lists = members.is_a?(Hash) ? members.stringify_keys : { 'include' => members } unknown = lists.keys - %w[include exclude] if unknown.any? - raise ArgumentError, "#{owner.class.name}##{config.name} partial reports are a list of members, or { include:, exclude: }, not #{unknown.join(', ')}. " \ + raise ArgumentError, "#{owner.class.name}##{config.name}: a partial list is given as a list of members or as { include:, exclude: }, not with #{unknown.join(', ')}. " \ 'To forget a member, record it with mode: :edit' end @@ -377,28 +378,28 @@ def assert_member_precedence!(key_hash, attributes, member_settings, settings) end # Returns an expiries item (see MemberList) for each of `members`, as `normalize_with_settings` returns them, whose - # settings expire otherwise than the report's `settings` + # settings give an expiry other than the member list's `settings` def member_expiries(members, settings, recorded_at) members.filter_map do |key_hash, _, member_settings| MemberList.expiry_item(key_hash, member_settings, recorded_at) if member_settings.expiring? && member_settings.expiry != settings.expiry end end - # Raises when the source's `previous` report cannot take the edit `action` of `member`, with `member_settings`: - # - forgetting a member of a complete report, which has an opinion on every member - # - excluding a member from a complete report with an expiry, since that exclusion cannot expire + # Raises when the source's `previous` member list cannot take the edit `action` of `member`, with `member_settings`: + # - forgetting a member of a complete list, which covers every member + # - excluding a member from a complete list with an expiry, since that exclusion cannot expire def assert_editable!(previous, member, action, member_settings) return if previous.partial? described = config.describe_key(config.key_of(member)) if action == :forget - raise ArgumentError, "#{owner.class.name}##{config.name}: the source's report is a complete list, which has an opinion on every member, so forgetting #{described} " \ - 'would exclude it rather than leave it to other sources. Record the list with mode: :partial first, or exclude the member' + raise ArgumentError, "#{owner.class.name}##{config.name}: the source has a complete list, which covers every member, so forgetting #{described} " \ + 'would exclude it rather than leave it to other sources. Record the member list with mode: :partial first, or exclude the member' end return unless action == :exclude && member_settings.expiring? - raise ArgumentError, "#{owner.class.name}##{config.name}: the source's report is a complete list, so excluding #{described} drops it from the list, " \ - 'which cannot expire. Record the list with mode: :partial, or exclude the member from another source' + raise ArgumentError, "#{owner.class.name}##{config.name}: the source has a complete list, which excludes #{described} by leaving it out, " \ + 'and leaving a member out cannot expire. Record the member list with mode: :partial, or exclude the member from another source' end # Adds the members an earlier forge removed in the owner's pending writes to `existing`, so this forge can include @@ -408,27 +409,27 @@ def include_removed_members end # Names each new member whose pending item the tombstone matcher matches to a tombstone by that tombstone's key in - # the reports, so membership decides it as the excluded member it was. With `save: true` the entries take the - # renamed reports too, to be saved with the owner. Leaves the reports as they are when the association has no - # tombstone matcher or no tombstones. - def recognise_tombstones(save:) + # the member lists, so membership decides it as the excluded member it was. With `save: true` the entries take the + # renamed member lists too, to be saved with the owner. Leaves the member lists as they are when the association has + # no tombstone matcher or no tombstones. + def recognize_tombstones(save:) return unless config.tombstone_matcher known = tombstones return if known.empty? - @reports = reports.map do |entry, report| - next [entry, report] if entry.marked_for_destruction? + @member_lists = member_lists.map do |entry, list| + next [entry, list] if entry.marked_for_destruction? - names = report.pending.each_with_object({}) do |item, result| + names = list.pending.each_with_object({}) do |item, result| next unless config.new_member?(item['key']) found = match_tombstone(unsaved_member(item['attributes']), known) result[config.key_of(item['key'])] = found.key if found end - next [entry, report] if names.empty? + next [entry, list] if names.empty? - value = report.renamed(config, names) + value = list.renamed(config, names) entry.value = value if save [entry, MemberList.new(value)] end @@ -459,7 +460,7 @@ def discard_created end # Returns a record for each member membership includes, in order, and records each one's pending items on it. An - # existing member is taken out of `existing`, and its removal by an earlier forge is cancelled (see `keep`). A member + # existing member is taken out of `existing`, and its removal by an earlier forge is canceled (see `keep`). A member # that does not exist is the one an earlier forge built, or is built now, and is left out when it is skipped. def included_records included.filter_map do |member| @@ -468,7 +469,7 @@ def included_records record = existing.delete(key)&.then { |kept| keep(kept) } || created[key] || add(member, described: items.any?) next unless record - recorded = items.map { |source_id, attributes, recorded_at, edit, settings| record.laforge.record_reported(attributes, source_id, recorded_at, edit: edit, settings: settings) } + recorded = items.map { |source_id, attributes, recorded_at, edit, settings| record.laforge.record_pending(attributes, source_id, recorded_at, edit: edit, settings: settings) } recorded_on << record if recorded.any? record end @@ -489,7 +490,7 @@ def keep(record) def add(member, described:) if config.primary_keyed? && !config.unsaved?(member) owner.logger&.warn("LaForge skipped #{config.klass.name} #{config.key_of(member).first.inspect} in #{owner.class.name} #{owner.id}'s #{config.name}: " \ - "membership includes it, but it is no longer one of them. Report it without its #{config.klass.primary_key} to create a new one") + "membership includes it, but it is no longer one of them. List it without its #{config.klass.primary_key} to create a new one") return end @@ -510,7 +511,7 @@ def build(member, described:) association.target.delete(record) if config.collection? owner.logger&.warn("LaForge skipped the #{config.klass.name} with #{config.describe_key(config.key_of(member))} in #{owner.class.name} #{owner.id}'s #{config.name}: " \ "membership includes it, but it does not exist and is invalid with its key alone (#{record.errors.full_messages.to_sentence}). " \ - 'It is created when a source next reports it with its attributes') + 'It is created when a source next lists it with its attributes') nil end @@ -526,64 +527,64 @@ def forge_members(records) (recorded_on + withdraw_from_members(records)).uniq.each(&:forge) end - # Withdraws each source whose report excludes or withdrew a member from that member, for each saved one of + # Withdraws each source whose member list excludes or withdrew a member from that member, for each saved one of # `records`. Returns the members a withdrawn source had entries on, which need forging. def withdraw_from_members(records) records.select do |record| next false if record.new_record? key = config.record_key(record) - withdrawing = reports.select { |_, report| report.decision(config, key) == :exclude || report.withdrew?(config, key) } + withdrawing = member_lists.select { |_, list| list.decision(config, key) == :exclude || list.withdrew?(config, key) } withdrawing.map { |entry, _| record.laforge.withdraw(entry.source_id, owner) }.any? end end - # Drops from the reports the pending items of the included members, which the forge has recorded on them. A report - # whose inclusion of the member has expired keeps them, as the forge did not record them. Each report keeps its - # recorded_at, since its source said nothing new. + # Drops from the member lists the pending items of the included members, which the forge has recorded on them. A + # member list whose inclusion of the member has expired keeps them, as the forge did not record them. Each member + # list keeps its recorded_at, since its source said nothing new. def clear_pending - reports.each do |entry, report| - remaining = report.pending.reject { |item| included_keys.include?(config.key_of(item['key'])) && !report.expired?(config, config.key_of(item['key'])) } - next if remaining.size == report.pending.size + member_lists.each do |entry, list| + remaining = list.pending.reject { |item| included_keys.include?(config.key_of(item['key'])) && !list.expired?(config, config.key_of(item['key'])) } + next if remaining.size == list.pending.size - entry.value = report.with(pending: remaining) + entry.value = list.with(pending: remaining) end end - # Empties each report's withdrawn list, which the forge has applied. A partial report left with no members and no + # Empties each member list's withdrawn list, which the forge has applied. A partial list left with no members and no # pending items is marked for destruction, to go with the owner's save, whether or not it withdrew any. def prune_withdrawn - reports.each do |entry, _| + member_lists.each do |entry, _| next if entry.marked_for_destruction? - report = MemberList.new(entry.value) - if report.vacant? + list = MemberList.new(entry.value) + if list.vacant? entry.mark_for_destruction - elsif report.withdrawn.any? - entry.value = report.with(withdrawn: []) + elsif list.withdrawn.any? + entry.value = list.with(withdrawn: []) end end end - # Gives a tombstone to each member left in `existing`, which the forge removes, that a partial report excludes by an - # exclusion that has not expired. Every report that has not expired names the member by the tombstone's key in place - # of its id, inactive sources' included so they still name it when their source is active again, and each excluding - # report keeps the tombstone. + # Gives a tombstone to each member left in `existing`, which the forge removes, that a partial list excludes by an + # exclusion that has not expired. Every member list that has not expired names the member by the tombstone's key in + # place of its id. That includes inactive sources' member lists, so they still name it when the source is active + # again. Each excluding member list keeps the tombstone. def entomb - all_reports = owner.laforge.association_reports(config, inactive: true) + all_member_lists = owner.laforge.association_member_lists(config, inactive: true) existing.each_value do |record| key = config.record_key(record) - excluding = reports.reject { |entry, _| entry.marked_for_destruction? }.select { |entry, _| MemberList.new(entry.value).then { |report| report.partial? && report.decision(config, key) == :exclude } } + excluding = member_lists.reject { |entry, _| entry.marked_for_destruction? }.select { |entry, _| MemberList.new(entry.value).then { |list| list.partial? && list.decision(config, key) == :exclude } } next if excluding.empty? tombstone = { AssociationConfig::TOMBSTONE => SecureRandom.hex(8) } item = MemberList.tombstone_item(tombstone, tombstone_attributes(record), Time.current) - all_reports.each do |entry, _| + all_member_lists.each do |entry, _| next if entry.marked_for_destruction? || entry.expired? - report = MemberList.new(MemberList.new(entry.value).renamed(config, { key => tombstone })) - tombstones = excluding.any? { |excluder, _| excluder.equal?(entry) } ? report.tombstones + [item] : report.tombstones - value = report.with(tombstones: tombstones) + list = MemberList.new(MemberList.new(entry.value).renamed(config, { key => tombstone })) + tombstones = excluding.any? { |excluder, _| excluder.equal?(entry) } ? list.tombstones + [item] : list.tombstones + value = list.with(tombstones: tombstones) entry.value = value unless value == entry.value end end @@ -592,7 +593,7 @@ def entomb # Returns the removed member's columns as JSON values, without its primary key, its timestamps and its link to the # owner def tombstone_attributes(record) - reported_columns(record.attributes).except('created_at', 'updated_at').transform_values(&:as_json) + member_columns(record.attributes).except('created_at', 'updated_at').transform_values(&:as_json) end # Removes each member left in `existing`, which the forge does not keep, per the association's `dependent:` option. @@ -633,15 +634,15 @@ def buildable?(member) pending.key?(config.key_of(member)) || new_member(config.build_attributes(member)).valid? end - # Returns the included `member`, which does not exist, as `diff` reports it added + # Returns the included `member`, which does not exist, as `diff` shows it added def added_member(member) return config.key_hash(member) unless config.unsaved?(member) - pending.fetch(config.key_of(member), []).inject({}) { |attributes, (_, reported)| attributes.merge(RecordingSettings.strip(reported)) } + pending.fetch(config.key_of(member), []).inject({}) { |attributes, (_, provided)| attributes.merge(RecordingSettings.strip(provided)) } end # Returns the member model's columns among `attributes`, without its primary key and its link to the owner - def reported_columns(attributes) + def member_columns(attributes) attributes.slice(*config.klass.column_names).except(config.klass.primary_key, config.owner_foreign_key, config.owner_type_column) end @@ -673,35 +674,35 @@ def undecided resolution.last end - # Returns [included, undecided], read from the reports and `existing` when first called: - # - included: the hashes of the members membership includes, the highest-precedence report's first, without those - # whose referenced record does not exist - # - undecided: the keys of the `existing` members a forge leaves as they are, which are those no report names, and - # those no report has an opinion on and none withdrew + # Returns [included, undecided], read from the member lists and `existing` when first called: + # - included: the hashes of the members membership includes, the highest-precedence member list's first, without + # those whose referenced record does not exist + # - undecided: the keys of the `existing` members a forge leaves as they are, which are those no member list names, + # and those no member list covers and none withdrew def resolution @resolution ||= begin - decision = ->(key) { reports.lazy.filter_map { |_, report| report.decision(config, key) }.first } - named = reports.flat_map { |_, report| report.reported_keys(config) }.map { |member| config.key_of(member) }.to_set + decision = ->(key) { member_lists.lazy.filter_map { |_, list| list.decision(config, key) }.first } + named = member_lists.flat_map { |_, list| list.listed_keys(config) }.map { |member| config.key_of(member) }.to_set - candidates = reports.flat_map { |_, report| report.include }.uniq { |member| config.key_of(member) } + candidates = member_lists.flat_map { |_, list| list.include }.uniq { |member| config.key_of(member) } included = config.drop_dangling(candidates.select { |member| decision.(config.key_of(member)) == :include }) included_keys = included.map { |member| config.key_of(member) } undecided = existing.keys.reject do |key| - included_keys.include?(key) || (named.include?(key) && (decision.(key) || reports.any? { |_, report| report.withdrew?(config, key) })) + included_keys.include?(key) || (named.include?(key) && (decision.(key) || member_lists.any? { |_, list| list.withdrew?(config, key) })) end [included, undecided] end end - # Returns the reports' pending items by member key, oldest first so they are recorded in the order the sources - # reported them, as { key => [[source id, attributes, recorded_at, edit, settings], ...] }. The items of a member - # whose inclusion in that report has expired are left out. + # Returns the member lists' pending items by member key, oldest first so they are recorded in the order the sources + # provided them, as { key => [[source id, attributes, recorded_at, edit, settings], ...] }. The items of a member + # whose inclusion in that member list has expired are left out. def pending @pending ||= begin - items = reports.flat_map do |entry, report| - RecordingSettings.without_deleted_sources(report.counting_pending(config)).map do |item| + items = member_lists.flat_map do |entry, list| + RecordingSettings.without_deleted_sources(list.counting_pending(config)).map do |item| [config.key_of(item['key']), [entry.source_id, item['attributes'], Time.iso8601(item['recorded_at']), item['edit'] == true, RecordingSettings.from_item(item)]] end end @@ -720,16 +721,17 @@ def created @created ||= owner.laforge.pending_writes.created_members(config) end - # Returns the source's report, or `MemberList.none` when it has none or its entry has expired. An expired report - # counts as if it did not exist, so one replacing it starts afresh and withdraws nothing. An entry marked for - # destruction gives a report that withdraws what it named, so a report replacing it removes those members. - def report(source) + # Returns the source's member list, or `MemberList.none` when it has none or its entry has expired. An expired + # member list counts as if it did not exist, so one replacing it starts afresh and withdraws nothing. An entry + # marked for destruction gives a member list that withdraws what it named, so a member list replacing it withdraws + # those members. + def member_list(source) entry = source_entry(source) entry && !entry.expired? ? MemberList.new(entry.value, marked: entry.marked_for_destruction?) : MemberList.none end - # Returns `settings` for the source's report entry an edit writes. They keep the entry's own expiry, unless it has - # expired, when the edit starts a new report with no expiry. + # Returns `settings` for the source's entry an edit writes. They keep the entry's own expiry, unless it has + # expired, when the edit starts a new member list with no expiry. def edit_entry_settings(source, settings) settings.with_entry_expiry(source_entry(source)&.expired? ? Expiring::NONE : nil) end @@ -738,9 +740,9 @@ def source_entry(source) owner.laforge.filter_data_entries_in_memory(attributes: config.name, sources: source).first end - # Returns the association's entries in precedence order, each paired with its report, as read when first called - def reports - @reports ||= owner.laforge.association_reports(config) + # Returns the association's entries in precedence order, each paired with its member list, as read when first called + def member_lists + @member_lists ||= owner.laforge.association_member_lists(config) end def association diff --git a/spec/lib/activerecord_spec.rb b/spec/lib/activerecord_spec.rb index c9426dc..9dc7f8d 100644 --- a/spec/lib/activerecord_spec.rb +++ b/spec/lib/activerecord_spec.rb @@ -440,7 +440,7 @@ record.save! end - it 'returns each source that supplied a data_entry, once' do + it 'returns each source with a data entry on the record, once' do expect(record.reload.data_sources).to contain_exactly(data_source, data_source2) end diff --git a/spec/lib/association_capabilities_spec.rb b/spec/lib/association_capabilities_spec.rb index 434375d..d688d65 100644 --- a/spec/lib/association_capabilities_spec.rb +++ b/spec/lib/association_capabilities_spec.rb @@ -1,9 +1,10 @@ require 'spec_helper' -# What association tracking does, one capability per example. Its vocabulary (report, member, membership, tombstone) -# is defined on `LaForge::TrackedAssociation`, `LaForge::MemberList` and `LaForge::AssociationConfig`. +# What association tracking does, one capability per example. Its vocabulary (member list, member, membership, +# tombstone) is in `.ai/language.md`, and is defined in code on `LaForge::MemberList`, `LaForge::TrackedAssociation` +# and `LaForge::Tombstone`. # -# The API is experimental. These names can change while the behaviour stays: +# The API is experimental. These names can change while the behavior stays: # - `laforged associations:` # - association names as keys in `record_data_entries` # - `source_values` @@ -13,14 +14,13 @@ # `noted:` marks an example's line that checks a removed row. The rows removed are: # - owned records (tenants) that membership no longer includes, per the association's `dependent:` option, as # `place.tenants = [...]` would. They are destroyed for `:destroy`, as the spec model's tenants are, deleted for -# `:delete_all`, and have their foreign key nullified otherwise. The entries other sources recorded on them go with -# them. +# `:delete_all`, and have their foreign key nullified otherwise. Other sources' entries on them go with them. # - join rows (place categories) that membership no longer includes, the same way, with the entries recorded on them. # - the join row of a member whose referenced record was deleted, since the forge drops the member (C30). # - the duplicate's join row that `LaForge.merge` merges into the survivor's (C102). # # A record keyed by `key:` columns and destroyed some other way is recreated by the next forge while membership includes -# it. One keyed by primary key is not, and a source reporting it without its id creates a new one. Entries alone never +# it. One keyed by primary key is not, and a source listing it without its id creates a new one. Entries alone never # recreate a record. # # Every example carries a `capability:` id. Retired ids (C09 C10 C11 C23 C35 C48 C66 C67 C76 C82 C83 C84) are not reused. @@ -121,7 +121,7 @@ def count_queries(&block) end describe 'membership' do - it 'takes the member list from the highest-precedence source that recorded one', capability: 'C06' do + it 'takes the member list from the highest-precedence source that has one', capability: 'C06' do place.record_data_entries({ place_categories: categories(cafe, bakery, bookstore) }, low) place.record_data_entries({ place_categories: categories(bakery, bookstore, florist) }, high) place.forge! @@ -129,7 +129,7 @@ def count_queries(&block) expect(categories_of(place)).to contain_exactly(bakery, bookstore, florist) end - it "replaces a source's previous list when that source records again", capability: 'C07' do + it "replaces a source's previous list when a list from that source is recorded again", capability: 'C07' do place.record_data_entries({ place_categories: categories(cafe) }, low) place.record_data_entries({ place_categories: categories(bakery, bookstore, florist) }, high) place.forge! @@ -154,18 +154,18 @@ def count_queries(&block) place.data_entries.with_source(high).with_attribute(:place_categories).destroy_all place.data_entries.reset place.forge! - expect(categories_of(place)).to eq([cafe, bakery, florist]) # no report names florist, so it is left alone, like a row from before tracking + expect(categories_of(place)).to eq([cafe, bakery, florist]) # no member list names florist, so it is left alone, like a row from before tracking end it "creates a join row with its columns when a lower-precedence source's list takes over", capability: 'C46' do - place.record_data_entries({ reported_parcel_links: [parcel_link(lot_14, occupies, active: false)] }, low) - place.record_data_entries({ reported_parcel_links: [parcel_link(lot_12, occupies)] }, high) + place.record_data_entries({ provided_parcel_links: [parcel_link(lot_14, occupies, active: false)] }, low) + place.record_data_entries({ provided_parcel_links: [parcel_link(lot_12, occupies)] }, high) place.forge! expect(parcel_link_row(place, lot_14, occupies)).to be_nil - place.data_entries.with_source(high).with_attribute(:reported_parcel_links).destroy_all + place.data_entries.with_source(high).with_attribute(:provided_parcel_links).destroy_all place.reload.forge! - expect(parcel_link_row(place, lot_14, occupies)).to have_attributes(active: false) # from the attributes low's report kept pending + expect(parcel_link_row(place, lot_14, occupies)).to have_attributes(active: false) # from the attributes low's member list kept pending end it "creates an owned record with the attributes a lower-precedence source's list gave it once that list takes over, and skips one named by an id that no longer exists", capability: 'C47' do @@ -192,7 +192,7 @@ def count_queries(&block) expect(log.string).to include(%(LaForge skipped Tenant #{harbor_books.id} in Place #{place.id}'s tenants: membership includes it, but it is no longer one of them)) end - it 'skips a member named by key alone that is invalid with its key alone, until a source reports it with its attributes', capability: 'C80' do + it 'skips a member named by key alone that is invalid with its key alone, until a source lists it with its attributes', capability: 'C80' do corner_cafe = place.tenants.create!(name: 'Corner Café') corner_cafe.record_data_entries({ opening_hours: [{ day: 1 }] }, low) log = StringIO.new @@ -225,8 +225,8 @@ def count_queries(&block) end end - describe 'partial reports' do - it "keeps every member that partial reports include, the highest-precedence report's members first", capability: 'C49' do + describe 'partial lists' do + it "keeps every member that partial lists include, the highest-precedence list's members first", capability: 'C49' do place.record_data_entries({ place_categories: categories(cafe) }, low, mode: :partial) place.record_data_entries({ place_categories: categories(bakery) }, high, mode: :partial) place.forge! @@ -256,7 +256,7 @@ def count_queries(&block) expect(categories_of(place)).to eq([bakery]) end - it 'lets a complete list exclude what lower partial reports include, but not what higher ones include', capability: 'C52' do + it 'lets a complete list exclude what lower partial lists include, but not what higher ones include', capability: 'C52' do place.record_data_entries({ place_categories: categories(florist) }, low, mode: :partial) place.record_data_entries({ place_categories: categories(cafe) }, high) place.record_data_entries({ place_categories: categories(bookstore) }, top, mode: :partial) @@ -265,7 +265,7 @@ def count_queries(&block) expect(categories_of(place)).to eq([bookstore, cafe]) end - it 'leaves rows that no source has reported alone when only partial reports exist', capability: 'C53' do + it 'leaves alone the rows no source has listed, when only partial lists exist', capability: 'C53' do place.place_categories.create!(category: bookstore) place.record_data_entries({ place_categories: categories(cafe) }, low, mode: :partial) place.forge! @@ -273,7 +273,7 @@ def count_queries(&block) expect(categories_of(place)).to eq([cafe, bookstore]) end - it 'withdraws everything a source reported when it stops reporting the record, keeping what others include', capability: 'C54' do + it 'withdraws everything a source listed when it stops providing the record, keeping what others include', capability: 'C54' do place.record_data_entries({ name: 'From Low', tenants: [tenant('T-1', name: 'Corner Café')] }, low, mode: :partial) place.record_data_entries({ tenants: [tenant('T-2', name: 'Harbor Books')] }, high, mode: :partial) place.forge! @@ -295,7 +295,7 @@ def count_queries(&block) expect(categories_of(place)).to eq([bakery]) end - it 'removes a member a complete report excludes, when excluded twice before a forge or created outside tracking', capability: 'C160' do + it 'removes a member a complete list excludes, when excluded twice before a forge or created outside tracking', capability: 'C160' do place.record_data_entries({ place_categories: categories(cafe, bakery) }, low) place.forge! @@ -309,7 +309,7 @@ def count_queries(&block) expect(categories_of(place)).to eq([bakery]) end - it 'withdraws what a removed complete report included, keeping the members another source includes', capability: 'C161' do + it 'withdraws what a removed complete list included, keeping the members another source includes', capability: 'C161' do place.record_data_entries({ place_categories: categories(cafe, bakery) }, low) place.record_data_entries({ place_categories: categories(cafe, florist) }, high) place.forge! @@ -319,7 +319,7 @@ def count_queries(&block) expect(categories_of(place)).to eq([cafe, bakery]) end - it "withdraws a removed report's members when the source records a new report before the forge", capability: 'C162' do + it "withdraws a removed member list's members when a new member list from the source is recorded before the forge", capability: 'C162' do place.record_data_entries({ place_categories: categories(cafe) }, low) place.forge! @@ -330,7 +330,7 @@ def count_queries(&block) expect(categories_of(place)).to eq([bakery]) end - it 'destroys a partial report left with nothing in it, as when its last exclusion is forgotten', capability: 'C163' do + it 'destroys a partial list left with nothing in it, as when its last exclusion is forgotten', capability: 'C163' do place.record_data_entries({ place_categories: categories(cafe) }, low) place.record_data_entries({ place_categories: { exclude: categories(cafe) } }, top, mode: :edit) place.forge! @@ -346,7 +346,7 @@ def count_queries(&block) expect(place.reload.data_entries.with_source(high)).to be_empty end - it 'adds members one at a time to a report without replacing the rest of it', capability: 'C55' do + it 'includes members one edit at a time, without replacing the rest of the member list', capability: 'C55' do place.record_data_entries({ place_categories: categories(cafe) }, low) place.record_data_entries({ place_categories: { include: [{ category_id: florist.id }] } }, top, mode: :edit) place.record_data_entries({ place_categories: { include: [{ category_id: bakery.id }] } }, top, mode: :edit) @@ -355,7 +355,7 @@ def count_queries(&block) expect(categories_of(place)).to eq([florist, bakery, cafe]) end - it 'includes some members and excludes others in one partial report', capability: 'C75' do + it 'includes some members and excludes others in one partial list', capability: 'C75' do place.record_data_entries({ place_categories: categories(cafe, bakery) }, low) place.record_data_entries({ place_categories: { include: categories(florist), exclude: categories(cafe) } }, high, mode: :partial) place.forge! @@ -363,7 +363,7 @@ def count_queries(&block) expect(categories_of(place)).to eq([florist, bakery]) end - it 'applies edits to only the members they name, at every nested level, keeping the rest of the report', capability: 'C87' do + it 'applies edits to only the members they name, at every nested level, keeping the rest of the member list', capability: 'C87' do place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café', opening_hours: week('08:00', days: [1, 2]))] }, low) place.forge! corner_cafe = place.tenants.sole @@ -387,13 +387,13 @@ def count_queries(&block) expect { place.record_data_entries({ tenants: [] }, top, mode: :edits) }.to raise_error(ArgumentError, /Unknown mode :edits; pass :complete, :partial or :edit/) end - it 'rejects a partial report given as a hash with keys other than include: and exclude:', capability: 'C168' do + it 'rejects a partial list given as a hash with keys other than include: and exclude:', capability: 'C168' do expect { place.record_data_entries({ place_categories: { include: categories(cafe), forget: categories(bakery) } }, low, mode: :partial) } - .to raise_error(ArgumentError, /Place#place_categories partial reports are a list of members, or \{ include:, exclude: \}, not forget. To forget a member, record it with mode: :edit/) - expect { place.record_data_entries({ place_categories: { 'includes' => categories(cafe) } }, low, mode: :partial) }.to raise_error(ArgumentError, /not includes\./) + .to raise_error(ArgumentError, /Place#place_categories: a partial list is given as a list of members or as \{ include:, exclude: \}, not with forget. To forget a member, record it with mode: :edit/) + expect { place.record_data_entries({ place_categories: { 'includes' => categories(cafe) } }, low, mode: :partial) }.to raise_error(ArgumentError, /not with includes\./) end - it 'reports which sources include and which exclude a member', capability: 'C56' do + it 'shows which sources include and which exclude a member', capability: 'C56' do place.record_data_entries({ place_categories: categories(cafe) }, low) place.record_data_entries({ place_categories: { exclude: [{ category_id: cafe.id }] } }, top, mode: :edit) @@ -403,43 +403,43 @@ def count_queries(&block) describe 'member attributes' do it 'applies join model attributes', capability: 'C14' do - place.record_data_entries({ reported_parcel_links: [parcel_link(lot_12, occupies, active: false, external_id: 'CR-1')] }, low) + place.record_data_entries({ provided_parcel_links: [parcel_link(lot_12, occupies, active: false, external_id: 'CR-1')] }, low) place.forge! expect(parcel_link_row(place, lot_12, occupies)).to have_attributes(active: false, external_id: 'CR-1') end it 'resolves each attribute of a member independently, including from sources whose list did not win', capability: 'C15' do - place.record_data_entries({ reported_parcel_links: [parcel_link(lot_12, occupies, active: false, external_id: 'L-1')] }, low) - place.record_data_entries({ reported_parcel_links: [parcel_link(lot_12, occupies, external_id: 'H-1')] }, high) + place.record_data_entries({ provided_parcel_links: [parcel_link(lot_12, occupies, active: false, external_id: 'L-1')] }, low) + place.record_data_entries({ provided_parcel_links: [parcel_link(lot_12, occupies, external_id: 'H-1')] }, high) place.forge! expect(parcel_link_row(place, lot_12, occupies)).to have_attributes(active: false, external_id: 'H-1') end - it 'treats an omitted attribute as no opinion and an explicit nil as a value', capability: 'C16' do - place.record_data_entries({ reported_parcel_links: [parcel_link(lot_12, occupies, active: false, external_id: 'L-1')] }, low) - place.record_data_entries({ reported_parcel_links: [parcel_link(lot_12, occupies, external_id: nil)] }, high) + it 'treats an omitted attribute as not provided and an explicit nil as a value', capability: 'C16' do + place.record_data_entries({ provided_parcel_links: [parcel_link(lot_12, occupies, active: false, external_id: 'L-1')] }, low) + place.record_data_entries({ provided_parcel_links: [parcel_link(lot_12, occupies, external_id: nil)] }, high) place.forge! expect(parcel_link_row(place, lot_12, occupies)).to have_attributes(active: false, external_id: nil) end it 'updates existing join rows in place rather than recreating them', capability: 'C17' do - place.record_data_entries({ reported_parcel_links: [parcel_link(lot_12, occupies)] }, low) + place.record_data_entries({ provided_parcel_links: [parcel_link(lot_12, occupies)] }, low) place.forge! original_id = parcel_link_row(place, lot_12, occupies).id - place.record_data_entries({ reported_parcel_links: [parcel_link(lot_12, occupies, external_id: 'H-1')] }, high) + place.record_data_entries({ provided_parcel_links: [parcel_link(lot_12, occupies, external_id: 'H-1')] }, high) place.forge! expect(parcel_link_row(place, lot_12, occupies).id).to eq(original_id) end it 'identifies 3-way join members by every foreign key other than the owner', capability: 'C18' do - place.record_data_entries({ reported_parcel_links: [parcel_link(lot_12, occupies), parcel_link(lot_12, leases)] }, low) + place.record_data_entries({ provided_parcel_links: [parcel_link(lot_12, occupies), parcel_link(lot_12, leases)] }, low) place.forge! - place.record_data_entries({ reported_parcel_links: [parcel_link(lot_12, occupies, external_id: 'L-1'), parcel_link(lot_12, leases)] }, low) + place.record_data_entries({ provided_parcel_links: [parcel_link(lot_12, occupies, external_id: 'L-1'), parcel_link(lot_12, leases)] }, low) place.forge! expect(place.reload.parcel_links.count).to eq(2) @@ -463,7 +463,7 @@ def count_queries(&block) expect(corner_cafe.reload.name).to eq('Corner Café') end - it "withdraws a source's attributes from a member when its partial report stops including it while another source keeps it", capability: 'C62' do + it "withdraws a source's attributes from a member when its partial list stops including it while another source includes it", capability: 'C62' do place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café', unit: '1A')] }, low) place.forge! place.record_data_entries({ tenants: [tenant('T-1', unit: '2B')] }, high, mode: :partial) @@ -476,10 +476,10 @@ def count_queries(&block) expect(corner_cafe.reload.unit).to eq('1A') expect(corner_cafe.data_source?(high)).to be(false) - expect(place.reload.data_source?(high)).to be(false) # its report, settled and empty, is gone + expect(place.reload.data_source?(high)).to be(false) # the forge emptied its member list and destroyed it end - it "withdraws a source from a member its complete list leaves out while another source keeps it, including the member's own lists", capability: 'C85' do + it "withdraws a source from a member its complete list leaves out while another source includes it, including the member's own member lists", capability: 'C85' do place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café', unit: '1A', opening_hours: week('08:00', days: [1]))] }, low) place.forge! corner_cafe = place.tenants.sole @@ -496,7 +496,7 @@ def count_queries(&block) expect(LaForge::DataEntry.where(record_type: 'OpeningHour')).to be_empty # noted: the opening hour was destroyed per `dependent: :destroy` end - it 'withdraws a source from the members it reported when it is withdrawn from the owner, and drops its settled reports', capability: 'C86' do + it 'withdraws a source from the members it listed when it is withdrawn from the owner, and destroys its member lists the forge empties', capability: 'C86' do place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café', unit: '1A')] }, low) place.forge! corner_cafe = place.tenants.sole @@ -522,7 +522,7 @@ def count_queries(&block) expect(place.place_categories.map(&:position)).to eq([0, 1, 2]) end - it "writes member order over a position recorded on the member, when the owner's report also gives it attributes", capability: 'C165' do + it "writes member order over a position recorded on the member, when the owner's member list also gives it attributes", capability: 'C165' do stub_const('ListingMock', Class.new(ActiveRecord::Base)) ListingMock.table_name = 'active_record_mocks' ListingMock.has_many :children, -> { order(:count) }, class_name: 'ActiveRecordMock', foreign_key: :parent_id @@ -587,7 +587,7 @@ def count_queries(&block) expect(harbor_books.reload.source_precedences.values.flat_map(&:keys)).not_to include(top) # a record says nothing about its attributes end - it 'keeps a custom precedence when the same source records again without one', capability: 'C42' do + it 'keeps a custom precedence when values from the same source are recorded again without one', capability: 'C42' do place.record_data_entries({ place_categories: categories(florist) }, high) place.record_data_entries({ place_categories: categories(cafe) }, low, precedence: 50) place.record_data_entries({ place_categories: categories(cafe, bakery) }, low) @@ -708,7 +708,7 @@ def count_queries(&block) expect(detaching_place.parcel_links.map(&:parcel)).to eq([lot_12]) end - it 'removes a member that an earlier unsaved forge removed, when a forge of the same report follows', capability: 'C124' do + it 'removes a member that an earlier unsaved forge removed, when the owner is forged again with the same member list', capability: 'C124' do place.record_data_entries({ place_categories: categories(cafe, bakery) }, low) place.forge! place.reload.record_data_entries({ place_categories: categories(cafe) }, low) @@ -718,11 +718,11 @@ def count_queries(&block) expect(categories_of(place)).to eq([cafe]) end - it 'keeps a partial report that an earlier unsaved forge emptied, once the source includes a member again', capability: 'C126' do + it 'keeps a partial list that an earlier unsaved forge emptied, once the source includes a member again', capability: 'C126' do place.record_data_entries({ place_categories: categories(cafe) }, low, mode: :partial) place.forge! place.reload.record_data_entries({ place_categories: [] }, low, mode: :partial) - place.forge # withdraws the café and marks the emptied report for destruction, unsaved + place.forge # withdraws the café and marks the emptied member list for destruction, unsaved place.record_data_entries({ place_categories: categories(bakery) }, low, mode: :partial) place.forge! @@ -750,7 +750,7 @@ def count_queries(&block) expect(place.reload.marker).to eq(marker) # the same row, not a replacement end - it 'creates owned records and updates the same row when a source reports the same key again', capability: 'C20' do + it 'creates owned records and updates the same row when a source lists the same key again', capability: 'C20' do place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café')] }, low) place.forge! tenant_id = place.reload.tenants.first.id @@ -792,7 +792,7 @@ def count_queries(&block) expect(corner_cafe.reload.name).to eq('Edited') end - it "reports edits made directly on an owned record in the owner's per-source values", capability: 'C25' do + it "shows edits made directly on an owned record in the owner's per-source values", capability: 'C25' do place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café')] }, low) place.forge! @@ -806,7 +806,7 @@ def count_queries(&block) ) end - it "counts a source among an owned record's sources only when it records on the record, while the owner reports every source that lists it", capability: 'C68' do + it "counts a source among an owned record's sources only when it has entries on the record, while the owner shows every source that lists it", capability: 'C68' do place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café')] }, low) place.forge! corner_cafe = place.tenants.first @@ -817,13 +817,13 @@ def count_queries(&block) expect(corner_cafe.data_source?(top)).to be(false) expect(place.reload.member_sources(:tenants, id: corner_cafe.id)).to eq(membership: top, listed_by: [top, low], excluded_by: [], attributes: { 'external_id' => low, 'name' => low }) - # As an orphan check for low sees it, no other source records on the tenant, but top still lists it + # As an orphan check for low sees it, no other source has entries on the tenant, but top still lists it corner_cafe.withdraw_source(low) expect(corner_cafe.data_entries.without_source(low)).to be_empty expect(place.member_sources(:tenants, id: corner_cafe.id)[:listed_by] - [low]).to eq([top]) end - it 'leaves rows no report has named alone, even when a complete list leaves them out, and removes a member a complete list stops naming', capability: 'C69' do + it 'leaves alone the rows no member list has named, even when a complete list leaves them out, and removes a member a complete list stops naming', capability: 'C69' do added_by_hand = place.tenants.create!(name: 'Added By Hand') # as rows from before the association was tracked are place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café'), tenant('T-2', name: 'Harbor Books')] }, low) expect(place.forge_diff['tenants']).to include(removed: []) @@ -839,7 +839,7 @@ def count_queries(&block) place.forge! place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café')] }, low) place.forge! - expect(place.reload.tenants.pluck(:name)).to eq(['Corner Café']) # once a report names a row, a complete list can remove it + expect(place.reload.tenants.pluck(:name)).to eq(['Corner Café']) # once a member list names a row, a complete list can remove it end it "records a member's attributes on the member as the source's entries, keeping only its key in the list", capability: 'C63' do @@ -887,13 +887,13 @@ def count_queries(&block) expect { place.record_data_entries({ tenants: [Marker.create!(label: 'Pin')] }, low) }.to raise_error(LaForge::InvalidValue, /must be Tenant records/) end - it 'keeps member attributes in the report until a forge records them, so the owner can be saved and forged later', capability: 'C72' do + it 'keeps member attributes in the member list until a forge records them, so the owner can be saved and forged later', capability: 'C72' do place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café')] }, low) place.save! - report = place.data_entries.with_source(low).with_attribute(:tenants).sole.value - expect(report).to match( + member_list = place.data_entries.with_source(low).with_attribute(:tenants).sole.value + expect(member_list).to match( 'include' => [{ 'new' => String }], # a placeholder until the tenant exists - 'pending' => [{ 'key' => { 'new' => report['include'].first['new'] }, 'attributes' => { 'external_id' => 'T-1', 'name' => 'Corner Café' }, 'recorded_at' => String }] + 'pending' => [{ 'key' => { 'new' => member_list['include'].first['new'] }, 'attributes' => { 'external_id' => 'T-1', 'name' => 'Corner Café' }, 'recorded_at' => String }] ) place.reload.forge! @@ -923,7 +923,7 @@ def count_queries(&block) expect { place.record_data_entries({ tenants: [{ id: 0 }] }, low) }.to raise_error(LaForge::InvalidValue, /Tenant 0 is not one of/) end - it 'keeps what a report says about a member while membership excludes it, until the source reports it again or is withdrawn', capability: 'C78' do + it 'keeps what a member list says about a member while membership excludes it, until the source lists it again or is withdrawn', capability: 'C78' do harbor_books = place.tenants.create!(external_id: 'T-2', name: 'Harbor Books') place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café'), tenant('T-2', name: 'Harbor Books')] }, low) place.record_data_entries({ tenants: [tenant('T-2')] }, high) @@ -935,7 +935,7 @@ def count_queries(&block) place.save! expect(LaForge::MemberList.new(place.reload.data_entries.with_source(high).with_attribute(:tenants).sole.value).pending.map { |item| item['attributes'] }).to eq([{ 'name' => 'Harbor Books Ltd' }]) - place.record_data_entries({ tenants: [{ id: harbor_books.id }] }, low) # its next report no longer lists T-1 + place.record_data_entries({ tenants: [{ id: harbor_books.id }] }, low) # its next member list no longer includes T-1 place.save! expect(pending.()).to be_empty @@ -945,7 +945,7 @@ def count_queries(&block) expect(pending.()).to be_empty end - it 'does not let what an older report says about a member undo a newer value from the same source', capability: 'C79' do + it 'does not let what an older member list says about a member undo a newer value from the same source', capability: 'C79' do corner_cafe = travel_to(Time.utc(2025, 12, 1)) { place.tenants.create!(name: 'Corner Café') } travel_to(Time.utc(2026, 1, 1)) do # recorded, and the place saved without forging @@ -986,8 +986,8 @@ def count_queries(&block) place.forge! expect { place.reload.record_data_entries({ place_categories: { forget: categories(cafe) } }, high, mode: :edit) } - .to raise_error(ArgumentError, /Place#place_categories: the source's report is a complete list, which has an opinion on every member, so forgetting category_id #{cafe.id} would exclude it rather than leave it to other sources. Record the list with mode: :partial first, or exclude the member/) - place.record_data_entries({ place_categories: { forget: categories(cafe) } }, top, mode: :edit) # a source with no report, or a partial one, can + .to raise_error(ArgumentError, /Place#place_categories: the source has a complete list, which covers every member, so forgetting category_id #{cafe.id} would exclude it rather than leave it to other sources. Record the member list with mode: :partial first, or exclude the member/) + place.record_data_entries({ place_categories: { forget: categories(cafe) } }, top, mode: :edit) # a source with no member list, or a partial one, can end end @@ -1123,7 +1123,7 @@ def form_place_model(**options, &body) expect { place.record_data_entry(:tenants_attributes, [{ id: corner_cafe.id }], top) }.to raise_error(ArgumentError, /record it with mode: :edit/) expect(place.data_entries).to be_empty - place.record_data_entry(:tenants, [{ name: 'Harbor Books' }], top) # a tracked association is recorded as a report + place.record_data_entry(:tenants, [{ name: 'Harbor Books' }], top) # a tracked association is recorded as a member list place.forge! expect(place.reload.tenants.pluck(:name)).to contain_exactly('Corner Café', 'Harbor Books') end @@ -1137,7 +1137,7 @@ def exclude_corner_cafe place.forge! end - it 'destroys an excluded member and keeps a tombstone that keeps it out when a lower source reports it again', capability: 'C92' do + it 'destroys an excluded member and keeps a tombstone that keeps it out when a lower source lists it again', capability: 'C92' do exclude_corner_cafe expect(Tenant.pluck(:name)).to eq(['Harbor Books']) tombstone = place.reload.data_entries.tombstones(:tenants).sole @@ -1150,7 +1150,7 @@ def exclude_corner_cafe expect(place.reload.tenants.pluck(:name)).to eq(['Harbor Books']) end - it 'keeps a tombstone through the excluding source\'s later partial reports, and drops it when the source is withdrawn', capability: 'C93' do + it 'keeps a tombstone through the excluding source\'s later partial lists, and drops it when the source is withdrawn', capability: 'C93' do exclude_corner_cafe place.reload.record_data_entries({ tenants: [tenant('T-2')] }, top, mode: :partial) place.forge! @@ -1218,8 +1218,8 @@ def exclude_corner_cafe copy.forge! { copy.record_data_entries({ name: 'Harbor Bldg' }, low) } end - report = LaForge.merge(place, duplicate) - expect(report).to include(entries_moved: 0, entries_conflicting: [{ attribute: 'name', source: low, kept: :duplicate }]) + result = LaForge.merge(place, duplicate) + expect(result).to include(entries_moved: 0, entries_conflicting: [{ attribute: 'name', source: low, kept: :duplicate }]) expect(place.data_entries.sole.value).to eq('Harbor Bldg') expect(duplicate.data_entries).to be_empty # left for the application to archive @@ -1232,13 +1232,13 @@ def exclude_corner_cafe duplicate.update_attribute_source_precedence(:name, low, 50) duplicate.update_attribute_source_precedence(:name, high, 5) - report = LaForge.merge(place, duplicate) + result = LaForge.merge(place, duplicate) - expect(report).to include(precedence_rules_moved: 1, precedence_rules_conflicting: [{ attribute: 'name', source: low, kept: :duplicate }]) + expect(result).to include(precedence_rules_moved: 1, precedence_rules_conflicting: [{ attribute: 'name', source: low, kept: :duplicate }]) expect(place.source_precedences['name']).to eq(low => 50, high => 5) end - it 'moves owned members with their ids and what was recorded on them, combining each source\'s reports and tombstones', capability: 'C101' do + it 'moves owned members with their ids and what was recorded on them, combining each source\'s member lists and tombstones', capability: 'C101' do place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café')] }, low) place.forge! duplicate.record_data_entries({ tenants: [tenant('T-2', name: 'Harbor Books', opening_hours: week('08:00', days: [1])), { name: 'Florist' }] }, low) @@ -1247,11 +1247,11 @@ def exclude_corner_cafe duplicate.record_data_entries({ tenants: { exclude: [{ id: duplicate.tenants.find_by!(name: 'Florist').id }] } }, top, mode: :edit) duplicate.forge! - report = LaForge.merge(place, duplicate) + result = LaForge.merge(place, duplicate) - expect(report[:members_moved]).to eq('tenants' => [harbor_books.id]) - low_report = LaForge::MemberList.new(place.data_entries.with_source(low).with_attribute(:tenants).sole.value) - expect(low_report.include.filter_map { |member| member['id'] }).to contain_exactly(place.tenants.find_by!(name: 'Corner Café').id, harbor_books.id) + expect(result[:members_moved]).to eq('tenants' => [harbor_books.id]) + low_list = LaForge::MemberList.new(place.data_entries.with_source(low).with_attribute(:tenants).sole.value) + expect(low_list.include.filter_map { |member| member['id'] }).to contain_exactly(place.tenants.find_by!(name: 'Corner Café').id, harbor_books.id) place.forge! expect(place.reload.tenants.pluck(:name)).to contain_exactly('Corner Café', 'Harbor Books') expect(harbor_books.reload.opening_hours.pluck(:day)).to eq([1]) @@ -1263,17 +1263,17 @@ def exclude_corner_cafe end it 'merges a join row both records have into the survivor\'s, and moves the rest', capability: 'C102' do - place.record_data_entries({ reported_parcel_links: [parcel_link(lot_12, occupies, external_id: 'P-1')] }, low) + place.record_data_entries({ provided_parcel_links: [parcel_link(lot_12, occupies, external_id: 'P-1')] }, low) place.forge! - duplicate.record_data_entries({ reported_parcel_links: [parcel_link(lot_12, occupies, external_id: 'D-1'), parcel_link(lot_14, leases)] }, low) + duplicate.record_data_entries({ provided_parcel_links: [parcel_link(lot_12, occupies, external_id: 'D-1'), parcel_link(lot_14, leases)] }, low) duplicate.forge! kept = parcel_link_row(place, lot_12, occupies) removed = parcel_link_row(duplicate, lot_12, occupies) - report = LaForge.merge(place, duplicate) + result = LaForge.merge(place, duplicate) place.forge! - expect(report[:members_merged]).to eq('reported_parcel_links' => [{ kept: kept.id, removed: removed.id }]) + expect(result[:members_merged]).to eq('provided_parcel_links' => [{ kept: kept.id, removed: removed.id }]) expect(place.reload.parcel_links.map { |row| [row.parcel, row.parcel_link_type, row.external_id] }).to contain_exactly([lot_12, occupies, 'D-1'], [lot_14, leases, nil]) expect(ParcelLink.exists?(removed.id)).to be(false) # noted: the duplicate's row is destroyed once merged expect(duplicate.reload.parcel_links).to be_empty @@ -1294,7 +1294,7 @@ def exclude_corner_cafe expect(Marker.where(markable_type: 'LabelledPlace', markable_id: survivor.id).count).to eq(1) end - it "raises before moving anything when a source's unforged has_one reports on both records name different members", capability: 'C158' do + it "raises before moving anything when a source's unforged member lists for a has_one on both records name different members", capability: 'C158' do stub_const('LabelledPlace', Class.new(ActiveRecord::Base)) LabelledPlace.table_name = 'places' LabelledPlace.has_one :marker, as: :markable, dependent: :destroy @@ -1305,7 +1305,7 @@ def exclude_corner_cafe copy.record_data_entries({ marker: { label: 'Quay' } }, low) copy.save! - expect { LaForge.merge(survivor, copy) }.to raise_error(ArgumentError, /Cannot merge LabelledPlace #{copy.id} into LabelledPlace #{survivor.id}: low's reports of their marker name label "Quay" and label "Harbor", and LabelledPlace#marker holds one. Remove one of the reports, or give the members the same key so the merge combines them, then merge again/) + expect { LaForge.merge(survivor, copy) }.to raise_error(ArgumentError, /Cannot merge LabelledPlace #{copy.id} into LabelledPlace #{survivor.id}: low's member lists for their marker name label "Quay" and label "Harbor", and LabelledPlace#marker holds one. Remove one of the member lists, or give the members the same key so the merge combines them, then merge again/) expect(copy.data_entries.reload.count).to eq(1) expect(survivor.data_entries.reload.count).to eq(1) end @@ -1339,7 +1339,7 @@ def exclude_corner_cafe describe 'attribute groups and inactive sources on members' do it 'checks and resolves an attribute group on members as on the record', capability: 'C104' do expect { place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café', street: '1 Quay')] }, low) } - .to raise_error(ArgumentError, /Tenant members of Place#tenants: street, postcode are attribute group postal_address, which a source records together \(missing: postcode\)/) + .to raise_error(ArgumentError, /Tenant members of Place#tenants: street, postcode are attribute group postal_address, which a source provides together \(missing: postcode\)/) place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café', street: '1 Quay', postcode: 'H1')] }, low) place.forge! @@ -1406,8 +1406,8 @@ def exclude_corner_cafe end end - describe 'expiring reports' do - it 'leaves an expired report out of the next forge, so a member only it named stays, as a row no report names', capability: 'C181' do + describe 'expiring member lists' do + it 'leaves an expired member list out of the next forge, so a member only it named stays, as a row no member list names', capability: 'C181' do place.record_data_entries({ place_categories: categories(cafe) }, low) place.record_data_entries({ place_categories: categories(bakery) }, high, mode: :partial, expires_at: 1.week.from_now) place.forge! @@ -1422,9 +1422,9 @@ def exclude_corner_cafe expect(place.data_entries.with_source(high).sole.value).to include('include' => categories(bakery).map(&:stringify_keys)) end - it 'removes a member another report excludes once the expired report no longer keeps it in', capability: 'C182' do + it 'removes a member another member list excludes once the expired one no longer includes it', capability: 'C182' do place.record_data_entries({ place_categories: categories(bakery) }, low, mode: :partial) - place.record_data_entries({ place_categories: categories(cafe) }, high) # a complete report, which excludes the bakery + place.record_data_entries({ place_categories: categories(cafe) }, high) # a complete list, which excludes the bakery place.record_data_entries({ place_categories: categories(bakery) }, top, mode: :partial, expires_at: 1.week.from_now) place.forge! expect(categories_of(place)).to contain_exactly(cafe, bakery) @@ -1433,7 +1433,7 @@ def exclude_corner_cafe place.reload.forge! expect(categories_of(place)).to eq([cafe]) - place.reload.record_data_entries({ place_categories: categories(florist) }, top, mode: :edit) # an edit to an expired report starts afresh + place.reload.record_data_entries({ place_categories: categories(florist) }, top, mode: :edit) # an edit to an expired member list starts afresh place.forge! end @@ -1460,7 +1460,7 @@ def exclude_corner_cafe end end - it 'expires a report the next time the source it expires on reports the association, whatever it reports', capability: 'C184' do + it 'expires a member list the next time the association is recorded from the source it expires on, whatever that source provides', capability: 'C184' do place.record_data_entries({ place_categories: categories(bakery) }, low, mode: :partial) place.record_data_entries({ place_categories: categories(cafe) }, high) place.record_data_entries({ place_categories: categories(bakery) }, top, mode: :partial, expires_on: high) @@ -1476,7 +1476,7 @@ def exclude_corner_cafe expect(place.data_entries.with_source(top).sole).to have_attributes(expires_at: be_within(1.minute).of(Time.current), expires_on_source: high) end - it 'records what an expiring report says about members with its expiry, so their attributes fall back with it', capability: 'C129' do + it 'records what an expiring member list says about members with its expiry, so their attributes fall back with it', capability: 'C129' do place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café')] }, low) place.forge! place.reload.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café & Bakery')] }, high, mode: :partial, expires_at: 1.week.from_now) @@ -1503,17 +1503,17 @@ def exclude_corner_cafe expect(name).to have_attributes(value: 'Corner Café & Bakery', expires_at: be_within(1.second).of(expires_at)) expect(Tenant.sole.precedence_rules.find_by!(attribute_name: 'name')).to have_attributes(source_id: high.id, precedence: 5, expires_at: be_within(1.second).of(expires_at)) expect(place.precedence_rules.find_by!(attribute_name: 'tenants')).to have_attributes(source_id: high.id, precedence: 5, expires_at: be_within(1.second).of(expires_at)) - expect(place.data_entries.with_source(high).sole.expires_at).to be_nil # the report itself does not expire + expect(place.data_entries.with_source(high).sole.expires_at).to be_nil # the member list itself does not expire place.reload.record_data_entries({ place_categories: { include: [] } }, top, mode: :edit, precedence: 7, expires_on: low) expect(place.source_precedences['place_categories']).to eq(top => 7) expect(place.precedence_rules.find { |rule| rule.attribute_name == 'place_categories' }.expires_on_source_id).to eq(low.id) end - it "gives a complete or partial report's precedence and expiry to the rules it sets on members", capability: 'C132' do + it "gives a complete or partial list's precedence and expiry to the rules it sets on members", capability: 'C132' do expires_at = 1.week.from_now place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café')] }, low, precedence: 5, expires_at: expires_at) - place.record_data_entries({ reported_parcel_links: [parcel_link(lot_12, occupies, external_id: 'P-1')] }, high, mode: :partial, precedence: 6, expires_at: expires_at) + place.record_data_entries({ provided_parcel_links: [parcel_link(lot_12, occupies, external_id: 'P-1')] }, high, mode: :partial, precedence: 6, expires_at: expires_at) place.forge! expect(Tenant.sole.precedence_rules.find_by!(attribute_name: 'name')).to have_attributes(source_id: low.id, precedence: 5, expires_at: be_within(1.second).of(expires_at)) @@ -1521,12 +1521,12 @@ def exclude_corner_cafe .to have_attributes(source_id: high.id, precedence: 6, expires_at: be_within(1.second).of(expires_at)) end - it 'expires a change on a source at a recording made after it, kept pending or not, but not at a report the source made before it', capability: 'C133' do + it 'expires a change on a source at a recording from it made after the change, kept pending or not, but not at one made before', capability: 'C133' do renamed = -> { Tenant.sole.data_entries.with_source(top).sole } travel_to(Time.utc(2026, 1, 1)) { place.forge! { place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café')] }, low) } } travel_to(Time.utc(2026, 2, 1)) { place.reload.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Cafe')] }, low) && place.save! } # kept pending travel_to(Time.utc(2026, 3, 1)) { Tenant.sole.then { |corner_cafe| corner_cafe.forge! { corner_cafe.record_data_entries({ name: 'Corner Café (renamed)' }, top, expires_on: low) } } } - travel_to(Time.utc(2026, 4, 1)) { place.reload.forge! } # records the pending report on the tenant + travel_to(Time.utc(2026, 4, 1)) { place.reload.forge! } # records the member list's pending items on the tenant expect(Tenant.sole.reload.name).to eq('Corner Café (renamed)') expect(renamed.call.expires_at).to be_nil @@ -1538,19 +1538,19 @@ def exclude_corner_cafe expect(renamed.call).to have_attributes(expires_at: Time.utc(2026, 5, 1), expires_on_source: low) end - it 'keeps the unexpired entry when merging, and adds nothing from an expired report', capability: 'C130' do + it 'keeps the unexpired entry when merging, and adds nothing from an expired member list', capability: 'C130' do duplicate = Place.create!(name: 'Harbor Bldg') place.forge! { place.record_data_entries({ name: 'Harbor Building', place_categories: categories(cafe) }, low) } duplicate.forge! { duplicate.record_data_entries({ name: 'Harbor Bldg', place_categories: categories(bakery) }, low, expires_at: 1.week.from_now) } travel 8.days do - report = LaForge.merge(place, duplicate) - expect(report[:entries_conflicting]).to eq([{ attribute: 'name', source: low, kept: :survivor }]) + result = LaForge.merge(place, duplicate) + expect(result[:entries_conflicting]).to eq([{ attribute: 'name', source: low, kept: :survivor }]) place.forge! end expect(place.reload.name).to eq('Harbor Building') - expect(categories_of(place)).to contain_exactly(cafe, bakery) # the duplicate's row moved, and no report names it + expect(categories_of(place)).to contain_exactly(cafe, bakery) # the duplicate's row moved, and no member list names it expect(place.data_entries.with_attribute(:place_categories).sole).to have_attributes(value: categories(cafe).map(&:stringify_keys), expires_at: nil) end end @@ -1569,26 +1569,26 @@ def tenant_names = place.reload.tenants.pluck(:name) travel_to(Time.utc(2026, 1, 4)) { place.reload.forge! { place.record_data_entries({ tenants: [{ name: 'Bakery Pop-up' }] }, top, mode: :edit, expires_at: Time.utc(2026, 2, 1)) } } expect(tenant_names).to contain_exactly('Harbor Books', 'Florist', 'Bakery Pop-up') - expect(place.data_entries.with_source(top).sole.expires_at).to be_nil # an edit's expiry is its members', not the report's + expect(place.data_entries.with_source(top).sole.expires_at).to be_nil # an edit's expiry is its members', not the member list's travel_to(Time.utc(2026, 2, 2)) { import.call } - expect(tenant_names).to contain_exactly('Harbor Books', 'Florist') # the pop-up's inclusion has expired, so low's complete report excludes it + expect(tenant_names).to contain_exactly('Harbor Books', 'Florist') # the pop-up's inclusion has expired, so low's complete list excludes it expect(Tenant.find_by!(name: 'Harbor Books').unit).to eq('2B') travel_to(Time.utc(2026, 3, 2)) { import.call } expect(tenant_names).to contain_exactly('Corner Café', 'Harbor Books', 'Florist') end - it "expires a member's inclusion or exclusion on a source's next report, and replaces or clears it when the member is edited again", capability: 'C135' do + it "expires a member's inclusion or exclusion on the next recording from a source, and replaces or clears it when the member is edited again", capability: 'C135' do place.record_data_entries({ place_categories: categories(cafe, bakery) }, low, mode: :partial) place.record_data_entries({ place_categories: { exclude: categories(bakery), include: categories(florist) } }, top, mode: :edit, expires_on: low) place.save! place.forge! expect(categories_of(place)).to contain_exactly(cafe, florist) - place.reload.record_data_entries({ place_categories: categories(cafe, bakery) }, low, mode: :partial) # the same report, after the edit + place.reload.record_data_entries({ place_categories: categories(cafe, bakery) }, low, mode: :partial) # the same member list, after the edit place.forge! - expect(categories_of(place)).to contain_exactly(cafe, bakery, florist) # no report keeps the florist in, and none excludes it + expect(categories_of(place)).to contain_exactly(cafe, bakery, florist) # no member list includes the florist, and none excludes it expect(place.data_entries.membership_expiries(:place_categories).map { |item| item[:expires_at] }).to all(be_within(1.minute).of(Time.current)) place.reload.record_data_entries({ place_categories: { exclude: categories(bakery) } }, top, mode: :edit, expires_at: 1.week.from_now) @@ -1601,7 +1601,7 @@ def tenant_names = place.reload.tenants.pluck(:name) expect(place.data_entries.membership_expiries(:place_categories).map { |item| item[:key] }).to eq([{ 'category_id' => florist.id }]) end - it 'records nothing an expired inclusion said about a member, and keeps it pending in the report', capability: 'C140' do + it 'records nothing an expired inclusion said about a member, and keeps it pending in the member list', capability: 'C140' do place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café')] }, low) place.forge! place.reload.record_data_entries({ tenants: [tenant('T-1', unit: '1A')] }, top, mode: :edit, expires_at: 1.week.from_now) @@ -1615,7 +1615,7 @@ def tenant_names = place.reload.tenants.pluck(:name) expect(place.data_entries.with_source(top).sole.value['pending'].sole['attributes']).to include('unit' => '1A') end - it "keeps a member's expiry when merging the report it is in", capability: 'C142' do + it "keeps a member's expiry when merging the member list it is in", capability: 'C142' do duplicate = Place.create!(name: 'Harbor Bldg') place.forge! { place.record_data_entries({ place_categories: categories(cafe, bakery) }, low, mode: :partial) } duplicate.forge! { duplicate.record_data_entries({ place_categories: { exclude: categories(bakery) } }, low, mode: :edit, expires_at: 1.week.from_now) } @@ -1627,7 +1627,7 @@ def tenant_names = place.reload.tenants.pluck(:name) expect(categories_of(place)).to eq([cafe]) end - it "keeps a newer recording's expiry when a forge records the same value from an older report kept pending on the owner", capability: 'C144' do + it "keeps a newer recording's expiry when a forge records the same value from an older member list kept pending on the owner", capability: 'C144' do travel_to(Time.utc(2026, 1, 1)) { place.forge! { place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café')] }, low) } } corner_cafe = Tenant.sole travel_to(Time.utc(2026, 1, 2)) { place.reload.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café')] }, low) && place.save! } # kept pending @@ -1648,7 +1648,7 @@ def tenant_names = place.reload.tenants.pluck(:name) expect(place.reload.data_entries.with_source(top).sole.expires_at).to be_nil end - it "expires a member's saved exclusion on a source's report though another member of the report was edited since", capability: 'C146' do + it "expires a member's saved exclusion on a source's member list though another member of the list was edited since", capability: 'C146' do place.record_data_entries({ place_categories: categories(cafe, bakery) }, low, mode: :partial) place.record_data_entries({ place_categories: { exclude: categories(bakery) } }, top, mode: :edit, expires_on: low) place.save! @@ -1703,7 +1703,7 @@ def tenant_names = place.reload.tenants.pluck(:name) expect([corner_cafe.data_entries.with_source(top).sole.expires_at, corner_cafe.precedence_rules.sole.expires_at]).to eq([nil, nil]) end - it "expires a change recorded from a pending report on the source's later report, both replayed by one forge", capability: 'C149' do + it "expires a change recorded from a member list's pending items on the source's later list, both replayed by one forge", capability: 'C149' do travel_to(Time.utc(2026, 1, 1)) { place.forge! { place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café')] }, low) } } travel_to(Time.utc(2026, 1, 2)) { place.reload.record_data_entries({ tenants: [tenant('T-1', unit: LaForge.with_settings('1A', expires_on: low))] }, high, mode: :partial) && place.save! } travel_to(Time.utc(2026, 1, 3)) { place.reload.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café', unit: '2B')] }, low) && place.save! } @@ -1713,7 +1713,7 @@ def tenant_names = place.reload.tenants.pluck(:name) expect(Tenant.sole.unit).to eq('2B') end - it "expires a precedence recorded from a pending report on the source's later report, both replayed by one forge", capability: 'C152' do + it "expires a precedence recorded from a member list's pending items on the source's later list, both replayed by one forge", capability: 'C152' do travel_to(Time.utc(2026, 1, 1)) { place.forge! { place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café')] }, low) } } travel_to(Time.utc(2026, 1, 2)) { place.reload.record_data_entries({ tenants: [tenant('T-1', unit: LaForge.with_settings('1A', precedence: 100, expires_on: low))] }, high, mode: :partial) && place.save! } travel_to(Time.utc(2026, 1, 3)) { place.reload.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café', unit: '2B')] }, low) && place.save! } @@ -1722,19 +1722,19 @@ def tenant_names = place.reload.tenants.pluck(:name) expect(Tenant.sole.precedence_rules.find_by!(attribute_name: 'unit', source: high).expires_at).to eq(Time.utc(2026, 1, 3)) end - it "expires a saved report on the source's recording though an unsaved forge since cleared what it kept pending", capability: 'C154' do + it "expires a saved member list on a recording from the source it expires on, though an unsaved forge since cleared what it kept pending", capability: 'C154' do travel_to(Time.utc(2026, 1, 1)) { place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café')] }, high, expires_on: low) && place.save! } travel_to(Time.utc(2026, 1, 2)) do - place.reload.forge # records Corner Café on its member and clears it from the report, without saving + place.reload.forge # records Corner Café's pending item on the tenant and clears it from the member list, without saving place.record_data_entries({ tenants: [tenant('T-2', name: 'Harbor Books')] }, low) place.forge! end expect(place.data_entries.with_source(high).sole.expires_at).to eq(Time.utc(2026, 1, 2)) - expect(place.reload.tenants.map(&:name)).to include('Harbor Books') # high's complete report has expired, so it does not exclude it + expect(place.reload.tenants.map(&:name)).to include('Harbor Books') # high's complete list has expired, so it does not exclude it end - it "expires a change recorded from a pending report, and saved, on the source's report made after it and replayed by a later forge", capability: 'C155' do + it "expires a change recorded from a member list's pending items, and saved, on the source's list made after it and replayed by a later forge", capability: 'C155' do travel_to(Time.utc(2026, 1, 1)) { place.forge! { place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café')] }, low) } } travel_to(Time.utc(2026, 1, 2)) { place.reload.record_data_entries({ tenants: [tenant('T-1', unit: LaForge.with_settings('1A', expires_on: low))] }, high, mode: :partial) && place.save! } travel_to(Time.utc(2026, 1, 3)) { place.reload.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café', unit: '2B')] }, low) && place.save! } @@ -1744,14 +1744,14 @@ def tenant_names = place.reload.tenants.pluck(:name) end travel_to(Time.utc(2026, 1, 5)) do low.update!(active: true) - place.reload.forge! # records low's report of January 3 + place.reload.forge! # records low's member list of January 3 end expect(Tenant.sole.data_entries.with_source(high).with_attribute(:unit).sole.expires_at).to eq(Time.utc(2026, 1, 3)) expect(Tenant.sole.unit).to eq('2B') end - it "expires a precedence recorded from a pending report, and saved, on the source's report made after it and replayed by a later forge", capability: 'C174' do + it "expires a precedence recorded from a member list's pending items, and saved, on the source's list made after it and replayed by a later forge", capability: 'C174' do travel_to(Time.utc(2026, 1, 1)) { place.forge! { place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café')] }, low) } } travel_to(Time.utc(2026, 1, 2)) { place.reload.record_data_entries({ tenants: [tenant('T-1', unit: LaForge.with_settings('1A', precedence: 100, expires_on: low))] }, high, mode: :partial) && place.save! } travel_to(Time.utc(2026, 1, 3)) { place.reload.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café', unit: '2B')] }, low) && place.save! } @@ -1761,14 +1761,14 @@ def tenant_names = place.reload.tenants.pluck(:name) end travel_to(Time.utc(2026, 1, 5)) do low.update!(active: true) - place.reload.forge! # records low's report of January 3 + place.reload.forge! # records low's member list of January 3 end expect(Tenant.sole.data_entries.with_source(high).with_attribute(:unit).sole.expires_at).to eq(Time.utc(2026, 1, 3)) expect(Tenant.sole.precedence_rules.find_by!(attribute_name: 'unit', source: high)).to have_attributes(recorded_at: Time.utc(2026, 1, 2), expires_at: Time.utc(2026, 1, 3)) end - it "expires a change recorded from a pending report on the source's recording on the member made after the report and saved before the forge", capability: 'C177' do + it "expires a change recorded from a member list's pending items on a recording from the source on the member, made after the list and saved before the forge", capability: 'C177' do travel_to(Time.utc(2026, 1, 1)) { place.forge! { place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café', unit: '1A')] }, low) } } corner_cafe = Tenant.sole travel_to(Time.utc(2026, 1, 2)) do @@ -1785,7 +1785,7 @@ def tenant_names = place.reload.tenants.pluck(:name) expect(corner_cafe.data_entries.with_source(low).with_attribute(:unit).sole).to have_attributes(value: '2B', recorded_at: Time.utc(2026, 1, 3)) end - it "expires a saved exclusion on the source's recording though an unsaved forge since renamed its member to a tombstone", capability: 'C178' do + it "expires a saved exclusion on a recording from the source it expires on, though an unsaved forge since renamed its member to a tombstone", capability: 'C178' do travel_to(Time.utc(2026, 1, 1)) { place.forge! { place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café')] }, low) } } travel_to(Time.utc(2026, 1, 2)) { place.reload.record_data_entries({ tenants: { exclude: [Tenant.sole] } }, top, mode: :edit, expires_on: low) && place.save! } travel_to(Time.utc(2026, 1, 3)) do @@ -1811,7 +1811,7 @@ def tenant_names = place.reload.tenants.pluck(:name) expect(Tenant.find_by!(external_id: 'T-1').opening_hours.pluck(:day, :opens_at)).to eq([[1, '08:00'], [2, '09:00']]) end - it 'forges a pending report whose expiry names a source deleted since, keeping the expiry at a time', capability: 'C180' do + it "forges a member list's pending items whose expiry names a source deleted since, keeping the expiry at a time", capability: 'C180' do geocoder = LaForge::DataSource.create!(name: 'geocoder', precedence: 5) expires_at = 1.week.from_now unit = LaForge.with_settings('1A', precedence: 100, expires_on: geocoder, expires_at: expires_at) @@ -1827,7 +1827,7 @@ def tenant_names = place.reload.tenants.pluck(:name) expect(corner_cafe.precedence_rules.sole).to have_attributes(precedence: 100, expires_at: be_within(1.second).of(expires_at), expires_on_source_id: nil) end - it "expires the precedence of a nested list edit recorded from a pending report, empty or not, on the source's later report, both replayed by one forge", capability: 'C175' do + it "expires the precedence of a nested list edit recorded from a member list's pending items, empty or not, on the source's later list, both replayed by one forge", capability: 'C175' do travel_to(Time.utc(2026, 1, 1)) { place.forge! { place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café'), tenant('T-2', name: 'Harbor Books')] }, low) } } travel_to(Time.utc(2026, 1, 2)) do hours = ->(list) { LaForge.with_settings(list, precedence: 100, expires_on: low) } @@ -1850,11 +1850,11 @@ def tenant_names = place.reload.tenants.pluck(:name) expect { place.record_data_entries({ category_ids: LaForge.with_settings([cafe.id], expires_at: 1.week.from_now) }, top, mode: :edit) }.not_to raise_error end - it 'raises for an exclusion with an expiry from a source whose report is a complete list', capability: 'C136' do + it 'raises for an exclusion with an expiry from a source with a complete list', capability: 'C136' do place.record_data_entries({ place_categories: categories(cafe, bakery) }, low) expect { place.record_data_entries({ place_categories: { exclude: categories(bakery) } }, low, mode: :edit, expires_at: 1.week.from_now) } - .to raise_error(ArgumentError, /the source's report is a complete list, so excluding category_id #{bakery.id} drops it from the list, which cannot expire. Record the list with mode: :partial/) + .to raise_error(ArgumentError, /the source has a complete list, which excludes category_id #{bakery.id} by leaving it out, and leaving a member out cannot expire. Record the member list with mode: :partial/) end end @@ -1896,7 +1896,7 @@ def snapshot(record) expect(snapshot(place).last.map(&:second)).to contain_exactly(x, y) end - it 'gives a nested list and a member attribute their own settings, through the report kept pending on the owner', capability: 'C138' do + it 'gives a nested list and a member attribute their own settings, through the member list kept pending on the owner', capability: 'C138' do place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café', unit: LaForge.with_settings('1A', expires_at: 1.week.from_now), opening_hours: LaForge.with_settings(week('08:00', days: [1]), expires_on: high))] }, low) place.save! @@ -1929,13 +1929,13 @@ def snapshot(record) expect { place.record_data_entries({ tenants: { exclude: [LaForge.with_settings({ id: corner_cafe.id }, precedence: 9)] } }, top, mode: :edit) } .to raise_error(ArgumentError, /Place#tenants: LaForge.with_settings\(precedence:\) around id #{corner_cafe.id}, which has no attributes, would rank the source for the whole association/) expect { place.record_data_entries({ tenants: [{ name: 'Florist', street: LaForge.with_settings('1 Quay', expires_at: later), postcode: 'H1' }] }, top, mode: :edit) } - .to raise_error(ArgumentError, /Tenant members of Place#tenants: street, postcode are attribute group postal_address, which a source records together with the same settings/) + .to raise_error(ArgumentError, /Tenant members of Place#tenants: street, postcode are attribute group postal_address, whose values from a source are recorded together with the same settings/) expect { place.record_data_entries({ place_categories: [{ category_id: LaForge.with_settings(cafe.id, expires_at: later) }] }, top) } .to raise_error(ArgumentError, /Place#place_categories: LaForge.with_settings around category_id, part of a member's key, which cannot carry settings; wrap the whole member instead/) expect { place.record_data_entries({ name: ['Harbor', LaForge.with_settings('Building', expires_at: later)] }, top) } .to raise_error(ArgumentError, /Recording Place#name: LaForge.with_settings can wrap an attribute's whole value, a member or a nested list, not part of a value/) expect { place.record_data_entries({ name: LaForge.with_settings('Harbor Building', expires_on: top) }, top) } - .to raise_error(ArgumentError, /Recording Place: LaForge.with_settings\(expires_on:\) names the source recording/) + .to raise_error(ArgumentError, /Recording Place: LaForge.with_settings\(expires_on:\) names the source this value comes from/) expect { place.record_data_entries({ tenants: { include: LaForge.with_settings([{ name: 'Florist' }], expires_at: later) } }, top, mode: :edit) } .to raise_error(ArgumentError, /Place#tenants: LaForge.with_settings can wrap each member, or the association's whole value, not the include list/) expect { place.record_data_entries({ tenants: { forget: [LaForge.with_settings({ id: corner_cafe.id }, expires_at: later)] } }, top, mode: :edit) } @@ -1944,7 +1944,7 @@ def snapshot(record) expect { LaForge.with_settings('Harbor Building', expires_at: '2030-01-01') }.to raise_error(ArgumentError, /LaForge.with_settings: expires_at: must be a time/) end - it 'raises for a recorded hash using the key LaForge reserves for settings in a pending report', capability: 'C153' do + it "raises for a recorded hash using the key LaForge reserves for settings in a member list's pending items", capability: 'C153' do reserved = { '_laforge_settings' => {}, 'value' => 'Harbor Hall', 'other' => 1 } expect { place.record_data_entry(:name, reserved, top) }.to raise_error(ArgumentError, /Recording Place: _laforge_settings is a key LaForge reserves/) @@ -1955,7 +1955,7 @@ def snapshot(record) end describe 'has_one' do - it 'forges the single member from every source that reports it, updating the same row on later runs', capability: 'C57' do + it 'forges the single member from every source that lists it, updating the same row on later runs', capability: 'C57' do place.record_data_entries({ marker: { label: 'Harbor', icon: 'pin' } }, low) place.record_data_entries({ marker: { label: 'Harbor Building' } }, high) place.forge! @@ -1968,7 +1968,7 @@ def snapshot(record) expect(place.reload.marker).to have_attributes(id: marker_id, label: 'Harbor Building', icon: 'star') end - it 'removes the member when the winning source reports none, writing nothing until the owner is saved', capability: 'C58' do + it 'removes the member when the winning source lists none, writing nothing until the owner is saved', capability: 'C58' do place.record_data_entries({ marker: { label: 'Harbor' } }, low) place.forge! @@ -2010,7 +2010,7 @@ def snapshot(record) expect(Tenant.count).to eq(0) end - it 'rejects more than one member and partial reports', capability: 'C59' do + it 'rejects more than one member and partial lists', capability: 'C59' do expect { place.record_data_entries({ marker: [{}, {}] }, low) }.to raise_error(ArgumentError) expect { place.record_data_entries({ marker: { label: 'A' } }, low, mode: :partial) }.to raise_error(ArgumentError) end @@ -2028,7 +2028,7 @@ def snapshot(record) it 'never touches rows outside the scope of a tracked scoped association', capability: 'C26' do computed_row = place.parcel_links.create!(parcel: lot_14, parcel_link_type: computed) - place.record_data_entries({ reported_parcel_links: [parcel_link(lot_12, occupies)] }, low) + place.record_data_entries({ provided_parcel_links: [parcel_link(lot_12, occupies)] }, low) place.forge! expect(computed_row.reload).to have_attributes(active: true) @@ -2101,11 +2101,11 @@ def snapshot(record) end it "returns only members the tracked association's scope includes", capability: 'C118' do - link = ParcelLink.create!(place: place, parcel: lot_12, parcel_link_type: computed) # outside reported_parcel_links + link = ParcelLink.create!(place: place, parcel: lot_12, parcel_link_type: computed) # outside provided_parcel_links link.record_data_entries({ external_id: 'C-1' }, low) link.save! - expect(place.source_values(:reported_parcel_links)).to eq({}) + expect(place.source_values(:provided_parcel_links)).to eq({}) end it "returns each source's members for many records in a constant number of queries", capability: 'C32' do @@ -2123,35 +2123,35 @@ def snapshot(record) expect(Place.source_values(:place_categories, records: places).keys).to match_array(places.map(&:id)) end - it 'reports which list decided a membership and which source supplied each member attribute', capability: 'C33' do - place.record_data_entries({ reported_parcel_links: [parcel_link(lot_12, occupies, active: false)] }, low) - place.record_data_entries({ reported_parcel_links: [parcel_link(lot_12, occupies, external_id: 'H-1')] }, high) + it "shows which source's member list decided a member's membership, and which source provided each member attribute", capability: 'C33' do + place.record_data_entries({ provided_parcel_links: [parcel_link(lot_12, occupies, active: false)] }, low) + place.record_data_entries({ provided_parcel_links: [parcel_link(lot_12, occupies, external_id: 'H-1')] }, high) place.forge! - provenance = place.member_sources(:reported_parcel_links, parcel_id: lot_12.id, parcel_link_type_id: occupies.id) + provenance = place.member_sources(:provided_parcel_links, parcel_id: lot_12.id, parcel_link_type_id: occupies.id) expect(provenance[:membership]).to eq(high) expect(provenance[:listed_by]).to contain_exactly(low, high) expect(provenance[:attributes]).to eq('active' => low, 'external_id' => high) end - it 'reports added, removed and changed members in forge_diff without writing', capability: 'C34' do - place.record_data_entries({ reported_parcel_links: [parcel_link(lot_12, occupies, active: false), parcel_link(lot_14, occupies)] }, low) + it 'shows added, removed and changed members in forge_diff without writing', capability: 'C34' do + place.record_data_entries({ provided_parcel_links: [parcel_link(lot_12, occupies, active: false), parcel_link(lot_14, occupies)] }, low) place.forge! - place.reload.record_data_entries({ reported_parcel_links: [parcel_link(lot_12, occupies, active: false, external_id: 'H-1'), parcel_link(lot_12, leases)] }, low) - diff = place.forge_diff['reported_parcel_links'] + place.reload.record_data_entries({ provided_parcel_links: [parcel_link(lot_12, occupies, active: false, external_id: 'H-1'), parcel_link(lot_12, leases)] }, low) + diff = place.forge_diff['provided_parcel_links'] expect(diff[:added]).to eq([{ 'parcel_id' => lot_12.id, 'parcel_link_type_id' => leases.id }]) expect(diff[:removed]).to eq([{ 'parcel_id' => lot_14.id, 'parcel_link_type_id' => occupies.id }]) expect(diff[:changed]).to eq([{ 'parcel_id' => lot_12.id, 'parcel_link_type_id' => occupies.id, 'external_id' => [nil, 'H-1'] }]) - expect(place.reported_parcel_links.flat_map { |row| row.data_entries.target }.map { |entry| [entry.attribute_name, entry.value, entry.changed?] }).to eq([['active', false, false]]) + expect(place.provided_parcel_links.flat_map { |row| row.data_entries.target }.map { |entry| [entry.attribute_name, entry.value, entry.changed?] }).to eq([['active', false, false]]) expect(place.reload.parcel_links.count).to eq(2) end end describe 'integration with attribute tracking' do - it "leaves no precedence rule behind on a member after forge_diff previews a report's precedence", capability: 'C109' do + it "leaves no precedence rule behind on a member after forge_diff previews a member list's precedence", capability: 'C109' do place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café')] }, low) place.forge! place.reload.record_data_entries({ tenants: [tenant('T-1', unit: '2B')] }, low, precedence: 50) @@ -2172,7 +2172,7 @@ def snapshot(record) expect(place.forge_diff(attributes: :place_categories).keys).to eq(['place_categories']) end - it "leaves a member's waiting expiries as they were after forge_diff previews a report from the source they expire on", capability: 'C143' do + it "leaves a member's waiting expiries as they were after forge_diff previews a member list from the source they expire on", capability: 'C143' do travel_to(Time.utc(2026, 1, 1)) { place.forge! { place.record_data_entries({ tenants: [tenant('T-1', name: 'Corner Café')] }, low) } } travel_to(Time.utc(2026, 1, 2)) do corner_cafe = Tenant.sole @@ -2263,7 +2263,7 @@ def numbered_place_model numbered_place.record_data_entries({ tenants: [{ external_id: 'T-1', name: 'Corner Café' }] }, low) numbered_place.forge! corner_cafe = numbered_place.tenants.sole - corner_cafe.record_data_entries({ unit: '2B' }, high) # a source with entries on the member and no report + corner_cafe.record_data_entries({ unit: '2B' }, high) # a source with entries on the member and no member list corner_cafe.save! values = numbered_place.reload.source_values(:tenants) @@ -2284,7 +2284,7 @@ def numbered_place_model it 'derives join model keys from the association reflection', capability: 'C39' do expect(Place.laforge.association(:place_categories).key).to eq(%w[category_id]) - expect(Place.laforge.association(:reported_parcel_links).key).to eq(%w[parcel_id parcel_link_type_id]) + expect(Place.laforge.association(:provided_parcel_links).key).to eq(%w[parcel_id parcel_link_type_id]) end it 'raises for a method the model already defines that would hide LaForge\'s, and lets one defined after it wrap LaForge\'s', capability: 'C98' do diff --git a/spec/lib/association_importer_scenarios_spec.rb b/spec/lib/association_importer_scenarios_spec.rb index e7be060..112ded9 100644 --- a/spec/lib/association_importer_scenarios_spec.rb +++ b/spec/lib/association_importer_scenarios_spec.rb @@ -2,9 +2,9 @@ # Association assignment as importers drive it, end to end across several import runs. # -# Each run records everything the source currently says about a record, whole association lists included, then forges. -# A tenant's attributes, reported in the place's list, are recorded on the tenant itself. An orphan sweep withdraws a -# source from a record it no longer reports, without forging, and closes the record only when no other source has +# Each run records everything the source currently says about a record, whole member lists included, then forges. A +# tenant's attributes, provided in the place's member list, are recorded on the tenant itself. An orphan sweep withdraws +# a source from a record it no longer provides, without forging, and closes the record only when no other source has # entries on it. # # Sources, lowest to highest precedence: cadastre, business directory, business registry, and the map editor, which @@ -62,7 +62,7 @@ def tenant(external_id, **fields) end def tenants_of(record) = record.reload.tenants.order(:external_id).pluck(:external_id, :name, :unit) - describe 'a single importer reporting a category list each run' do + describe "a single importer recording a place's categories each run" do it 'adds, removes, re-adds and reorders categories as the list changes from run to run' do import(place, cadastre, place_categories: categories(cafe, bakery)) expect(categories_of(place)).to eq([cafe, bakery]) @@ -85,7 +85,7 @@ def tenants_of(record) = record.reload.tenants.order(:external_id).pluck(:extern expect(place.reload.place_categories.find_by!(category: cafe).id).to eq(cafe_row.id) end - it 'clears every category when the source reports none' do + it 'clears every category when the source lists none' do import(place, cadastre, place_categories: categories(cafe, bakery)) import(place.reload, cadastre, place_categories: []) @@ -94,7 +94,7 @@ def tenants_of(record) = record.reload.tenants.order(:external_id).pluck(:extern end end - describe 'two importers reporting categories for the same place' do + describe 'two importers listing categories for the same place' do before do import(place, cadastre, place_categories: categories(cafe, bakery, bookstore)) import(place.reload, business_directory, place_categories: categories(cafe, florist)) @@ -106,7 +106,7 @@ def tenants_of(record) = record.reload.tenants.order(:external_id).pluck(:extern expect(categories_of(place)).to eq([cafe, florist]) end - it "falls back to the lower importer's list on the next forge after the higher one stops reporting the place" do + it "falls back to the lower importer's member list on the next forge after the higher one stops providing the place" do sweep(place.reload, business_directory) expect(categories_of(place)).to eq([cafe, florist]) # the sweep does not forge @@ -159,10 +159,10 @@ def tenants_of(record) = record.reload.tenants.order(:external_id).pluck(:extern let!(:computed_row) { place.parcel_links.create!(parcel: lot_14, parcel_link_type: computed) } it 'adds, updates in place and removes parcel links without touching computed ones' do - import(place.reload, cadastre, reported_parcel_links: [parcel_link(lot_12, occupies, external_id: 'E-1'), parcel_link(lot_14, leases)]) + import(place.reload, cadastre, provided_parcel_links: [parcel_link(lot_12, occupies, external_id: 'E-1'), parcel_link(lot_14, leases)]) occupies_row = place.parcel_links.find_by!(parcel: lot_12, parcel_link_type: occupies) - import(place.reload, cadastre, reported_parcel_links: [parcel_link(lot_12, occupies, external_id: 'E-2', active: false)]) + import(place.reload, cadastre, provided_parcel_links: [parcel_link(lot_12, occupies, external_id: 'E-2', active: false)]) expect(parcel_links_of(place)).to eq([[lot_14, computed, true, nil], [lot_12, occupies, false, 'E-2']]) expect(place.parcel_links.find_by!(parcel: lot_12, parcel_link_type: occupies).id).to eq(occupies_row.id) @@ -170,23 +170,23 @@ def tenants_of(record) = record.reload.tenants.order(:external_id).pluck(:extern end it 'replaces the row when only the link type changes, since the type is part of the identity' do - import(place.reload, cadastre, reported_parcel_links: [parcel_link(lot_12, occupies, external_id: 'E-1')]) - import(place.reload, cadastre, reported_parcel_links: [parcel_link(lot_12, leases, external_id: 'E-1')]) + import(place.reload, cadastre, provided_parcel_links: [parcel_link(lot_12, occupies, external_id: 'E-1')]) + import(place.reload, cadastre, provided_parcel_links: [parcel_link(lot_12, leases, external_id: 'E-1')]) expect(parcel_links_of(place)).to eq([[lot_14, computed, true, nil], [lot_12, leases, true, 'E-1']]) end - it "keeps a map editor's attribute on a parcel link the importer keeps reporting" do - import(place.reload, cadastre, reported_parcel_links: [parcel_link(lot_12, occupies, active: true)]) - import(place.reload, map_editor, reported_parcel_links: [parcel_link(lot_12, occupies, active: false)]) - import(place.reload, cadastre, reported_parcel_links: [parcel_link(lot_12, occupies, active: true, external_id: 'E-3')]) + it "keeps a map editor's attribute on a parcel link the importer keeps listing" do + import(place.reload, cadastre, provided_parcel_links: [parcel_link(lot_12, occupies, active: true)]) + import(place.reload, map_editor, provided_parcel_links: [parcel_link(lot_12, occupies, active: false)]) + import(place.reload, cadastre, provided_parcel_links: [parcel_link(lot_12, occupies, active: true, external_id: 'E-3')]) expect(parcel_links_of(place).last).to eq([lot_12, occupies, false, 'E-3']) end end describe 'tenants listed by a business directory and enriched by a business registry' do - # Records on the tenant as the business registry does, giving the registration number precedence 100 so that it + # Records values from the business registry on the tenant, giving the registration number precedence 100 so that it # outranks every other source def register(tenant, registration_number, **fields) tenant.record_data_entries({ registration_number: registration_number }, business_registry, precedence: 100) @@ -208,7 +208,7 @@ def register(tenant, registration_number, **fields) expect(t1.reload).to have_attributes(registration_number: 'R-10', name: 'Harbor Café', unit: '2B') end - it "clears the registration number and restores the importer's unit once the business registry stops reporting the business" do + it "clears the registration number and restores the importer's unit once the business registry stops providing the business" do register(t1, 'R-10', unit: '3C') # Blanks the registration number and lets the unit fall back to the importer's @@ -230,7 +230,7 @@ def register(tenant, registration_number, **fields) expect(t1.reload.status).to eq('open') end - it 'removes a tenant, including the registration number the business registry recorded on it, when the business directory drops it' do + it 'removes a tenant, including its registration number from the business registry, when the business directory drops it' do register(t1, 'R-10') import(place.reload, business_directory, tenants: [tenant('T-2', unit: '1A')]) @@ -259,7 +259,7 @@ def register(tenant, registration_number, **fields) end describe 'a nightly import that names the tenants it created before by id' do - it 'keeps the same tenants, and what other sources recorded on them, from night to night' do + it "keeps the same tenants, and other sources' entries on them, from night to night" do import(place, business_directory, tenants: [tenant('T-1', unit: '1A'), tenant('T-2', unit: '2A')]) corner_cafe, harbor_books = %w[T-1 T-2].map { |external_id| Tenant.find_by!(external_id: external_id) } corner_cafe.forge! { corner_cafe.record_data_entries({ unit: '1B' }, map_editor) } @@ -286,7 +286,7 @@ def register(tenant, registration_number, **fields) place.forge! end - it 'destroys the tenant and keeps it out of later imports that report it again as new, while new tenants still arrive' do + it 'destroys the tenant and keeps it out of later imports that list it again as new, while new tenants still arrive' do expect(tenants_of(place).map(&:first)).to eq(%w[T-1]) # noted: destroyed per `dependent: :destroy` expect(place.reload.data_entries.tombstones(:tenants).map { |tombstone| [tombstone.source, tombstone.attributes['name']] }).to eq([[map_editor, 'Tenant T-2']]) @@ -376,7 +376,7 @@ def listed(external_id) = { external_id: external_id, name: "Tenant #{external_i end end - describe 'an importer that reports only the tenants it knows about' do + describe 'an importer that lists only the tenants it knows about' do before do import(place, business_directory, tenants: [tenant('T-1')]) place.reload.record_data_entries({ tenants: [tenant('T-9', unit: '9Z')] }, business_registry, mode: :partial) @@ -415,15 +415,15 @@ def listed(external_id) = { external_id: external_id, name: "Tenant #{external_i end end - describe 'a dry run that reports changes before applying them (change tracking)' do - it 'reports the categories and parcel links an import would add, remove and change' do - import(place, cadastre, place_categories: categories(cafe, bakery), reported_parcel_links: [parcel_link(lot_12, occupies)]) + describe 'a dry run that shows changes before applying them (change tracking)' do + it 'shows the categories and parcel links an import would add, remove and change' do + import(place, cadastre, place_categories: categories(cafe, bakery), provided_parcel_links: [parcel_link(lot_12, occupies)]) - place.reload.record_data_entries({ place_categories: categories(cafe, bookstore), reported_parcel_links: [parcel_link(lot_12, occupies, external_id: 'E-1')] }, cadastre) + place.reload.record_data_entries({ place_categories: categories(cafe, bookstore), provided_parcel_links: [parcel_link(lot_12, occupies, external_id: 'E-1')] }, cadastre) diff = place.forge_diff expect(diff['place_categories']).to include(added: [{ 'category_id' => bookstore.id }], removed: [{ 'category_id' => bakery.id }]) - expect(diff['reported_parcel_links'][:changed]).to eq([{ 'parcel_id' => lot_12.id, 'parcel_link_type_id' => occupies.id, 'external_id' => [nil, 'E-1'] }]) + expect(diff['provided_parcel_links'][:changed]).to eq([{ 'parcel_id' => lot_12.id, 'parcel_link_type_id' => occupies.id, 'external_id' => [nil, 'E-1'] }]) expect(categories_of(place)).to eq([cafe, bakery]) end end diff --git a/spec/lib/usage_spec.rb b/spec/lib/usage_spec.rb index 264985c..932d5f6 100644 --- a/spec/lib/usage_spec.rb +++ b/spec/lib/usage_spec.rb @@ -62,7 +62,7 @@ def phone_number=(number) expect(record.name).to eq('From Map Editor') end - it 'reports the winning value of every recorded attribute, so changes can be tracked before saving' do + it 'returns the winning value of every recorded attribute, so changes can be tracked before saving' do record.record_data_entries({ name: 'From Cadastre', active: false }, cadastre) record.record_data_entries({ name: 'From Business Directory' }, business_directory) @@ -192,7 +192,7 @@ def phone_number=(number) expect(record.data_entries.sole).to have_attributes(attribute_name: 'name', value: 'Approved', source_id: map_editor.id) end - it 'updates the existing entry when the same source records the attribute again' do + it 'updates the existing entry when a value from the same source is recorded for the attribute again' do record.record_data_entries({ name: 'First' }, map_editor) record.save! record.reload.record_data_entries({ name: 'Second' }, map_editor) @@ -223,7 +223,7 @@ def phone_number=(number) expect(record.reload.name).to eq('From Business Directory') end - it 'keeps the custom precedence when the same source records the attribute again without one' do + it 'keeps the custom precedence when a value from the same source is recorded for the attribute again without one' do record.record_data_entries({ name: 'From Map Editor' }, map_editor) record.record_data_entries({ name: 'From Business Directory' }, business_directory, precedence: 100) record.forge! @@ -245,7 +245,7 @@ def phone_number=(number) expect(record.name).to eq('From Cadastre') end - it 'can be set before the source has recorded the attribute, and applies once it does' do + it 'can be set before the source has an entry for the attribute, and applies once it has one' do record.update_attribute_source_precedence(:name, cadastre, 100) record.record_data_entries({ name: 'From Map Editor' }, map_editor) record.forge! @@ -381,14 +381,14 @@ def import(rows) end end - describe 'a source that stops reporting a record' do + describe 'a source that stops providing a record' do before do record.record_data_entries({ name: 'From Cadastre', count: 5 }, cadastre) record.record_data_entries({ name: 'From Business Directory', count: 7, active: false }, business_directory) record.forge! end - it 'blanks the attributes the source was supplying by recording nil, even where a lower source has a value' do + it 'blanks the attributes the source was providing by recording nil, even where a lower source has a value' do record.record_data_entries({ name: nil, count: nil }, business_directory) record.forge! @@ -450,7 +450,7 @@ def still_claimed?(record, source, neutral_sources) expect(still_claimed?(record, cadastre.name, [])).to be(true) end - it 'ignores sources that only supply attribute values' do + it 'ignores sources that only provide attribute values' do expect(still_claimed?(record, cadastre.name, ['Geocoder'])).to be(false) expect(record.data_entries.reload.map(&:source)).to eq([geocoder]) end @@ -579,7 +579,7 @@ def still_claimed?(record, source, neutral_sources) expect(counted.reload.subject).to eq(parent) end - it 'clears the association when the winning source records nil' do + it 'clears the association when the winning source provides nil' do record.record_data_entries({ parent: parent }, cadastre) record.record_data_entries({ parent: nil }, business_directory) record.forge! @@ -673,9 +673,9 @@ def self.name = 'ActiveRecordMock' expect(grouped.reload).to have_attributes(name: 'From Map Editor', count: nil) end - it 'raises when a source records part of a group, naming what is missing' do + it 'raises when a source provides part of a group, naming what is missing' do expect { grouped.record_data_entries({ name: 'From Cadastre' }, cadastre) } - .to raise_error(ArgumentError, /name, count are attribute group listing, which a source records together \(missing: count\)/) + .to raise_error(ArgumentError, /name, count are attribute group listing, which a source provides together \(missing: count\)/) expect { grouped.record_data_entry(:count, 5, cadastre) }.to raise_error(ArgumentError, /missing: name/) end @@ -730,7 +730,7 @@ def self.name = 'ActiveRecordMock' it "falls back to the next source, and keeps a value no other source has, once the record is forged again" do record.update!(active: false) - record.record_data_entries({ active: true }, map_editor) # an attribute only the map editor reports + record.record_data_entries({ active: true }, map_editor) # an attribute only the map editor provides record.forge! map_editor.update!(active: false) @@ -955,7 +955,7 @@ def self.name = 'ActiveRecordMock' end end - it 'expires a value the next time the source it expires on records the attribute, whatever that source reports' do + it 'expires a value the next time the attribute is recorded from the source it expires on, whatever that source provides' do record.record_data_entries({ name: 'Renamed in the Map Editor' }, map_editor, expires_on: cadastre) record.forge! record.reload.record_data_entries({ count: 6 }, cadastre) # another attribute @@ -1107,7 +1107,7 @@ def self.name = 'ActiveRecordMock' it 'raises for an expiry it cannot apply, saying what to pass' do expect { record.record_data_entries({ name: 'Renamed' }, map_editor, expires_at: '2030-01-01') }.to raise_error(ArgumentError, /expires_at: must be a time, such as 2.weeks.from_now, not "2030-01-01"/) - expect { record.record_data_entries({ name: 'Renamed' }, map_editor, expires_on: map_editor) }.to raise_error(ArgumentError, /expires_on: names the source recording.*Pass the source whose next report should end the change/) + expect { record.record_data_entries({ name: 'Renamed' }, map_editor, expires_on: map_editor) }.to raise_error(ArgumentError, /expires_on: names the source these values come from.*Pass another source, whose next values should end the change/) expect { record.update_precedence_order(:name, [], expires_at: 1.week.from_now) }.to raise_error(ArgumentError, /update_precedence_order\(:name\) with no sources removes it, so there is nothing to expire/) expect { record.update_attribute_source_precedence(:name, cadastre, nil, expires_on: map_editor) }.to raise_error(ArgumentError, /with a nil precedence removes it/) expect { LaForge::PrecedenceRule.create!(record_type: 'ActiveRecordMock', attribute_name: 'name', source: cadastre, precedence: 1, expires_at: 1.week.from_now) } diff --git a/spec/support/association_capabilities.rb b/spec/support/association_capabilities.rb index 38bca95..af034ec 100644 --- a/spec/support/association_capabilities.rb +++ b/spec/support/association_capabilities.rb @@ -129,7 +129,7 @@ def self.define_models has_many :parcel_links, dependent: :destroy # Leaves out computed links, which another process derives from parcel geometry, so forging never touches them - has_many :reported_parcel_links, + has_many :provided_parcel_links, -> { where.not(parcel_link_type_id: ParcelLinkType.where(key: 'computed').select(:id)) }, class_name: 'ParcelLink' @@ -140,9 +140,9 @@ def self.define_models laforged associations: { place_categories: {}, # key derived from the reflection, `category_id` - reported_parcel_links: {}, # key derived, `parcel_id` and `parcel_link_type_id` - # Tenants are keyed by primary key. An excluded tenant is destroyed, and recognised by its name when a source - # reports it again. + provided_parcel_links: {}, # key derived, `parcel_id` and `parcel_link_type_id` + # Tenants are keyed by primary key. An excluded tenant is destroyed, and recognized by its name when a source + # lists it again. tenants: { tombstone_matcher: ->(tenant, tombstones) { tombstones.find { |tombstone| tombstone.name == tenant.name } } }, marker: {} # a has_one with no key, as there is only ever one } From 1ab662b7e3e07fb287503d6718296e87ad41c09b Mon Sep 17 00:00:00 2001 From: Nicholas Jakobsen Date: Tue, 27 Oct 2026 11:47:29 -0700 Subject: [PATCH 3/3] docs: call a source's association data its member list in the README, as the language guide does Issue: the README called what a source provides for a tracked association its "report", said sources "report" or "supply" values, called `LaForge.merge`'s return value a report, said a member list has an "opinion" on a member, and spelled "recognise" the British way, while `.ai/language.md` calls these a member list, provides, a result, and covers. The README now uses the guide's words, says once that members are Rails' associated records, and calls the merged association's outcome its membership, and says a member list covers a member. Headings are renamed to match, with every in-README link updated: "When a source stops reporting a record" is now "When a source stops providing a record", "Expiring an association report" is now "Expiring a member list", "Partial reports" is now "Partial lists", "Removed members that sources report again" is now "Removed members that sources list again", and "When a source stops reporting" is now "When a source stops listing members". Co-Authored-By: Claude Opus 5.5 --- Readme.md | 184 +++++++++++++++++++++++++++--------------------------- 1 file changed, 92 insertions(+), 92 deletions(-) diff --git a/Readme.md b/Readme.md index e6cc221..bd1d4a9 100644 --- a/Readme.md +++ b/Readme.md @@ -69,12 +69,12 @@ Key features: |---|---| | `DataSource.create!(name:, priority:)`, `source.priority` | `DataSource.create!(name:, precedence:)`, `source.precedence` | | `record_data_entries(…, priority:)`, `record_data_entry(…, priority:)` | `record_data_entries(…, precedence:)`, `record_data_entry(…, precedence:)` | - | `record.update_attribute_source_priority` | `record.update_attribute_source_precedence`, which also works before the source has recorded the attribute | + | `record.update_attribute_source_priority` | `record.update_attribute_source_precedence`, which also works before the source has an entry for the attribute | | `DataEntry#priority_with_fallback`, `DataEntry#source_priority` | `DataEntry#effective_precedence`, `DataEntry#source_precedence` | 4. Check the precedences set on one record. A source with a precedence for an attribute of one record, such as one given with `priority:` when recording, now outranks every source without one for that attribute of that record, whatever the numbers. Sources rank by the most specific level they have a rule at, and numbers are compared only within a level (see [Precedence](#precedence)). Where a per-record priority was meant to rank a source above some sources but not others, set an order that lists them all instead, such as `record.update_precedence_order(:name, [map_editor, cadastre, business_directory])`. -5. Record `nil` where a removal was meant to blank an attribute. Removing entries inside `forge!` used to blank the removed attributes for that forge, and the next forge then fell back to lower sources. A forge now leaves removed entries out, so each attribute falls back to the next source straight away, or keeps its value where no other source has one (see [When a source stops reporting a record](#when-a-source-stops-reporting-a-record)). Where a source should blank what it supplied, record `nil` for those attributes instead. +5. Record `nil` where a removal was meant to blank an attribute. Removing entries inside `forge!` used to blank the removed attributes for that forge, and the next forge then fell back to lower sources. A forge now leaves removed entries out, so each attribute falls back to the next source straight away, or keeps its value where no other source has one (see [When a source stops providing a record](#when-a-source-stops-providing-a-record)). Where a source should blank what it provided, record `nil` for those attributes instead. ```ruby # Before @@ -148,7 +148,7 @@ Sources rank by level first. A source with a rule at a more specific level outra foo.forge! # => Saves the model with the attributes { height: 19, width: 5 } ``` -A record's precedence for an attribute is kept when the source records the attribute again without passing one, and removed by passing `precedence: nil`. It does not depend on the source's entries, so it can be set before the source has recorded the attribute, and it survives the source's entries being destroyed. Pass `nil` to any of the setters above to remove a rule. +A record's precedence for an attribute is kept when a value from the source is recorded for the attribute again without passing one, and removed by passing `precedence: nil`. It does not depend on the source's entries, so it can be set before the source has an entry for the attribute, and it survives the source's entries being destroyed. Pass `nil` to any of the setters above to remove a rule. ```ruby foo.record_data_entries({ height: 18 }, source2) # Still precedence 3 @@ -224,7 +224,7 @@ A ruleset selector names each record's ruleset, as a method name or a Proc given ##### When the selector is called -The selector is called when a record is forged, by `forge`, `forge_attributes` or `forge_associations`, and by `forge_diff`, `precedence_order` and `member_sources`, never when the app boots. `member_sources` calls it for the owner and for the member it reports on. `forge` and `forge_diff` call it once for each record, before any attribute is assigned, so a forge that changes what the selector reads ranks attributes and associations alike, and a `member_sources` call made during the forge, such as from a setter, ranks by that same selection. A forge forges a record's tracked associations before assigning its attributes, so a member whose selector reads its owner ranks by the owner as the forge found it, which is also what `forge_diff` previews. Other ranking helpers, such as `effective_precedence` and `DataEntry.in_precedence_order`, reuse the last selection until the record is forged again or reloaded. +The selector is called when a record is forged, by `forge`, `forge_attributes` or `forge_associations`, and by `forge_diff`, `precedence_order` and `member_sources`, never when the app boots. `member_sources` calls it for the owner and for the member it describes. `forge` and `forge_diff` call it once for each record, before any attribute is assigned, so a forge that changes what the selector reads ranks attributes and associations alike, and a `member_sources` call made during the forge, such as from a setter, ranks by that same selection. A forge forges a record's tracked associations before assigning its attributes, so a member whose selector reads its owner ranks by the owner as the forge found it, which is also what `forge_diff` previews. Other ranking helpers, such as `effective_precedence` and `DataEntry.in_precedence_order`, reuse the last selection until the record is forged again or reloaded. ##### Sources created or destroyed later @@ -236,7 +236,7 @@ Set rulesets up from one writer at a time. Two processes copying a ruleset at on ### Querying ```ruby - foo.data_sources # => The sources that supplied at least one of foo's data entries, each listed once + foo.data_sources # => The sources that provided at least one of foo's data entries, each listed once MyModel.joins(:data_sources).where(laforge_data_sources: { name: 'Wikipedia' }) # => Records with an entry from Wikipedia MyModel.with_source(source1) # => All records from source1 @@ -266,7 +266,7 @@ The same scopes are available on a record's entries and on `LaForge::DataEntry`. ## Examples -These cover common ways of merging several feeds and manual edits into one record, and are covered by `spec/lib/usage_spec.rb`. They follow a map that places entities from several sources: a cadastre supplying land records, a business directory that outranks it, a map editor whose edits outrank both, and a geocoder that only adds coordinates. +These cover common ways of merging several feeds and manual edits into one record, and are covered by `spec/lib/usage_spec.rb`. They follow a map that places entities from several sources: a cadastre providing land records, a business directory that outranks it, a map editor whose edits outrank both, and a geocoder that only adds coordinates. ```ruby cadastre = LaForge::DataSource.create!(name: 'Cadastre', precedence: 1) @@ -277,7 +277,7 @@ These cover common ways of merging several feeds and manual edits into one recor ### Importing -Each import run records everything the source currently says about a record, then forges. Attributes no source has recorded are left as they are, so an importer can still write some columns directly. +Each import run records everything the source currently says about a record, then forges. Attributes no source has an entry for are left as they are, so an importer can still write some columns directly. ```ruby place.record_data_entries({ land_use: 'Commercial', floor_area: 420 }, cadastre) @@ -285,7 +285,7 @@ Each import run records everything the source currently says about a record, the place.forge! ``` -To report what an import would change before saving, use `forge_attributes` (the winning value of every recorded attribute) or `forge_diff` (only those that differ from the record). +To see what an import would change before saving, use `forge_attributes` (the winning value of every recorded attribute) or `forge_diff` (only those that differ from the record). ```ruby place.record_data_entries({ land_use: 'Mixed Use' }, cadastre) @@ -346,16 +346,16 @@ To keep the map editor on top while the cadastre outranks the other importers on place.save! ``` -### When a source stops reporting a record +### When a source stops providing a record -To blank what the source supplied, record `nil` for it. Attributes a higher source supplies are unaffected. +To blank what the source provided, record `nil` for it. Attributes a higher source provides are unaffected. ```ruby place.record_data_entries({ name: nil, phone_number: nil }, business_directory) place.forge! ``` -To stop applying what the source supplied instead, remove its entries. Each attribute falls back to the next source, or keeps its value where no other source has one. +To stop applying what the source provided instead, remove its entries. Each attribute falls back to the next source, or keeps its value where no other source has one. ```ruby place.forge! { place.remove_data_entries(attribute_names: %i[name phone_number], sources: business_directory) } @@ -368,7 +368,7 @@ To save an explicit value instead, such as for a column that cannot be blank, re place.update!(listed: false) ``` -Whether another source still has entries on the record tells whether anything else still claims it exists. A source that only supplies values, such as the geocoder, can be left out of that check. For a member of a tracked association, also check the sources that list it (see [Inspecting associations](#inspecting-associations)). +Whether another source still has entries on the record tells whether anything else still claims it exists. A source that only provides values, such as the geocoder, can be left out of that check. For a member of a tracked association, also check the sources that list it (see [Inspecting associations](#inspecting-associations)). ```ruby place.data_entries.with_source(business_directory).destroy_all @@ -394,7 +394,7 @@ Whether another source still has entries on the record tells whether anything el `effective_precedence` on an entry is the precedence of its most specific rule. Since sources rank by level first, compare entries with `in_precedence_order` rather than by that number. -A source that lists a member of a tracked association without saying anything about it has no entries on the member, so it isn't among the member's sources; `member_sources` on the owner reports it (see [Inspecting associations](#inspecting-associations)). +A source that lists a member of a tracked association without saying anything about it has no entries on the member, so it isn't among the member's sources; `member_sources` on the owner names it (see [Inspecting associations](#inspecting-associations)). ### Fields that come from one source together @@ -406,8 +406,8 @@ Some attributes only make sense from one source together, such as the lines of a end ``` -- **Forging** takes every attribute of the group from the source whose entry for any of them ranks highest. An attribute that source has no entry for is `nil`. -- **Recording** some of a group's attributes but not all of them raises, naming the missing ones, in every mode and for members reported through an owner. Pass `nil` for a value the source doesn't have. +- **Forging** takes every attribute of the group from the source whose entry for any of them ranks highest. An attribute that source has no entry for is `nil` rather than falling back to another source's value. +- **Recording** some of a group's attributes but not all of them raises, naming the missing ones, in every mode and for members recorded through an owner. Pass `nil` for a value the source doesn't have. - **Precedence** is set on the group, as `place.update_precedence_order(:postal_address, [...])`. A rule on one of its attributes raises, naming the group. `precedence:` given while recording goes to the group. - **Removing and forging** one attribute of a group, with `remove_data_entries(attribute_names:)` or `forge_attributes(attributes:)`, covers the whole group. @@ -420,11 +420,11 @@ A source that is no longer used can be deactivated. From each record's next forg place.forge! # coordinates fall back to the next source, or stay as they are ``` -In a tracked association, an inactive source's reports are left out too. Members only it listed become rows no report has named, which are left alone, and its exclusions and tombstones stop applying until it is active again. A tenant it excluded that another source reported as new meanwhile is removed, per `dependent:`, by the first forge after it is activated again. A tenant the application created itself meanwhile, after `LaForge.tombstone_blocking` allowed it, keeps its id and is not matched against the tombstone, so the exclusion does not apply to it. +In a tracked association, an inactive source's member lists are left out too. Members only it listed become rows no member list has named, which are left alone, and its exclusions and tombstones stop applying until it is active again. A tenant it excluded that another source listed as new meanwhile is removed, per `dependent:`, by the first forge after it is activated again. A tenant the application created itself meanwhile, after `LaForge.tombstone_blocking` allowed it, keeps its id and is not matched against the tombstone, so the exclusion does not apply to it. ### Expiring a change -A change to one record can be made to expire, so the record goes back to what its sources say without it and no one has to remember to undo it. A recorded value, a record's precedence order, a single precedence, or several of them in one save, take `expires_at:` (a time) or `expires_on:` (a source, so the change expires the next time that source records the same attribute or association for the record, whatever it reports, which sets `expires_at` to when it was recorded). +A change to one record can be made to expire, so the record goes back to what its sources say without it and no one has to remember to undo it. A recorded value, a record's precedence order, a single precedence, or several of them in one save, take `expires_at:` (a time) or `expires_on:` (a source, so the change expires the next time a value from that source is recorded for the same attribute or association of the record, whatever that source provides, which sets `expires_at` to when it was recorded). ```ruby # Rename the place until the next cadastre import @@ -440,7 +440,7 @@ A change to one record can be made to expire, so the record goes back to what it - **An expiry on part of an attribute group applies to the whole group.** - **Only changes to one record expire.** Model-wide and global rules are configuration; giving one an expiry raises. -**Timing is the application's job.** Like any precedence change, an expiry takes effect at the record's next forge; nothing changes in between. An expiry on a source takes effect at the forge that follows that source's recording, which an import does anyway. For expiries at a time, list what expires and forge those records on schedule, or warn ahead of time. +**Timing is the application's job.** Like any precedence change, an expiry takes effect at the record's next forge; nothing changes in between. An expiry on a source takes effect at the forge that follows the recording from that source, which an import does anyway. For expiries at a time, list what expires and forge those records on schedule, or warn ahead of time. ```ruby LaForge::DataEntry.expiring(before: 1.week.from_now) # expired already, or expiring within the week @@ -452,17 +452,17 @@ LaForge publishes nothing when a change expires by time, and the forge writes no #### Which recordings end an expiry on a source -Only a recording made after the change counts, so a report the source made earlier, kept pending on an owner until the owner is forged, does not end it. A value can't expire on the source recording it, since that source's next recording replaces it anyway. +Only a recording made after the change counts, so a member list recorded from the source earlier, kept pending on an owner until the owner is forged, does not end it. A value can't expire on the source it comes from, since the next recording from that source replaces it anyway. -A change kept pending on an owner is made when its report was, so a recording the source makes after that report and before the owner is forged ends it, whether that is a later report forged with it or a recording on the member itself. A recording on the member shows only in the entries it changed, so one that changed nothing there does not end the change. If the source is deleted before the owner is forged, the change keeps only its expiry at a time, when it has one. +A change kept pending on an owner is made when its member list was recorded, so a recording from the source made after that and before the owner is forged ends it, whether that is a later member list forged with it or a recording on the member itself. A recording on the member shows only in the entries it changed, so one that changed nothing there does not end the change. If the source is deleted before the owner is forged, the change keeps only its expiry at a time, when it has one. -#### Expiring an association report +#### Expiring a member list -An expired association report counts as if it did not exist, exactly as an inactive source's report, and nothing is removed because of the expiry itself. A report recorded whole, as a complete or partial list, expires as a whole with the call's expiry. A member only it named becomes a row no report names, which is left alone; a member another report excludes, such as a higher-ranked complete list that leaves it out, is removed once the expired report no longer keeps it in. An expired exclusion or tombstone stops keeping its member out, so a source can bring the member back. What the report says about members is recorded on them with the same expiry, so their attributes fall back with it. +An expired member list counts as if it did not exist, exactly as an inactive source's member list, and nothing is removed because of the expiry itself. A member list recorded whole, as a complete or partial list, expires as a whole with the call's expiry. A member only it named becomes a row no member list names, which is left alone; a member another member list excludes, such as a higher-ranked complete list that leaves it out, is removed once the expired list no longer keeps it in. An expired exclusion or tombstone stops keeping its member out, so a source can bring the member back. What the member list says about members is recorded on them with the same expiry, so their attributes fall back with it. #### Expiring one member -In an edit (`mode: :edit`), the expiry is given to each member the edit includes or excludes, not to the source's report, so editing one member leaves the expiries earlier edits gave the others as they are. Editing the same member again replaces its expiry, and leaving the options out makes it permanent. +In an edit (`mode: :edit`), the expiry is given to each member the edit includes or excludes, not to the source's member list, so editing one member leaves the expiries earlier edits gave the others as they are. Editing the same member again replaces its expiry, and leaving the options out makes it permanent. ```ruby # Close the corner café until March; a later edit of another tenant leaves that alone @@ -470,11 +470,11 @@ In an edit (`mode: :edit`), the expiry is given to each member the edit includes place.record_data_entries({ tenants: [{ id: harbor_books.id, unit: '2B' }] }, map_editor, mode: :edit) ``` -- **An expired inclusion or exclusion is no opinion on that member**, as an expired report is no opinion on any. An expired inclusion no longer keeps the member in, so a complete list from another source that leaves it out removes it, and an expired exclusion or its tombstone stops keeping the member out. A complete list's own inclusion that has expired excludes the member, as the list no longer lists it. What an expired inclusion said about the member is not recorded on it. -- **An expiry on a source** fires the next time that source records the association for the record, under the same rule as for a row, so only a recording made after the edit counts. +- **A partial list no longer covers a member whose inclusion or exclusion has expired**, as an expired member list covers none. An expired inclusion no longer keeps the member in, so a complete list from another source that leaves it out removes it, and an expired exclusion or its tombstone stops keeping the member out. A complete list's own inclusion that has expired excludes the member, as the list no longer lists it. What an expired inclusion said about the member is not recorded on it. +- **An expiry on a source** fires the next time the association is recorded from that source for the record, under the same rule as for a row, so only a recording made after the edit counts. - **A complete list can't exclude a member with an expiry.** Excluding from it drops the member from the list. Record the source's list with `mode: :partial`, or exclude from another source. -- **Edits don't change the report's own expiry.** A report recorded whole with an expiry expires as a whole, edits since included. An edit over a report whose expiry has passed starts a new report, which does not expire. A `precedence:` passed with an edit takes its expiry, as everything else it sets does. -- **The `expiring` scopes can't see these expiries**, which are kept in the report's value. `place.data_entries.membership_expiries(:tenants)` lists them, as `[{ source_id:, key:, expires_at:, expires_on_source_id: }]`. +- **Edits don't change the member list's own expiry.** A member list recorded whole with an expiry expires as a whole, edits since included. An edit over a member list whose expiry has passed starts a new member list, which does not expire. A `precedence:` passed with an edit takes its expiry, as everything else it sets does. +- **The `expiring` scopes can't see these expiries**, which are kept in the member list's value. `place.data_entries.membership_expiries(:tenants)` lists them, as `[{ source_id:, key:, expires_at:, expires_on_source_id: }]`. #### Settings for part of a payload @@ -525,18 +525,18 @@ It costs every write, so add it only where those counts are needed. When two records turn out to be the same, merge what LaForge holds for the duplicate into the survivor, then forge the survivor. ```ruby - report = LaForge.merge(survivor, duplicate) # on a conflict, the duplicate's entry wins, as the argument does in Hash#merge - report = LaForge.reverse_merge(survivor, duplicate) # on a conflict, the survivor's entry wins + result = LaForge.merge(survivor, duplicate) # on a conflict, the duplicate's entry wins, as the argument does in Hash#merge + result = LaForge.reverse_merge(survivor, duplicate) # on a conflict, the survivor's entry wins survivor.forge! duplicate.archive! # the duplicate is left to the application ``` - **Attribute entries and per-record precedence rules** move to the survivor. Where both records have one for the same attribute and source, that's a conflict, and the function decides which is kept, unless only one of them has expired, in which case the one that still counts is kept. The other is destroyed. -- **Members of tracked associations** move to the survivor with their ids and everything recorded on them. Where both records have a member with the same key, as two join rows naming the same category do, the survivor's row is kept, and the duplicate's row is merged into it the same way, forged, and destroyed. Where both have a has_one member with different keys, such as markers with different labels, the merge raises `ArgumentError` before anything moves, since the survivor can hold only one. Remove one of them, or give them the same key, then merge again. The same goes for one source's has_one reports on both records that name members with different keys, as before either record is forged. -- **Each source's reports** on the two records are combined, so the survivor's report names every member either named, and keeps both records' tombstones. Where the reports disagree about a member, the winner's opinion is kept. +- **Members of tracked associations** move to the survivor with their ids and everything recorded on them. Where both records have a member with the same key, as two join rows naming the same category do, the survivor's row is kept, and the duplicate's row is merged into it the same way, forged, and destroyed. Where both have a has_one member with different keys, such as markers with different labels, the merge raises `ArgumentError` before anything moves, since the survivor can hold only one. Remove one of them, or give them the same key, then merge again. The same goes for one source's has_one lists on both records that name members with different keys, as before either record is forged. +- **Each source's member lists** on the two records are combined, so the survivor's list names every member either named, and keeps both records' tombstones. Where the lists cover a member differently, the winner's coverage of it is kept. - **Not touched:** associations LaForge doesn't track, and the duplicate itself, which is left with no LaForge data for the application to archive or destroy. -The report says what moved and what conflicted, as `{ entries_moved:, entries_conflicting: [{ attribute:, source:, kept: }], precedence_rules_moved:, precedence_rules_conflicting: [...], members_moved: { 'tenants' => [ids] }, members_merged: { 'place_categories' => [{ kept:, removed: }] } }`. Both records must be saved, of the same model, and have no unsaved entries or rules. The merge runs in a transaction and is saved immediately. +The result says what moved and what conflicted, as `{ entries_moved:, entries_conflicting: [{ attribute:, source:, kept: }], precedence_rules_moved:, precedence_rules_conflicting: [...], members_moved: { 'tenants' => [ids] }, members_merged: { 'place_categories' => [{ kept:, removed: }] } }`. Both records must be saved, of the same model, and have no unsaved entries or rules. The merge runs in a transaction and is saved immediately. ### Pitfalls @@ -547,7 +547,7 @@ The report says what moved and what conflicted, as `{ entries_moved:, entries_co LaForge can track associations alongside attributes. The API is experimental and may change. -Recording never touches an association. Each source's report is stored as a data entry on the owner, and only `forge` resolves the reports into the association's final state and assigns it, as assigning the collection yourself would. +Recording never touches an association. Each source's member list is stored as a data entry on the owner, and only `forge` resolves the member lists into the association's membership and assigns it, as assigning the collection yourself would. ### belongs_to @@ -559,14 +559,14 @@ A `belongs_to` association needs no declaration. Record the associated record, o ### Declaring has_many and has_one associations -Tracking a `has_many` or `has_one` association is about which records belong to it, its membership. Every member is identified by a key: +Tracking a `has_many` or `has_one` association is about which members (Rails' associated records) belong to it, its membership. Every member is identified by a key: - The rows of a join model, such as those behind a `has_many :through`, are identified by the records they reference. LaForge derives that key from the association's reflection. It is every `belongs_to` on the member other than the one pointing back at the owner, with its type column for a polymorphic one. - Other records, such as tenants, are identified by their primary key. A source names an existing member by its `id`, and a new member by its attributes alone (see [Naming members](#naming-members)). - `key:` names other columns instead, for a natural key such as a weekday, or an identifier every source shares. It also overrides a derived key, as `key: :id` does for an owned model that happens to belong to another record. -- A `has_one` with no key has a single member, which every source reports. +- A `has_one` with no key has a single member, which every source lists. - Member order is written to the integer column the association's scope orders by, such as `position` for `-> { order(:position) }`, in the scope's direction, unless that column identifies the member or its owner. `order_column:` names the column instead, written ascending, which a scope ordering by an expression such as `Arel.sql(...)` needs, and `order_column: false` writes no order. The list's order wins over a position recorded on the member itself. -- `tombstone_matcher:` recognises a member a source excluded when a source reports it again (see [Removed members that sources report again](#removed-members-that-sources-report-again)). +- `tombstone_matcher:` recognizes a member a source excluded when a source lists it again (see [Removed members that sources list again](#removed-members-that-sources-list-again)). ```ruby class Place < ApplicationRecord @@ -601,10 +601,10 @@ An association declared with `primary_key:`, such as `has_many :tenants, as: :oc ### Recording a source's members -A source reports an association's members as a list of hashes. Each hash holds the member's key and whatever the source says about the member, including the member's own tracked associations. By default the list is complete, meaning the source says these are all the members, which excludes every other member that a report has named. Rows no report has named are left alone (see [How membership is decided](#how-membership-is-decided)). `mode:` says otherwise: `:partial` for the members a source knows about (see [Partial reports](#partial-reports)), and `:edit` for changes to the members named (see [Edits](#edits)). +A source provides an association's members as a list of hashes. Each hash holds the member's key and whatever the source says about the member, including the member's own tracked associations. By default the list is complete, meaning the source says these are all the members, which excludes every other member that a member list has named. Rows no member list has named are left alone (see [How membership is decided](#how-membership-is-decided)). `mode:` says otherwise: `:partial` for the members a source knows about (see [Partial lists](#partial-lists)), and `:edit` for changes to the members named (see [Edits](#edits)). ```ruby - # A business directory run. The importer finds the tenants it reported before by its own means, and names them by id. + # A business directory run. The importer finds the tenants it listed before by its own means, and names them by id. tenants = feed['tenants'].map do |row| attributes = { name: row['name'], opening_hours: row['hours'].map { |hours| { day: hours['day'], opens_at: hours['opens'] } } } existing = place.tenants.find_by(external_id: row['id']) @@ -615,18 +615,18 @@ A source reports an association's members as a list of hashes. Each hash holds t place.forge! ``` -The report stored on the place lists the members by key, and keeps what the source says about each one until a forge of the place includes that member. That forge records the member's attributes on the member itself as that source's entries, exactly as if the source had recorded them on the member directly, and removes them from the report. It then forges the member, which applies the member's own tracked associations in turn. So: +The member list stored on the place names the members by key, and keeps what the source says about each one until a forge of the place includes that member. That forge records the member's attributes on the member itself as that source's entries, exactly as if they had been recorded from the source on the member directly, and removes them from the list. It then forges the member, which applies the member's own tracked associations in turn. So: - A member's attributes are resolved on the member, with every other source's entries on it, under its precedence rules. -- Nothing is created for a member that membership excludes. What the source said about it waits in the source's report, and is used if membership later includes it, until the source's next report replaces it. +- Nothing is created for a member that membership excludes. What the source said about it waits in the source's member list, and is used if membership later includes it, until the source's next list replaces it. - The owner can be saved after recording and forged later, and nothing is lost in between. - A new member is built as the association's own `build` would, with the association scope's attributes. - The owner's forge forges only the members it records attributes on. A member edited in some other way is forged on its own. - Members appear in the order of the highest precedence list that includes them. If the association's scope orders by an integer column, LaForge writes the order there, so lists cannot set that column. -#### When a source also records on the member directly +#### When values from a source are also recorded on the member directly -An attribute the member already has a newer value for from the same source, recorded on the member directly, is not overwritten by an older report. An entry's `recorded_at` is when its value was recorded, and `updated_at` keeps its usual meaning. +An attribute the member already has a newer value for from the same source, recorded on the member directly, is not overwritten by an older member list. An entry's `recorded_at` is when its value was recorded, and `updated_at` keeps its usual meaning. ### Naming members @@ -646,21 +646,21 @@ A saved record suits a member created or edited on its own. place.record_data_entries({ tenants: [tenant] }, map_editor, mode: :partial) ``` -Sources rarely know the host application's ids, so an importer finds each record it reported before by its own means, such as the identifier its feed uses, and names it by `id`. +Sources rarely know the host application's ids, so an importer finds each record it listed before by its own means, such as the identifier its feed uses, and names it by `id`. An `id` must be one of the owner's current members. LaForge does not move a record from one owner to another, nor create a record with a given `id`, so naming a record that belongs to another owner, or that does not exist, raises `LaForge::InvalidValue`. Move it outside LaForge, by saving its foreign key, before an import names it under its new owner. #### When a new member is created twice -Until a forge creates a new member, its source's report names it by a placeholder. Once the owner is saved, the placeholder is replaced by the new row's `id`. Forging again before saving reuses the member already built, rather than building another. Recording twice before a forge creates the member once, since a source's report replaces its previous one. A new member is created twice only when: +Until a forge creates a new member, its source's member list names it by a placeholder. Once the owner is saved, the placeholder is replaced by the new row's `id`. Forging again before saving reuses the member already built, rather than building another. Recording twice before a forge creates the member once, since a source's member list replaces its previous one. A new member is created twice only when: -- an importer doesn't find a member it reported before, so it reports it as new again. A complete list then also removes the old row, along with everything else recorded on it; -- two sources each report the same new member before either has been forged. LaForge cannot know they mean the same record, so the second source should find the member the first created; +- an importer doesn't find a member it listed before, so it lists it as new again. A complete list then also removes the old row, along with everything else recorded on it; +- two sources each list the same new member before either has been forged. LaForge cannot know they mean the same record, so the second source should find the member the first created; - a source adds the same new member twice in single-member edits before a forge. -### Partial reports +### Partial lists -A source that knows only some of the members records a partial report, with `mode: :partial`. It includes, or excludes, the members it names and leaves the rest to other sources. When its next report no longer includes a member, that member is removed unless another source includes it. +A source that knows only some of the members provides a partial list, recorded with `mode: :partial`. It includes, or excludes, the members it names and leaves the rest to other sources. When its next list no longer includes a member, that member is removed unless another source includes it. ```ruby # A business registry that knows only the businesses it has registered @@ -672,7 +672,7 @@ A source that knows only some of the members records a partial report, with `mod ### Edits -An editor changes only what it names. `mode: :edit` applies a payload as single-member edits, at every level. Each member listed under `include:` is added to the source's report, with the attributes given, each under `exclude:` is removed even if other sources include it, and each under `forget:` drops the source's opinion on it. A plain list includes each member. The rest of the source's report stays as it is. +An editor changes only what it names. `mode: :edit` applies a payload as single-member edits, at every level. Each member listed under `include:` is added to the source's member list, with the attributes given, each under `exclude:` is removed even if other sources include it, and the source's member list stops covering each under `forget:`. A plain list includes each member. The rest of the source's member list stays as it is. ```ruby place.record_data_entries({ @@ -688,7 +688,7 @@ An editor changes only what it names. `mode: :edit` applies a payload as single- place.forge! ``` -A member's nested lists in an edit are edits too, so `opening_hours` above changes two days and leaves the others to the sources that report them. +A member's nested lists in an edit are edits too, so `opening_hours` above changes two days and leaves the others to the sources that provide them. One member at a time is the same call with one member in it. @@ -707,7 +707,7 @@ Edits made before a forge add up. Each keeps what it said about a member, and th #### What an edit can't do -- **Forget a member of a complete list.** A complete list has an opinion on every member, so forgetting one raises. Record the source's list with `mode: :partial` first. +- **Forget a member of a complete list.** A complete list covers every member, so forgetting one raises. Record the source's list with `mode: :partial` first. - **Exclude a member keyed by primary key before it exists.** Exclusions name members by key only. ### Recording a Rails form @@ -730,11 +730,11 @@ Edits also accept the shapes a Rails form submits, at every level, so a controll - Ids may be strings, as forms submit them. - `*_attributes` or `*_ids` for an association that is not tracked raise `LaForge::InvalidAttributeName`, rather than being recorded as an attribute LaForge cannot forge. Save such associations outside LaForge. In another mode, these shapes raise `ArgumentError`, since they describe edits. -Anything else in the form, such as an association's own metadata stored outside the member, is up to the controller. A has_one takes a complete report, so it can't be edited. +Anything else in the form, such as an association's own metadata stored outside the member, is up to the controller. A has_one takes a complete list, so it can't be edited. -### Removed members that sources report again +### Removed members that sources list again -Excluding a member keyed by primary key destroys it, per `dependent:`, like any removed member. Its exclusion has to outlast the row. Otherwise, when the business directory next reports the tenant, its importer finds nothing, reports it as new, and it comes back. So the exclusion keeps a **tombstone** in the row's place, holding what the member was, and the association declares how to recognise a tombstone in what a source reports. +Excluding a member keyed by primary key destroys it, per `dependent:`, like any removed member. Its exclusion has to outlast the row. Otherwise, when the business directory next lists the tenant, its importer finds nothing, lists it as new, and it comes back. So the exclusion keeps a **tombstone** in the row's place, holding what the member was, and the association declares how to recognize a tombstone in what a source lists. ```ruby class Place < ApplicationRecord @@ -746,32 +746,32 @@ Excluding a member keyed by primary key destroys it, per `dependent:`, like any end ``` -The matcher receives an unsaved member built from what a source reported, and an unsaved member for each tombstone, built from what it was. It returns the tombstone the member is, or nil. It can use a fuzzy matcher, such as one that scores addresses. +The matcher receives an unsaved member built from what a source provided, and an unsaved member for each tombstone, built from what it was. It returns the tombstone the member is, or nil. It can use a fuzzy matcher, such as one that scores addresses. ```ruby place.record_data_entries({ tenants_attributes: [{ id: 12, _destroy: '1' }] }, map_editor, mode: :edit) - place.forge! # Tenant 12 is destroyed, and the map editor's report keeps its tombstone + place.forge! # Tenant 12 is destroyed, and the map editor's list keeps its tombstone place.data_entries.tombstones(:tenants) # => [# 'Corner Café', ... }, excluded_at: ...>] ``` -- **Each forge** checks the new members that reports name against the tombstones, before membership is decided. A member a tombstone matches is decided as the excluded member, so a source that ranks below the excluding source cannot add it again, and a source that ranks above it can, which creates it and drops the tombstone. -- **Code that creates members itself,** such as an importer that builds and saves a tenant rather than reporting it on the place, asks before saving a new one. `LaForge.tombstone_blocking(tenant, source)` returns the tombstone that keeps it out, with the same precedence rule, or nil. -- **A tombstone lasts** through the excluding source's later reports, until the source forgets it with `place.record_data_entries({ tenants: { forget: [tombstone] } }, map_editor, mode: :edit)`, which lets sources add the member again. Withdrawing the source drops its tombstones too. -- **Associations that need no tombstones:** join rows and associations with `key:` name something that outlasts the row, so their exclusions hold already, and a has_one takes only complete reports. Declaring `tombstone_matcher:` on one raises. +- **Each forge** checks the new members that member lists name against the tombstones, before membership is decided. A member a tombstone matches is decided as the excluded member, so a source that ranks below the excluding source cannot add it again, and a source that ranks above it can, which creates it and drops the tombstone. +- **Code that creates members itself,** such as an importer that builds and saves a tenant rather than listing it on the place, asks before saving a new one. `LaForge.tombstone_blocking(tenant, source)` returns the tombstone that keeps it out, with the same precedence rule, or nil. +- **A tombstone lasts** through the excluding source's later lists, until the source forgets it with `place.record_data_entries({ tenants: { forget: [tombstone] } }, map_editor, mode: :edit)`, which lets sources add the member again. Withdrawing the source drops its tombstones too. +- **Associations that need no tombstones:** join rows and associations with `key:` name something that outlasts the row, so their exclusions hold already, and a has_one takes only complete lists. Declaring `tombstone_matcher:` on one raises. Excluding a member keyed by primary key without a `tombstone_matcher` raises `ArgumentError` when the exclusion is recorded, naming what to declare, rather than letting the member come back at the next import. ### How membership is decided -Membership is decided member by member. The highest precedence report with an opinion on a member decides it. A complete list has an opinion on every member, and a partial report only on the members it names. So a complete list from a higher precedence source excludes members that lower partial reports add, and a source that only adds members needs to rank above the complete lists it adds to. +Membership is decided member by member. The highest precedence member list that covers a member decides it. A complete list covers every member, and a partial list only the members it names. So a complete list from a higher precedence source excludes members that lower partial lists add, and a source that only adds members needs to rank above the complete lists it adds to. -**Rows no report has named are left alone,** by complete lists as well as partial reports. LaForge cannot tell a row created before an association was tracked from one created outside LaForge since, so it removes neither. A complete list only excludes members that some report has named, by its key, in an include, exclude or withdrawal. +**Rows no member list has named are left alone,** by complete lists as well as partial lists. LaForge cannot tell a row created before an association was tracked from one created outside LaForge since, so it removes neither. A complete list only excludes members that some member list has named, by its key, in an include, exclude or withdrawal. -- **Adopting LaForge on an existing model needs no backfill.** An importer's first complete report names the rows it finds, by `id`, and from then on a complete list that stops naming one removes it. Rows that no feed mentions stay until someone removes them explicitly, such as an editor's exclusion. -- **A row nobody reports stays,** such as one added by hand outside LaForge, until a source names it. -- **Destroying a report forgets what it named.** Its members are then rows no report has named, and are left alone. To retract what a source said, so that its members are removed unless another source includes them, withdraw it instead (see [When a source stops reporting](#when-a-source-stops-reporting)). +- **Adopting LaForge on an existing model needs no backfill.** An importer's first complete list names the rows it finds, by `id`, and from then on a complete list that stops naming one removes it. Rows that no feed mentions stay until someone removes them explicitly, such as an editor's exclusion. +- **A row nobody lists stays,** such as one added by hand outside LaForge, until a source names it. +- **Destroying a member list forgets what it named.** Its members are then rows no member list has named, and are left alone. To retract what a source said, so that its members are removed unless another source includes them, withdraw it instead (see [When a source stops listing members](#when-a-source-stops-listing-members)). -A `precedence:` given while recording covers everything the payload says, as it does for attributes recorded directly: the report's say on membership, and every attribute of every member hash, down through nested lists. When the forge records a member's attributes, it gives each one a per-record rule for that source. +A `precedence:` given while recording covers everything the payload says, as it does for attributes recorded directly: the member list's say on membership, and every attribute of every member hash, down through nested lists. When the forge records a member's attributes, it gives each one a per-record rule for that source. ```ruby place.record_data_entries({ tenants: [{ id: 7, registration_number: 'R-10', unit: '2B' }] }, business_registry, precedence: 100) @@ -788,30 +788,30 @@ A `precedence:` given while recording covers everything the payload says, as it ### Keys -- A report cannot name the same member twice. -- With a key other than the primary key, every member a report names needs a value for each key column. A current row missing a key value cannot be identified across sources, so forging leaves it alone, even when a complete list excludes every other member. +- A member list cannot name the same member twice. +- With a key other than the primary key, every member a member list names needs a value for each key column. A current row missing a key value cannot be identified across sources, so forging leaves it alone, even when a complete list excludes every other member. - Forging raises `LaForge::DuplicateMemberKey` when several current rows share a key, since LaForge cannot tell which one a source means. A unique index on the owner and key columns prevents this. -- Keys other than the primary key must be stable. A row whose key is edited becomes a different member. A source reporting the old key gets a new row, and a complete list removes the edited one. -- With a key other than the primary key, a member belongs to one owner. When a source starts reporting a member under another owner, the new owner gets a new row, built from what the source reports, and the old owner's row is removed when its sources stop including it, along with everything else recorded on it. +- Keys other than the primary key must be stable. A row whose key is edited becomes a different member. A source listing the old key gets a new row, and a complete list removes the edited one. +- With a key other than the primary key, a member belongs to one owner. When a source starts listing a member under another owner, the new owner gets a new row, built from what the source provides, and the old owner's row is removed when its sources stop including it, along with everything else recorded on it. ### When a member can't be created -When membership includes a member named by `id` that is no longer one of the owner's members, because it was destroyed or moved after the source named it, the forge skips it, logs a warning and lists it under `skipped:` in `forge_diff`. LaForge does not recreate a record with a given `id`; a source that reports it without one creates a new member. +When membership includes a member named by `id` that is no longer one of the owner's members, because it was destroyed or moved after the source named it, the forge skips it, logs a warning and lists it under `skipped:` in `forge_diff`. LaForge does not recreate a record with a given `id`; a source that lists it without one creates a new member. -With a key other than the primary key, when membership includes a member that does not exist and no report says anything about it besides its key, the forge builds the member from its key. This happens when a source lists a member by key alone, and when a member was removed and destroyed, taking what was recorded on it, and membership later includes it again, such as when a map editor undoes an exclusion. If that member is invalid, as an opening hour without a time would be, the forge skips it in the same way, until a source reports it with its attributes. +With a key other than the primary key, when membership includes a member that does not exist and no member list says anything about it besides its key, the forge builds the member from its key. This happens when a source lists a member by key alone, and when a member was removed and destroyed, taking what was recorded on it, and membership later includes it again, such as when a map editor undoes an exclusion. If that member is invalid, as an opening hour without a time would be, the forge skips it in the same way, until a source lists it with its attributes. -### When a source stops reporting +### When a source stops listing members A member that membership no longer includes is removed the way assigning the collection without it would remove it, following the association's `dependent:` option: destroyed for `:destroy`, deleted for `:delete_all`, and detached by nullifying its foreign key otherwise. Everything recorded on a destroyed member goes with it. -When a source's report drops a member that another source keeps, because its complete list leaves the member out, or its partial report excludes it or stops including it, the forge withdraws the source from that member. Its entries on the member are destroyed when the owner is saved, and its reports for the member's own tracked associations are withdrawn in turn, down every level. Each of the member's attributes then falls back to the other sources' entries. An attribute no other source has an entry for keeps its value, as it does when `withdraw_source` destroys a record's entries, so withdrawing never leaves a member invalid. +When a source's member list drops a member that another source keeps, because its complete list leaves the member out, or its partial list excludes it or stops including it, the forge withdraws the source from that member. Its entries on the member are destroyed when the owner is saved, and its member lists for the member's own tracked associations are withdrawn in turn, down every level. Each of the member's attributes then falls back to the other sources' entries. An attribute no other source has an entry for keeps its value, as it does when `withdraw_source` destroys a record's entries, so withdrawing never leaves a member invalid. -A source that records on a member directly, and also reports a complete list on its owner, should list that member, since its complete list is its whole say on which members it describes. +When values from a source are recorded on a member directly and a complete list from the same source is recorded on its owner, that list should include the member, since a complete list covers every member. -When an import no longer reports a record at all, choose between two things, as for attributes: +When an import no longer provides a record at all, choose between two things, as for attributes: -- **Retract what the source said:** `withdraw_source` destroys the source's attribute entries and withdraws every member it reported, so the next forge removes those members unless another source includes them. What its reports said about members not yet recorded on them is dropped, and the next forge withdraws the source from every member another source keeps, as above. -- **Stop tracking what the source said:** destroying the source's entries leaves the association as it is. The members the source added stay, as attribute values stay when their entries are destroyed, and are treated from then on as rows no report has named, which every report leaves alone. +- **Retract what the source said:** `withdraw_source` destroys the source's attribute entries and withdraws every member it listed, so the next forge removes those members unless another source includes them. What its member lists said about members not yet recorded on them is dropped, and the next forge withdraws the source from every member another source keeps, as above. +- **Stop tracking what the source said:** destroying the source's entries leaves the association as it is. The members the source added stay, as attribute values stay when their entries are destroyed, and are treated from then on as rows no member list has named, which every member list leaves alone. ```ruby place.withdraw_source(business_registry) @@ -820,32 +820,32 @@ When an import no longer reports a record at all, choose between two things, as ### How it works -Each source's report for an association is one data entry on the owner, named after the association: +Each source's member list for an association is one data entry on the owner, named after the association: -- A complete report is stored as an array of member keys, such as `[{ 'id' => 7 }, { 'id' => 9 }]`, or as `{ 'include' => [...], 'withdrawn' => [...] }` while it remembers members it used to include. -- A partial report is stored as `{ 'partial' => true, 'include' => [...], 'exclude' => [...], 'withdrawn' => [...] }`, with `'tombstones' => [...]` when it has excluded members whose rows are gone; they are excluded by `{ 'tombstone' => '' }`. `withdrawn` holds the keys of members the source used to include, until the next forge has acted on them; see below. -- What a source said about a member besides its key waits under `'pending'`, with when it was said, until a forge records it on the member. A complete report with anything pending is stored as `{ 'include' => [...], 'pending' => [...] }`. +- A complete list is stored as an array of member keys, such as `[{ 'id' => 7 }, { 'id' => 9 }]`, or as `{ 'include' => [...], 'withdrawn' => [...] }` while it remembers members it used to include. +- A partial list is stored as `{ 'partial' => true, 'include' => [...], 'exclude' => [...], 'withdrawn' => [...] }`, with `'tombstones' => [...]` when it has excluded members whose rows are gone; they are excluded by `{ 'tombstone' => '' }`. `withdrawn` holds the keys of members the source used to include, until the next forge has acted on them; see below. +- What a source said about a member besides its key waits under `'pending'`, with when it was said, until a forge records it on the member. A complete list with anything pending is stored as `{ 'include' => [...], 'pending' => [...] }`. - A new member is keyed `{ 'new' => '' }` until the forge that creates it is saved. Everything else is ordinary data entries on the members themselves. A forge of the owner then: -1. Names each new member that the association's tombstone matcher recognises by that tombstone's key. -2. Decides membership member by member, from the highest-precedence report with an opinion on each. -3. Finds or builds each included member, and records on it what the reports hold pending for it, oldest first. -4. Withdraws each source whose report excludes or withdrew a member it keeps from that member. +1. Names each new member that the association's tombstone matcher recognizes by that tombstone's key. +2. Decides membership member by member, from the highest-precedence member list that covers each. +3. Finds or builds each included member, and records on it what the member lists hold pending for it, oldest first. +4. Withdraws each source whose member list excludes or withdrew a member it keeps from that member. 5. Forges the members it recorded on or withdrew from, which applies their own tracked associations the same way. -6. Removes what it recorded from the reports' pending, and drops the withdrawn keys it has acted on. A partial report left with nothing in it is destroyed. -7. Keeps a tombstone for each member keyed by primary key that it removes because a report excludes it, and names it by the tombstone's key in every report. +6. Removes what it recorded from the member lists' pending, and drops the withdrawn keys it has acted on. A partial list left with nothing in it is destroyed. +7. Keeps a tombstone for each member keyed by primary key that it removes because a member list excludes it, and names it by the tombstone's key in every member list. 8. Removes the members membership no longer includes, per the association's `dependent:` option. -Nothing is written until the owner is saved. Once it is, the reports name new members by their ids, and the entries step 4 withdrew are destroyed. +Nothing is written until the owner is saved. Once it is, the member lists name new members by their ids, and the entries step 4 withdrew are destroyed. -The withdrawn keys exist because LaForge leaves alone rows that no report has named. A member that a report stops naming, or that `withdraw_source` withdraws, would otherwise look like such a row, and stay. The withdrawn keys mark it for one forge, which removes it unless another source includes it, and withdraws the source from it if it is kept. Then they go. +The withdrawn keys exist because LaForge leaves alone rows that no member list has named. A member that a member list stops naming, or that `withdraw_source` withdraws, would otherwise look like such a row, and stay. The withdrawn keys mark it for one forge, which removes it unless another source includes it, and withdraws the source from it if it is kept. Then they go. ### Inspecting associations ```ruby - place.source_values(:tenants) # => { business_directory => [{ 'id' => 7, 'name' => 'Corner Café' }], ... }, including what reports hold for members not yet recorded on + place.source_values(:tenants) # => { business_directory => [{ 'id' => 7, 'name' => 'Corner Café' }], ... }, including what member lists hold for members not yet recorded on Place.source_values(:tenants, records: places) # the same for many records, in a constant number of queries place.member_sources(:tenants, id: 7) # => { membership: map_editor, listed_by: [map_editor, business_directory], excluded_by: [], @@ -853,7 +853,7 @@ The withdrawn keys exist because LaForge leaves alone rows that no report has na place.forge_diff['tenants'] # => { added: [...], removed: [...], changed: [...] }, and skipped: when any would be; a new member is added as its attributes ``` -A member's own `data_source_names` and `data_source?` count only the sources with entries on the member. A source that lists the member without saying anything about it, such as a map editor including it by key, is not among them; `member_sources` on the owner reports it. So an orphan check that asks whether another source still claims a member should check both. +A member's own `data_source_names` and `data_source?` count only the sources with entries on the member. A source that lists the member without saying anything about it, such as a map editor including it by key, is not among them; `member_sources` on the owner names it. So an orphan check that asks whether another source still claims a member should check both. ```ruby tenant.withdraw_source(business_registry)