feat: let a record's recorded values and precedence orders expire, at a time or on a source's next recording by njakobsen · Pull Request #18 · combinaut/laforge

njakobsen · 2026-10-22T18:37:51Z

Stacked on #16. Lets an editor's change to one record expire, so a value set by hand or a custom source order holds only for a while, or until a source's next recording replaces it.

Changes

What can expire: data entries (values, association reports and their tombstones) and record-level precedence rules take expires_at: (a time) and expires_on: (a source) in record_data_entries, record_data_entry, update_precedence_order and update_attribute_source_precedence. Model-wide and global rules raise. Recording or setting the order again replaces the expiry; leaving the options out clears it.
expires_on: ends the change the next time that source records the same attribute, group or association for the record, counting only recordings made after the change: a change not saved yet, or made again since it was saved, a new expiry included, counts as made now, and one recorded from a report kept pending on an owner as made when the report was. Recording a value again with a new expiry counts as recording it.
Passive: expired rows are skipped by the forge and kept. An expired association report counts as if it did not exist, like an inactive source's: nothing is removed because of the expiry, and its exclusions and tombstones stop blocking.
Spread over the payload: precedence and expiry travel as one LaForge::RecordingSettings value, applied in one place to every entry and rule a call records, member attributes and the precedence rule an edit writes included. An expiry on part of an attribute group covers the whole group.
Per member in edits: each member an edit includes or excludes keeps its own expiry in the report's expiries, listed by data_entries.membership_expiries(:name).
LaForge.with_settings(value, precedence:, expires_at:, expires_on:) gives part of a payload its own settings, so one call can record an edit form with mixed settings, a Rails form's *_attributes rows included, whose reject_if: sees their plain values. Misuses raise, saying where, such as settings inside a *_ids list.
Following changes: data_entry_changed.laforge carries expires_at and expires_on_source_id, and also publishes when only the expiry changes. Nothing is published when a change expires by time; scheduling re-forges is the application's, using the expiring(before:), expired and unexpired scopes.
Migration: rails generate laforge:expiry adds the columns to laforge_data_entries and laforge_precedence_rules, adds recorded_at to precedence rules, filled from updated_at, and gives LaForge's timestamps sub-second precision (slow on large tables). New installs get sub-second timestamps whatever the Rails default.

Follow-ups

When a report's expires_on fires, the member entries recorded from that report keep their own expiry, which fires only when that source records the member's attribute; propagating the firing needs a link between them.
A membership expiry inside a member's nested list, materialised from a report kept pending on the owner, counts as made at the forge rather than at its report.
LaForge.merge combining two reports where an inclusion or exclusion has expired can keep an expired opinion's expiry on the other report's live one.

UAT Steps

bundle exec rspec: 379 examples, 0 failures, 2 pending.

Closes

Unreported

Special Handling

This PR requires user testing
Include this PR in the changelog

🤖 Generated with Claude Code

Copilot

🟡 Changes recommended

Unresolved correctness issues affect expiry timing, member propagation, merge behavior and wrapped form payloads.

6 open issues

Treat expiry changes as recordings · New
Track unsaved changes to existing expiry rules · New
Propagate association expiry to member changes · New
Preserve unchanged expiry recording times · New
Resolve effective opinions before selecting expiry metadata · New
Unwrap nested settings before processing attributes · New

What changed in this PR

Adds expiry to LaForge’s recorded values and record-level precedence rules, extending association tracking with per-member expiry and mixed settings within a payload.

Changes:

Supports time-based and source-triggered expiry while retaining expired data.
Carries settings through association reports, member attributes, merges and change events.
Adds upgrade migrations, documentation and expiry-focused specs.

File	Description
`spec/spec_helper.rb`	Adds expiry columns and precise timestamps to the test schema.
`spec/lib/usage_spec.rb`	Covers value and precedence expiry, validation and events.
`spec/lib/association_capabilities_spec.rb`	Covers report expiry, member settings, merges and previews.
`Readme.md`	Documents expiry behavior, settings and migration steps.
`lib/laforge/recording_settings.rb`	Introduces shared recording settings and payload wrappers.
`lib/laforge/precedence_rule.rb`	Filters expired rules and restricts expiry to individual records.
`lib/laforge/payload.rb`	Preserves wrappers during payload translation.
`lib/laforge/merge.rb`	Adds expiry-aware conflict handling and membership metadata.
`lib/laforge/member_list.rb`	Stores and evaluates per-member expiries.
`lib/laforge/expiring.rb`	Defines shared expiry validation, scopes and behavior.
`lib/laforge/data_entry.rb`	Adds expiry support and expiry-aware change events.
`lib/laforge/association_tracking.rb`	Applies settings and expiry throughout association processing.
`lib/laforge/association_config.rb`	Validates settings placement in member payloads.
`lib/laforge/activerecord.rb`	Integrates expiry into recording, forging and precedence APIs.
`lib/laforge.rb`	Loads the new expiry and settings modules.
`lib/generators/laforge/templates/precedence_rules_migration.rb.tt`	Uses sub-second precedence-rule timestamps.
`lib/generators/laforge/templates/migration.rb.tt`	Adds expiry columns and precise timestamps for new installs.
`lib/generators/laforge/templates/expiry_migration.rb.tt`	Adds the expiry schema upgrade and timestamp conversion.
`lib/generators/laforge/expiry_generator.rb`	Exposes the expiry migration generator.

_{Review effort: Balanced}

Copilot

🟡 Changes recommended

Expiry renewal, pending-report timing, and wrapped form-payload handling have unresolved correctness issues.

4 open issues

6 resolved since last review

_{Review effort: Balanced}

Copilot

🟡 Changes recommended

Pending-report replay uses incorrect change times for precedence rules and nested membership expiries, leaving them active after a later triggering recording.

2 open issues

4 resolved since last review

_{Review effort: Balanced}

Copilot

🟡 Changes recommended

Source-triggered expiry timing and transient state left by previews can cause overrides to expire incorrectly or remain active.

3 open issues

2 resolved since last review

_{Review effort: Balanced}

Copilot

🟡 Changes recommended

Ordinary JSON values can lose data, and source-triggered expiry can fail after forge bookkeeping or delayed report replay.

6 open issues

_{Review effort: Balanced}

Copilot

🔵 Needs a closer look

The interactions among expiry, deferred recordings, association membership, and schema upgrades warrant human review before approval.

9 open issues

_{Review effort: Balanced}

Copilot

🟡 Changes recommended

Pending-report replay and nested settings validation still have correctness issues that can retain expired overrides or prevent forging.

12 open issues

_{Review effort: Balanced}

Copilot

🔵 Needs a closer look

Cross-cutting expiry timing, pending-report replay, and schema migration behavior warrant final human review.

12 open issues

_{Review effort: Balanced}

Copilot

🟡 Changes recommended

Renewing an entry or precedence rule after an unsaved expiry fires can leave its old change time intact and cause immediate re-expiry.

13 open issues

_{Review effort: Balanced}

njakobsen · 2026-10-26T22:17:07Z

+    # Sets the entry's `recorded_at` and notes the change made again (see Expiring#remade) when this recording creates
+    # the entry or changes its value or expiry. A recording replayed from a pending report makes it at `recorded_at`.
+    def note_recorded(data_entry, recorded_at)
+      return unless data_entry.new_record? || %w[value expires_at expires_on_source_id].any? { |column| data_entry.will_save_change_to_attribute?(column) }


Fixed in 40037ed: recording an entry or rule again compares its value, precedence and expiry with their in-memory state as well as the saved one, so setting back an expiry that fired in memory counts as renewing it.

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

… a time or on a source's next recording Issue: an editor's change to one record, such as a value set by hand or a custom source order for a field, had to be undone by someone remembering to do it, even when it was only meant to hold for a while or until a source's next import replaced it. A raised source order also does nothing unless that source has a value, so the expiry is needed on both kinds of change. Data entries (values, association reports and their tombstones) and precedence rules on one record now take `expires_at:` (a time) and `expires_on:` (a source) wherever the change is made: `record_data_entries`, `record_data_entry`, `record.update_precedence_order` and `record.update_attribute_source_precedence`, alone or together in one save. A precedence passed with `precedence:` while recording takes the same expiry. Recording the value or setting the order again replaces the expiry, and leaving the options out clears it. `expires_on:` expires the change the next time that source records the same attribute, attribute group or association for the record, whatever it reports, by setting `expires_at` to the recording's `recorded_at` and keeping `expires_on_source_id`. Only a recording made after the change counts, the change being made when an entry was last recorded, its `recorded_at`, which a later save such as a forge's bookkeeping does not move, or when a rule's precedence or expiry was last set, its new `recorded_at`, or now while it is unsaved or a recording has made it again since, a new expiry included, or when its report was for an entry or precedence rule recorded from a report kept pending on an owner, so a report the source made earlier and kept pending on an owner does not end a change made since when the owner is forged. Model-wide and global rules cannot expire, so the precedence cache stays independent of the clock; giving one an expiry raises an `ArgumentError` saying so, as do an expiry that is not a time, one on the source recording, and one on a removal. Expired rows are left out of forging and kept, like an inactive source's: an attribute falls back to the next source, `PrecedenceRule::Resolver` skips an expired record rule so the next level's order applies, and `source_precedences` lists only rules that still count. An expired association report counts as if it did not exist, exactly as an inactive source's report: nothing is removed because of the expiry itself, so a member only it named is left alone as a row no report names, and a member another report excludes is removed once the expired report no longer keeps it in. Its exclusions and tombstones stop counting, so they stop blocking, `LaForge.tombstone_blocking` included, and a report recorded or edited over an expired one starts afresh. What an expiring report says about members is recorded on them with the same expiry, through the pending items, as a precedence passed with it is. The precedence and expiry travel together as one `LaForge::RecordingSettings` value, built once where recording starts, passed down through attributes, attribute groups, belongs_to columns, reports, edits, nested lists and pending items unchanged, and applied in one place to each entry and to the record's precedence rule; pending items store it through one `to_item`/`from_item` pair, keeping the existing `precedence` key, so a later setting for a whole payload only changes that object. An expiry on part of an attribute group applies to the whole group. `LaForge.merge` keeps a row that still counts over an expired winner, and adds nothing from an expired report to one that still counts. In an edit, the expiry is given to each member it includes or excludes rather than to the source's report: each inclusion and exclusion keeps its own expiry in the report's `expiries`, as `{ 'key', 'recorded_at', 'expires_at', 'expires_on_source_id' }`, so editing one member leaves the expiries earlier edits gave the others alone, and editing the same member again replaces or clears its own. An expired inclusion or exclusion is no opinion on that member, so an expired inclusion no longer keeps the member in and an expired exclusion and its tombstone stop keeping it out; a complete list's expired inclusion excludes the member, and what an expired inclusion said about the member is not recorded on it. Its `expires_on` fires when the named source next records the association, under the same newer-than-the-change rule, each item keeping its own recording time until it is changed. Edits leave the report entry's own expiry, which belongs to a report recorded whole, and an edit over an expired report starts a new one; a `precedence:` passed with an edit takes its expiry, as everything else the edit sets does, through `RecordingSettings#with_entry_expiry`, which lets the report entry keep its own. A complete list cannot exclude a member with an expiry, which raises saying to record the list with `mode: :partial`. `data_entries.membership_expiries(:name)` lists these expiries, which the `expiring` scopes cannot see, and `LaForge.merge` keeps them with the member's opinion. Recording a value again with a new expiry counts as recording it, so an older report kept pending on an owner does not undo it. `LaForge.with_settings(value, precedence:, expires_at:, expires_on:)` gives any part of a `record_data_entries` or `record_data_entry` payload settings of its own, a Rails form's `*_attributes` rows included, whose `reject_if:` sees their values with the settings taken off, which replace the call's for everything recorded from it: an attribute's value, a belongs_to, a member with its attributes, nested lists and inclusion, an exclusion, or a nested list. It is unwrapped in one place, `RecordingSettings#for`, where the payload is split, and a wrapper inside what a report keeps pending on an owner is stored as JSON and read back the same way. A wrapper's precedence on a member applies to the member's own attributes, since precedence ranks a source for the whole association; wrapping a member with no attributes in a different precedence, wrapping an attribute group's attributes differently, wrapping part of a value, part of a member's key, a list rather than its members, or a member to forget, and naming the recording source in `expires_on:`, each raise an `ArgumentError` saying where, as does a recorded hash with the `_laforge_settings` key, which a pending report keeps a wrapper's settings under. Expiry takes effect at the record's next forge; scheduling is the application's. `DataEntry` and `PrecedenceRule` gain the `expiring(before:)`, `expired` and `unexpired` scopes, from the new `LaForge::Expiring` concern. LaForge publishes nothing when a change expires by time, and the forge writes nothing because of an expiry: `data_entry_changed.laforge` carries the entry's `expires_at` and `expires_on_source_id` in its payload, taken when the write is made and published once it is committed like every change event, so the application's change tracking can note when the change it records will expire. An entry now also publishes when only its expiry changes, such as when the same value is recorded again with a new or cleared expiry, or when an expiry on a source fires and the waiting entry is saved. `forge_diff`'s preview of a member's pending report restores the expiry columns of the member's entries and precedence rules along with the rest, since recording expires the changes waiting for the source. `rails generate laforge:expiry` adds `expires_at` (indexed) and `expires_on_source_id` (a foreign key to `laforge_data_sources`, nulled when the source is deleted) to `laforge_data_entries` and `laforge_precedence_rules` for existing installs, and `recorded_at` to `laforge_precedence_rules`, filled from `updated_at`, and gives `created_at` and `updated_at` on `laforge_data_sources`, `laforge_data_entries` and `laforge_precedence_rules` sub-second precision, reversibly, matching the `recorded_at` of entries and rules that an expiry on a source compares; `laforge:install` and `laforge:precedence_rules` now create sub-second timestamps whatever the Rails default. The README documents it under "Expiring a change". Further decisions: - A precedence rule records when its precedence or expiry was last set in its own `recorded_at` column, as a data entry does, rather than taking its `updated_at`. A rule recorded from a pending report keeps the report's time through later saves, so a change to an entry and its precedence in one call expires as one. `laforge:install` creates the column, and `laforge:expiry` adds it and fills it from `updated_at`. - A nested list edit recorded from a pending report makes its precedence rule at the report's time, for an empty edit too, so the source's later report in the same forge expires it. A direct edit makes it now. - A change recorded from a pending report with an expiry on a source checks that source's entries for the same attribute, attribute group or association on the record. When one was recorded after the report, the change expires at the earliest such time, since that recording found nothing waiting. The source's entries are left as they are. - An exclusion's saved expiry keeps its time when an unsaved forge renames the member to a tombstone, so the source's next recording still expires it. A renamed item matches the saved one in everything but its key. - A misplaced wrapper inside a nested list raises when the call records, as at the top level. Only a wrapper around the whole list is taken off before its members are checked, and each member is read through its own wrapper. - A pending report whose expiry names a source deleted since is still forged. The deleted source's id is dropped from the items and their stored wrappers, with one lookup of the ids they name, and `expires_at` is kept. - Recording a change again as it was saved, after its expiry on a source fired in memory, makes it again, for entries and rules alike. The value, precedence and expiry are compared with what they were in memory as well as with what is saved. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Issue: `Expiring a change` mixed situations that only arise in unusual cases into its opening paragraph and bullets, such as a report kept pending on an owner, a value expiring on the source recording it and an expired association report, and several bullets used a colon inside a sentence. The section now opens with the common case, and those situations follow in `Which recordings end an expiry on a source` and `Expiring an association report`. The bullets that used a colon inside a sentence now start a new sentence instead. Refs #18 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot

🔵 Needs a closer look

Cross-cutting replay, membership, and schema changes require human review, with unresolved expiry correctness issues remaining.

5 open issues

11 resolved since last review

_{Review effort: Balanced}

+      raise ArgumentError, "#{where} with #{removing} removes it, so there is nothing to expire; pass expires_at: or expires_on: only when setting it" if removing
+      raise ArgumentError, "#{where}: expires_at: must be a time, such as 2.weeks.from_now, not #{expires_at.inspect}" unless expires_at.nil? || expires_at.acts_like?(:time)
+
+      expires_on_source_id = expires_on && DataSource.normalize_source_id(expires_on)


-        rule.precedence = precedence
+        columns = %w[precedence expires_at expires_on_source_id]
+        before = rule.attributes.slice(*columns) # in memory, so setting back an expiry that fired since the save is a change
+        settings.apply_to_rule(rule)


+        else MemberList::Complete.replacing(report(source), config, include: keys, pending: pending, expiries: expiries)
        end
-      owner.laforge.write_data_entry(config.name, value, source, precedence: precedence, recorded_at: recorded_at)
+      owner.laforge.write_data_entry(config.name, value, source, settings: settings, recorded_at: recorded_at)


Copilot AI balanced review requested due to automatic review settings October 22, 2026 18:37

njakobsen added this pull request to stack #17 October 22, 2026 18:38

Copilot started reviewing on behalf of njakobsen October 22, 2026 18:38

View session

njakobsen mentioned this pull request Oct 22, 2026

feat: track association membership across sources #16

Merged

1 of 2 tasks

Copilot AI reviewed Oct 22, 2026

View reviewed changes

This was referenced Oct 23, 2026

feat: rank groups of records by rulesets, named copies of the model-wide and global rules #19

Merged

docs: usage examples and specs for merging several sources into one record #14

Merged

Copilot AI balanced review requested due to automatic review settings October 23, 2026 07:41

njakobsen force-pushed the feat/expiring-changes branch from a60f5ff to 256eb20 Compare October 23, 2026 07:41

Copilot started reviewing on behalf of njakobsen October 23, 2026 07:41

View session

Copilot AI reviewed Oct 23, 2026

View reviewed changes

Comment thread lib/laforge/activerecord.rb Outdated

Comment thread lib/laforge/activerecord.rb Outdated

Comment thread lib/laforge/payload.rb Outdated

Comment thread lib/laforge/payload.rb Outdated

Copilot AI balanced review requested due to automatic review settings October 23, 2026 08:07

njakobsen force-pushed the feat/expiring-changes branch from 256eb20 to ab0410c Compare October 23, 2026 08:07

Copilot started reviewing on behalf of njakobsen October 23, 2026 08:07

View session

Copilot AI reviewed Oct 23, 2026

View reviewed changes

Comment thread lib/laforge/activerecord.rb Outdated

Comment thread lib/laforge/member_list.rb Outdated

Copilot AI balanced review requested due to automatic review settings October 23, 2026 08:22

njakobsen force-pushed the feat/expiring-changes branch from ab0410c to ea7bad5 Compare October 23, 2026 08:22

Copilot started reviewing on behalf of njakobsen October 23, 2026 08:23

View session

Copilot AI reviewed Oct 23, 2026

View reviewed changes

Comment thread lib/laforge/activerecord.rb Outdated

Comment thread lib/laforge/activerecord.rb Outdated

Comment thread lib/laforge/expiring.rb Outdated

Copilot AI balanced review requested due to automatic review settings October 23, 2026 18:46

njakobsen force-pushed the feat/expiring-changes branch from ea7bad5 to 46fc66c Compare October 23, 2026 18:46

Copilot started reviewing on behalf of njakobsen October 23, 2026 18:46

View session

Copilot AI reviewed Oct 23, 2026

View reviewed changes

Comment thread lib/laforge/recording_settings.rb

Comment thread lib/laforge/activerecord.rb Outdated

Comment thread lib/laforge/activerecord.rb Outdated

Copilot AI balanced review requested due to automatic review settings October 23, 2026 19:33

njakobsen force-pushed the feat/expiring-changes branch from 46fc66c to 0468538 Compare October 23, 2026 19:33

njakobsen force-pushed the feat/expiring-changes branch from f9e6681 to d663679 Compare October 23, 2026 22:40

Copilot AI balanced review requested due to automatic review settings October 23, 2026 22:40

Copilot started reviewing on behalf of njakobsen October 23, 2026 22:40

View session

Copilot AI reviewed Oct 23, 2026

View reviewed changes

njakobsen force-pushed the feat/expiring-changes branch from d663679 to 82b0e85 Compare October 24, 2026 01:27

Copilot AI balanced review requested due to automatic review settings October 24, 2026 01:27

Copilot started reviewing on behalf of njakobsen October 24, 2026 01:28

View session

njakobsen force-pushed the feat/expiring-changes branch from 82b0e85 to 04758a8 Compare October 24, 2026 01:35

Copilot AI reviewed Oct 24, 2026

View reviewed changes

Comment thread lib/laforge/association_config.rb Outdated

Comment thread lib/laforge/recording.rb Outdated

Comment thread lib/laforge/tracked_association.rb Outdated

Copilot AI balanced review requested due to automatic review settings October 24, 2026 01:36

Copilot started reviewing on behalf of njakobsen October 24, 2026 01:36

View session

Copilot AI reviewed Oct 24, 2026

View reviewed changes

Copilot AI balanced review requested due to automatic review settings October 24, 2026 08:14

njakobsen force-pushed the feat/expiring-changes branch from 04758a8 to a781a90 Compare October 24, 2026 08:14

Copilot started reviewing on behalf of njakobsen October 24, 2026 08:15

View session

Copilot AI reviewed Oct 24, 2026

View reviewed changes

njakobsen force-pushed the feat/expiring-changes branch from a781a90 to c7b579e Compare October 26, 2026 18:11

Copilot AI balanced review requested due to automatic review settings October 26, 2026 18:11

Copilot started reviewing on behalf of njakobsen October 26, 2026 18:12

View session

Copilot AI reviewed Oct 26, 2026

njakobsen and others added 2 commits October 26, 2026 12:36

Copilot AI balanced review requested due to automatic review settings October 26, 2026 22:11

njakobsen force-pushed the feat/expiring-changes branch from c7b579e to 2f6faa7 Compare October 26, 2026 22:11

Copilot started reviewing on behalf of njakobsen October 26, 2026 22:12

View session

Copilot AI reviewed Oct 26, 2026

View reviewed changes

njakobsen merged commit bb41c7e into master Oct 26, 2026
3 checks passed

Conversation

njakobsen commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Follow-ups

UAT Steps

Closes

Special Handling

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

njakobsen Oct 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

njakobsen commented Oct 22, 2026 •

edited

Loading