diff --git a/.changeset/full-battery-stop.md b/.changeset/full-battery-stop.md new file mode 100644 index 00000000..ebf07ccb --- /dev/null +++ b/.changeset/full-battery-stop.md @@ -0,0 +1,5 @@ +--- +"ftw": patch +--- + +Pause charging for each battery that reports 100% and allow it again at 99% or lower. Keep discharge available and apply the stop to planned and manual charging, including during ramp limits and dispatch waits. Show the full-battery stop separately from measurement confidence and retain warnings when the device ignores the stop. diff --git a/.changeset/measured-control-feedback.md b/.changeset/measured-control-feedback.md new file mode 100644 index 00000000..115bd5dc --- /dev/null +++ b/.changeset/measured-control-feedback.md @@ -0,0 +1,9 @@ +--- +"ftw": minor +--- + +Show whether each device FTW controls does what FTW asked. A tap on the battery, charger or solar bubble answers "Are we in control?" first: Following FTW, Waiting, Limited, Not following, No contact or Not controlled, with one sentence and, when you can act, the next step. Manual override moves below it. + +"How FTW knows" lists the evidence: sent, accepted, measured and confirmed by a separate grid meter, with numbers and response curves for experts. Support reports and the API carry the same status and evidence. + +The overview stays quiet while FTW is in control and marks only devices that need a look (amber) or attention now (red). Proof follows a device through targets retuned every tick and slow cloud sources, a confirmed step survives later household loads, and a battery that takes less power as it fills is explained instead of flagged. diff --git a/VISION.md b/VISION.md index ba20a594..4ebd1db6 100644 --- a/VISION.md +++ b/VISION.md @@ -34,6 +34,54 @@ Core validates every plan and command, including plans supplied by another system. Fuse, equipment, SoC, freshness and other quantified safety limits always apply. Stale required site-meter data stops dispatch. +## Are we in control? + +**Trust what you read, not what you sent.** + +For each device function FTW commands, such as a battery, a car charger, a +solar cap or V2X, the normal view answers one question: is the device doing +what FTW asked? A successful driver call, an API reply or an echoed setpoint +does not prove a physical effect. + +Core decides the answer and its urgency; clients only render it: + +- **Following FTW:** fresh readings show the device doing what FTW asked. +- **Waiting:** a response is still due, or there is nothing to verify yet. +- **Limited:** the device follows within a known limit, such as a nearly full + battery, the main fuse or a charger setting. +- **Not following:** fresh readings disagree with the command after the + device's response time. +- **No contact:** readings or commands fail. Losing a device FTW had measured + is an alarm. +- **Not controlled:** FTW only reads the device or has handed control back. + +Show the evidence as a receipt, not as the headline: sent, accepted, measured +and confirmed. Accepted means the driver took the command. Measured means +fresh, distinct device readings across the response window. Confirmed means a +separate physical meter saw the matching change. Keep the target separate: a +battery can be confirmed at 4.4 kW against a 5 kW command, and the shortfall +stays visible. + +- Freshness follows the source. A slow cloud source stays current for as long + as it declares; a repeated or cached sample never extends proof. +- Compare each reading with the commands that could still be in force during + the response time. Retuning every tick is normal control, not a new test. + A material step starts a new comparison. +- Confirmation needs a separate physical sensor, aligned readings and a clear + step. Subtract other measured flows, and never let an unmeasured load + disappear into an average. A confirmed step stays confirmed while the device + keeps following with fresh readings. +- Explain a shortfall only with a fresh, relevant fact, such as state of + charge or a reported limit. Say when the cause is unknown. +- Keep the overview quiet while FTW is in control. Mark warnings amber and + alarms red, and clear them when fresh evidence shows recovery. +- Never change a power target just to create a test signal without the + owner's consent. +- Give authorized agents and support reports the same evidence. + +This is the product direction. Each implementation states which devices, +paths and physical outcomes it has verified. + ## Trust through visible behaviour The live view is a core product feature. It must feel local and fast, and @@ -68,6 +116,25 @@ verified device limits where available and learn the usable response within safe bounds. Advanced users may set limits explicitly. Do not treat an observed power level as proof of an absolute hardware or installation limit. +Develop the per-device battery and inverter model into a digital twin grounded +in normal operation. Learn charge and discharge response, delay, usable power +and losses under the observed state of charge, temperature and device mode. +Keep energy capacity in kWh separate from power in kW. A 10 kWh battery may +have a 5 kW inverter; reaching 5 kW says nothing by itself about its capacity. +If user input conflicts with repeated measurements or device ratings, explain +the conflict and propose a correction rather than silently changing that input. + +Keep user input, reported limits and learned estimates distinct, with source, +age, tested conditions and uncertainty. Prefer independently confirmed samples +for learning, label device-only samples, and never train on an inferred site +effect as though it were another meter. A plateau at one state of charge is +evidence for those conditions, not a permanent nameplate limit. Learn capacity +only from suitable energy and state-of-charge observations. Detect changed +behaviour and rebuild confidence after equipment or mode changes. Use the model +to plan achievable work within verified safety limits; learned estimates must +never raise those limits or replace fresh measurements as proof of an effect. +This is the target for the model, not a claim that the full twin has shipped. + For solar, the target is that "I have solar" is enough to start learning. Installed kWp is an optional starting estimate. Approximate user input must not permanently constrain a model when measurements support a better fit. diff --git a/docs/roadmap.md b/docs/roadmap.md index c74d2e95..93d63425 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -48,6 +48,14 @@ clear result that the owner can review. | 5 | Complete external authority and fair value as separate focused changes. | Temporary control expires to a defined local default; durable goals persist; schedule/plan access can be revoked; cloud MCP does not expose data to the relay. Separately, the validated self-consumption comparison reports FTW's incremental value and missing evidence honestly. | | Later | Bounded thermal control and further expert extensions. | A named tank/hot-water use case meets comfort, hardware and failure requirements without making ordinary household setup harder. Native expansion still follows its existing Pair + Now verification gates. | +Battery and inverter learning should build on the existing per-battery response +model. The digital-twin target needs tests for separate kW and kWh inputs, +reported versus observed limits, partial delivery confirmed by the site meter, +charge/discharge and state-of-charge-dependent response, temperature and mode +changes, stale samples and model recovery. Show source and uncertainty before +using an estimate in planning. A model prediction must never verify itself or +raise a safety limit. These are acceptance requirements, not shipped support. + Necessary safety, security, recovery and support fixes continue throughout. Correct misleading value labels when their scope is known; that need not wait for the new counterfactual model. Reading and analysis access for agents can @@ -69,7 +77,7 @@ or reopen closed issues. |---|---|---| | Simple setup and first-day value | Discover mixed equipment. Confirm the main fuse and site meter. Read battery capacity where possible and ask for kWh when needed. Power settings and solar kWp are optional where safe device information and learning allow. Provide useful initial load and PV forecasts. | A fresh install reaches useful automatic operation without panel drawings or expert settings. Missing data and wrong start estimates have tested behaviour. Record device identity, known limits and uncertainty. Validate on named hardware combinations as well as simulators. | | A clear commissioning result | Check commands and measured response within known limits. Distinguish working telemetry from working control. Exclude failed control from both the plan and dispatch. | Show request, device response, measured effect and timing. Cover delayed response, refusal, disconnect, stale site data and recovery. A short commissioning test does not claim full hardware qualification. | -| Live control that earns trust | Keep the fast local feel. Make request, acceptance, command, response, physical effect and freshness visible in the normal Flow experience. | Browser review on desktop and mobile, timing measurements on a target box, and traces with different sampling rates. A pending command or old reading never appears as completed or fresh. Examine existing Live and Flow views before deciding their final layout. | +| Verified control: “Are we in control?” | Keep the fast local feel. Make request, acceptance, command, response, physical effect and freshness visible in the normal Flow experience. | Browser review on desktop and mobile, timing measurements on a target box, and traces with different sampling rates. A successful call, echoed setpoint or old reading never appears as a verified physical result. Test accepted, measured and confirmed evidence per device, and the status each one shows. Confirm one device while another is offline; keep unavailable background flows in the residual. Show overview status, alarm on lost measured proof after the response wait, and clear it on measured recovery. Include direction, tolerance, response delay, simultaneous load/solar changes, shared measurement sources, overridden setpoints, device limits and recovery. A reduced request remains visibly limited even when the device follows the reduced command. Examine existing Live and Flow views before deciding their final layout. | | Good automatic planning | Use site physics and charge/discharge efficiency. Default wear cost is zero; users may opt in. Keep hard SoC limits separate from forecast-based caution and explicit backup needs. | Cold-start and learned forecasts, stale inputs, multiple assets and unavailable optimizer paths have tests. Backtests use held-out periods and report uncertainty. Defaults and persisted settings agree across UI, Core and worker. Site runs establish practical benefit. | | Reliable daily charging | Persistent weekday target and deadline. Offline-car estimates. Direct SoC slider after connection, no extra save, prompt replanning, one-action Charge now and notifications when action is needed. | Test from app intent to charger/car response and delivered energy, including missed-goal risk, absent vehicle cloud, unknown SoC, reconnect and restart. Test notifications with the app closed. Confirm physical charging separately from simulation. | | Useful analysis and fair savings | Keep enough provenance to explain plans and outcomes. Main savings target compares with ordinary self-consumption on the same installation. | Actual cost reconciles with measured import/export and prices. Specify EV behaviour, initial and final stored-energy accounting, efficiency and coverage. Show missing and negative results. Label the current no-PV/no-battery comparison as total site value until replacement is verified. | diff --git a/drivers/BUNDLED_SOURCE.json b/drivers/BUNDLED_SOURCE.json index a01d7afe..6707abf4 100644 --- a/drivers/BUNDLED_SOURCE.json +++ b/drivers/BUNDLED_SOURCE.json @@ -16,7 +16,7 @@ "from the signed channel. Run scripts/sync-bundled-drivers.sh to update." ], "repository": "srcfl/device-drivers", - "commit": "7e572fb499dda7acf6f1212b8b0b78235b9d3ba9", + "commit": "d74ace6a6e5fc335177c40696ac3d449b1517057", "source_dir": "drivers/lua", "drivers": [ "ambibox_v2x", "ctek", "ctek_hybrid", "ctek_v2", "deye", "easee_cloud", diff --git a/go/cmd/ftw/main.go b/go/cmd/ftw/main.go index f51afb17..6a421517 100644 --- a/go/cmd/ftw/main.go +++ b/go/cmd/ftw/main.go @@ -2457,6 +2457,9 @@ func main() { SiteDispatchBlocked: func() string { return siteDispatchNow(tel, cfg, cfgMu, ctrl, ctrlMu, time.Now()).Reason }, + SiteMeasurementSources: func() telemetry.ForecastOptions { + return forecastSettings.Snapshot().Options + }, HA: haOwner.Bridge, Registry: reg, DriverRepository: driverRepository, diff --git a/go/internal/api/api.go b/go/internal/api/api.go index d4c36921..4628be87 100644 --- a/go/internal/api/api.go +++ b/go/internal/api/api.go @@ -73,6 +73,9 @@ const ( // One instance is shared across all handlers; mutations use the contained // mutexes from each package. type Deps struct { + // SiteMeasurementSources reuses the configured physical-flow inventory. + // These are source declarations, never forecast values or derived load. + SiteMeasurementSources func() telemetry.ForecastOptions // MutationPolicy protects every state-changing route at the shared // Handler boundary. Production requires tokens for non-local hostnames; @@ -1220,6 +1223,7 @@ func (s *Server) handleStatus(w http.ResponseWriter, r *http.Request) { // diagnostic — incremented when actual fleet delivery diverges // from the plan's BatteryEnergyWh by > 50 % (over) or < 50 % // (under). Idle slots (|planned| ≤ 50 Wh) are ignored. + "control_feedback": s.controlFeedback(time.Now()), "slot_delivery_stats": ctrl.SlotDeliveryStats, } // A stale or missing site meter is not 0 W. Publishing zero made the @@ -3677,7 +3681,7 @@ func (s *Server) handleLoadpoints(w http.ResponseWriter, r *http.Request) { writeJSON(w, 200, map[string]any{ "enabled": true, "vehicle_limit_goal_supported": true, - "loadpoints": states, + "loadpoints": s.loadpointsWithFeedback(states), }) } diff --git a/go/internal/api/api_control_feedback.go b/go/internal/api/api_control_feedback.go new file mode 100644 index 00000000..0e7d1ced --- /dev/null +++ b/go/internal/api/api_control_feedback.go @@ -0,0 +1,552 @@ +package api + +import ( + "encoding/json" + "math" + "sort" + "time" + + "github.com/srcfl/ftw/go/internal/config" + "github.com/srcfl/ftw/go/internal/control" + "github.com/srcfl/ftw/go/internal/loadpoint" + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// ControlFeedback answers "Are we in control?" for one device function in +// every mode. Status and severity are Core's verdict; clients choose words and +// colours for them and never derive their own. Evidence names the strongest +// proof behind it: accepted (the driver took the call), measured (fresh device +// readings) or confirmed (a separate meter saw the change). Missing numbers +// are null, never invented zeros. Reasons describe only what Core or the +// device reported, not a guessed physical cause. +type ControlFeedback struct { + Driver string `json:"driver"` + Kind string `json:"kind"` + Mode string `json:"mode"` + Status string `json:"status"` + Reason string `json:"reason"` + Severity string `json:"severity"` + Evidence string `json:"evidence"` + ReadingsFresh bool `json:"readings_fresh"` + ConfirmedAtMs int64 `json:"confirmed_at_ms,omitempty"` + SiteConfirmation string `json:"site_confirmation"` + SiteMeter string `json:"site_meter,omitempty"` + SiteEvidence *ControlComparison `json:"site_evidence,omitempty"` + ToleranceW *float64 `json:"tolerance_w"` + // Internal steps of the verdict; the API exposes their result above. + State string `json:"-"` + VerificationTier *int `json:"-"` + VerificationLost bool `json:"-"` + Response string `json:"-"` + SiteBeforeW *float64 `json:"-"` + SiteAfterW *float64 `json:"-"` + SiteBeforeAtMs int64 `json:"-"` + SiteAfterAtMs int64 `json:"-"` + SiteDeltaW *float64 `json:"-"` + DeviceDeltaW *float64 `json:"-"` + VerifiedAtMs int64 `json:"-"` + RequestedW *float64 `json:"requested_w"` + SentW *float64 `json:"sent_w"` + ReadbackW *float64 `json:"readback_w"` + ActualW *float64 `json:"actual_w"` + BatterySoC *float64 `json:"battery_soc,omitempty"` + ChargeResumeSoC *float64 `json:"charge_resume_soc,omitempty"` + RequestedA *float64 `json:"requested_a"` + OfferedA *float64 `json:"offered_a"` + DeviceLimitA *float64 `json:"device_limit_a"` + DeviceReason string `json:"device_reason,omitempty"` + SinceMs int64 `json:"since_ms,omitempty"` + CommandAtMs int64 `json:"command_at_ms,omitempty"` + ObservedAtMs int64 `json:"observed_at_ms,omitempty"` +} + +type feedbackReading struct { + SetpointW *float64 `json:"setpoint_w"` + MaxA *float64 `json:"max_a"` + DeviceLimitA *float64 `json:"device_limit_a"` + DeviceLimitAgeS *float64 `json:"device_limit_age_s"` + Reason string `json:"reason_no_current_label"` + Connected *bool `json:"connected"` + Online *bool `json:"is_online"` +} + +func watts(v float64) *float64 { + if math.IsNaN(v) || math.IsInf(v, 0) { + return nil + } + return &v +} + +func (s *Server) controlFeedback(now time.Time) []ControlFeedback { + out := []ControlFeedback{} + if s.deps.Tel == nil { + return out + } + mode, meter := "", "" + clamped := map[string]bool{} + full := map[string]bool{} + targeted := map[string]bool{} + var hold control.BatteryManualHold + var held bool + if s.deps.Ctrl != nil && s.deps.CtrlMu != nil { + s.deps.CtrlMu.Lock() + mode = string(s.deps.Ctrl.Mode) + meter = s.deps.Ctrl.SiteMeterDriver + for _, t := range s.deps.Ctrl.LastTargets { + clamped[t.Driver] = t.Clamped + full[t.Driver] = s.deps.Ctrl.BatteryChargePaused(t.Driver) + targeted[t.Driver] = true + } + hold, held = s.deps.Ctrl.GetBatteryManualHold(now) + s.deps.CtrlMu.Unlock() + } + observe, disabled := map[string]bool{}, map[string]bool{} + configuredBattery := map[string]bool{} + if s.deps.Cfg != nil && s.deps.CfgMu != nil { + s.deps.CfgMu.RLock() + observe = config.ObserveOnlyDriverSet(s.deps.Cfg) + for _, d := range s.deps.Cfg.Drivers { + disabled[d.Name] = d.Disabled + configuredBattery[d.Name] = d.BatteryCapacityWh > 0 && !d.BatteryTelemetryOnly + } + s.deps.CfgMu.RUnlock() + } + lps := map[string]loadpoint.State{} + if s.deps.Loadpoints != nil { + states := s.deps.Loadpoints.States() + s.decorateLoadpointsWithManual(states) + for _, lp := range states { + lps[lp.DriverName] = lp + } + } + blocked := "" + if s.deps.SiteDispatchBlocked != nil { + blocked = s.deps.SiteDispatchBlocked() + } + for _, kind := range []telemetry.DerType{telemetry.DerBattery, telemetry.DerEV, telemetry.DerPV, telemetry.DerV2X} { + for _, rd := range s.deps.Tel.ReadingsByType(kind) { + cmd, commanded := s.deps.Tel.CommandEvidence(rd.Driver, kind.String()) + _, loadpointOwned := lps[rd.Driver] + manualBattery := held && kind == telemetry.DerBattery && (hold.Driver == "" || hold.Driver == rd.Driver) + // Measurements inform the site comparison, but only controlled + // functions receive a command result. PV becomes one on curtailment. + if !commanded && !(kind == telemetry.DerBattery && (targeted[rd.Driver] || configuredBattery[rd.Driver])) && !(kind == telemetry.DerEV && loadpointOwned) && !manualBattery { + continue + } + if kind == telemetry.DerPV && !commanded { + continue + } + f := ControlFeedback{Driver: rd.Driver, Kind: kind.String(), Mode: mode, State: "waiting", Reason: "no_command", Severity: "info", Response: "unconfirmed"} + var d feedbackReading + _ = json.Unmarshal(rd.Data, &d) + f.ReadbackW, f.OfferedA, f.DeviceReason = d.SetpointW, d.MaxA, d.Reason + if d.DeviceLimitAgeS != nil && *d.DeviceLimitAgeS >= 0 && *d.DeviceLimitAgeS <= 120 { + f.DeviceLimitA = d.DeviceLimitA + } + f.ObservedAtMs = rd.UpdatedAt.UnixMilli() + fresh := now.Sub(rd.UpdatedAt) <= time.Minute && !rd.UpdatedAt.After(now) + if h := s.deps.Tel.DriverHealth(rd.Driver); h != nil { + fresh = fresh && h.TelemetryLive() + } + if d.Online != nil && !*d.Online { + fresh = false + } + measurement, powerKnown := telemetry.ControlPowerObservation(rd.RawW, rd.Data, rd.UpdatedAt) + if !measurement.At.IsZero() { + f.ObservedAtMs = measurement.Seen().UnixMilli() + } + if fresh && powerKnown && now.Sub(measurement.Seen()) <= telemetry.ControlPowerMaxAge(rd.Data) { + f.ActualW = watts(measurement.PowerW) + } + if fresh && kind == telemetry.DerBattery && rd.SoC != nil && + !rd.SoCUpdatedAt.After(now) && now.Sub(rd.SoCUpdatedAt) <= telemetry.BatterySoCMaxAge && *rd.SoC >= 0 && *rd.SoC <= 1 { + f.BatterySoC = watts(*rd.SoC) + } + + if commanded { + f.SentW, f.RequestedW = cmd.PowerW, cmd.PowerW + if cmd.PowerW != nil { + f.ToleranceW = watts(telemetry.ControlToleranceW(*cmd.PowerW)) + } + f.CommandAtMs, f.SinceMs = cmd.At.UnixMilli(), cmd.Since.UnixMilli() + if cmd.Result == "accepted" { + tier := 0 + f.VerificationTier = &tier + } + } + coreReason := "" + if clamped[rd.Driver] && kind == telemetry.DerBattery { + coreReason = "core_limit" + } + if full[rd.Driver] && kind == telemetry.DerBattery && f.BatterySoC != nil { + coreReason = "battery_full" + f.ChargeResumeSoC = watts(control.BatteryChargeResumeSoC) + } + if held && kind == telemetry.DerBattery && (hold.Driver == "" || hold.Driver == rd.Driver) { + f.Mode = "manual" + f.RequestedW = watts(hold.PowerW) + } + if lp, ok := lps[rd.Driver]; ok && kind == telemetry.DerEV { + f.Mode = "plan" + if lp.SurplusOnly { + f.Mode = "solar" + } + if lp.ManualActive { + f.Mode = "manual" + } + if lp.CommandedKnown { + f.RequestedW = watts(lp.CommandedW) + coreReason = lp.CommandedReason + } + if lp.ManualActive { + f.RequestedW = watts(lp.ManualChargeW) + } + if f.RequestedW != nil && lp.Phases > 0 && lp.VoltageV > 0 { + f.RequestedA = watts(*f.RequestedW / float64(lp.Phases) / lp.VoltageV) + } + if lp.PowerUnavailable { + fresh = false + f.ActualW = nil + } + } + // Evidence describes measured power, not target fulfilment. A device + // delivering 4.4 kW against a 5 kW command can still be site-confirmed. + if commanded && cmd.Result == "accepted" && fresh && f.ActualW != nil && f.SentW != nil && cmd.HasMeasuredPower(now) { + f.Response = "device_reported" + tier := 1 + f.VerificationTier = &tier + f.VerifiedAtMs = cmd.LastObservation.UnixMilli() + } + f.SiteConfirmation = "device_response_unconfirmed" + if f.VerificationTier != nil && *f.VerificationTier == 1 { + from := now.Add(-telemetry.ControlWindowDuration) + if cmd.Since.After(from) { + from = cmd.Since + } + f.SiteMeter = meter + after, at := s.deps.Tel.ControlWindows(from, now), now + if cmd.StepAfter != nil { + // A completed step keeps the verdict its own window supported. + after, at = cmd.StepAfter, cmd.StepAfterAt + } + comparison := s.controlResponse(cmd, meter, after, at) + f.SiteEvidence = &comparison + f.SiteConfirmation, f.DeviceDeltaW, f.SiteDeltaW = comparison.Reason, comparison.DeviceDeltaW, comparison.SiteDeltaW + if f.SiteDeltaW != nil { + f.SiteBeforeW, f.SiteAfterW = watts(comparison.BeforeW), watts(comparison.AfterW) + f.SiteBeforeAtMs, f.SiteAfterAtMs = comparison.BeforeAt.UnixMilli(), comparison.AfterAt.UnixMilli() + } + if h := s.deps.Tel.DriverHealth(meter); h == nil || !h.TelemetryLive() || h.DeviceFault || blocked != "" { + f.SiteConfirmation = "waiting_for_meter" + f.SiteDeltaW, f.SiteBeforeW, f.SiteAfterW = nil, nil, nil + f.SiteEvidence = nil + } + if f.SiteConfirmation == "confirmed" { + tier := 2 + f.VerificationTier = &tier + f.Response = "site_confirmed" + f.ConfirmedAtMs = comparison.AfterAt.UnixMilli() + } + } + classifyControlFeedback(&f, cmd, commanded, fresh, coreReason, d.Connected, now) + if h := s.deps.Tel.DriverHealth(rd.Driver); h != nil && h.DeviceFault { + f.State, f.Reason, f.Severity = "blocked", "device_fault", "warning" + f.DeviceReason = h.DeviceFaultReason + } + if s.deps.Registry != nil { + if cs, ok := s.deps.Registry.ControlStatus(rd.Driver); ok && cs.Blocked { + f.State, f.Reason, f.Severity = "blocked", "default_failed", "warning" + } + } + if blocked != "" { + f.State, f.Reason, f.Severity = "blocked", blocked, "warning" + } + if observe[rd.Driver] { + f.State, f.Reason, f.Severity = "observing", "observe_only", "info" + } + if disabled[rd.Driver] { + f.State, f.Reason, f.Severity = "blocked", "disabled", "info" + } + if observe[rd.Driver] || disabled[rd.Driver] { + f.VerificationTier, f.SiteEvidence, f.Response = nil, nil, "unconfirmed" + f.SiteConfirmation = "not_controlling" + f.SiteDeltaW, f.DeviceDeltaW, f.SiteBeforeW, f.SiteAfterW = nil, nil, nil, nil + f.SiteBeforeAtMs, f.SiteAfterAtMs = 0, 0 + } + f.VerificationLost = controlVerificationLost(f, cmd, now) + if f.VerificationLost { + f.Reason = "readings_lost" + } + f.ReadingsFresh = fresh + out = append(out, f) + } + } + seen := map[string]bool{} + for _, f := range out { + seen[f.Driver] = true + } + for driver, lp := range lps { + if seen[driver] { + continue + } + mode := "plan" + if lp.ManualActive { + mode = "manual" + } + if lp.SurplusOnly { + mode = "solar" + } + out = append(out, ControlFeedback{Driver: driver, Kind: "ev", Mode: mode, State: "unknown", Reason: "telemetry_stale", Severity: "warning"}) + seen[driver] = true + } + for driver, h := range s.deps.Tel.AllHealth() { + if seen[driver] || !h.DeviceFault { + continue + } + controlled := targeted[driver] + for _, kind := range []string{"battery", "ev", "pv", "v2x_charger"} { + if c, ok := s.deps.Tel.CommandEvidence(driver, kind); ok && c.Result != "released" { + controlled = true + } + } + if !controlled { + continue + } + out = append(out, ControlFeedback{Driver: driver, Kind: "device", Mode: mode, State: "blocked", Reason: "device_fault", Severity: "warning", DeviceReason: h.DeviceFaultReason}) + } + for i := range out { + setControlStatus(&out[i]) + } + sort.Slice(out, func(i, j int) bool { + if out[i].Driver == out[j].Driver { + return out[i].Kind < out[j].Kind + } + return out[i].Driver < out[j].Driver + }) + return out +} + +// setControlStatus answers "Are we in control?" from the classified reason. +// Following needs measured readings; a driver's acknowledgement alone waits. +// Info needs no action, warning asks the owner to look and alarm means FTW +// cannot see or steer a device it is meant to control. +func setControlStatus(f *ControlFeedback) { + status, severity := "waiting", "info" + switch f.Reason { + case "power_observed", "solar_below_ceiling", "idle", "plan", "manual_hold", "pv_surplus", + "vehicle_complete", "vehicle_limit_completion", "vehicle_not_requesting": + status = "following" + case "battery_full", "battery_nearly_full", "battery_nearly_empty", "core_limit", + "fuse_limit", "fuse_cooldown", "charger_limit": + status = "limited" + case "device_limit", "offered_current_lower": + status, severity = "limited", "warning" + case "setpoint_changed", "power_below_target", "power_above_target", "power_wrong_direction", + "no_power_response", "power_while_idle": + status, severity = "not_following", "warning" + case "device_fault": + status, severity = "not_following", "alarm" + case "telemetry_stale", "command_failed", "command_unconfirmed", "response_unknown": + status, severity = "no_contact", "warning" + case "readings_lost", "default_failed": + status, severity = "no_contact", "alarm" + case "observe_only", "disabled", "device_control": + status = "not_controlled" + case "site_meter_stale", "site_phase_currents_stale": + status, severity = "not_controlled", "warning" + } + if status == "following" && (f.VerificationTier == nil || *f.VerificationTier < 1) { + status = "waiting" + } + f.Status, f.Severity = status, severity + f.Evidence = "none" + if f.VerificationTier != nil { + f.Evidence = []string{"accepted", "measured", "confirmed"}[min(max(*f.VerificationTier, 0), 2)] + } +} + +// An acknowledgement is not an alarm during the normal response wait. A +// function that previously supplied measured proof must recover that proof. +// This state comes from telemetry, not from whether a browser was open. +func controlVerificationLost(f ControlFeedback, cmd telemetry.CommandEvidence, now time.Time) bool { + // Failed or released calls have their own status. They do not by themselves + // mean that a previously measured device stopped supplying power readings. + if cmd.Result != "accepted" || cmd.LastMeasuredAt.IsZero() || f.VerificationTier != nil && *f.VerificationTier >= 1 { + return false + } + switch f.Reason { + case "observe_only", "disabled", "device_control", "not_connected", "idle", "no_command": + return false + } + if f.Reason == "telemetry_stale" { + return true + } + grace := 30 * time.Second + if f.Kind == "ev" { + grace = 2 * time.Minute + } + return !cmd.Since.IsZero() && now.Sub(cmd.Since) >= grace && now.Sub(cmd.LastMeasuredAt) >= grace +} + +func classifyControlFeedback(f *ControlFeedback, cmd telemetry.CommandEvidence, commanded, fresh bool, coreReason string, connected *bool, now time.Time) { + set := func(state, reason, severity string) { f.State, f.Reason, f.Severity = state, reason, severity } + if !fresh { + f.ActualW, f.ReadbackW, f.OfferedA, f.DeviceLimitA = nil, nil, nil, nil + set("unknown", "telemetry_stale", "warning") + return + } + if commanded && cmd.Result == "default_failed" { + set("blocked", "default_failed", "warning") + return + } + if commanded && cmd.Result == "released" { + set("idle", "device_control", "info") + return + } + if commanded && cmd.Result == "failed" { + set("blocked", "command_failed", "warning") + return + } + if commanded && cmd.Result == "unconfirmed" { + set("unknown", "command_unconfirmed", "warning") + return + } + if f.Kind == "ev" && connected != nil && !*connected { + set("waiting", "not_connected", "info") + return + } + if f.RequestedA != nil && f.DeviceLimitA != nil && *f.RequestedA > *f.DeviceLimitA+0.5 { + set("limited", "device_limit", "warning") + return + } + if coreReason == "site_meter_stale" || coreReason == "site_phase_currents_stale" || coreReason == "fuse_cooldown" || coreReason == "fuse_limit" || coreReason == "charger_limit" { + set("limited", coreReason, "warning") + return + } + if !commanded { + if coreReason != "" { + set("waiting", coreReason, "info") + } + return + } + grace := 30 * time.Second + if f.Kind == "ev" { + grace = 2 * time.Minute + } + settled := !cmd.Since.IsZero() && now.Sub(cmd.Since) >= grace + if !cmd.ReadbackMismatchSince.IsZero() && now.Sub(cmd.ReadbackMismatchSince) >= 30*time.Second && f.ReadbackW != nil { + set("warning", "setpoint_changed", "warning") + return + } + if f.Kind == "ev" && f.SentW != nil && f.RequestedA != nil && f.OfferedA != nil && *f.SentW > 100 && *f.OfferedA+0.5 < *f.RequestedA && settled { + set("limited", "offered_current_lower", "warning") + return + } + // This is a known Core stop, not a BMS diagnosis or new measurement proof. + // Continued power, changed setpoints, stale readings and faults keep their + // own verdicts. A full battery is still allowed to discharge. + if coreReason == "battery_full" && cmd.Result == "accepted" && f.SentW != nil && math.Abs(*f.SentW) < 1 && + f.ActualW != nil && math.Abs(*f.ActualW) <= telemetry.ControlToleranceW(0) { + set("limited", "battery_full", "info") + return + } + if cmd.Result == "pending" || (!settled && f.Response == "unconfirmed") || f.ObservedAtMs < cmd.Since.UnixMilli() { + set("waiting", "waiting_response", "info") + return + } + if f.SentW == nil || f.ActualW == nil { + set("unknown", "response_unknown", "info") + return + } + // The gap is measured against every command that could still be in force, + // so a target retuned each tick does not read as a missed response. + gap := math.Abs(*f.SentW - *f.ActualW) + if f.Kind == "pv" { + gap = math.Max(0, math.Abs(*f.ActualW)-math.Abs(*f.SentW)) + } + if cmd.LastGapW != nil { + gap = *cmd.LastGapW + } + if !cmd.PowerMismatchSince.IsZero() && now.Sub(cmd.PowerMismatchSince) >= grace && gap > telemetry.ControlToleranceW(*f.SentW) { + if reason := batteryChargeLevelLimit(f); reason != "" { + set("limited", reason, "info") + return + } + reason := "power_above_target" + switch { + case math.Abs(*f.SentW) < 100: + reason = "power_while_idle" + case math.Abs(*f.ActualW) < 100: + reason = "no_power_response" + case f.Kind != "pv" && (*f.SentW < 0) != (*f.ActualW < 0): + reason = "power_wrong_direction" + case math.Abs(*f.ActualW) < math.Abs(*f.SentW): + reason = "power_below_target" + } + set("warning", reason, "warning") + return + } + if coreReason == "core_limit" { + set("limited", "core_limit", "info") + return + } + if math.Abs(*f.SentW) < 100 { + if coreReason == "" || coreReason == "manual_hold" { + coreReason = "idle" + } + set("idle", coreReason, "info") + return + } + if gap > telemetry.ControlToleranceW(*f.SentW) { + set("waiting", "waiting_response", "info") + return + } + if f.Kind == "pv" && math.Abs(*f.ActualW) < math.Abs(*f.SentW)-telemetry.ControlToleranceW(*f.SentW) { + set("unknown", "solar_below_ceiling", "info") + return + } + if f.Response != "device_reported" && f.Response != "site_confirmed" || cmd.PowerMatchSince.IsZero() || cmd.LastObservation.Sub(cmd.PowerMatchSince) < 10*time.Second { + set("waiting", "waiting_response", "info") + return + } + set("following", "power_observed", "info") +} + +// A battery accepts less charge as it fills and less discharge as it empties. +// Its own fresh SoC explains that shortfall; wrong direction still warns. +func batteryChargeLevelLimit(f *ControlFeedback) string { + if f.Kind != "battery" || f.BatterySoC == nil || f.SentW == nil || f.ActualW == nil { + return "" + } + soc, sent, actual := *f.BatterySoC, *f.SentW, *f.ActualW + tolerance := telemetry.ControlToleranceW(sent) + switch { + case sent > tolerance && soc >= 0.9 && actual > -tolerance && actual < sent: + return "battery_nearly_full" + case sent < -tolerance && soc <= 0.1 && actual < tolerance && actual > sent: + return "battery_nearly_empty" + } + return "" +} + +// Keep status and the EV panel on the same explanation. This wrapper adds +// evidence to the existing endpoint without changing stored loadpoint state. +type loadpointFeedbackView struct { + loadpoint.State + ControlFeedback *ControlFeedback `json:"control_feedback,omitempty"` +} + +func (s *Server) loadpointsWithFeedback(states []loadpoint.State) []loadpointFeedbackView { + feedback := s.controlFeedback(time.Now()) + byDriver := map[string]*ControlFeedback{} + for i := range feedback { + if feedback[i].Kind == "ev" { + byDriver[feedback[i].Driver] = &feedback[i] + } + } + out := make([]loadpointFeedbackView, 0, len(states)) + for _, st := range states { + out = append(out, loadpointFeedbackView{st, byDriver[st.DriverName]}) + } + return out +} diff --git a/go/internal/api/api_control_feedback_test.go b/go/internal/api/api_control_feedback_test.go new file mode 100644 index 00000000..1bd271b8 --- /dev/null +++ b/go/internal/api/api_control_feedback_test.go @@ -0,0 +1,312 @@ +package api + +import ( + "encoding/json" + "github.com/srcfl/ftw/go/internal/config" + "github.com/srcfl/ftw/go/internal/control" + "github.com/srcfl/ftw/go/internal/telemetry" + "math" + "sync" + "testing" + "testing/synctest" + "time" +) + +func TestControlFeedbackReasons(t *testing.T) { + now := time.Now() + connected := true + for _, tc := range []struct { + name, kind, result, core, want string + fresh bool + sent, actual, readback, requestedA, limitA float64 + readMismatch, powerMismatch bool + }{ + {"driver accepted but no measured effect", "battery", "accepted", "", "no_power_response", true, 1000, 0, 1000, 0, 0, false, true}, + {"Pixii overwritten while zero requested", "battery", "accepted", "", "setpoint_changed", true, 0, -1500, -1600, 0, 0, true, true}, + {"8 amp charger limit while charging", "ev", "accepted", "", "device_limit", true, 11040, 5500, 11040, 16, 8, false, true}, + {"fuse cap before dispatch", "ev", "accepted", "fuse_limit", "fuse_limit", true, 5000, 5000, 5000, 16, 0, false, false}, + {"driver refuses", "battery", "failed", "", "command_failed", true, 1000, 0, 0, 0, 0, false, false}, + {"unknown call outcome", "battery", "unconfirmed", "", "command_unconfirmed", true, 1000, 0, 0, 0, 0, false, false}, + {"default failed", "battery", "default_failed", "", "default_failed", true, 1000, 0, 0, 0, 0, false, false}, + {"device now autonomous", "battery", "released", "", "device_control", true, 1000, 0, 0, 0, 0, false, false}, + {"stale power cannot verify effect", "battery", "accepted", "", "telemetry_stale", false, 1000, 1000, 1000, 0, 0, false, false}, + {"device follows command", "battery", "accepted", "", "power_observed", true, 1000, 1000, 1000, 0, 0, false, false}, + {"Core clamp stays visible when followed", "battery", "accepted", "core_limit", "core_limit", true, 1000, 1000, 1000, 0, 0, false, false}, + } { + t.Run(tc.name, func(t *testing.T) { + c := telemetry.CommandEvidence{Result: tc.result, Since: now.Add(-3 * time.Minute), PowerMatchSince: now.Add(-time.Minute), LastObservation: now} + if tc.readMismatch { + c.ReadbackMismatchSince = now.Add(-time.Minute) + } + if tc.powerMismatch { + c.PowerMismatchSince = now.Add(-3 * time.Minute) + } + f := ControlFeedback{Kind: tc.kind, SentW: watts(tc.sent), ActualW: watts(tc.actual), ReadbackW: watts(tc.readback), ObservedAtMs: now.UnixMilli(), Response: "device_reported"} + if tc.requestedA > 0 { + f.RequestedA = watts(tc.requestedA) + } + if tc.limitA > 0 { + f.DeviceLimitA = watts(tc.limitA) + } + classifyControlFeedback(&f, c, true, tc.fresh, tc.core, &connected, now) + if f.Reason != tc.want { + t.Fatalf("got %+v, want %s", f, tc.want) + } + if !tc.fresh && (f.ActualW != nil || f.ReadbackW != nil) { + t.Fatal("stale numbers kept as current") + } + }) + } +} + +func TestControlFeedbackEveryModeUsesEvidence(t *testing.T) { + for _, mode := range control.AllModes() { + t.Run(string(mode), func(t *testing.T) { + tel := telemetry.NewStore() + now := time.Now() + c := tel.BeginCommand("battery", []byte(`{"action":"battery","power_w":1000}`), now.Add(-time.Minute)) + tel.CompleteCommand(c, "failed") + tel.Update("battery", telemetry.DerBattery, 0, nil, []byte(`{}`)) + ctrl := &control.State{Mode: mode} + srv := New(&Deps{Tel: tel, Ctrl: ctrl, CtrlMu: &sync.Mutex{}, Cfg: &config.Config{}, CfgMu: &sync.RWMutex{}}) + feedback := srv.controlFeedback(time.Now()) + if len(feedback) != 1 || feedback[0].Reason != "command_failed" || feedback[0].VerificationTier != nil { + t.Fatalf("mode %s: %+v", mode, feedback) + } + }) + } +} + +func TestControlFeedbackRequiresMeasuredPowerNotSetpoint(t *testing.T) { + tel := telemetry.NewStore() + now := time.Now() + c := tel.BeginCommand("battery", []byte(`{"action":"battery","power_w":1000}`), now) + tel.CompleteCommand(c, "accepted") + tel.Update("battery", telemetry.DerBattery, 0, nil, []byte(`{"setpoint_w":1000}`)) + srv := New(&Deps{Tel: tel}) + f := srv.controlFeedback(time.Now())[0] + if f.VerificationTier == nil || *f.VerificationTier != 0 { + t.Fatalf("setpoint echo reached measured tier: %+v", f) + } + tel.EndCommandControl("battery", false) + f = srv.controlFeedback(time.Now())[0] + if f.VerificationTier != nil || f.Reason != "device_control" { + t.Fatalf("old command survived default: %+v", f) + } +} + +func TestControlFeedbackDoesNotVerifyOneCachedSourceSample(t *testing.T) { + tel := telemetry.NewStore() + now := time.Now() + c := tel.BeginCommand("battery", []byte(`{"action":"battery","power_w":1000}`), now.Add(-time.Minute)) + tel.CompleteCommand(c, "accepted") + data := []byte(`{"power_observed_at":"` + now.Add(-10*time.Second).Format(time.RFC3339Nano) + `"}`) + tel.Update("battery", telemetry.DerBattery, 1000, nil, data) + tel.Update("battery", telemetry.DerBattery, 1000, nil, data) + srv := New(&Deps{Tel: tel}) + f := srv.controlFeedback(now)[0] + if f.VerificationTier == nil || *f.VerificationTier != 0 { + t.Fatalf("one source sample reached measured tier: %+v", f) + } +} + +func TestControlTiersBelongToCommandedFunctions(t *testing.T) { + tel := telemetry.NewStore() + tel.Update("solar", telemetry.DerPV, -3000, nil, []byte(`{}`)) + tel.Update("monitored-battery", telemetry.DerBattery, 0, nil, []byte(`{}`)) + srv := New(&Deps{Tel: tel}) + if got := srv.controlFeedback(time.Now()); len(got) != 0 { + t.Fatalf("measurement sources received control verdicts: %+v", got) + } + cmd := tel.BeginCommand("solar", []byte(`{"action":"curtail","power_w":-1000}`), time.Now()) + tel.CompleteCommand(cmd, "accepted") + got := srv.controlFeedback(time.Now()) + if len(got) != 1 || got[0].Driver != "solar" || got[0].VerificationTier == nil || *got[0].VerificationTier != 0 { + t.Fatalf("curtailment missing command verdict: %+v", got) + } +} + +func TestControlVerificationLossAlarm(t *testing.T) { + now := time.Now() + one, zero := 1, 0 + for _, tc := range []struct { + name, reason, kind string + tier *int + verified, commandAge time.Duration + want bool + }{ + {"startup", "waiting_response", "battery", &zero, 0, time.Minute, false}, + {"new command settling", "waiting_response", "battery", &zero, time.Minute, 5 * time.Second, false}, + {"lost measured response", "waiting_response", "battery", &zero, time.Minute, time.Minute, true}, + {"device offline", "telemetry_stale", "battery", &zero, 20 * time.Second, time.Minute, true}, + {"EV still settling", "waiting_response", "ev", &zero, time.Minute, time.Minute, false}, + {"EV lost response", "response_unknown", "ev", &zero, 3 * time.Minute, 3 * time.Minute, true}, + {"unplugged", "not_connected", "ev", &zero, 3 * time.Minute, 3 * time.Minute, false}, + {"released", "device_control", "battery", nil, time.Minute, time.Minute, false}, + {"device recovers", "power_observed", "battery", &one, time.Minute, time.Minute, false}, + } { + t.Run(tc.name, func(t *testing.T) { + cmd := telemetry.CommandEvidence{Result: "accepted", Since: now.Add(-tc.commandAge)} + if tc.verified > 0 { + cmd.LastMeasuredAt = now.Add(-tc.verified) + } + f := ControlFeedback{Reason: tc.reason, Kind: tc.kind, VerificationTier: tc.tier} + if got := controlVerificationLost(f, cmd, now); got != tc.want { + t.Fatalf("alarm=%v want %v", got, tc.want) + } + }) + } +} + +func TestConfiguredBatteryVisibleBeforeFirstCommand(t *testing.T) { + tel := telemetry.NewStore() + tel.Update("battery", telemetry.DerBattery, -300, nil, []byte(`{}`)) + cfg := &config.Config{Drivers: []config.Driver{{Name: "battery", BatteryCapacityWh: 10000}}} + srv := New(&Deps{Tel: tel, Cfg: cfg, CfgMu: &sync.RWMutex{}}) + got := srv.controlFeedback(time.Now()) + if len(got) != 1 || got[0].Reason != "no_command" || got[0].VerificationTier != nil || got[0].VerificationLost { + t.Fatalf("pre-command battery needs a neutral status, got %+v", got) + } +} + +// Exercise telemetry-to-API with a fake clock. The site sees actual device +// watts, not requested watts; no UI poll supplies the proof window. +func TestControlEvidenceIsIndependentOfTargetFulfilment(t *testing.T) { + for _, tc := range []struct { + name, action, reason, site string + kind telemetry.DerType + target, actual, before, extraLoad float64 + seconds, tier int + }{ + {"battery discharge shortfall", "battery", "power_below_target", "confirmed", telemetry.DerBattery, -5000, -4400, 0, 0, 42, 2}, + {"battery charge shortfall", "battery", "power_below_target", "confirmed", telemetry.DerBattery, 5000, 4400, 0, 0, 42, 2}, + {"EV shortfall", "ev_set_current", "power_below_target", "confirmed", telemetry.DerEV, 11000, 5500, 0, 0, 132, 2}, + {"V2X shortfall", "v2x_set_power", "power_below_target", "confirmed", telemetry.DerV2X, -5000, -4400, 0, 0, 42, 2}, + {"shortfall with unexplained load", "battery", "power_below_target", "site_change_differs", telemetry.DerBattery, -5000, -4400, 0, 2000, 42, 1}, + {"already at lower output", "battery", "power_below_target", "no_clear_change", telemetry.DerBattery, -5000, -4400, -4400, 0, 42, 1}, + {"no response is measured but not followed", "battery", "no_power_response", "no_clear_change", telemetry.DerBattery, -5000, 0, 0, 0, 42, 1}, + {"opposite response is confirmed but wrong", "battery", "power_wrong_direction", "confirmed", telemetry.DerBattery, -5000, 4400, 0, 0, 42, 2}, + {"overshoot is confirmed but wrong", "battery", "power_above_target", "confirmed", telemetry.DerBattery, -5000, -6000, 0, 0, 42, 2}, + {"ignored idle is confirmed but wrong", "battery", "power_while_idle", "confirmed", telemetry.DerBattery, 0, -4400, 0, 0, 42, 2}, + {"PV below ceiling is not proven curtailment", "curtail", "solar_below_ceiling", "confirmed", telemetry.DerPV, -3000, -1000, -5000, 0, 42, 2}, + {"ordinary response wait", "battery", "waiting_response", "device_response_unconfirmed", telemetry.DerBattery, -5000, -4400, 0, 0, 6, 0}, + {"matching response", "battery", "power_observed", "confirmed", telemetry.DerBattery, -5000, -5000, 0, 0, 42, 2}, + } { + t.Run(tc.name, func(t *testing.T) { + synctest.Test(t, func(t *testing.T) { + tel := telemetry.NewStore() + read := func(power, extra float64) { + tel.Update("hybrid", tc.kind, power, nil, []byte(`{}`)) + tel.Update("hybrid", telemetry.DerMeter, 800+power+extra, nil, []byte(`{"power_origin":"external_meter"}`)) + tel.RecordDriverSuccess("hybrid") + } + for range 5 { + read(tc.before, 0) + time.Sleep(3 * time.Second) + } + request, _ := json.Marshal(map[string]any{"action": tc.action, "power_w": tc.target}) + c := tel.BeginCommand("hybrid", request, time.Now()) + tel.CompleteCommand(c, "accepted") + for elapsed := 0; elapsed <= tc.seconds; elapsed += 3 { + read(tc.actual, tc.extraLoad) + time.Sleep(3 * time.Second) + } + srv := New(&Deps{Tel: tel, Ctrl: &control.State{SiteMeterDriver: "hybrid"}, CtrlMu: &sync.Mutex{}}) + f := srv.controlFeedback(time.Now())[0] + if f.VerificationTier == nil || *f.VerificationTier != tc.tier || f.Reason != tc.reason || f.SiteConfirmation != tc.site || f.VerificationLost { + t.Fatalf("got %+v; want tier %d, %s, %s with no lost-measurement alarm", f, tc.tier, tc.reason, tc.site) + } + if f.ActualW == nil || *f.ActualW != tc.actual || f.DeviceReason != "" { + t.Fatalf("changed watts or invented a cause: %+v", f) + } + if tc.tier == 2 && (f.DeviceDeltaW == nil || math.Abs(*f.DeviceDeltaW-(tc.actual-tc.before)) > 1) { + t.Fatalf("confirmed target instead of actual change: %+v", f) + } + if tc.reason == "power_below_target" && tc.tier == 2 { + tel.Update("hybrid", tc.kind, 0, nil, []byte(`{"control_power_available":false}`)) + time.Sleep(3 * time.Minute) + lost := srv.controlFeedback(time.Now())[0] + if lost.VerificationTier == nil || *lost.VerificationTier != 0 || !lost.VerificationLost || lost.ActualW != nil { + t.Fatalf("missing power kept proof: %+v", lost) + } + for range 6 { + read(tc.target, 0) + time.Sleep(3 * time.Second) + } + recovered := srv.controlFeedback(time.Now())[0] + if recovered.VerificationTier == nil || *recovered.VerificationTier != 2 || recovered.Reason != "power_observed" || recovered.VerificationLost { + t.Fatalf("fresh recovery failed: %+v", recovered) + } + } + }) + }) + } +} + +func TestFailedCallIsNotLostMeasurement(t *testing.T) { + now := time.Now() + for _, result := range []string{"failed", "unconfirmed", "default_failed", "pending", "released"} { + cmd := telemetry.CommandEvidence{Result: result, Since: now.Add(-time.Minute), LastMeasuredAt: now.Add(-time.Minute)} + if controlVerificationLost(ControlFeedback{Reason: "command_failed", ActualW: watts(-4400)}, cmd, now) { + t.Fatalf("%s call was relabelled as lost measurements", result) + } + } +} + +func TestFullBatteryFeedbackKeepsEvidenceAndFaultsSeparate(t *testing.T) { + for _, tc := range []struct { + name string + target, actual float64 + data, reason string + fault, stale bool + }{ + {name: "full stop", target: 0, actual: 0, data: `{"setpoint_w":0}`, reason: "battery_full"}, + {name: "stop ignored", target: 0, actual: 500, data: `{"setpoint_w":0}`, reason: "power_while_idle"}, + {name: "discharge ignored", target: -1000, actual: 0, data: `{"setpoint_w":-1000}`, reason: "no_power_response"}, + {name: "discharge allowed", target: -1000, actual: -1000, data: `{"setpoint_w":-1000}`, reason: "power_observed"}, + {name: "fault", target: 0, actual: 0, data: `{"setpoint_w":0}`, reason: "device_fault", fault: true}, + {name: "stale", target: 0, actual: 0, data: `{"setpoint_w":0}`, reason: "readings_lost", stale: true}, + {name: "setpoint changed", target: 0, actual: 0, data: `{"setpoint_w":1000}`, reason: "setpoint_changed"}, + {name: "power unknown", target: 0, actual: 0, data: `{"setpoint_w":0,"control_power_available":false}`, reason: "response_unknown"}, + } { + t.Run(tc.name, func(t *testing.T) { + synctest.Test(t, func(t *testing.T) { + tel := telemetry.NewStore() + soc := 1.0 + tel.Update("battery", telemetry.DerBattery, 0, &soc, []byte(`{}`)) + tel.RecordDriverSuccess("battery") + tel.Update("meter", telemetry.DerMeter, -1000, nil, []byte(`{}`)) + tel.RecordDriverSuccess("meter") + st := control.NewState(0, 50, "meter") + st.Mode = control.ModeCharge + targets := control.ComputeDispatch(tel, st, map[string]float64{"battery": 9600}, 13800) + if len(targets) != 1 || targets[0].TargetW != 0 { + t.Fatalf("full battery not stopped: %+v", targets) + } + request, _ := json.Marshal(map[string]any{"action": "battery", "power_w": tc.target}) + c := tel.BeginCommand("battery", request, time.Now()) + tel.CompleteCommand(c, "accepted") + for range 16 { + tel.Update("battery", telemetry.DerBattery, tc.actual, &soc, []byte(tc.data)) + time.Sleep(3 * time.Second) + } + if tc.fault { + tel.SetDriverDeviceFault("battery", true, "test fault") + } + if tc.stale { + time.Sleep(2 * time.Minute) + } + srv := New(&Deps{Tel: tel, Ctrl: st, CtrlMu: &sync.Mutex{}}) + f := srv.controlFeedback(time.Now())[0] + if f.Reason != tc.reason { + t.Fatalf("got %+v want %s", f, tc.reason) + } + if tc.reason == "battery_full" { + if f.Severity != "info" || f.VerificationTier == nil || *f.VerificationTier != 1 || f.ActualW == nil || *f.ActualW != 0 || f.BatterySoC == nil || *f.BatterySoC != 1 || f.ChargeResumeSoC == nil || *f.ChargeResumeSoC != .99 { + t.Fatalf("full stop invented proof or lost context: %+v", f) + } + } + }) + }) + } +} diff --git a/go/internal/api/api_control_scenarios_test.go b/go/internal/api/api_control_scenarios_test.go new file mode 100644 index 00000000..36595786 --- /dev/null +++ b/go/internal/api/api_control_scenarios_test.go @@ -0,0 +1,368 @@ +package api + +// Site traces replayed through telemetry, command evidence and the API on a +// fake clock. They cover the verdict over time, not one classified instant. + +import ( + "fmt" + "math" + "math/rand" + "sync" + "testing" + "testing/synctest" + "time" + + "github.com/srcfl/ftw/go/internal/config" + "github.com/srcfl/ftw/go/internal/control" + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// A hybrid inverter that relays its own grid meter, as on the Sungrow SH. +type hybridTrace struct { + tel *telemetry.Store + srv *Server + houseW, pvW, batW, cmdW, socV float64 + verdicts []ControlFeedback + at []time.Duration + start time.Time +} + +func newHybridTrace() *hybridTrace { + tel := telemetry.NewStore() + tel.DriverHealthMut("hybrid") + ctrl := &control.State{Mode: control.ModeSelfConsumption, SiteMeterDriver: "hybrid"} + srv := New(&Deps{Tel: tel, Ctrl: ctrl, CtrlMu: &sync.Mutex{}, Cfg: &config.Config{}, CfgMu: &sync.RWMutex{}}) + return &hybridTrace{tel: tel, srv: srv, socV: 0.5, start: time.Now()} +} + +func (h *hybridTrace) poll() { + h.tel.Update("hybrid", telemetry.DerMeter, h.houseW+h.pvW+h.batW, nil, []byte(`{"power_origin":"external_meter"}`)) + h.tel.Update("hybrid", telemetry.DerPV, h.pvW, nil, []byte(`{}`)) + soc := h.socV + h.tel.Update("hybrid", telemetry.DerBattery, h.batW, &soc, []byte(fmt.Sprintf(`{"setpoint_w":%g}`, h.cmdW))) +} + +func (h *hybridTrace) command(w float64) { + h.cmdW = w + c := h.tel.BeginCommand("hybrid", []byte(fmt.Sprintf(`{"action":"battery","power_w":%g}`, w)), time.Now()) + h.tel.CompleteCommand(c, "accepted") +} + +func (h *hybridTrace) sample() { + for _, f := range h.srv.controlFeedback(time.Now()) { + if f.Kind == "battery" { + h.verdicts = append(h.verdicts, f) + h.at = append(h.at, time.Since(h.start)) + } + } +} + +// share is the fraction of samples taken after `from` that match. +func (h *hybridTrace) share(from time.Duration, match func(ControlFeedback) bool) float64 { + n, hit := 0, 0 + for i, f := range h.verdicts { + if h.at[i] < from { + continue + } + n++ + if match(f) { + hit++ + } + } + if n == 0 { + return 0 + } + return float64(hit) / float64(n) +} + +func (h *hybridTrace) changes() int { + n := 0 + for i := 1; i < len(h.verdicts); i++ { + if h.verdicts[i].Status != h.verdicts[i-1].Status { + n++ + } + } + return n +} + +// Self-consumption retunes the target every tick. A battery that follows each +// command must read as following, not as a stream of new response waits. +func TestTrackingTargetStaysFollowing(t *testing.T) { + synctest.Test(t, func(t *testing.T) { + rng := rand.New(rand.NewSource(7)) + h := newHybridTrace() + h.houseW, h.pvW = 700, -2500 + for range 15 * 60 / 5 { + h.houseW = math.Max(150, h.houseW+rng.NormFloat64()*120) + if rng.Float64() < 0.03 { + h.houseW += 1500 * (rng.Float64() - 0.3) + } + h.command(math.Max(-5000, math.Min(5000, -(h.houseW+h.pvW)))) + time.Sleep(2500 * time.Millisecond) + h.batW = h.cmdW + rng.NormFloat64()*15 + h.poll() + h.sample() + time.Sleep(2500 * time.Millisecond) + h.poll() + h.sample() + } + following := h.share(time.Minute, func(f ControlFeedback) bool { return f.Status == "following" }) + if following < 0.9 || h.changes() > 12 { + t.Fatalf("following %.0f%% with %d status changes", following*100, h.changes()) + } + }) +} + +// The same retuning must not hide a battery that stops following. +func TestTrackingTargetWarnsWhenIgnored(t *testing.T) { + synctest.Test(t, func(t *testing.T) { + rng := rand.New(rand.NewSource(11)) + h := newHybridTrace() + h.houseW, h.pvW = 700, -2500 + for range 15 * 60 / 5 { + h.houseW = math.Max(150, h.houseW+rng.NormFloat64()*120) + h.command(math.Max(-5000, math.Min(5000, -(h.houseW+h.pvW)))) + time.Sleep(2500 * time.Millisecond) + h.batW = h.cmdW + rng.NormFloat64()*15 + if time.Since(h.start) >= 5*time.Minute { + h.batW = 0 + } + h.poll() + h.sample() + time.Sleep(2500 * time.Millisecond) + h.poll() + h.sample() + } + warned := h.share(6*time.Minute, func(f ControlFeedback) bool { + return f.Status == "not_following" && f.Severity == "warning" + }) + if warned < 0.95 { + t.Fatalf("ignored commands warned in %.0f%% of samples", warned*100) + } + }) +} + +// A kettle after a confirmed step changes the house, not the battery. +func TestConfirmedStepSurvivesLaterHouseholdLoad(t *testing.T) { + synctest.Test(t, func(t *testing.T) { + h := newHybridTrace() + h.houseW, h.pvW = 600, -4800 + for range 30 { + time.Sleep(time.Second) + h.poll() + } + h.command(-2250) + for sec := range 15 * 60 { + time.Sleep(time.Second) + if sec >= 2 { + h.batW = -2250 + } + switch sec { + case 180: + h.houseW += 2000 + case 360: + h.houseW -= 2000 + } + h.pvW = -4800 + 300*math.Sin(float64(sec)/90) + if sec%5 == 0 { + h.poll() + } + if sec%30 == 0 { + h.command(-2250) + } + h.sample() + } + confirmed := h.share(2*time.Minute, func(f ControlFeedback) bool { + return f.Status == "following" && f.Evidence == "confirmed" && f.ConfirmedAtMs > 0 + }) + if confirmed < 0.99 { + t.Fatalf("confirmed in %.0f%% of samples after the step", confirmed*100) + } + }) +} + +// A battery takes less charge as it fills. Its fresh SoC explains that. +func TestChargeTaperNearFullIsInformation(t *testing.T) { + synctest.Test(t, func(t *testing.T) { + h := newHybridTrace() + h.houseW, h.pvW, h.socV = 500, -4500, 0.95 + for range 30 { + time.Sleep(time.Second) + h.poll() + } + h.command(3000) + for sec := range 20 * 60 { + time.Sleep(time.Second) + h.socV = math.Min(0.995, 0.95+float64(sec)/20/60*0.045) + h.batW = 3000 + if h.socV > 0.96 { + h.batW = math.Max(300, 3000*(1-(h.socV-0.96)/0.035)) + } + if sec%5 == 0 { + h.poll() + } + if sec%30 == 0 { + h.command(3000) + } + h.sample() + } + if warned := h.share(0, func(f ControlFeedback) bool { return f.Severity != "info" }); warned > 0 { + t.Fatalf("normal taper warned in %.0f%% of samples", warned*100) + } + if full := h.share(10*time.Minute, func(f ControlFeedback) bool { return f.Reason == "battery_nearly_full" }); full < 0.9 { + t.Fatalf("taper explained in %.0f%% of samples", full*100) + } + }) +} + +// A cloud charger reports less often than a local inverter. Its declared +// power_max_age_s, not a local ten-second rule, decides freshness. +func TestCloudChargerReachesMeasuredFollowing(t *testing.T) { + synctest.Test(t, func(t *testing.T) { + tel := telemetry.NewStore() + tel.DriverHealthMut("cloud") + srv := New(&Deps{Tel: tel}) + start := time.Now() + var observedAt time.Time + var verdicts []ControlFeedback + var at []time.Duration + for sec := range 20 * 60 { + if sec%60 == 0 { + c := tel.BeginCommand("cloud", []byte(`{"action":"ev_set_current","power_w":11000}`), time.Now()) + tel.CompleteCommand(c, "accepted") + } + time.Sleep(time.Second) + if sec%60 == 30 { + observedAt = time.Now().Add(-5 * time.Second) + } + if sec%10 == 0 && !observedAt.IsZero() { + data := fmt.Sprintf(`{"connected":true,"control_power_observed_at":%q,"power_max_age_s":180}`, observedAt.Format(time.RFC3339Nano)) + tel.Update("cloud", telemetry.DerEV, 10900, nil, []byte(data)) + } + for _, f := range srv.controlFeedback(time.Now()) { + verdicts = append(verdicts, f) + at = append(at, time.Since(start)) + } + } + n, following := 0, 0 + for i, f := range verdicts { + if at[i] < 3*time.Minute { + continue + } + n++ + if f.Status == "following" && f.Evidence == "measured" { + following++ + } + } + if n == 0 || float64(following)/float64(n) < 0.95 { + t.Fatalf("cloud charger following in %d of %d samples", following, n) + } + }) +} + +func TestControlStatusAnswersEveryReason(t *testing.T) { + measured := 1 + for reason, want := range map[string][2]string{ + "power_observed": {"following", "info"}, + "idle": {"following", "info"}, + "vehicle_complete": {"following", "info"}, + "waiting_response": {"waiting", "info"}, + "not_connected": {"waiting", "info"}, + "no_plan_budget": {"waiting", "info"}, + "battery_full": {"limited", "info"}, + "battery_nearly_full": {"limited", "info"}, + "fuse_limit": {"limited", "info"}, + "device_limit": {"limited", "warning"}, + "power_below_target": {"not_following", "warning"}, + "setpoint_changed": {"not_following", "warning"}, + "device_fault": {"not_following", "alarm"}, + "telemetry_stale": {"no_contact", "warning"}, + "command_failed": {"no_contact", "warning"}, + "readings_lost": {"no_contact", "alarm"}, + "default_failed": {"no_contact", "alarm"}, + "observe_only": {"not_controlled", "info"}, + "device_control": {"not_controlled", "info"}, + "site_meter_stale": {"not_controlled", "warning"}, + "unknown_future_cause": {"waiting", "info"}, + } { + f := ControlFeedback{Reason: reason, VerificationTier: &measured} + setControlStatus(&f) + if f.Status != want[0] || f.Severity != want[1] || f.Evidence != "measured" { + t.Errorf("%s: got %s/%s/%s, want %s/%s", reason, f.Status, f.Severity, f.Evidence, want[0], want[1]) + } + } + accepted := 0 + f := ControlFeedback{Reason: "power_observed", VerificationTier: &accepted} + if setControlStatus(&f); f.Status != "waiting" || f.Evidence != "accepted" { + t.Fatalf("a driver reply alone read as following: %+v", f) + } +} + +// Easee's cloud records power only when it changes. While the cloud still +// hears from the charger, each poll confirms the unchanged value, so a steady +// charge stays measured instead of turning into a lost-control alarm. +func TestChangeOnlyChargerStaysCurrentWhileSteady(t *testing.T) { + synctest.Test(t, func(t *testing.T) { + tel := telemetry.NewStore() + tel.DriverHealthMut("cloud") + srv := New(&Deps{Tel: tel}) + start := time.Now() + changedAt := time.Time{} + power := 0.0 + var late []ControlFeedback + for sec := range 15 * 60 { + if sec%60 == 0 { + c := tel.BeginCommand("cloud", []byte(`{"action":"ev_set_current","power_w":11000}`), time.Now()) + tel.CompleteCommand(c, "accepted") + } + time.Sleep(time.Second) + // The charger ramps once, 40 s after the first command, then holds. + if sec == 40 { + power, changedAt = 10900, time.Now().Add(-3*time.Second) + } + if sec%10 == 0 && !changedAt.IsZero() { + data := fmt.Sprintf(`{"connected":true,"control_power_observed_at":%q,"control_power_confirmed":true,"power_max_age_s":180}`, changedAt.Format(time.RFC3339Nano)) + tel.Update("cloud", telemetry.DerEV, power, nil, []byte(data)) + } + if time.Since(start) >= 3*time.Minute { + late = append(late, srv.controlFeedback(time.Now())...) + } + } + for _, f := range late { + if f.Status != "following" || f.Evidence != "measured" || f.Severity != "info" { + t.Fatalf("steady confirmed charging read as %s/%s/%s (%s)", f.Status, f.Evidence, f.Severity, f.Reason) + } + } + }) +} + +// A confirmed unchanged value after a new target is evidence too: a charger +// that keeps 11 kW when asked for 4 kW is not following. +func TestChangeOnlyChargerIgnoringNewTargetWarns(t *testing.T) { + synctest.Test(t, func(t *testing.T) { + tel := telemetry.NewStore() + tel.DriverHealthMut("cloud") + srv := New(&Deps{Tel: tel}) + changedAt := time.Now() + target := 11000.0 + var last ControlFeedback + for sec := range 10 * 60 { + if sec == 5*60 { + target = 4000 + } + if sec%60 == 0 || sec == 5*60 { + c := tel.BeginCommand("cloud", []byte(fmt.Sprintf(`{"action":"ev_set_current","power_w":%g}`, target)), time.Now()) + tel.CompleteCommand(c, "accepted") + } + time.Sleep(time.Second) + if sec%10 == 0 { + data := fmt.Sprintf(`{"connected":true,"control_power_observed_at":%q,"control_power_confirmed":true,"power_max_age_s":180}`, changedAt.Format(time.RFC3339Nano)) + tel.Update("cloud", telemetry.DerEV, 11000, nil, []byte(data)) + } + last = srv.controlFeedback(time.Now())[0] + } + if last.Status != "not_following" || last.Reason != "power_above_target" { + t.Fatalf("ignored target read as %s/%s", last.Status, last.Reason) + } + }) +} diff --git a/go/internal/api/api_control_verification.go b/go/internal/api/api_control_verification.go new file mode 100644 index 00000000..be845591 --- /dev/null +++ b/go/internal/api/api_control_verification.go @@ -0,0 +1,317 @@ +package api + +import ( + "encoding/json" + "math" + "sort" + "time" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +type ControlResponsePoint struct { + AtMS int64 `json:"at_ms"` + DeviceW float64 `json:"device_change_w"` + SiteW float64 `json:"adjusted_site_change_w"` +} + +// ControlComparison keeps the measured correction and residual visible. It does +// not estimate which household appliance changed or assign a confidence score. +type ControlComparison struct { + BeforeW float64 `json:"-"` + AfterW float64 `json:"-"` + GridBeforeW *float64 `json:"grid_before_w,omitempty"` + GridAfterW *float64 `json:"grid_after_w,omitempty"` + BeforeAt time.Time `json:"-"` + AfterAt time.Time `json:"-"` + Reason string `json:"-"` + DeviceDeltaW *float64 `json:"device_change_w"` + SiteDeltaW *float64 `json:"grid_change_w"` + OtherDeltaW *float64 `json:"other_change_w"` + AdjustedDeltaW *float64 `json:"adjusted_site_change_w"` + ResidualW *float64 `json:"unexplained_change_w"` + ToleranceW float64 `json:"tolerance_w"` + Samples int `json:"samples"` + WindowS float64 `json:"window_s"` + MaxSkewMS *int64 `json:"max_skew_ms"` + UnmeasuredFlows []string `json:"unmeasured_flows,omitempty"` + Trace []ControlResponsePoint `json:"trace,omitempty"` +} + +type alignedControlPoint struct { + at time.Time + device, grid, other float64 + skew time.Duration +} + +// Compare the shape of grid-minus-other-measured-flows with the device's own +// curve. The command never supplies a measured watt. A before-command baseline +// is the hypothesis to test, not a derived house-load value reused as evidence. +func independentResponse(cmd telemetry.CommandEvidence, meter string, separate bool, after map[string]telemetry.ControlBaseline, now time.Time) ControlComparison { + out := ControlComparison{} + stop := func(reason string) ControlComparison { out.Reason = reason; return out } + if meter == "" { + return stop("no_site_meter") + } + if !separate { + return stop("independent_source_unknown") + } + target := cmd.Driver + ":" + cmd.Kind + gridKey := meter + ":meter" + if _, ok := cmd.Baseline[target]; !ok { + return stop("no_baseline") + } + if _, ok := cmd.Baseline[gridKey]; !ok { + return stop("no_baseline") + } + // Only the commanded function and site meter are required measurements. + // Every other flow is a correction when both windows support it. Otherwise + // its real effect stays in the residual; missing data never means zero power. + for _, k := range []string{target, gridKey} { + current, ok := after[k] + if !ok || !current.Window.Usable(now) { + return stop("waiting_for_meter") + } + if current.Window.First.Before(cmd.Since) { + return stop("waiting_for_meter") + } + } + keys := []string{target} + before := alignControlPoints(cmd.Baseline, gridKey, target, keys) + current := alignControlPoints(after, gridKey, target, keys) + if len(before) < 3 || len(current) < 3 { + return stop("readings_not_aligned") + } + if !controlSpanReady(before, cmd.Since) || !controlSpanReady(current, now) { + return stop("waiting_for_meter") + } + candidates := map[string]bool{} + for _, flows := range []map[string]telemetry.ControlBaseline{cmd.Baseline, after} { + for k, flow := range flows { + if k != target && flow.Kind != telemetry.DerMeter && flow.Kind != telemetry.DerVehicle { + candidates[k] = true + } + } + } + ordered := make([]string, 0, len(candidates)) + for k := range candidates { + ordered = append(ordered, k) + } + sort.Strings(ordered) + for _, k := range ordered { + old, next := cmd.Baseline[k], after[k] + trial := append(append([]string(nil), keys...), k) + b := alignControlPoints(cmd.Baseline, gridKey, target, trial) + a := alignControlPoints(after, gridKey, target, trial) + // Select by sample availability and timing only, never by whether a + // correction makes the power curves agree. Use one set on both sides. + if !old.Window.Usable(cmd.Since) || !next.Window.Usable(now) || next.Window.First.Before(cmd.Since) || !controlSpanReady(b, cmd.Since) || !controlSpanReady(a, now) { + out.UnmeasuredFlows = append(out.UnmeasuredFlows, k) + continue + } + keys, before, current = trial, b, a + } + values := func(points []alignedControlPoint, f func(alignedControlPoint) float64) []float64 { + v := make([]float64, len(points)) + for i, p := range points { + v[i] = f(p) + } + return v + } + device := func(p alignedControlPoint) float64 { return p.device } + grid := func(p alignedControlPoint) float64 { return p.grid } + other := func(p alignedControlPoint) float64 { return p.other } + adjusted := func(p alignedControlPoint) float64 { return p.grid - p.other } + residual := func(p alignedControlPoint) float64 { return p.grid - p.other - p.device } + baseDevice := meanControlPower(values(before, device)) + baseAdjusted := meanControlPower(values(before, adjusted)) + baseResidual := meanControlPower(values(before, residual)) + // A moving household baseline or moving actuator cannot identify this step. + if spread(values(before, residual)) > 200 || spread(values(before, device)) > 200 { + return stop("flows_changing") + } + deltaDevice := meanControlPower(values(current, device)) - baseDevice + out.BeforeW, out.AfterW = meanControlPower(values(before, grid)), meanControlPower(values(current, grid)) + out.GridBeforeW, out.GridAfterW = watts(out.BeforeW), watts(out.AfterW) + out.BeforeAt, out.AfterAt = before[len(before)-1].at, current[len(current)-1].at + out.DeviceDeltaW = watts(deltaDevice) + out.SiteDeltaW = watts(out.AfterW - out.BeforeW) + out.OtherDeltaW = watts(meanControlPower(values(current, other)) - meanControlPower(values(before, other))) + out.AdjustedDeltaW = watts(meanControlPower(values(current, adjusted)) - baseAdjusted) + out.ResidualW = watts(meanControlPower(values(current, residual)) - baseResidual) + out.ToleranceW = math.Max(150, math.Abs(deltaDevice)*0.15) + out.Samples = len(current) + out.WindowS = current[len(current)-1].at.Sub(current[0].at).Seconds() + maxSkewMS := int64(0) + out.MaxSkewMS = &maxSkewMS + for _, p := range before { + if p.skew.Milliseconds() > maxSkewMS { + maxSkewMS = p.skew.Milliseconds() + } + } + // All points must support the same change. Means alone could hide a household + // load starting and stopping inside the window. Do not widen tolerance to fit it. + matches := true + for _, p := range current { + out.Trace = append(out.Trace, ControlResponsePoint{p.at.UnixMilli(), p.device - baseDevice, p.grid - p.other - baseAdjusted}) + if p.skew.Milliseconds() > maxSkewMS { + maxSkewMS = p.skew.Milliseconds() + } + if math.Abs(residual(p)-baseResidual) > out.ToleranceW { + matches = false + } + } + // Unmeasured background may include a disconnected battery or solar source, + // so its absolute sign cannot establish an impossible energy balance. Its + // change must still agree at every point; a load pulse cannot be averaged out. + if math.Abs(deltaDevice) < 500 { + return stop("no_clear_change") + } + if !matches { + return stop("site_change_differs") + } + return stop("confirmed") +} + +// Use only distinct real samples. Never interpolate a missing transition or +// reuse a slow source to manufacture a dense curve. Bound skew across ALL flows, +// not just target and grid, because moving solar makes stale alignment unsafe. +func alignControlPoints(series map[string]telemetry.ControlBaseline, gridKey, target string, keys []string) []alignedControlPoint { + var out []alignedControlPoint + used := map[string]time.Time{} + for _, g := range series[gridKey].Points { + p := alignedControlPoint{at: g.At, grid: g.PowerW} + first, last := g.At, g.At + chosen := map[string]telemetry.ControlObservation{} + for _, k := range keys { + var nearest telemetry.ControlObservation + distance := 3 * time.Second + for _, v := range series[k].Points { + d := v.At.Sub(g.At) + if d < 0 { + d = -d + } + if d < distance { + nearest, distance = v, d + } + } + if nearest.At.IsZero() || distance > 2*time.Second || !nearest.At.After(used[k]) { + break + } + chosen[k] = nearest + if nearest.At.Before(first) { + first = nearest.At + } + if nearest.At.After(last) { + last = nearest.At + } + if k == target { + p.device = nearest.PowerW + } else { + p.other += nearest.PowerW + } + } + if len(chosen) != len(keys) || last.Sub(first) > 2*time.Second { + continue + } + p.skew = last.Sub(first) + out = append(out, p) + for k, v := range chosen { + used[k] = v.At + } + } + return out +} +func meanControlPower(values []float64) float64 { + total := 0.0 + for _, value := range values { + total += value + } + return total / float64(len(values)) +} + +func spread(values []float64) float64 { + lo, hi := values[0], values[0] + for _, v := range values { + lo = math.Min(lo, v) + hi = math.Max(hi, v) + } + return hi - lo +} + +func (s *Server) separateMeterSource(driver, meter string) bool { + if s.deps.Tel == nil { + return false + } + reading := s.deps.Tel.Get(meter, telemetry.DerMeter) + if reading == nil { + return false + } + var data struct { + PowerOrigin string `json:"power_origin"` + } + if json.Unmarshal(reading.Data, &data) != nil || data.PowerOrigin == "derived" { + return false + } + // A driver may relay an external physical meter. Its documented sensor + // origin matters here, not whether it shares the inverter's connection. + if driver == meter { + return data.PowerOrigin == "external_meter" + } + if s.deps.Registry == nil { + return false + } + a, b := s.runningDriverDevice(driver), s.runningDriverDevice(meter) + if a.Serial == "" && s.deps.OCPPChargers != nil { + if c, ok := s.deps.OCPPChargers()[driver]; ok { + a.Serial = c.Serial + a.DeviceID = c.Vendor + ":" + c.Serial + } + } + return a.Serial != "" && b.Serial != "" && a.Serial != b.Serial && a.DeviceID != "" && b.DeviceID != "" && a.DeviceID != b.DeviceID +} + +// Source inventory helps avoid double counting; it does not require every +// device to be online. Control proof belongs to one function at a time. +func (s *Server) controlResponse(cmd telemetry.CommandEvidence, meter string, after map[string]telemetry.ControlBaseline, now time.Time) ControlComparison { + opts := telemetry.ForecastOptions{} + if s.deps.SiteMeasurementSources != nil { + opts = s.deps.SiteMeasurementSources() + } + counts := map[string]int{} + for _, f := range opts.ExpectedFlows { + if f.FlowID != "" { + counts[f.FlowID]++ + } + k := f.Driver + ":" + f.DerType.String() + if _, exists := after[k]; !exists { + after[k] = telemetry.ControlBaseline{Driver: f.Driver, Kind: f.DerType} + } + } + duplicates := map[string]bool{} + for _, f := range opts.ExpectedFlows { + if f.FlowID != "" && counts[f.FlowID] > 1 { + duplicates[f.Driver+":"+f.DerType.String()] = true + } + } + target := cmd.Driver + ":" + cmd.Kind + for k, f := range after { + if k == target || f.Kind == telemetry.DerMeter || f.Kind == telemetry.DerVehicle { + continue + } + _, ev := after[f.Driver+":ev"] + _, v2x := after[f.Driver+":v2x_charger"] + ambiguous := duplicates[k] || (ev && v2x && (f.Kind == telemetry.DerEV || f.Kind == telemetry.DerV2X)) + unidentified := opts.HouseholdInvalidReason != "" || f.Kind == telemetry.DerPV && opts.PVInvalidReason != "" + h := s.deps.Tel.DriverHealth(f.Driver) + if h == nil || !h.TelemetryLive() || h.DeviceFault || ambiguous || unidentified { + f.Window, f.Points = telemetry.ControlWindow{}, nil + after[k] = f + } + } + return independentResponse(cmd, meter, s.separateMeterSource(cmd.Driver, meter), after, now) +} + +func controlSpanReady(points []alignedControlPoint, now time.Time) bool { + return len(points) >= 3 && points[len(points)-1].at.Sub(points[0].at) >= 8*time.Second && !points[len(points)-1].at.After(now) && now.Sub(points[len(points)-1].at) <= 10*time.Second +} diff --git a/go/internal/api/api_control_verification_test.go b/go/internal/api/api_control_verification_test.go new file mode 100644 index 00000000..46bbe571 --- /dev/null +++ b/go/internal/api/api_control_verification_test.go @@ -0,0 +1,265 @@ +package api + +import ( + "math" + "math/rand" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +func responseSeries(driver string, kind telemetry.DerType, at time.Time, values []float64) telemetry.ControlBaseline { + r := telemetry.ControlBaseline{Driver: driver, Kind: kind} + for i, v := range values { + r.Points = append(r.Points, telemetry.ControlObservation{PowerW: v, At: at.Add(time.Duration(i) * 3 * time.Second)}) + } + r.Window = telemetry.ControlWindow{MeanW: values[0], MinW: values[0], MaxW: values[0], First: r.Points[0].At, Last: r.Points[len(r.Points)-1].At, Count: len(values)} + return r +} + +func TestIndependentControlConfirmation(t *testing.T) { + now := time.Now() + flat := func(v float64) []float64 { return []float64{v, v, v, v, v} } + for _, tc := range []struct { + name string + device, grid, pv []float64 + separate bool + want string + }{ + {"one kW charge", flat(1000), flat(1400), flat(-1200), true, "confirmed"}, + {"one kW discharge", flat(-1000), flat(-600), flat(-1200), true, "confirmed"}, + {"noise within tolerance", flat(990), []float64{1420, 1380, 1440, 1400, 1410}, flat(-1200), true, "confirmed"}, + {"solar offsets charging at the meter", flat(1000), flat(400), flat(-2200), true, "confirmed"}, + {"cloud moves while charging ramps", []float64{400, 700, 1000, 1000, 1000}, []float64{600, 200, 100, 600, 1100}, []float64{-1400, -2100, -2500, -2000, -1500}, true, "confirmed"}, + {"household load starts", flat(1000), flat(2600), flat(-1200), true, "site_change_differs"}, + {"household load stops", flat(1000), flat(200), flat(-1200), true, "site_change_differs"}, + {"pulse disappears in the mean", flat(1000), []float64{1400, 2400, 1400, 400, 1400}, flat(-1200), true, "site_change_differs"}, + {"reported solar change absent from grid", flat(1000), flat(1400), flat(-2200), true, "site_change_differs"}, + {"same physical sensor", flat(1000), flat(1400), flat(-1200), false, "independent_source_unknown"}, + {"small change", flat(100), flat(500), flat(-1200), true, "no_clear_change"}, + {"impossible balance", flat(5000), flat(0), flat(-1200), true, "site_change_differs"}, + } { + t.Run(tc.name, func(t *testing.T) { + base := map[string]telemetry.ControlBaseline{ + "battery:battery": responseSeries("battery", telemetry.DerBattery, now.Add(-30*time.Second), flat(0)), + "grid:meter": responseSeries("grid", telemetry.DerMeter, now.Add(-30*time.Second), flat(400)), + "solar:pv": responseSeries("solar", telemetry.DerPV, now.Add(-30*time.Second), flat(-1200)), + } + after := map[string]telemetry.ControlBaseline{ + "battery:battery": responseSeries("battery", telemetry.DerBattery, now.Add(-12*time.Second), tc.device), + "grid:meter": responseSeries("grid", telemetry.DerMeter, now.Add(-12*time.Second), tc.grid), + "solar:pv": responseSeries("solar", telemetry.DerPV, now.Add(-12*time.Second), tc.pv), + } + cmd := telemetry.CommandEvidence{Driver: "battery", Kind: "battery", Since: now.Add(-15 * time.Second), Baseline: base} + got := independentResponse(cmd, "grid", tc.separate, after, now) + if got.Reason != tc.want { + t.Fatalf("got %+v, want %s", got, tc.want) + } + if tc.want != "confirmed" { + return + } + if got.Samples != 5 || got.WindowS != 12 || len(got.Trace) != 5 { + t.Fatalf("missing trace: %+v", got) + } + for _, missing := range []string{"grid:meter", "battery:battery"} { + saved := after[missing] + delete(after, missing) + if independentResponse(cmd, "grid", true, after, now).Reason == "confirmed" { + t.Fatalf("confirmed without %s", missing) + } + after[missing] = saved + } + // A delayed or reused source cannot manufacture aligned samples. + saved := after["solar:pv"] + delayed := saved + delayed.Points = append([]telemetry.ControlObservation(nil), saved.Points...) + for i := range delayed.Points { + delayed.Points[i].At = now.Add(-30 * time.Second) + } + after["solar:pv"] = delayed + if got := independentResponse(cmd, "grid", true, after, now); len(got.UnmeasuredFlows) != 1 || got.UnmeasuredFlows[0] != "solar:pv" { + t.Fatalf("cached solar counted as fresh: %+v", got) + } + after["solar:pv"] = saved + if independentResponse(cmd, "grid", true, after, now.Add(time.Minute)).Reason == "confirmed" { + t.Fatal("stale site confirmed") + } + }) + } +} + +func TestIndependentResponseReplaysChangingSites(t *testing.T) { + rng := rand.New(rand.NewSource(42)) + now := time.Now() + for run := 0; run < 250; run++ { + direction := 1.0 + if run%2 == 1 { + direction = -1 + } + step := direction * (600 + rng.Float64()*4400) + house := 500 + rng.Float64()*4000 + baseSolar := -500 - rng.Float64()*6000 + beforeGrid, beforePV, zero, device, grid, pv := []float64{}, []float64{}, []float64{}, []float64{}, []float64{}, []float64{} + for i := 0; i < 5; i++ { + solar := baseSolar + rng.Float64()*400 + beforePV = append(beforePV, solar) + beforeGrid = append(beforeGrid, house+solar) + zero = append(zero, 0) + solar = baseSolar - rng.Float64()*1500 + power := step * math.Min(1, float64(i+1)/3) + device = append(device, power) + pv = append(pv, solar) + grid = append(grid, house+solar+power+(rng.Float64()-.5)*40) + } + cmd := telemetry.CommandEvidence{Driver: "battery", Kind: "battery", Since: now.Add(-15 * time.Second), Baseline: map[string]telemetry.ControlBaseline{ + "battery:battery": responseSeries("battery", telemetry.DerBattery, now.Add(-30*time.Second), zero), + "grid:meter": responseSeries("grid", telemetry.DerMeter, now.Add(-30*time.Second), beforeGrid), + "solar:pv": responseSeries("solar", telemetry.DerPV, now.Add(-30*time.Second), beforePV), + }} + after := map[string]telemetry.ControlBaseline{ + "battery:battery": responseSeries("battery", telemetry.DerBattery, now.Add(-12*time.Second), device), + "grid:meter": responseSeries("grid", telemetry.DerMeter, now.Add(-12*time.Second), grid), + "solar:pv": responseSeries("solar", telemetry.DerPV, now.Add(-12*time.Second), pv), + } + if got := independentResponse(cmd, "grid", true, after, now); got.Reason != "confirmed" { + t.Fatalf("replay %d: %+v", run, got) + } + grid[2] += 2000 // A load pulse cannot disappear into an average or solar correction. + after["grid:meter"] = responseSeries("grid", telemetry.DerMeter, now.Add(-12*time.Second), grid) + if got := independentResponse(cmd, "grid", true, after, now); got.Reason == "confirmed" { + t.Fatalf("replay %d accepted an unexplained pulse", run) + } + } +} + +func TestExternalMeterThroughSameDriver(t *testing.T) { + now := time.Now() + tel := telemetry.NewStore() + srv := New(&Deps{Tel: tel}) + for _, tc := range []struct { + data string + want bool + }{ + {`{}`, false}, {`{"power_origin":"derived"}`, false}, + {`{"power_origin":"inverter"}`, false}, {`{"power_origin":"external_meter"}`, true}, + } { + tel.Update("hybrid", telemetry.DerMeter, 1400, nil, []byte(tc.data)) + if got := srv.separateMeterSource("hybrid", "hybrid"); got != tc.want { + t.Fatalf("origin %s: separate=%v", tc.data, got) + } + } + flat := func(v float64) []float64 { return []float64{v, v, v, v, v} } + cmd := telemetry.CommandEvidence{Driver: "hybrid", Kind: "battery", Since: now.Add(-15 * time.Second), Baseline: map[string]telemetry.ControlBaseline{ + "hybrid:battery": responseSeries("hybrid", telemetry.DerBattery, now.Add(-30*time.Second), flat(0)), + "hybrid:meter": responseSeries("hybrid", telemetry.DerMeter, now.Add(-30*time.Second), flat(400)), + "hybrid:pv": responseSeries("hybrid", telemetry.DerPV, now.Add(-30*time.Second), flat(-1200)), + }} + after := map[string]telemetry.ControlBaseline{ + "hybrid:battery": responseSeries("hybrid", telemetry.DerBattery, now.Add(-12*time.Second), flat(1000)), + "hybrid:meter": responseSeries("hybrid", telemetry.DerMeter, now.Add(-12*time.Second), flat(1400)), + "hybrid:pv": responseSeries("hybrid", telemetry.DerPV, now.Add(-12*time.Second), flat(-1200)), + } + separate := srv.separateMeterSource("hybrid", "hybrid") + for _, skew := range []time.Duration{0, 250 * time.Millisecond} { + solar := responseSeries("hybrid", telemetry.DerPV, now.Add(-12*time.Second-skew), flat(-1200)) + after["hybrid:pv"] = solar + got := independentResponse(cmd, "hybrid", separate, after, now) + if got.Reason != "confirmed" || got.MaxSkewMS == nil || *got.MaxSkewMS != skew.Milliseconds() { + t.Fatalf("skew %s: %+v", skew, got) + } + } + missing := cmd + missing.Baseline = nil + if got := independentResponse(missing, "hybrid", separate, after, now); got.MaxSkewMS != nil { + t.Fatalf("missing baseline invented zero timing: %+v", got) + } + delete(after, "hybrid:meter") + if got := independentResponse(cmd, "hybrid", separate, after, now); got.Reason == "confirmed" || got.MaxSkewMS != nil { + t.Fatalf("external sensor identity replaced fresh meter evidence: %+v", got) + } +} + +func TestControlResponseUsesPerDeviceResidual(t *testing.T) { + now := time.Now() + flat := func(v float64) []float64 { return []float64{v, v, v, v, v} } + for _, tc := range []struct { + name string + missing, offline, late, pulse, unknownGeneration bool + }{ + {name: "charger never emitted", missing: true}, + {name: "charger offline", offline: true}, + {name: "charger returns after command", late: true}, + {name: "unknown load pulse", missing: true, pulse: true}, + {name: "unmeasured battery exports", missing: true, unknownGeneration: true}, + } { + t.Run(tc.name, func(t *testing.T) { + tel := telemetry.NewStore() + tel.Update("hybrid", telemetry.DerMeter, 1400, nil, []byte(`{"power_origin":"external_meter"}`)) + tel.RecordDriverSuccess("hybrid") + bgKind := telemetry.DerEV + gridBefore, gridAfter := 400.0, 1400.0 + if tc.unknownGeneration { + bgKind = telemetry.DerBattery + gridBefore, gridAfter = -4000, -3000 + } + bgKey := "other:" + bgKind.String() + srv := New(&Deps{Tel: tel, SiteMeasurementSources: func() telemetry.ForecastOptions { + return telemetry.ForecastOptions{ExpectedFlows: []telemetry.ForecastFlow{{Driver: "hybrid", DerType: telemetry.DerBattery}, {Driver: "hybrid", DerType: telemetry.DerPV}, {Driver: "other", DerType: bgKind}}} + }}) + cmd := telemetry.CommandEvidence{Driver: "hybrid", Kind: "battery", Since: now.Add(-15 * time.Second), Baseline: map[string]telemetry.ControlBaseline{ + "hybrid:battery": responseSeries("hybrid", telemetry.DerBattery, now.Add(-30*time.Second), flat(0)), + "hybrid:meter": responseSeries("hybrid", telemetry.DerMeter, now.Add(-30*time.Second), flat(gridBefore)), + "hybrid:pv": responseSeries("hybrid", telemetry.DerPV, now.Add(-30*time.Second), flat(-1200)), + }} + grid := flat(gridAfter) + if tc.pulse { + grid[2] += 2000 + } + after := map[string]telemetry.ControlBaseline{ + "hybrid:battery": responseSeries("hybrid", telemetry.DerBattery, now.Add(-12*time.Second), flat(1000)), + "hybrid:meter": responseSeries("hybrid", telemetry.DerMeter, now.Add(-12*time.Second), grid), + "hybrid:pv": responseSeries("hybrid", telemetry.DerPV, now.Add(-12*time.Second), flat(-1200)), + } + if !tc.missing { + after[bgKey] = responseSeries("other", bgKind, now.Add(-12*time.Second), flat(6000)) + if !tc.late { + cmd.Baseline[bgKey] = responseSeries("other", bgKind, now.Add(-30*time.Second), flat(3000)) + } + if !tc.offline { + tel.RecordDriverSuccess("other") + } + } + got := srv.controlResponse(cmd, "hybrid", after, now) + want := "confirmed" + if tc.pulse { + want = "site_change_differs" + } + if got.Reason != want || len(got.UnmeasuredFlows) != 1 || got.UnmeasuredFlows[0] != bgKey { + t.Fatalf("got %+v, want %s with background %s", got, want, bgKey) + } + if *got.OtherDeltaW != 0 { + t.Fatalf("invented correction from missing source: %+v", got) + } + }) + } +} + +func TestUsableOtherFlowCannotBeDroppedToMakeCurvesAgree(t *testing.T) { + now := time.Now() + flat := func(v float64) []float64 { return []float64{v, v, v, v, v} } + cmd := telemetry.CommandEvidence{Driver: "battery", Kind: "battery", Since: now.Add(-15 * time.Second), Baseline: map[string]telemetry.ControlBaseline{ + "battery:battery": responseSeries("battery", telemetry.DerBattery, now.Add(-30*time.Second), flat(0)), + "grid:meter": responseSeries("grid", telemetry.DerMeter, now.Add(-30*time.Second), flat(400)), + "ev:ev": responseSeries("ev", telemetry.DerEV, now.Add(-30*time.Second), flat(0)), + }} + after := map[string]telemetry.ControlBaseline{ + "battery:battery": responseSeries("battery", telemetry.DerBattery, now.Add(-12*time.Second), flat(1000)), + "grid:meter": responseSeries("grid", telemetry.DerMeter, now.Add(-12*time.Second), flat(1400)), + "ev:ev": responseSeries("ev", telemetry.DerEV, now.Add(-12*time.Second), flat(2000)), + } + got := independentResponse(cmd, "grid", true, after, now) + if got.Reason != "site_change_differs" || len(got.UnmeasuredFlows) != 0 { + t.Fatalf("ignored available conflicting evidence: %+v", got) + } +} diff --git a/go/internal/api/api_support_report.go b/go/internal/api/api_support_report.go index 582cd08d..f2796359 100644 --- a/go/internal/api/api_support_report.go +++ b/go/internal/api/api_support_report.go @@ -27,6 +27,7 @@ package api import ( "context" + "encoding/json" "fmt" "github.com/srcfl/ftw/go/internal/units" @@ -232,6 +233,11 @@ func (s *Server) buildSupportReport(ctx context.Context, now time.Time) string { var b strings.Builder writeReportHeader(&b, s.deps.Version, now) writeFindings(&b, findings) + if feedback, err := json.MarshalIndent(s.controlFeedback(now), "", " "); err == nil { + b.WriteString("## Command feedback\n\nDriver-call results, readbacks and measured power are separate evidence. Missing values are unknown.\n\n```json\n") + b.Write(feedback) + b.WriteString("\n```\n\n") + } writeRightNow(&b, ctrl, snap, activeSlot, targets, slotEnergy, now) writePlanControlDecision(&b, plan, slotEnergy) writePlanSection(&b, plan, lastReplanAt, lastReplanReason, now) diff --git a/go/internal/control/battery_full.go b/go/internal/control/battery_full.go new file mode 100644 index 00000000..b7c35840 --- /dev/null +++ b/go/internal/control/battery_full.go @@ -0,0 +1,81 @@ +package control + +import ( + "math" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// A one percentage point gap prevents repeated starts at a rounded 100%. +// This limits charging only; the plan can discharge a full battery at once. +const BatteryChargeResumeSoC = 0.99 + +// BatteryChargePaused reports the dispatcher's full-battery stop. Callers use +// the same lock as ComputeDispatch; this state is local to the running device. +func (s *State) BatteryChargePaused(driver string) bool { + return s != nil && s.batteryChargeFull[driver] +} + +func (s *State) updateBatteryChargeFull(store *telemetry.Store, capacities map[string]float64) { + if s.batteryChargeFull == nil { + s.batteryChargeFull = map[string]bool{} + } + for name := range s.batteryChargeFull { + if _, ok := capacities[name]; !ok { + delete(s.batteryChargeFull, name) + } + } + for name := range capacities { + r := store.Get(name, telemetry.DerBattery) + h := store.DriverHealth(name) + if r == nil || r.SoC == nil || h == nil || !h.IsOnline() { + continue + } + age := s.now().Sub(r.SoCUpdatedAt) + soc := *r.SoC + if age < 0 || age > telemetry.BatterySoCMaxAge || math.IsNaN(soc) || math.IsInf(soc, 0) || soc < 0 || soc > 1 { + continue + } + if soc == 1 { + s.batteryChargeFull[name] = true + } else if soc <= BatteryChargeResumeSoC { + delete(s.batteryChargeFull, name) + } + } +} + +func (s *State) floorFullBatteryCharge(targets []DispatchTarget) []DispatchTarget { + for i := range targets { + if targets[i].TargetW > 0 && s.BatteryChargePaused(targets[i].Driver) { + targets[i].TargetW = 0 + targets[i].Clamped = true + } + } + return targets +} + +// A holdoff or quiet meter must not leave an earlier charging command active. +// Preserve any fuse-relief discharge and stop only online, controlled batteries. +func (s *State) stopFullBatteries(targets []DispatchTarget, store *telemetry.Store, capacities map[string]float64) []DispatchTarget { + if s == nil { + return targets + } + seen := map[string]bool{} + for _, target := range targets { + seen[target.Driver] = true + } + for name := range capacities { + if seen[name] || !s.BatteryChargePaused(name) { + continue + } + r, h := store.Get(name, telemetry.DerBattery), store.DriverHealth(name) + if r == nil || h == nil || !h.IsOnline() || s.PrevTargets[name] <= 0 && r.RawW <= 100 { + continue + } + targets = append(targets, DispatchTarget{Driver: name, TargetW: 0, Clamped: true}) + if s.PrevTargets != nil { + s.PrevTargets[name] = 0 + } + } + return s.floorFullBatteryCharge(targets) +} diff --git a/go/internal/control/battery_full_test.go b/go/internal/control/battery_full_test.go new file mode 100644 index 00000000..585b1f93 --- /dev/null +++ b/go/internal/control/battery_full_test.go @@ -0,0 +1,126 @@ +package control + +import ( + "testing" + "testing/synctest" + "time" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +func fullBatterySite(soc, power float64) (*telemetry.Store, *State, map[string]float64) { + tel := seedStore(-1000, []struct { + name string + currentW, soc float64 + }{{"battery", power, soc}}) + st := NewState(0, 50, "ferroamp") + st.MinDispatchIntervalS = 0 + st.Mode = ModeCharge + return tel, st, map[string]float64{"battery": 9600} +} + +func TestFullBatteryStopsChargingAcrossModes(t *testing.T) { + for _, mode := range AllModes() { + t.Run(string(mode), func(t *testing.T) { + tel, st, capacities := fullBatterySite(1, 2000) + st.Mode = mode + st.UseEnergyDispatch = true + st.SlewEnabled, st.SlewRateW = true, 10 + st.SlotDirective = func(now time.Time) (SlotDirective, bool) { + return SlotDirective{SlotStart: now, SlotEnd: now.Add(15 * time.Minute), BatteryEnergyWh: 500, Strategy: "arbitrage"}, true + } + st.PlanTarget = func(time.Time) (string, float64, string, bool) { return string(ModeCharge), 2000, "test", true } + got := ComputeDispatch(tel, st, capacities, 13800) + if len(got) == 0 { + t.Fatal("full battery still charging: no stop command") + } + for _, target := range got { + if target.TargetW > 0 { + t.Fatalf("charge survived full stop: %+v", got) + } + } + }) + } +} + +func TestFullBatteryManualStopDischargeAndSibling(t *testing.T) { + tel, st, capacities := fullBatterySite(1, 0) + tel.Update("other", telemetry.DerBattery, 0, ptrF64(.5), nil) + tel.RecordDriverSuccess("other") + capacities["other"] = 9600 + st.SlewEnabled = false + st.SetBatteryManualHold(BatteryManualHold{PowerW: 3000, ExpiresAt: time.Now().Add(time.Hour)}) + got := ComputeDispatch(tel, st, capacities, 13800) + for _, target := range got { + if target.Driver == "battery" && target.TargetW != 0 { + t.Fatalf("full battery received charge: %+v", got) + } + if target.Driver == "other" && target.TargetW < 2900 { + t.Fatalf("capable sibling lost charge: %+v", got) + } + } + st.SetBatteryManualHold(BatteryManualHold{PowerW: -2000, ExpiresAt: time.Now().Add(time.Hour)}) + got = ComputeDispatch(tel, st, capacities, 13800) + for _, target := range got { + if target.TargetW >= 0 { + t.Fatalf("full stop prevented discharge: %+v", got) + } + } +} + +func TestFullBatteryHysteresisAndMissingSoC(t *testing.T) { + synctest.Test(t, func(t *testing.T) { + tel, st, capacities := fullBatterySite(.999, 0) + check := func(wantPaused bool) { + t.Helper() + got := ComputeDispatch(tel, st, capacities, 13800) + if st.BatteryChargePaused("battery") != wantPaused || len(got) != 1 || (got[0].TargetW == 0) != wantPaused { + t.Fatalf("paused=%v targets=%+v want paused=%v", st.BatteryChargePaused("battery"), got, wantPaused) + } + } + check(false) // A near-full reading alone does not prove full. + for _, soc := range []float64{1, .999, .995, 1, .991} { + tel.Update("battery", telemetry.DerBattery, 0, &soc, nil) + check(true) + } + time.Sleep(6 * time.Minute) + tel.Update("battery", telemetry.DerBattery, 0, nil, nil) + tel.RecordDriverSuccess("battery") + check(true) // Missing SoC cannot release a known full stop. + tel.Update("battery", telemetry.DerBattery, 0, ptrF64(.99), nil) + check(false) + }) +} + +func TestFullBatteryStopsPreviousChargeDuringHoldoffAndDeadband(t *testing.T) { + for _, holdoff := range []bool{false, true} { + tel, st, capacities := fullBatterySite(1, 0) + st.Mode = ModeSelfConsumption + tel.Update("ferroamp", telemetry.DerMeter, 0, nil, nil) + // The BMS has already stopped power, but the previous command is still positive. + st.PrevTargets["battery"] = 530 + st.LastTargets = []DispatchTarget{{Driver: "battery", TargetW: 530}} + if holdoff { + now := time.Now() + st.LastDispatch = &now + st.MinDispatchIntervalS = 60 + } + got := ComputeDispatch(tel, st, capacities, 13800) + if len(got) != 1 || got[0].TargetW != 0 || st.LastTargets[0].TargetW != 0 { + t.Fatalf("holdoff=%v left prior charge active: %+v", holdoff, got) + } + } +} + +func TestFullBatteryStillProvidesFuseRelief(t *testing.T) { + tel, st, capacities := fullBatterySite(1, 0) + tel = seedStore(16000, []struct { + name string + currentW, soc float64 + }{{"battery", 0, 1}}) + st.Mode = ModeIdle + got := ComputeDispatch(tel, st, capacities, 13800) + if len(got) != 1 || got[0].TargetW >= 0 { + t.Fatalf("full stop blocked fuse relief: %+v", got) + } +} diff --git a/go/internal/control/dispatch.go b/go/internal/control/dispatch.go index bafe3a20..16ce0f92 100644 --- a/go/internal/control/dispatch.go +++ b/go/internal/control/dispatch.go @@ -481,6 +481,7 @@ type State struct { MinDispatchIntervalS int LastDispatch *time.Time PrevTargets map[string]float64 + batteryChargeFull map[string]bool // clock is nil in production. Tests may set it to keep a dispatch // scenario on one instant even when the test runner is delayed. @@ -1339,6 +1340,7 @@ func ComputeDispatch( // main battery decision. state.controlSlotDecisionID = "" tickAt := state.now() + state.updateBatteryChargeFull(store, driverCapacities) state.controlPlanSnapshot = controlPlanSnapshot{active: true, tickAt: tickAt} defer func() { state.controlPlanSnapshot = controlPlanSnapshot{} }() // ---- Per-slot Wh delivery observability (path-agnostic) ---- @@ -1695,7 +1697,7 @@ func ComputeDispatch( // "use MaxCommandW" sentinel. Feed it into the allocator as well as // the clamps so capable siblings receive the blocked battery's share. dischargeBlocked = dischargeBlocked || lim.dischargeCap() == 0 - chargeBlocked = chargeBlocked || lim.chargeCap() == 0 + chargeBlocked = chargeBlocked || lim.chargeCap() == 0 || state.BatteryChargePaused(name) batteries = append(batteries, batteryInfo{ driver: name, capacityWh: cap, @@ -2708,6 +2710,7 @@ func applyDispatchSafetyPipeline( targets = clampTargetsToPowerLimits(targets, state.DriverLimits) } targets = floorMissingSoCDischarge(targets, store) + targets = state.floorFullBatteryCharge(targets) republishFuseEVCapAfterFuseDischarge(targets, store, state, fuseMaxW) recordDispatchTargets(targets, state, opts.updatePrevTargets, opts.recordDispatch) return targets @@ -4457,6 +4460,7 @@ func fuseSaverEarlyExit( fuseMaxW float64, ) []DispatchTarget { out := fuseSaverFromLivePower(store, state, driverCapacities, fuseMaxW) + out = state.stopFullBatteries(out, store, driverCapacities) if out == nil || state == nil { return out } diff --git a/go/internal/drivers/registry.go b/go/internal/drivers/registry.go index 290124a2..ca1f5a9e 100644 --- a/go/internal/drivers/registry.go +++ b/go/internal/drivers/registry.go @@ -819,6 +819,9 @@ func (r *Registry) runLoop(rd *runningDriver) { invalidateCommandSequence() defaultCtx, cancel := context.WithTimeout(context.Background(), defaultRecoveryTimeout) defaultErr := rd.driver.DefaultMode(defaultCtx) + if r.tel != nil { + r.tel.EndCommandControl(rd.cfg.Name, defaultErr != nil) + } cancel() if defaultErr != nil { scheduleRecovery() @@ -851,6 +854,9 @@ func (r *Registry) runLoop(rd *runningDriver) { cmdCtx, cancel = context.WithTimeout(context.Background(), defaultRecoveryTimeout) } err := rd.driver.DefaultMode(cmdCtx) + if r.tel != nil { + r.tel.EndCommandControl(rd.cfg.Name, err != nil) + } if cancel != nil { cancel() } @@ -977,10 +983,24 @@ func (r *Registry) runLoop(rd *runningDriver) { err = rejection break } + var evidence telemetry.CommandEvidence + if r.tel != nil { + evidence = r.tel.BeginCommand(rd.cfg.Name, cmd.payload, time.Now()) + } err = rd.driver.Command(commandCtx, cmd.payload) if err == nil { err = commandContextError(cmdCtx, commandCtx) } + if r.tel != nil { + result := "accepted" + if err != nil { + result = "failed" + } + if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) { + result = "unconfirmed" + } + r.tel.CompleteCommand(evidence, result) + } if err != nil { err = restoreAfterCommand(err) } else { diff --git a/go/internal/ocpp/control.go b/go/internal/ocpp/control.go index a39fd015..2d59bab1 100644 --- a/go/internal/ocpp/control.go +++ b/go/internal/ocpp/control.go @@ -114,7 +114,20 @@ var ErrNotConnected = errors.New("ocpp: charger not connected") // Command applies an EV control command to a connected charge point. The // signature matches drivers.Registry.Send so it can back a loadpoint // SenderFunc directly. -func (s *Server) Command(ctx context.Context, id string, payload []byte) error { +func (s *Server) Command(ctx context.Context, id string, payload []byte) (resultErr error) { + if s != nil && s.handler != nil && s.handler.tel != nil { + record := s.handler.tel.BeginCommand(id, payload, time.Now()) + defer func() { + result := "accepted" + if resultErr != nil { + result = "failed" + } + if errors.Is(resultErr, context.Canceled) || errors.Is(resultErr, context.DeadlineExceeded) { + result = "unconfirmed" + } + s.handler.tel.CompleteCommand(record, result) + }() + } if s == nil || s.cs == nil { return errors.New("ocpp: server not running") } diff --git a/go/internal/telemetry/command_evidence.go b/go/internal/telemetry/command_evidence.go new file mode 100644 index 00000000..bdd10588 --- /dev/null +++ b/go/internal/telemetry/command_evidence.go @@ -0,0 +1,356 @@ +package telemetry + +import ( + "encoding/json" + "math" + "time" +) + +// CommandEvidence records a driver call, not proof that hardware obeyed it. +// Keep one command per device function so a hybrid's PV cap cannot replace +// its battery command. The registry clears these records on driver restart. +// +// A device keeps one measurement record while FTW sends it the same action. +// Modes that retune the target every tick must not restart the proof on each +// command, so a reading is compared with every command that could still be in +// force during the device's response delay. Only a material step starts a new +// run with its own response wait and site comparison. +type CommandEvidence struct { + Driver string + Kind string + Action string + PowerW *float64 + At time.Time + Since time.Time + Result string + ReadbackMismatchSince time.Time + PowerMismatchSince time.Time + PowerMatchSince time.Time + PowerObservedSince time.Time + LastObservation time.Time + LastMeasuredAt time.Time + // LastGapW is the latest fresh reading's distance from the commands in + // force during the response delay. Nil until such a reading arrives. + LastGapW *float64 + // MaxAge is the power freshness the source declares; zero when it declares none. + MaxAge time.Duration + Recent []CommandPoint + Baseline map[string]ControlBaseline + // StepAfter freezes the completed window after a material step, so a later + // household load cannot rewrite the site comparison for that step. + StepAfter map[string]ControlBaseline + StepAfterAt time.Time +} + +// CommandPoint is one command FTW sent, kept for the response-delay check. +type CommandPoint struct { + PowerW float64 + At time.Time +} + +func commandKind(action string) string { + switch action { + case "battery": + return "battery" + case "ev_set_current", "ev_pause", "ev_resume", "ev_start": + return "ev" + case "curtail", "curtail_disable": + return "pv" + case "v2x_set_power": + return "v2x_charger" + default: + return "" + } +} + +// ResponseDelay bounds how long a device may still show an earlier command. +// Cloud chargers often report a new current a minute or more after the call. +func ResponseDelay(kind string) time.Duration { + if kind == "ev" { + return 2 * time.Minute + } + return 15 * time.Second +} + +// A material step needs its own response wait and site comparison. Smaller +// retuning continues the current run. +func materialStep(old, next *float64) bool { + if old == nil || next == nil { + return old != nil || next != nil + } + return math.Abs(*next-*old) >= math.Max(500, ControlToleranceW(*old)) +} + +func (s *Store) BeginCommand(driver string, payload []byte, now time.Time) CommandEvidence { + var request struct { + Action string `json:"action"` + PowerW *float64 `json:"power_w"` + } + if json.Unmarshal(payload, &request) != nil { + return CommandEvidence{} + } + if request.Action == "ev_pause" { + zero := 0.0 + request.PowerW = &zero + } + kind := commandKind(request.Action) + if kind == "" || (request.PowerW != nil && !finite(*request.PowerW)) { + return CommandEvidence{} + } + c := CommandEvidence{Driver: driver, Kind: kind, Action: request.Action, PowerW: request.PowerW, At: now, Since: now, Result: "pending"} + s.mu.Lock() + defer s.mu.Unlock() + if s.commands == nil { + s.commands = map[string]CommandEvidence{} + } + k := driver + ":" + kind + old, ok := s.commands[k] + if ok && old.Result != "released" { + c.LastMeasuredAt = old.LastMeasuredAt + } + // Retuning the same action continues one measurement record. A material + // step, a failed or unknown call, or a return to device control needs new + // proof against the new target. + if ok && old.Action == c.Action && (old.Result == "accepted" || old.Result == "pending") { + c.Recent, c.MaxAge = append([]CommandPoint(nil), old.Recent...), old.MaxAge + if !materialStep(old.PowerW, c.PowerW) { + c.Since, c.Baseline, c.StepAfter, c.StepAfterAt = old.Since, old.Baseline, old.StepAfter, old.StepAfterAt + c.ReadbackMismatchSince, c.PowerMismatchSince, c.PowerMatchSince = old.ReadbackMismatchSince, old.PowerMismatchSince, old.PowerMatchSince + c.PowerObservedSince, c.LastObservation, c.LastGapW = old.PowerObservedSince, old.LastObservation, old.LastGapW + } + } + if c.PowerW != nil { + c.Recent = append(c.Recent, CommandPoint{PowerW: *c.PowerW, At: now}) + } + if len(c.Recent) > 8 { + c.Recent = c.Recent[len(c.Recent)-8:] + } + if c.Baseline == nil { + c.Baseline = s.controlBaseline(now) + } + s.commands[k] = c + // The completion token never owns the stored windows or history. + c.Baseline, c.StepAfter, c.Recent = nil, nil, nil + if c.PowerW != nil { + v := *c.PowerW + c.PowerW = &v + } + return c +} + +// CompleteCommand never retains raw driver errors: they may contain vendor +// URLs or credentials. Detailed errors remain in the existing private logs. +func (s *Store) CompleteCommand(c CommandEvidence, result string) { + if c.Kind == "" { + return + } + s.mu.Lock() + defer s.mu.Unlock() + k := c.Driver + ":" + c.Kind + if latest, ok := s.commands[k]; ok && latest.At.Equal(c.At) { + latest.Result = result + s.commands[k] = latest + } +} + +func copyWindows(in map[string]ControlBaseline) map[string]ControlBaseline { + if in == nil { + return nil + } + out := make(map[string]ControlBaseline, len(in)) + for k, v := range in { + v.Points = append([]ControlObservation(nil), v.Points...) + out[k] = v + } + return out +} + +func (s *Store) CommandEvidence(driver, kind string) (CommandEvidence, bool) { + s.mu.RLock() + defer s.mu.RUnlock() + c, ok := s.commands[driver+":"+kind] + c.Baseline, c.StepAfter = copyWindows(c.Baseline), copyWindows(c.StepAfter) + c.Recent = append([]CommandPoint(nil), c.Recent...) + for _, p := range []**float64{&c.PowerW, &c.LastGapW} { + if *p != nil { + v := **p + *p = &v + } + } + return c, ok +} + +func (s *Store) ClearCommandEvidence(driver string) { + s.mu.Lock() + defer s.mu.Unlock() + for k, c := range s.commands { + if c.Driver == driver { + delete(s.commands, k) + } + } +} + +// EndCommandControl invalidates comparisons with an earlier command when the +// actor returns the device to its own control. Keep failed attempts visible. +func (s *Store) EndCommandControl(driver string, failed bool) { + s.mu.Lock() + defer s.mu.Unlock() + for k, c := range s.commands { + if c.Driver != driver { + continue + } + if failed { + c.Result = "default_failed" + } else if c.Result == "accepted" || c.Result == "pending" || c.Result == "default_failed" { + c.Result = "released" + } + c.ReadbackMismatchSince, c.PowerMismatchSince, c.PowerMatchSince = time.Time{}, time.Time{}, time.Time{} + c.PowerObservedSince, c.LastObservation, c.LastGapW = time.Time{}, time.Time{}, nil + c.Recent, c.StepAfter, c.StepAfterAt = nil, nil, time.Time{} + if !failed { + c.LastMeasuredAt = time.Time{} + } + s.commands[k] = c + } +} + +// commandGap is a reading's distance from the commands that could still be in +// force at its source time: the last command sent before the response delay +// and every later one. A PV command is a ceiling, so lower output is no gap. +func (c CommandEvidence) commandGap(power float64, at time.Time) float64 { + cutoff := at.Add(-ResponseDelay(c.Kind)) + lo, hi := math.Inf(1), math.Inf(-1) + var inForce *CommandPoint + for i := range c.Recent { + p := c.Recent[i] + if p.At.After(at) { + continue + } + if !p.At.After(cutoff) { + inForce = &c.Recent[i] + continue + } + lo, hi = math.Min(lo, p.PowerW), math.Max(hi, p.PowerW) + } + if inForce != nil { + lo, hi = math.Min(lo, inForce.PowerW), math.Max(hi, inForce.PowerW) + } + if lo > hi { + return math.NaN() + } + if c.Kind == "pv" { + return math.Max(0, math.Abs(power)-math.Max(math.Abs(lo), math.Abs(hi))) + } + if power < lo { + return lo - power + } + if power > hi { + return power - hi + } + return 0 +} + +// Called under mu only when telemetry arrives. UI polling never starts or +// advances a mismatch timer. A gap in readings requires new evidence. +func (s *Store) observeCommand(driver string, kind DerType, power float64, data json.RawMessage, now time.Time) { + k := driver + ":" + kind.String() + c, ok := s.commands[k] + if !ok || c.Result != "accepted" || c.PowerW == nil { + return + } + var d struct { + SetpointW *float64 `json:"setpoint_w"` + } + if json.Unmarshal(data, &d) != nil { + c.ReadbackMismatchSince, c.PowerMismatchSince, c.PowerMatchSince = time.Time{}, time.Time{}, time.Time{} + c.PowerObservedSince, c.LastObservation = time.Time{}, time.Time{} + s.commands[k] = c + return + } + c.MaxAge = controlPowerDeclaredMaxAge(data) + if now.Sub(c.LastObservation) > max(time.Minute, c.MaxAge) { + c.ReadbackMismatchSince, c.PowerMismatchSince, c.PowerMatchSince = time.Time{}, time.Time{}, time.Time{} + c.PowerObservedSince = time.Time{} + } + mark := func(since *time.Time, mismatch bool, at time.Time) { + if !mismatch { + *since = time.Time{} + } else if since.IsZero() { + *since = at + } + } + readbackMismatch := false + if d.SetpointW != nil && finite(*d.SetpointW) { + setpoint := *d.SetpointW + if kind == DerPV { + setpoint = -math.Abs(setpoint) + } + gap := c.commandGap(setpoint, now) + readbackMismatch = math.IsNaN(gap) || gap > math.Max(100, math.Abs(*c.PowerW)*0.05) + if kind == DerPV { + readbackMismatch = math.Abs(math.Abs(*d.SetpointW)-math.Abs(*c.PowerW)) > math.Max(100, math.Abs(*c.PowerW)*0.05) + } + } + mark(&c.ReadbackMismatchSince, readbackMismatch, now) + observation, fresh := ControlPowerObservation(power, data, now) + // A change-only source that confirms an unchanged sample states the power + // at the confirmation time; an unconfirmed repeat adds nothing new. + power, observedAt := observation.PowerW, observation.Seen() + // A cached sample from before the run cannot show its response. Repeated + // delivery of one source sample also cannot extend a measured time window. + if !fresh || !finite(power) || observedAt.Before(c.Since) { + c.PowerMismatchSince, c.PowerMatchSince, c.LastObservation = time.Time{}, time.Time{}, time.Time{} + c.PowerObservedSince, c.LastGapW = time.Time{}, nil + s.commands[k] = c + return + } + if !observedAt.After(c.LastObservation) { + s.commands[k] = c + return + } + c.LastObservation = observedAt + mark(&c.PowerObservedSince, true, observedAt) + gap := c.commandGap(power, observedAt) + c.LastGapW = nil + if finite(gap) { + c.LastGapW = &gap + } + tolerance := ControlToleranceW(*c.PowerW) + mismatch := !finite(gap) || gap > tolerance + mark(&c.PowerMismatchSince, mismatch, observedAt) + follows := !mismatch + if kind == DerPV { + // Weak sun alone cannot prove that a ceiling took effect. + follows = PowerFollowsCommand(c.Kind, *c.PowerW, power) + } + mark(&c.PowerMatchSince, follows, observedAt) + if c.HasMeasuredPower(now) { + c.LastMeasuredAt = observedAt + } + s.commands[k] = c +} + +// HasMeasuredPower confirms a window of distinct, fresh device readings, even +// when they show a shortfall, no response or the wrong direction. Whether the +// device meets the command is a separate verdict; a setpoint echo is not power. +// A slow source counts as fresh for as long as it declares. +func (c CommandEvidence) HasMeasuredPower(now time.Time) bool { + return !c.PowerObservedSince.IsZero() && + c.LastObservation.Sub(c.PowerObservedSince) >= 10*time.Second && + !c.LastObservation.After(now) && now.Sub(c.LastObservation) <= max(10*time.Second, c.MaxAge) +} + +// ControlToleranceW is the response tolerance, not a device rating or safety limit. +func ControlToleranceW(sent float64) float64 { return math.Max(100, math.Abs(sent)*0.1) } + +// PowerFollowsCommand requires a measured target response. PV below its ceiling +// is not a failure, but weak sun alone cannot prove that curtailment took effect. +func PowerFollowsCommand(kind string, sent, actual float64) bool { + if !finite(sent) || !finite(actual) { + return false + } + if kind == "pv" { + return math.Abs(math.Abs(actual)-math.Abs(sent)) <= ControlToleranceW(sent) + } + return math.Abs(actual-sent) <= ControlToleranceW(sent) +} diff --git a/go/internal/telemetry/command_evidence_test.go b/go/internal/telemetry/command_evidence_test.go new file mode 100644 index 00000000..1e2c7410 --- /dev/null +++ b/go/internal/telemetry/command_evidence_test.go @@ -0,0 +1,180 @@ +package telemetry + +import ( + "encoding/json" + "testing" + "time" +) + +func TestCommandEvidenceTracksPhysicalResponse(t *testing.T) { + s := NewStore() + start := time.Now() + c := s.BeginCommand("pixii", []byte(`{"action":"battery","power_w":0}`), start) + s.CompleteCommand(c, "accepted") + got, _ := s.CommandEvidence("pixii", "battery") + if !got.PowerMatchSince.IsZero() { + t.Fatal("driver acceptance claimed measured success") + } + // Writes succeeded, but another writer or internal control changes the setpoint. + sample := func(at time.Time, data string, power float64) { + s.mu.Lock() + defer s.mu.Unlock() + s.observeCommand("pixii", DerBattery, power, json.RawMessage(data), at) + } + sample(start.Add(time.Second), `{"setpoint_w":-1500,"control_power_w":-1400}`, -1300) + sample(start.Add(35*time.Second), `{"setpoint_w":-1600,"control_power_w":-1500}`, -1400) + got, _ = s.CommandEvidence("pixii", "battery") + if got.ReadbackMismatchSince != start.Add(time.Second) || got.PowerMismatchSince != start.Add(time.Second) || !got.PowerMatchSince.IsZero() { + t.Fatalf("mismatch evidence: %+v", got) + } + // Fresh repeated observations, not polling status, establish recovery. + sample(start.Add(40*time.Second), `{"setpoint_w":0,"control_power_w":30}`, 30) + sample(start.Add(52*time.Second), `{"setpoint_w":0,"control_power_w":20}`, 20) + got, _ = s.CommandEvidence("pixii", "battery") + if !got.ReadbackMismatchSince.IsZero() || !got.PowerMismatchSince.IsZero() || got.PowerMatchSince != start.Add(40*time.Second) { + t.Fatalf("recovery: %+v", got) + } + sample(start.Add(3*time.Minute), `{"setpoint_w":0,"control_power_w":0}`, 0) + got, _ = s.CommandEvidence("pixii", "battery") + if got.PowerMatchSince != start.Add(3*time.Minute) { + t.Fatal("gap retained proof of live response") + } +} + +func TestCommandEvidenceDoesNotConfuseACWithDCOrMissingPower(t *testing.T) { + s := NewStore() + now := time.Now() + c := s.BeginCommand("hybrid", []byte(`{"action":"battery","power_w":-1000}`), now) + s.CompleteCommand(c, "accepted") + s.mu.Lock() + s.observeCommand("hybrid", DerBattery, -300, json.RawMessage(`{"control_power_w":-990}`), now.Add(time.Second)) + s.mu.Unlock() + got, _ := s.CommandEvidence("hybrid", "battery") + if got.PowerMatchSince.IsZero() { + t.Fatal("compared AC request with DC measurement") + } + s.mu.Lock() + s.observeCommand("hybrid", DerBattery, -1000, json.RawMessage(`{"control_power_available":false}`), now.Add(2*time.Second)) + s.mu.Unlock() + got, _ = s.CommandEvidence("hybrid", "battery") + if !got.PowerMatchSince.IsZero() || !got.PowerMismatchSince.IsZero() { + t.Fatal("missing measurement is not proof or failure") + } +} + +func TestCommandEvidenceLifecycleAndIsolation(t *testing.T) { + s := NewStore() + now := time.Now() + c := s.BeginCommand("hybrid", []byte(`{"action":"battery","power_w":1000}`), now) + *c.PowerW = 9999 + got, _ := s.CommandEvidence("hybrid", "battery") + if *got.PowerW != 1000 { + t.Fatal("caller mutated evidence") + } + s.CompleteCommand(c, "accepted") + next := s.BeginCommand("hybrid", []byte(`{"action":"battery","power_w":1020}`), now.Add(time.Second)) + s.CompleteCommand(c, "failed") // late result from an old call + got, _ = s.CommandEvidence("hybrid", "battery") + if got.Result != "pending" || got.Since != now { + t.Fatalf("late result or small correction: %+v", got) + } + s.CompleteCommand(next, "accepted") + pv := s.BeginCommand("hybrid", []byte(`{"action":"curtail","power_w":-3000}`), now.Add(2*time.Second)) + s.CompleteCommand(pv, "accepted") + got, _ = s.CommandEvidence("hybrid", "battery") + if *got.PowerW != 1020 { + t.Fatal("PV command replaced battery evidence") + } + s.EndCommandControl("hybrid", false) + got, _ = s.CommandEvidence("hybrid", "battery") + if got.Result != "released" { + t.Fatal("default retained old command") + } + s.Remove("hybrid") + if _, ok := s.CommandEvidence("hybrid", "battery"); ok { + t.Fatal("restart retained evidence") + } +} + +func TestCommandEvidencePVIsCeilingAndStalePowerIsNotProof(t *testing.T) { + s := NewStore() + now := time.Now() + c := s.BeginCommand("pv", []byte(`{"action":"curtail","power_w":-3000}`), now) + s.CompleteCommand(c, "accepted") + s.mu.Lock() + s.observeCommand("pv", DerPV, -500, json.RawMessage(`{}`), now.Add(time.Second)) + s.mu.Unlock() + got, _ := s.CommandEvidence("pv", "pv") + if !got.PowerMismatchSince.IsZero() { + t.Fatal("weak sun is not failed curtailment") + } + if !got.PowerMatchSince.IsZero() { + t.Fatal("weak sun alone proved curtailment") + } + s.mu.Lock() + s.observeCommand("pv", DerPV, -500, json.RawMessage(`{"power_observed_at":"2020-01-01T00:00:00Z"}`), now.Add(2*time.Second)) + s.mu.Unlock() + got, _ = s.CommandEvidence("pv", "pv") + if !got.PowerMatchSince.IsZero() { + t.Fatal("stale source claimed proof") + } +} + +func TestCommandEvidenceUsesSourceTimeForResponse(t *testing.T) { + s := NewStore() + start := time.Now() + c := s.BeginCommand("charger", []byte(`{"action":"ev_set_current","power_w":1000}`), start) + s.CompleteCommand(c, "accepted") + sample := func(sourceAt, receivedAt time.Time) CommandEvidence { + data, err := json.Marshal(map[string]any{"power_observed_at": sourceAt.Format(time.RFC3339Nano)}) + if err != nil { + t.Fatal(err) + } + s.mu.Lock() + s.observeCommand("charger", DerEV, 1000, data, receivedAt) + s.mu.Unlock() + got, _ := s.CommandEvidence("charger", "ev") + return got + } + got := sample(start.Add(-time.Second), start.Add(time.Second)) + if !got.PowerMatchSince.IsZero() { + t.Fatal("a pre-command sample claimed a response") + } + first := start.Add(2 * time.Second) + got = sample(first, start.Add(3*time.Second)) + got = sample(first, start.Add(15*time.Second)) + if !got.PowerMatchSince.Equal(first) || !got.LastObservation.Equal(first) { + t.Fatalf("repeated cached sample advanced proof: %+v", got) + } + last := start.Add(16 * time.Second) + got = sample(last, start.Add(17*time.Second)) + if !got.PowerMatchSince.Equal(first) || !got.LastObservation.Equal(last) { + t.Fatalf("fresh source samples did not extend proof: %+v", got) + } +} + +func TestLastMeasuredResponseSurvivesNewCommandButNotRelease(t *testing.T) { + s := NewStore() + now := time.Now() + c := s.BeginCommand("battery", []byte(`{"action":"battery","power_w":1000}`), now) + s.CompleteCommand(c, "accepted") + s.mu.Lock() + s.observeCommand("battery", DerBattery, 1000, []byte(`{}`), now.Add(time.Second)) + s.observeCommand("battery", DerBattery, 1000, []byte(`{}`), now.Add(12*time.Second)) + s.mu.Unlock() + got, _ := s.CommandEvidence("battery", "battery") + if !got.LastMeasuredAt.Equal(now.Add(12 * time.Second)) { + t.Fatalf("missing measured proof: %+v", got) + } + c = s.BeginCommand("battery", []byte(`{"action":"battery","power_w":2000}`), now.Add(15*time.Second)) + s.CompleteCommand(c, "accepted") + next, _ := s.CommandEvidence("battery", "battery") + if next.LastMeasuredAt != got.LastMeasuredAt || !next.PowerMatchSince.IsZero() { + t.Fatalf("new command confused past and current proof: %+v", next) + } + s.EndCommandControl("battery", false) + next, _ = s.CommandEvidence("battery", "battery") + if !next.LastMeasuredAt.IsZero() { + t.Fatal("intentional release kept loss alarm armed") + } +} diff --git a/go/internal/telemetry/control_observations.go b/go/internal/telemetry/control_observations.go new file mode 100644 index 00000000..536406e3 --- /dev/null +++ b/go/internal/telemetry/control_observations.go @@ -0,0 +1,223 @@ +package telemetry + +import ( + "encoding/json" + "math" + "time" +) + +// Short, bounded raw observations for independent response checks. These are +// never model estimates or the house-load value derived from energy balance. +type ControlObservation struct { + PowerW float64 + At time.Time + // ConfirmedAt is when a source that reports only changes last confirmed + // that this sample is still current. It keeps the sample fresh; it never + // makes a repeated sample count as a new measurement. + ConfirmedAt time.Time +} +type ControlWindow struct { + MeanW, MinW, MaxW float64 + First, Last time.Time + Count int +} +type ControlBaseline struct { + Driver string + Kind DerType + Window ControlWindow + Points []ControlObservation +} + +const ControlWindowDuration = 20 * time.Second + +// Caller holds mu. At most one point per second and 64 points per DER. +func (s *Store) recordControlObservation(driver string, kind DerType, power float64, data json.RawMessage, now time.Time) { + if kind == DerVehicle { + return + } + k := key(driver, kind) + invalidate := func() { delete(s.controlObservations, k) } + p, known := ControlPowerObservation(power, data, now) + if !known || now.Sub(p.At) > 10*time.Second { + invalidate() + return + } + at := p.At + if s.controlObservations == nil { + s.controlObservations = map[string][]ControlObservation{} + } + points := s.controlObservations[k] + if len(points) > 0 && !at.After(points[len(points)-1].At) { + return + } + if len(points) > 0 && at.Unix() == points[len(points)-1].At.Unix() { + points[len(points)-1] = p + } else { + points = append(points, p) + } + if len(points) > 64 { + copy(points, points[len(points)-64:]) + points = points[:64] + } + s.controlObservations[k] = points +} + +func observationWindow(points []ControlObservation, from, now time.Time) ControlWindow { + var w ControlWindow + for _, p := range points { + if p.At.Before(from) || p.At.After(now) { + continue + } + if w.Count == 0 { + w.MinW, w.MaxW, w.First = p.PowerW, p.PowerW, p.At + } + w.MeanW += p.PowerW + w.MinW = math.Min(w.MinW, p.PowerW) + w.MaxW = math.Max(w.MaxW, p.PowerW) + w.Last = p.At + w.Count++ + } + if w.Count > 0 { + w.MeanW /= float64(w.Count) + } + return w +} +func (w ControlWindow) Usable(now time.Time) bool { + return w.Count >= 2 && w.Last.Sub(w.First) >= 5*time.Second && !w.Last.After(now) && now.Sub(w.Last) <= 10*time.Second +} +func (s *Store) controlBaseline(now time.Time) map[string]ControlBaseline { + result := map[string]ControlBaseline{} + for k, rd := range s.readings { + if rd.DerType == DerVehicle { + continue + } + from := now.Add(-ControlWindowDuration) + w := observationWindow(s.controlObservations[k], from, now) + if w.Usable(now) { + result[k] = ControlBaseline{rd.Driver, rd.DerType, w, copyControlPoints(s.controlObservations[k], from, now)} + } + } + return result +} +func (s *Store) ControlWindows(from, now time.Time) map[string]ControlBaseline { + s.mu.RLock() + defer s.mu.RUnlock() + result := map[string]ControlBaseline{} + for k, rd := range s.readings { + if rd.DerType == DerVehicle { + continue + } + result[k] = ControlBaseline{rd.Driver, rd.DerType, observationWindow(s.controlObservations[k], from, now), copyControlPoints(s.controlObservations[k], from, now)} + } + return result +} + +func copyControlPoints(points []ControlObservation, from, until time.Time) []ControlObservation { + var out []ControlObservation + for _, p := range points { + if !p.At.Before(from) && !p.At.After(until) { + out = append(out, p) + } + } + return out +} + +// ControlPowerObservation preserves measurement time even when a cloud/status +// poll is fresh. OCPP status frames and cached vendor observations are not new +// power samples. Legacy local drivers retain their fresh-read receipt contract. +func ControlPowerObservation(power float64, data json.RawMessage, receivedAt time.Time) (ControlObservation, bool) { + var d struct { + ControlPowerW *float64 `json:"control_power_w"` + ControlPowerAvailable *bool `json:"control_power_available"` + ControlPowerConfirmed bool `json:"control_power_confirmed"` + ControlPowerObservedAt string `json:"control_power_observed_at"` + PowerObservedAt string `json:"power_observed_at"` + ForecastPower *ForecastPowerSample `json:"forecast_power"` + } + p := ControlObservation{PowerW: power, At: receivedAt} + if len(data) > 0 && json.Unmarshal(data, &d) != nil { + return p, false + } + if d.ControlPowerAvailable != nil && !*d.ControlPowerAvailable { + return p, false + } + if d.ForecastPower != nil { + f := d.ForecastPower + if f.Version != 1 || !f.Known || f.MeasuredAtMS <= 0 || f.ReceivedAtMS < f.MeasuredAtMS || f.ReceivedAtMS > receivedAt.UnixMilli() { + return p, false + } + p.PowerW, p.At = f.Watts, time.UnixMilli(f.MeasuredAtMS) + } + if d.ControlPowerW != nil { + p.PowerW = *d.ControlPowerW + } + stamp := d.PowerObservedAt + if d.ControlPowerObservedAt != "" { + stamp = d.ControlPowerObservedAt + } + if stamp != "" { + at, err := time.Parse(time.RFC3339Nano, stamp) + if err != nil { + return p, false + } + p.At = at + } + if d.ControlPowerConfirmed { + p.ConfirmedAt = receivedAt + } + age := ControlPowerMaxAge(data) + return p, finite(p.PowerW) && !p.At.After(receivedAt) && receivedAt.Sub(p.Seen()) <= age +} + +// Seen is the latest time the source vouched for this sample. +func (p ControlObservation) Seen() time.Time { + if p.ConfirmedAt.After(p.At) { + return p.ConfirmedAt + } + return p.At +} + +// ControlPowerMaxAge is how long a power observation stays current: the +// source's power_max_age_s when it declares one, at most three minutes, +// otherwise one minute. +func ControlPowerMaxAge(data json.RawMessage) time.Duration { + if age := controlPowerDeclaredMaxAge(data); age > 0 { + return age + } + return time.Minute +} + +func controlPowerDeclaredMaxAge(data json.RawMessage) time.Duration { + var d struct { + PowerMaxAgeS float64 `json:"power_max_age_s"` + } + if len(data) == 0 || json.Unmarshal(data, &d) != nil { + return 0 + } + age := time.Duration(d.PowerMaxAgeS * float64(time.Second)) + if age <= 0 || age > 3*time.Minute { + return 0 + } + return age +} + +// freezeStepWindows keeps each completed post-step window with its command, +// so the site comparison neither depends on when a client asks nor changes +// when a household load starts later. Caller holds mu. +func (s *Store) freezeStepWindows(now time.Time) { + for k, c := range s.commands { + if c.StepAfter != nil || c.Since.IsZero() || now.Sub(c.Since) < ControlWindowDuration { + continue + } + end := c.Since.Add(ControlWindowDuration) + windows := map[string]ControlBaseline{} + for rk, rd := range s.readings { + if rd.DerType == DerVehicle { + continue + } + windows[rk] = ControlBaseline{rd.Driver, rd.DerType, observationWindow(s.controlObservations[rk], c.Since, end), copyControlPoints(s.controlObservations[rk], c.Since, end)} + } + c.StepAfter, c.StepAfterAt = windows, end + s.commands[k] = c + } +} diff --git a/go/internal/telemetry/control_observations_test.go b/go/internal/telemetry/control_observations_test.go new file mode 100644 index 00000000..43f66da2 --- /dev/null +++ b/go/internal/telemetry/control_observations_test.go @@ -0,0 +1,83 @@ +package telemetry + +import ( + "fmt" + "testing" + "time" +) + +func TestControlWindowsAreBoundedAndMissingDataBreaksProof(t *testing.T) { + s := NewStore() + now := time.Now() + for i := 0; i < 1000; i++ { + s.recordControlObservation("battery", DerBattery, 1000, []byte(`{}`), now.Add(time.Duration(i)*time.Second)) + } + points := s.controlObservations["battery:battery"] + if len(points) != 64 { + t.Fatalf("unbounded series: %d", len(points)) + } + at := now.Add(999 * time.Second) + w := observationWindow(points, at.Add(-12*time.Second), at) + if !w.Usable(at) || w.MeanW != 1000 { + t.Fatalf("stable window: %+v", w) + } + s.recordControlObservation("battery", DerBattery, 1000, []byte(`{"control_power_available":false}`), at.Add(time.Second)) + if len(s.controlObservations["battery:battery"]) != 0 { + t.Fatal("missing power retained an apparently complete window") + } + s.recordControlObservation("battery", DerBattery, 1000, []byte(`{"power_observed_at":"2020-01-01T00:00:00Z"}`), at.Add(2*time.Second)) + if len(s.controlObservations["battery:battery"]) != 0 { + t.Fatal("stale source became a new measurement") + } +} + +func TestProofDoesNotDependOnStatusPolling(t *testing.T) { + s := NewStore() + start := time.Now() + // Let the site settle before requesting a change. + for i := 0; i <= 6; i++ { + at := start.Add(time.Duration(i) * time.Second) + s.mu.Lock() + s.readings["battery:battery"] = &DerReading{Driver: "battery", DerType: DerBattery} + s.recordControlObservation("battery", DerBattery, 0, []byte(`{}`), at) + s.mu.Unlock() + } + c := s.BeginCommand("battery", []byte(`{"action":"battery","power_w":1000}`), start.Add(7*time.Second)) + s.CompleteCommand(c, "accepted") + got, _ := s.CommandEvidence("battery", "battery") + if got.Baseline["battery:battery"].Window.Count != 7 { + t.Fatalf("no before-command window: %+v", got) + } + for i := 8; i <= 20; i++ { + s.mu.Lock() + s.observeCommand("battery", DerBattery, 1000, []byte(`{}`), start.Add(time.Duration(i)*time.Second)) + s.mu.Unlock() + } + got, _ = s.CommandEvidence("battery", "battery") + if got.PowerMatchSince.IsZero() || got.LastObservation.Sub(got.PowerMatchSince) < 10*time.Second { + t.Fatal("no proof without an API poll") + } +} + +func TestControlPowerKeepsCloudAndOCPPSourceTimes(t *testing.T) { + now := time.Now().UTC().Truncate(time.Millisecond) + old := now.Add(-20 * time.Second) + cases := []struct { + name, data string + known bool + at time.Time + }{ + {"cloud replay", `{"control_power_observed_at":"` + old.Format(time.RFC3339Nano) + `"}`, true, old}, + {"unavailable", `{"control_power_available":false}`, false, now}, + {"OCPP status replay", fmt.Sprintf(`{"forecast_power":{"version":1,"known":true,"watts":1000,"measured_at_ms":%d,"received_at_ms":%d}}`, old.UnixMilli(), old.UnixMilli()), true, old}, + {"OCPP no meter yet", `{"forecast_power":{"version":1,"known":false}}`, false, now}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + p, known := ControlPowerObservation(1000, []byte(tc.data), now) + if known != tc.known || !p.At.Equal(tc.at) { + t.Fatalf("%+v known=%v", p, known) + } + }) + } +} diff --git a/go/internal/telemetry/store.go b/go/internal/telemetry/store.go index dffc0211..2f24abc0 100644 --- a/go/internal/telemetry/store.go +++ b/go/internal/telemetry/store.go @@ -290,10 +290,12 @@ type MetricSample struct { // Store is the central telemetry sink that drivers emit into and that the // control loop reads from. Thread-safe. type Store struct { - mu sync.RWMutex - readings map[string]*DerReading // key = driver + ":" + der_type - filters map[string]*KalmanFilter1D - health map[string]*DriverHealth + mu sync.RWMutex + readings map[string]*DerReading // key = driver + ":" + der_type + filters map[string]*KalmanFilter1D + health map[string]*DriverHealth + commands map[string]CommandEvidence + controlObservations map[string][]ControlObservation processNoise float64 measurementNoise float64 @@ -455,6 +457,10 @@ func (s *Store) Update(driver string, t DerType, rawW float64, soc *float64, dat UpdatedAt: now, } + s.observeCommand(driver, t, rawW, data, now) + s.recordControlObservation(driver, t, rawW, data, now) + s.freezeStepWindows(now) + // Auto-buffer the standard fields (raw, not smoothed — we store ground // truth and let consumers smooth as they like). tsMs := now.UnixMilli() @@ -740,6 +746,18 @@ func (s *Store) Remove(driver string) { delete(s.filters, k) } delete(s.health, driver) + for _, kind := range allDerTypes { + delete(s.controlObservations, key(driver, kind)) + } + for k, c := range s.commands { + c.Baseline, c.StepAfter = nil, nil + s.commands[k] = c + } + for k, command := range s.commands { + if command.Driver == driver { + delete(s.commands, k) + } + } s.mu.Unlock() prefix := driver + ":" diff --git a/web/app.js b/web/app.js index ee0b4e5e..60108111 100644 --- a/web/app.js +++ b/web/app.js @@ -2,6 +2,7 @@ (function () { "use strict"; + var controlFeedbackExpiry; const evPlanUI = import("/ev-plan.js").catch(function () { return null; }); const POLL_INTERVAL = 2000; // status poll cadence — snappier cards @@ -521,6 +522,7 @@ // replay when either the mapper or the element is ready, so the hero // does not sit in its loading skeleton until the next two-second poll. var lastFlowStatus = null; + var lastFlowLive = true; var lastFlowReadings = null; var flowUpgradeReplayQueued = false; @@ -552,7 +554,9 @@ } } - function paintEnergyFlow(data) { + function paintEnergyFlow(data, live) { + if (data !== lastFlowStatus) lastFlowLive = true; + if (typeof live === "boolean") lastFlowLive = live; lastFlowStatus = data; var flowEl = document.getElementById("energy-flow"); if (!flowEl) return; @@ -560,7 +564,8 @@ queueFlowReplay(); return; } - lastFlowReadings = window.ftwFlowReadingsFromStatus(data, energyFlowOpts(data)); + var opts = energyFlowOpts(data); opts.live = lastFlowLive; + lastFlowReadings = window.ftwFlowReadingsFromStatus(data, opts); (lastFlowReadings.planets || []).forEach(function (p) { if (p.role !== "ev" || p.placeholder || !p.name) return; var lpEv = loadpointsByDriver && loadpointsByDriver[p.name]; @@ -578,6 +583,7 @@ p.socStale = !!lpEv.vehicle_stale; }); if (typeof flowEl.setReadings === "function") { + if (typeof flowEl.toggleAttribute === "function") flowEl.toggleAttribute("static", !lastFlowLive); flowEl.setReadings(lastFlowReadings); } else { queueFlowReplay(); @@ -1012,6 +1018,11 @@ // Drivers renderDrivers(data.drivers || {}, dispatchByDriver); + if (window.FTWControlFeedback) { + updateControlFeedback(data, true); + clearTimeout(controlFeedbackExpiry); + controlFeedbackExpiry = setTimeout(function () { updateControlFeedback(data, false); paintEnergyFlow(data, false); }, 15000); + } // Dispatch renderDispatch(data.dispatch || []); @@ -2686,6 +2697,55 @@ var evModalBody = document.getElementById("ev-modal-body"); var evModalDriver = null; // captured from the planet click; sent on commands var energyFlowEl = document.getElementById("energy-flow"); + var controlProofModal = document.getElementById("control-proof-modal"); + var controlProofScope = null; + var controlProofLive = false; + var batteryProofDriver = ""; + var pvProofDriver = ""; + var openPlanetControls = function () {}; + // Each device sheet answers "Are we in control?" first. Core decides the + // status; the shared view only renders it. + function updateControlFeedback(data, live) { + controlProofLive = live; + var feedback = window.FTWControlFeedback; + if (!feedback) return; + feedback.render(document.getElementById("control-results"), data.control_feedback, live, {compact:true}); + feedback.render(document.getElementById("battery-control-proof"), feedback.forPlanet(data.control_feedback, {role:"battery", name:batteryProofDriver}), live); + feedback.render(document.getElementById("ev-control-proof"), feedback.forPlanet(data.control_feedback, {role:"ev", name:evModalDriver || ""}), live); + feedback.render(document.getElementById("pv-control-proof"), feedback.forPlanet(data.control_feedback, {role:"pv", name:pvProofDriver}), live); + if (controlProofModal && controlProofModal.hasAttribute("open")) { + feedback.render(document.getElementById("control-proof-details"), feedback.forPlanet(data.control_feedback, controlProofScope || {}), live); + } + } + function openControlProof(scope) { + var feedback = window.FTWControlFeedback; + if (!controlProofModal || !feedback || !lastStatusPayload) return false; + var rows = feedback.forPlanet(lastStatusPayload.control_feedback, scope); + if (!rows.length) return false; + controlProofScope = scope; + feedback.render(document.getElementById("control-proof-details"), rows, controlProofLive); + controlProofModal.open(); + return true; + } + var batteryControls = document.getElementById("battery-control"); + if (batteryControls) batteryControls.addEventListener("ftw-battery-scope", function (event) { + batteryProofDriver = event.detail.driver || ""; + if (lastStatusPayload) updateControlFeedback(lastStatusPayload, controlProofLive); + }); + var pvControls = document.getElementById("pv-control"); + if (pvControls) pvControls.addEventListener("ftw-pv-scope", function (event) { + pvProofDriver = event.detail.driver || ""; + if (lastStatusPayload) updateControlFeedback(lastStatusPayload, controlProofLive); + }); + var proofSummary = document.getElementById("control-results"); + if (proofSummary) proofSummary.addEventListener("click", function (event) { + var button = event.target.closest("button[data-driver]"); + if (button) { + var scope = {name:button.dataset.driver, role:button.dataset.kind === "v2x_charger" ? "ev" : button.dataset.kind}; + if (!openPlanetControls(scope)) openControlProof(scope); + } + }); + // Render the EV modal by building DOM nodes (textContent) rather than // concatenating strings into innerHTML — d.driver comes from driver @@ -4601,6 +4661,7 @@ if (evModal) { function openEvModal(driver) { evModalDriver = driver || null; + if (lastStatusPayload) updateControlFeedback(lastStatusPayload, controlProofLive); evModal.open(); refreshEvModal(); // Guard against stacked timers if the modal opens again while a @@ -4621,26 +4682,31 @@ if (energyFlowEl) { energyFlowEl.addEventListener("ftw-planet-click", function (e) { var d = (e && e.detail) || {}; - if (d.role === "ev") openEvModal(d.name || null); + if (d.id && d.id.indexOf("agg-") === 0) d = {role:d.role, id:d.id, name:""}; + if (!openPlanetControls(d)) openControlProof(d); + }); + } + openPlanetControls = function (d) { + if (d.id && d.id.indexOf("agg-") === 0) d = {role:d.role}; + if (d.role === "ev") { openEvModal(d.name || null); return true; } if (d.role === "battery") { var drv = (lastStatusPayload && lastStatusPayload.drivers) || {}; var clicked = drv[d.name || ""] || {}; if (clicked.observe_only) return; var bc = document.getElementById("battery-control"); - if (bc && typeof bc.open === "function") bc.open(d.name || d.id || ""); + if (bc && typeof bc.open === "function") { bc.open(d.name || d.id || ""); return true; } } if (d.role === "pv") { var pc = document.getElementById("pv-control"); if (pc && typeof pc.open === "function") { - // d.id is the driver id when the user clicked an expanded - // per-driver bubble; "" / undefined opens at the aggregate - // scope from the merged bubble. - pc.open(d.id || ""); + // The node id includes a role prefix; controls take the driver name. + pc.open(d.name || ""); + return true; } } - if (d.role === "grid" && gridModal) gridModal.open(); - }); - } + if (d.role === "grid" && gridModal) { gridModal.open(); return true; } + return false; + }; // Tile-mode (numeric cards) parity: when the operator toggles the // hero off, the energy-flow planets aren't on screen, so the diff --git a/web/components/energy-flow-readings.js b/web/components/energy-flow-readings.js index 2e176cae..7b6e705e 100644 --- a/web/components/energy-flow-readings.js +++ b/web/components/energy-flow-readings.js @@ -11,6 +11,8 @@ // component's window.FTW_FLOW_IDLE_W wins once that module has loaded; // 42 is the no-modules / unit-test fallback. Do not import the // component from here — that would pull Custom Elements into Node tests. +import {withControlMarks} from "../control-feedback.js"; + function idleW() { return (typeof window !== "undefined" && window.FTW_FLOW_IDLE_W) || 42; } @@ -220,7 +222,7 @@ export function flowReadingsFromStatus(status, opts) { return { load: loadW === null ? null : loadW / 1000, - planets, + planets: withControlMarks(planets, status?.control_feedback, opts?.live !== false), selfPoweredPctToday, }; } diff --git a/web/components/ftw-battery-control.js b/web/components/ftw-battery-control.js index 445468ac..866ea760 100644 --- a/web/components/ftw-battery-control.js +++ b/web/components/ftw-battery-control.js @@ -155,45 +155,57 @@ class FtwBatteryControl extends FtwElement { } .error.hidden { display: none; } - /* Footer buttons — primary CTA is the accent, on-accent text is - near-black per the shared design system. The Stop button is a ghost variant - until a hold is active, then it switches to the red status hue - to signal a destructive action. */ - .footer-btn { - flex: 1; + + /* The sheet answers "Are we in control?" first; manual override is a + deliberate second step, so its form stays folded until opened. */ + .override { + border-top: 1px solid var(--line); + margin-top: 14px; + padding-top: 12px; + } + .override > summary { + cursor: pointer; + font-family: var(--sans); + font-size: 14px; + font-weight: 500; + color: var(--fg-dim); + padding: 4px 0 8px; + } + .override > summary:hover { color: var(--fg); } + .override[open] > summary { margin-bottom: 8px; } + .install-btn { + width: 100%; padding: 11px 18px; border-radius: 8px; cursor: pointer; font-family: var(--sans); font-weight: 500; font-size: 14px; - transition: transform 80ms, border-color 120ms, color 120ms, background 120ms; - } - .footer-btn:disabled { - opacity: 0.45; - cursor: not-allowed; - } - .footer-btn[data-variant="install"] { - flex: 2; background: var(--accent-e); - color: #0a0a0a; + color: var(--on-accent, #0a0a0a); border: 1px solid var(--accent-e); + margin-top: 4px; } - .footer-btn[data-variant="install"]:hover:not(:disabled) { - transform: translateY(-1px); + .install-btn:disabled { opacity: 0.45; cursor: not-allowed; } + .active-banner { + display: flex; + align-items: center; + gap: 12px; } - .footer-btn[data-stop] { + .active-text { flex: 1; } + .stop-btn { + padding: 8px 14px; + border-radius: 8px; + cursor: pointer; + font-family: var(--sans); + font-weight: 500; + font-size: 14px; background: transparent; - color: var(--fg); - border: 1px solid var(--line); - } - .footer-btn[data-stop]:not(:disabled) { color: var(--red-e); - border-color: var(--red-e); - } - .footer-btn[data-stop]:hover:not(:disabled) { - background: color-mix(in srgb, var(--red-e) 12%, transparent); + border: 1px solid var(--red-e); } + .stop-btn:hover:not(:disabled) { background: color-mix(in srgb, var(--red-e) 12%, transparent); } + .stop-btn:disabled { opacity: 0.45; cursor: not-allowed; } `; constructor() { @@ -223,6 +235,8 @@ class FtwBatteryControl extends FtwElement { this._selectDirection(this._formState.direction); this._selectDuration(this._formState.holdS); this._renderScope(); + const title = this.shadowRoot.querySelector("[data-title]"); + if (title) title.textContent = driver ? "Battery · " + driver : "Batteries"; modal.open(); this._refresh(); if (this._refreshTimer) clearInterval(this._refreshTimer); @@ -232,13 +246,21 @@ class FtwBatteryControl extends FtwElement { render() { return ` - Battery control + Battery + + +
+ Manual override +
`; } @@ -332,6 +351,8 @@ class FtwBatteryControl extends FtwElement { button.classList.toggle("active", on); button.setAttribute("aria-checked", on ? "true" : "false"); }); + this.dispatchEvent(new CustomEvent('ftw-battery-scope', {bubbles:true, composed:true, + detail:{driver:mode === 'driver' ? this._formState.scopeDriver : ''}})); } _selectDirection(dir) { diff --git a/web/components/ftw-energy-flow.js b/web/components/ftw-energy-flow.js index 000ac263..e529839c 100644 --- a/web/components/ftw-energy-flow.js +++ b/web/components/ftw-energy-flow.js @@ -664,7 +664,8 @@ class FtwEnergyFlow extends FtwElement { socStale: !p.placeholder && !!p.socStale, socSource: p.placeholder ? null : p.socSource, radius: p._r, - clickable: p.clickable === false ? false : (!p.placeholder && !!p.role), + clickable: p.clickable === false ? false : (!!p.controlMark || !p.placeholder && !!p.role), + controlMark: p.controlMark, role: p.role || "", name: p.name || "", id: p.id, @@ -706,6 +707,14 @@ class FtwEnergyFlow extends FtwElement { super.update(); } + _syncLayerAccess() { + for (const layer of this.shadowRoot.querySelectorAll('.ef-layer')) { + const visible = layer.classList.contains(this._aggregated ? 'ef-layer-agg' : 'ef-layer-ind'); + layer.setAttribute('aria-hidden', visible ? 'false' : 'true'); + for (const node of layer.querySelectorAll('.ef-clickable')) node.setAttribute('tabindex', visible ? '0' : '-1'); + } + } + // Called by FtwElement after each render() replaces the shadow DOM. // We cancel any in-flight rAF, bind the freshly-rendered // elements to the particle-param list `render()` just built, and @@ -726,6 +735,7 @@ class FtwEnergyFlow extends FtwElement { this._aggregated = !this._aggregated; const svgEl = this.shadowRoot.querySelector("svg"); if (svgEl) svgEl.dataset.agg = this._aggregated ? "on" : "off"; + this._syncLayerAccess(); toggleBtn.setAttribute("aria-checked", this._aggregated ? "true" : "false"); toggleBtn.setAttribute("title", this._aggregated ? "Split multi-device corners into individual bubbles" @@ -738,6 +748,7 @@ class FtwEnergyFlow extends FtwElement { // `ftw-planet-click` so callers (app.js) can route per-role // (e.g. ev → open EV modal scoped to this driver). const svg = this.shadowRoot.querySelector('svg'); + this._syncLayerAccess(); if (svg) { const fire = (g) => { const role = g.getAttribute('data-role') || ''; @@ -1152,7 +1163,7 @@ class FtwEnergyFlow extends FtwElement { ` : ""} - PV control + Solar + + +
+ Manual limit + - -
- - -
+ +
`; } @@ -271,7 +303,7 @@ class FtwPvControl extends FtwElement { root.querySelectorAll(".chip").forEach((c) => c.addEventListener("click", () => this._selectDuration(Number(c.dataset.hold)))); const sel = root.querySelector("[data-driver]"); - if (sel) sel.addEventListener("change", (e) => { this._formState.driver = e.target.value; }); + if (sel) sel.addEventListener("change", (e) => { this._formState.driver = e.target.value; this._announceScope(); }); root.querySelector("[data-install]").addEventListener("click", () => this._install()); root.querySelector("[data-stop]").addEventListener("click", () => this._stop()); modal.addEventListener("ftw-modal-close", () => { @@ -325,6 +357,11 @@ class FtwPvControl extends FtwElement { .catch(() => { this._capableDrivers = []; }); } + _announceScope() { + this.dispatchEvent(new CustomEvent('ftw-pv-scope', {bubbles:true, composed:true, + detail:{driver:this._formState.driver || ''}})); + } + _renderDriverOptions() { const root = this.shadowRoot; const sel = root.querySelector("[data-driver]"); diff --git a/web/control-feedback-view.js b/web/control-feedback-view.js new file mode 100644 index 00000000..439f3f32 --- /dev/null +++ b/web/control-feedback-view.js @@ -0,0 +1,151 @@ +// Box renderer for Core's control status. The words live in +// control-feedback.js, which the app shares. This file owns the DOM and +// updates each device's block in place, so open disclosures and keyboard +// focus survive the two-second status refresh. +import {controlRows, controlStatus, controlReceipt, controlNumbers, controlCurve, controlForPlanet, controlSummary} from './control-feedback.js'; + +const ICONS = {ok: '✓', neutral: '•', warning: '▲', alarm: '!', stale: '…'}; +const KINDS = {battery: 'battery', ev: 'charger', v2x_charger: 'charger', pv: 'solar', device: 'device'}; + +function el(tag, parent, cls, text) { + const node = document.createElement(tag); + if (cls) node.className = cls; + if (text) node.textContent = text; + if (parent) parent.appendChild(node); + return node; +} + +function setText(node, text) { + node.textContent = text || ''; + node.hidden = !text; +} + +function keyed(root, selector, key, build) { + const found = Array.from(root.querySelectorAll(selector)).find(node => node.dataset.key === key); + return found || build(); +} + +function curveSvg(curve) { + const svg = document.createElementNS('http://www.w3.org/2000/svg', 'svg'); + svg.setAttribute('viewBox', '0 0 280 110'); + svg.setAttribute('role', 'img'); + svg.setAttribute('aria-label', curve.label); + svg.classList.add('control-curve'); + for (const [points, cls] of [['10,55 270,55', 'zero'], [curve.device, 'device'], [curve.site, 'site']]) { + const line = document.createElementNS(svg.namespaceURI, 'polyline'); + line.setAttribute('points', points); + line.setAttribute('class', cls); + line.setAttribute('fill', 'none'); + svg.appendChild(line); + } + return svg; +} + +function buildBlock(root, key) { + const section = el('section', root, 'cs-block'); + section.dataset.key = key; + const title = el('h3', section, 'cs-title'); + el('span', title, 'cs-icon').setAttribute('aria-hidden', 'true'); + el('span', title, 'cs-title-text'); + el('p', section, 'cs-text'); + el('p', section, 'cs-proof'); + el('p', section, 'cs-next'); + const receipt = el('details', section, 'cs-receipt'); + el('summary', receipt, '', 'How FTW knows'); + el('ol', receipt, 'cs-steps'); + const numbers = el('details', receipt, 'cs-numbers'); + el('summary', numbers, '', 'Numbers'); + el('dl', numbers); + el('div', numbers, 'cs-curve'); + return section; +} + +function fillBlock(section, row, live) { + const status = controlStatus(row, live); + section.dataset.tone = status.tone; + section.querySelector('.cs-icon').textContent = ICONS[status.tone] || ICONS.neutral; + section.querySelector('.cs-title-text').textContent = status.title; + setText(section.querySelector('.cs-text'), status.text); + setText(section.querySelector('.cs-proof'), status.proof); + setText(section.querySelector('.cs-next'), status.next); + const steps = section.querySelector('.cs-steps'); + steps.replaceChildren(...controlReceipt(row, live).map(step => { + const item = el('li'); + item.dataset.state = step.state; + el('span', item, 'cs-step', step.step); + el('span', item, 'cs-value', step.value); + return item; + })); + const numbers = controlNumbers(row, live), curve = controlCurve(row, live); + const more = section.querySelector('.cs-numbers'); + more.hidden = !numbers.length && !curve; + more.querySelector('dl').replaceChildren(...numbers.flatMap(([label, value]) => [el('dt', null, '', label), el('dd', null, '', value)])); + more.querySelector('.cs-curve').replaceChildren(...(curve ? [curveSvg(curve)] : [])); +} + +function fillList(root, rows, live) { + const summary = controlSummary(rows, live); + const head = keyed(root, '.cs-summary', 'summary', () => { + const node = el('p', root, 'cs-summary'); + node.dataset.key = 'summary'; + return node; + }); + head.textContent = summary.title; + head.dataset.tone = summary.tone; + const list = root.querySelector('.cs-list') || el('ul', root, 'cs-list'); + const keep = new Set(); + rows.forEach((row, index) => { + const key = `${row.driver}:${row.kind}`; + keep.add(key); + const item = keyed(list, 'li', key, () => { + const node = el('li', list); + node.dataset.key = key; + const button = el('button', node, 'cs-row'); + button.type = 'button'; + el('span', button, 'cs-device'); + el('span', button, 'cs-row-status'); + return node; + }); + if (list.children[index] !== item) list.insertBefore(item, list.children[index] || null); + const status = controlStatus(row, live); + const button = item.querySelector('button'); + button.dataset.driver = row.driver; + button.dataset.kind = row.kind; + button.dataset.tone = status.tone; + button.setAttribute('aria-label', `${row.driver} ${KINDS[row.kind] || row.kind}: ${status.title}. Open details`); + item.querySelector('.cs-device').textContent = `${row.driver} · ${KINDS[row.kind] || row.kind}`; + item.querySelector('.cs-row-status').textContent = status.title; + }); + for (const item of Array.from(list.children)) if (!keep.has(item.dataset.key)) item.remove(); +} + +// Sheet mode shows one answer block per device function; compact mode lists +// the devices with a one-line answer each. +export function renderControlStatus(root, value, live = true, {compact = false} = {}) { + if (!root) return; + const rows = controlRows(value); + root.hidden = rows.length === 0; + if (root.dataset.mode !== (compact ? 'compact' : 'sheet')) { + root.replaceChildren(); + root.dataset.mode = compact ? 'compact' : 'sheet'; + } + if (!rows.length) return; + if (!root.querySelector(':scope > .cs-question')) el('p', root, 'cs-question', 'Are we in control?'); + if (compact) { + fillList(root, rows, live); + return; + } + const keep = new Set(); + rows.forEach(row => { + const key = `${row.driver}:${row.kind}`; + keep.add(key); + fillBlock(keyed(root, ':scope > .cs-block', key, () => buildBlock(root, key)), row, live); + }); + for (const section of Array.from(root.querySelectorAll(':scope > .cs-block'))) { + if (!keep.has(section.dataset.key)) section.remove(); + } +} + +if (typeof window !== 'undefined') { + window.FTWControlFeedback = {render: renderControlStatus, forPlanet: controlForPlanet, status: controlStatus}; +} diff --git a/web/control-feedback.js b/web/control-feedback.js new file mode 100644 index 00000000..95fda5fa --- /dev/null +++ b/web/control-feedback.js @@ -0,0 +1,304 @@ +// Words for Core's answer to "Are we in control?". Core owns every verdict: +// status, severity, evidence and reason. This module only picks words, marks +// and receipt rows for them. The app copies this file byte for byte, so it +// stays free of DOM and window access. + +const num = value => typeof value === 'number' && Number.isFinite(value); + +export function controlRows(value) { + if (!Array.isArray(value)) return []; + return value.filter(row => row && typeof row === 'object' && typeof row.driver === 'string' && + typeof row.status === 'string' && typeof row.reason === 'string'); +} + +export function controlPower(watts) { + if (!num(watts)) return 'unknown'; + const size = Math.abs(watts); + if (size < 1) return '0 W'; + return size >= 1000 ? `${(size / 1000).toFixed(1)} kW` : `${Math.round(size)} W`; +} + +const NOUNS = {battery: 'battery', ev: 'charger', v2x_charger: 'charger', pv: 'solar inverter', device: 'device'}; +const noun = row => NOUNS[row.kind] || 'device'; +const percent = soc => num(soc) ? `${Math.round(soc * 100)}%` : 'its current level'; +const amps = value => num(value) ? `${Math.round(value)} A` : 'the current'; +const clock = ms => num(ms) && ms > 0 ? new Date(ms).toLocaleTimeString([], {hour: '2-digit', minute: '2-digit'}) : ''; +const capital = text => text.charAt(0).toUpperCase() + text.slice(1); + +// Site convention: positive power is drawn into the site, so a battery or car +// charges and solar is negative. +function doing(row, watts) { + if (row.kind === 'pv') return 'producing'; + if (!num(watts) || Math.abs(watts) < 100) return 'idle'; + return watts > 0 ? 'charging' : 'discharging'; +} + +export function controlFlow(row, watts) { + if (!num(watts)) return 'unknown'; + if (row.kind === 'pv') return `${controlPower(watts)} solar`; + if (Math.abs(watts) < 100) return row.kind === 'ev' ? 'no charging' : 'idle'; + return `${controlPower(watts)} ${watts > 0 ? 'charge' : 'discharge'}`; +} + +function target(row) { + if (!num(row.sent_w)) return 'the new target'; + return row.kind === 'pv' ? `a ${controlPower(row.sent_w)} cap` : controlFlow(row, row.sent_w); +} + +function source(row) { + if (row.mode === 'manual') return 'as you asked'; + if (row.mode === 'solar') return 'from spare solar'; + if (row.mode === 'plan' || String(row.mode || '').startsWith('planner')) return 'as planned'; + return 'as FTW asked'; +} + +const STATUS_TITLES = { + following: 'Following FTW', waiting: 'Waiting', limited: 'Limited', not_following: 'Not following', + no_contact: 'No contact', not_controlled: 'Not controlled by FTW', +}; + +// A reason title replaces the status word when the word alone says too little. +const REASON_TITLES = { + waiting_response: 'Adjusting', no_command: 'No command yet', not_connected: 'No car connected', + no_plan_budget: 'Waiting for the plan', pv_surplus_pause: 'Waiting for spare solar', wake_kick: 'Starting the charger', + vehicle_complete: 'Car is full', vehicle_limit_completion: 'Car reached its limit', vehicle_not_requesting: 'Car not taking power', + battery_full: 'Battery full', battery_nearly_full: 'Battery nearly full', battery_nearly_empty: 'Battery nearly empty', + core_limit: 'Limited by FTW', fuse_limit: 'Limited by the main fuse', fuse_cooldown: 'Waiting after a fuse limit', + charger_limit: 'Limited by FTW’s charger setting', device_limit: 'Charger limit', offered_current_lower: 'Charger offers less current', + device_fault: 'Device fault', telemetry_stale: 'No fresh readings', readings_lost: 'Lost control', + command_failed: 'Commands failing', command_unconfirmed: 'Command not confirmed', default_failed: 'Cannot hand back control', + response_unknown: 'No power reading', observe_only: 'Monitoring only', disabled: 'Disabled', device_control: 'Own control', + site_meter_stale: 'Paused for safety', site_phase_currents_stale: 'Paused for safety', +}; + +function sentence(row) { + const name = noun(row), actual = row.actual_w; + switch (row.reason) { + case 'power_observed': case 'plan': case 'manual_hold': case 'pv_surplus': case 'idle': { + const now = doing(row, actual); + return now === 'idle' ? `Idle ${source(row)}.` : `${capital(now)} ${controlPower(actual)} ${source(row)}.`; + } + case 'solar_below_ceiling': return `Producing ${controlPower(actual)}, below ${target(row)}.`; + case 'vehicle_complete': return 'Not charging: the car reports that it is full.'; + case 'vehicle_limit_completion': return 'Not charging: the car reached its own charge limit.'; + case 'vehicle_not_requesting': return 'The car is not asking for power right now.'; + case 'waiting_response': return `Moving to ${target(row)}. The ${name} usually responds within ${row.kind === 'ev' ? 'two minutes' : '15 seconds'}.`; + case 'no_command': return 'FTW has not sent a command since it started.'; + case 'not_connected': return 'Plug in a car to charge.'; + case 'no_plan_budget': return 'The plan has no charging in this period.'; + case 'pv_surplus_pause': return 'Charging starts when there is enough spare solar.'; + case 'wake_kick': return 'FTW is waking the charger.'; + case 'battery_full': return `FTW paused charging at ${percent(row.battery_soc)}. Charging resumes at ${percent(row.charge_resume_soc)} or lower; discharging works as normal.`; + case 'battery_nearly_full': return `Taking ${controlPower(actual)} of ${controlPower(row.sent_w)} at ${percent(row.battery_soc)}. A battery charges slower when it is nearly full.`; + case 'battery_nearly_empty': return `Giving ${controlPower(actual)} of ${controlPower(row.sent_w)} at ${percent(row.battery_soc)}. A battery limits discharge when it is nearly empty.`; + case 'core_limit': return `FTW’s safety limits set the target to ${target(row)}.`; + case 'fuse_limit': return `FTW lowered the rate to ${target(row)} to protect the main fuse. It rises when the house uses less.`; + case 'fuse_cooldown': return 'FTW waits a moment after a fuse limit before it raises the rate.'; + case 'charger_limit': return 'The charger limit in FTW’s settings caps the rate.'; + case 'device_limit': return `The charger’s own limit is ${amps(row.device_limit_a)}, below the ${amps(row.requested_a)} FTW asked for.`; + case 'offered_current_lower': return `The charger offers ${amps(row.offered_a)}, below the ${amps(row.requested_a)} FTW asked for.`; + case 'power_below_target': case 'power_above_target': return `Asked for ${target(row)}, delivering ${controlPower(actual)}.`; + case 'power_wrong_direction': return `Asked for ${target(row)}, but it is ${doing(row, actual)} ${controlPower(actual)}.`; + case 'no_power_response': return `Asked for ${target(row)}, but it delivers no power.`; + case 'power_while_idle': return `Asked to stay idle, but it is ${doing(row, actual)} ${controlPower(actual)}.`; + case 'setpoint_changed': return `The ${name} reports a target of ${controlFlow(row, row.readback_w)}, not the ${target(row)} FTW sent.`; + case 'device_fault': return row.device_reason ? `The ${name} reports a fault: ${row.device_reason}.` : `The ${name} reports a fault that stops control.`; + case 'telemetry_stale': return `FTW has no fresh readings from the ${name}.`; + case 'readings_lost': return clock(row.observed_at_ms) + ? `No usable readings from the ${name} since ${clock(row.observed_at_ms)}. FTW cannot see what its commands do.` + : `No usable readings from the ${name}. FTW cannot see what its commands do.`; + case 'command_failed': return `The last command to the ${name} failed. FTW keeps trying.`; + case 'command_unconfirmed': return `The last command timed out. It may still have reached the ${name}.`; + case 'default_failed': return `FTW could not return the ${name} to its own mode, so it sends no other commands until it can.`; + case 'response_unknown': return `FTW has no power reading from the ${name} to compare with its command.`; + case 'observe_only': return `FTW reads this ${name} but does not control it.`; + case 'disabled': return 'This device is disabled in FTW’s settings.'; + case 'device_control': return `The ${name} runs its own mode. FTW is not sending commands.`; + case 'site_meter_stale': return 'FTW stopped control because the grid meter readings are out of date.'; + case 'site_phase_currents_stale': return 'FTW stopped control because it lacks fresh phase currents for the main fuse check.'; + default: return `The box reports “${row.reason}”, which this version cannot describe.`; + } +} + +function nextStep(row) { + const name = noun(row); + switch (row.reason) { + case 'power_below_target': case 'power_above_target': case 'power_wrong_direction': + case 'no_power_response': case 'power_while_idle': + return `The ${name} has not said why. Check its app or display.`; + case 'setpoint_changed': return 'Another app or controller may be changing it.'; + case 'device_fault': return `Check the ${name}.`; + case 'telemetry_stale': case 'readings_lost': case 'command_failed': case 'default_failed': + return `Check the ${name}’s connection.`; + case 'site_meter_stale': case 'site_phase_currents_stale': return 'Check the grid meter’s connection.'; + case 'device_limit': return 'Raise the limit in the charger’s app if you want faster charging.'; + case 'offered_current_lower': return 'Check the charger’s load balancing and limits.'; + case 'observe_only': return 'Turn on control in the device settings if you want FTW to steer it.'; + default: return ''; + } +} + +function proofLine(row) { + if (row.evidence === 'confirmed' && clock(row.confirmed_at_ms)) return `The grid meter confirmed this at ${clock(row.confirmed_at_ms)}.`; + if (row.evidence === 'measured' && row.status === 'following') return `Based on the ${noun(row)}’s own readings.`; + return ''; +} + +// One answer per device function: a title, one sentence and, when the owner +// can act, the next step. Tone follows Core's severity and status. +export function controlStatus(row, live = true) { + if (!live) return {title: 'Status not current', text: 'FTW has not reported for a while.', proof: '', next: 'Waiting for a fresh report.', tone: 'stale'}; + const title = REASON_TITLES[row.reason] || STATUS_TITLES[row.status] || 'Checking'; + const tone = row.severity === 'alarm' ? 'alarm' : row.severity === 'warning' ? 'warning' : row.status === 'following' ? 'ok' : 'neutral'; + return {title, text: sentence(row), proof: proofLine(row), next: nextStep(row), tone}; +} + +const CONFIRMATION = { + no_site_meter: ['No grid meter to compare with', 'none'], + independent_source_unknown: ['The grid meter is not a separate sensor', 'none'], + not_controlling: ['Not needed while FTW does not control it', 'none'], + device_response_unconfirmed: ['Waits for the device’s measured response', 'wait'], + waiting_for_meter: ['Waiting for fresh grid meter readings', 'wait'], + no_baseline: ['Waiting for a clear change to compare', 'wait'], + readings_not_aligned: ['Waiting for readings close enough in time', 'wait'], + flows_changing: ['Other loads changed at the same time', 'wait'], + site_change_differs: ['The grid change did not match; another load may have changed', 'wait'], + no_clear_change: ['The change was too small to compare', 'wait'], +}; + +function modeName(mode) { + if (mode === 'manual') return 'manual'; + if (mode === 'solar') return 'solar only'; + if (mode === 'plan' || String(mode || '').startsWith('planner')) return 'plan'; + return String(mode || 'FTW').replace(/_/g, ' '); +} + +// The receipt shows how FTW knows: each step with its value and state +// (done, wait, fail or none). +export function controlReceipt(row, live = true) { + const steps = []; + const at = ms => clock(ms) ? ` · ${clock(ms)}` : ''; + const sent = row.sent_w, asked = row.requested_w; + if (num(asked) && num(sent) && Math.abs(asked - sent) > Math.max(50, Math.abs(sent) * 0.02)) { + steps.push({step: 'Asked', value: `${controlFlow(row, asked)} · ${modeName(row.mode)}`, state: 'done'}); + } + steps.push({step: 'Sent', value: num(sent) ? `${controlFlow(row, sent)}${at(row.command_at_ms)}` : 'No command', state: num(sent) ? 'done' : 'none'}); + const failed = ['command_failed', 'command_unconfirmed', 'default_failed'].includes(row.reason); + const accepted = ['accepted', 'measured', 'confirmed'].includes(row.evidence); + steps.push({step: 'Accepted', + value: failed ? REASON_TITLES[row.reason] : accepted + ? (live && num(row.readback_w) ? `The ${noun(row)} reports target ${controlFlow(row, row.readback_w)}` : 'The driver took the command') + : num(sent) ? 'Waiting for the driver' : '—', + state: failed ? 'fail' : accepted ? 'done' : num(sent) ? 'wait' : 'none'}); + const measured = ['measured', 'confirmed'].includes(row.evidence); + steps.push({step: 'Measured', + value: !live ? 'Not current' : measured ? `${controlFlow(row, row.actual_w)}${at(row.observed_at_ms)}` + : row.readings_fresh === false ? 'No fresh readings' : num(sent) ? 'Waiting for fresh readings' : '—', + state: !live ? 'none' : measured ? 'done' : row.readings_fresh === false && num(sent) ? 'fail' : num(sent) ? 'wait' : 'none'}); + const change = row.site_evidence && row.site_evidence.device_change_w; + const [why, state] = CONFIRMATION[row.site_confirmation] || ['Waiting for a clear change to compare', 'wait']; + steps.push({step: 'Confirmed', + value: !live ? 'Not current' : row.evidence === 'confirmed' + ? `The grid meter matched ${num(change) ? `a ${controlPower(change)} change` : 'the change'}${at(row.confirmed_at_ms)}` + : why, + state: !live ? 'none' : row.evidence === 'confirmed' ? 'done' : state}); + return steps; +} + +// Changes in words, never a bare sign: positive power is drawn into the site. +function deviceChange(row, watts) { + if (row.kind === 'pv') return `${controlPower(watts)} ${watts < 0 ? 'more' : 'less'} solar`; + if (row.kind === 'ev') return `${controlPower(watts)} ${watts < 0 ? 'less' : 'more'} charging`; + return `${controlPower(watts)} toward ${watts < 0 ? 'discharge' : 'charge'}`; +} +const gridChange = watts => `${controlPower(watts)} toward ${watts < 0 ? 'export' : 'import'}`; +const drawnChange = watts => `${controlPower(watts)} ${watts < 0 ? 'less' : 'more'} drawn`; + +// Numbers for experts and support. +export function controlNumbers(row, live = true) { + if (!live) return []; + const values = []; + if (num(row.battery_soc)) values.push(['Battery charge', percent(row.battery_soc)]); + if (num(row.tolerance_w)) values.push(['Response tolerance', `±${controlPower(row.tolerance_w)}`]); + if (num(row.requested_a)) values.push(['Requested current', amps(row.requested_a)]); + if (num(row.offered_a)) values.push(['Charger offer', amps(row.offered_a)]); + if (num(row.device_limit_a)) values.push(['Charger limit', amps(row.device_limit_a)]); + const e = row.site_evidence; + if (e && typeof e === 'object') { + for (const [label, key] of [['Grid before', 'grid_before_w'], ['Grid after', 'grid_after_w']]) { + if (num(e[key])) values.push([label, `${controlPower(e[key])} ${e[key] < 0 ? 'export' : 'import'}`]); + } + for (const [label, key, words] of [['Device change', 'device_change_w', w => deviceChange(row, w)], ['Grid change', 'grid_change_w', gridChange], + ['Other measured change', 'other_change_w', drawnChange], ['Unexplained change', 'unexplained_change_w', drawnChange]]) { + if (num(e[key])) values.push([label, words(e[key])]); + } + if (num(e.samples) && e.samples > 0) values.push(['Compared', `${e.samples} readings over ${Math.round(e.window_s)} s`]); + if (num(e.tolerance_w) && e.tolerance_w > 0) values.push(['Match tolerance', `±${controlPower(e.tolerance_w)}`]); + if (Array.isArray(e.unmeasured_flows) && e.unmeasured_flows.length) { + const kinds = {battery: 'battery', ev: 'car charger', v2x_charger: 'car charger', pv: 'solar', meter: 'meter'}; + values.push(['Left in the background', e.unmeasured_flows.filter(v => typeof v === 'string') + .map(v => { const [driver, kind] = v.split(':'); return kinds[kind] ? `${driver} (${kinds[kind]})` : driver; }).join(', ')]); + } + if (num(e.max_skew_ms) && e.samples > 0) values.push(['Largest time gap', `${Math.round(e.max_skew_ms)} ms`]); + } + return values; +} + +export function controlCurve(row, live = true) { + const trace = row.site_evidence?.trace; + if (!live || !Array.isArray(trace) || trace.length < 3 || trace.length > 64) return null; + if (!trace.every(p => p && num(p.at_ms) && num(p.device_change_w) && num(p.adjusted_site_change_w))) return null; + const start = trace[0].at_ms, span = trace[trace.length - 1].at_ms - start; + if (span <= 0 || trace.some((p, i) => i > 0 && p.at_ms <= trace[i - 1].at_ms)) return null; + const scale = Math.max(500, ...trace.flatMap(p => [Math.abs(p.device_change_w), Math.abs(p.adjusted_site_change_w)])); + const line = key => trace.map(p => `${(10 + (p.at_ms - start) / span * 260).toFixed(1)},${(55 - p[key] / scale * 40).toFixed(1)}`).join(' '); + return {device: line('device_change_w'), site: line('adjusted_site_change_w'), + label: `Measured changes over ${Math.round(span / 1000)} seconds. Device: solid line. Grid after other measured flows: dashed line.`, + scale: controlPower(scale), duration: `${Math.round(span / 1000)} s`}; +} + +// Rows for one bubble or device sheet. A combined bubble includes every device. +export function controlForPlanet(value, planet = {}) { + const name = planet.id?.startsWith('agg-') ? '' : planet.name; + return controlRows(value).filter(row => (!name || row.driver === name) && + (!planet.role || row.kind === planet.role || row.kind === 'device' || planet.role === 'ev' && row.kind === 'v2x_charger')); +} + +export function controlSummary(value, live = true) { + const rows = controlRows(value); + if (!live) return {title: 'Status not current', tone: 'stale'}; + const count = (n, one, many) => `${n} ${n === 1 ? one : many}`; + const alarms = rows.filter(row => row.severity === 'alarm').length; + const warnings = rows.filter(row => row.severity === 'warning').length; + if (alarms) return {title: `${count(alarms, 'device needs', 'devices need')} attention now`, tone: 'alarm'}; + if (warnings) return {title: `${count(warnings, 'device needs', 'devices need')} a look`, tone: 'warning'}; + const controlled = rows.filter(row => row.status !== 'not_controlled'); + if (!controlled.length) return {title: 'FTW controls no device right now', tone: 'neutral'}; + return {title: 'Yes, FTW is in control', tone: 'ok'}; +} + +// Overview marks stay off while all is well. Only Core's warning and alarm +// severities draw one; a device that stopped reporting keeps its bubble. +export function withControlMarks(planets, value, live = true) { + const result = planets.map(p => ({...p})); + const corners = {battery: 'top-right', ev: 'bottom-right', pv: 'top-left', v2x_charger: 'bottom-right'}; + const titles = {battery: 'BATTERY', pv: 'SOLAR', ev: 'EV CHARGER'}; + for (const row of controlRows(value)) { + const role = row.kind === 'v2x_charger' ? 'ev' : row.kind; + let planet = result.find(p => p.name === row.driver && (row.kind === 'device' || p.role === role)); + if (!planet) { + if (!corners[row.kind]) continue; + planet = {id: `control-${row.kind}-${row.driver}`, name: row.driver, role, corner: corners[row.kind], + title: titles[role], kw: 0, toHub: false, color: 'var(--fg-muted)', sub: 'no data', placeholder: true}; + result.push(planet); + } + if (live && row.readings_fresh === false && row.status !== 'not_controlled') { + Object.assign(planet, {placeholder: true, kw: 0, sub: 'no data', color: 'var(--fg-muted)'}); + } + if (row.status !== 'not_controlled') planet.clickable = true; + if (!live || !['warning', 'alarm'].includes(row.severity)) continue; + const mark = {tone: row.severity, label: controlStatus(row).title}; + if (!planet.controlMark || mark.tone === 'alarm' && planet.controlMark.tone !== 'alarm') planet.controlMark = mark; + } + return result; +} diff --git a/web/control-feedback.test.mjs b/web/control-feedback.test.mjs new file mode 100644 index 00000000..00654f00 --- /dev/null +++ b/web/control-feedback.test.mjs @@ -0,0 +1,142 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import {readFileSync} from 'node:fs'; +import {controlRows, controlStatus, controlReceipt, controlNumbers, controlCurve, controlForPlanet, + controlSummary, withControlMarks} from './control-feedback.js'; + +const row = extra => ({driver: 'bat', kind: 'battery', mode: 'planner_arbitrage', status: 'following', reason: 'power_observed', + severity: 'info', evidence: 'measured', readings_fresh: true, sent_w: -2250, requested_w: -2250, actual_w: -2249, + readback_w: -2250, command_at_ms: 1, observed_at_ms: 2, site_confirmation: 'device_response_unconfirmed', ...extra}); + +// Every reason Core can classify must have words, or the owner sees a code. +test('every reason Core emits has a title and a sentence', () => { + const go = readFileSync(new URL('../go/internal/api/api_control_feedback.go', import.meta.url), 'utf8'); + const body = go.slice(go.indexOf('func setControlStatus'), go.indexOf('f.Status, f.Severity = status, severity')); + const reasons = [...body.matchAll(/"([a-z_]+)"/g)].map(m => m[1]) + .filter(r => !['waiting', 'info', 'following', 'limited', 'warning', 'not_following', 'alarm', 'no_contact', 'not_controlled'].includes(r)); + assert.ok(reasons.length > 30, `found only ${reasons.length} reasons`); + for (const reason of reasons) { + const words = controlStatus(row({reason, status: 'waiting'})); + assert.doesNotMatch(words.text, /cannot describe/, reason); + assert.ok(words.title && words.title !== 'Checking', reason); + } +}); + +test('the default view never shows tier numbers or internal codes', () => { + for (const [status, reason, severity] of [['following', 'power_observed', 'info'], ['waiting', 'waiting_response', 'info'], + ['limited', 'battery_nearly_full', 'info'], ['not_following', 'power_below_target', 'warning'], + ['no_contact', 'readings_lost', 'alarm'], ['not_controlled', 'observe_only', 'info']]) { + const words = controlStatus(row({status, reason, severity, evidence: 'confirmed', confirmed_at_ms: 3, battery_soc: 0.97})); + const shown = [words.title, words.text, words.proof, words.next].join(' '); + assert.doesNotMatch(shown, /tier|_|undefined|NaN|null/i, `${reason}: ${shown}`); + } +}); + +test('the answer follows Core: following, not following and lost control', () => { + const following = controlStatus(row()); + assert.equal(following.title, 'Following FTW'); + assert.equal(following.text, 'Discharging 2.2 kW as planned.'); + assert.equal(following.tone, 'ok'); + const ignored = controlStatus(row({status: 'not_following', reason: 'no_power_response', severity: 'warning', sent_w: 3000, actual_w: 0})); + assert.equal(ignored.title, 'Not following'); + assert.equal(ignored.text, 'Asked for 3.0 kW charge, but it delivers no power.'); + assert.match(ignored.next, /has not said why/); + assert.equal(ignored.tone, 'warning'); + const lost = controlStatus(row({status: 'no_contact', reason: 'readings_lost', severity: 'alarm', evidence: 'accepted'})); + assert.equal(lost.title, 'Lost control'); + assert.equal(lost.tone, 'alarm'); + assert.equal(controlStatus(row(), false).tone, 'stale'); +}); + +test('expected limits stay calm and explain themselves', () => { + const taper = controlStatus(row({status: 'limited', reason: 'battery_nearly_full', sent_w: 3000, actual_w: 900, battery_soc: 0.98})); + assert.equal(taper.title, 'Battery nearly full'); + assert.equal(taper.text, 'Taking 900 W of 3.0 kW at 98%. A battery charges slower when it is nearly full.'); + assert.equal(taper.tone, 'neutral'); + const full = controlStatus(row({status: 'limited', reason: 'battery_full', sent_w: 0, actual_w: 0, battery_soc: 1, charge_resume_soc: 0.99})); + assert.match(full.text, /paused charging at 100%\. Charging resumes at 99% or lower/); + const car = controlStatus(row({kind: 'ev', reason: 'vehicle_complete', mode: 'plan'})); + assert.equal(car.title, 'Car is full'); + const limit = controlStatus(row({kind: 'ev', status: 'limited', reason: 'device_limit', severity: 'warning', device_limit_a: 8, requested_a: 16})); + assert.equal(limit.text, 'The charger’s own limit is 8 A, below the 16 A FTW asked for.'); +}); + +test('the receipt walks from sent to confirmed without inventing proof', () => { + const confirmed = controlReceipt(row({evidence: 'confirmed', confirmed_at_ms: 3, site_confirmation: 'confirmed', site_evidence: {device_change_w: -836}})); + assert.deepEqual(confirmed.map(s => [s.step, s.state]), [['Sent', 'done'], ['Accepted', 'done'], ['Measured', 'done'], ['Confirmed', 'done']]); + assert.match(confirmed[3].value, /grid meter matched a 836 W change/); + const limited = controlReceipt(row({requested_w: -5000, mode: 'manual'})); + assert.deepEqual(limited[0], {step: 'Asked', value: '5.0 kW discharge · manual', state: 'done'}); + const accepted = controlReceipt(row({evidence: 'accepted', status: 'waiting', reason: 'waiting_response'})); + assert.deepEqual(accepted.map(s => s.state), ['done', 'done', 'wait', 'wait']); + const stale = controlReceipt(row({evidence: 'accepted', readings_fresh: false, actual_w: null})); + assert.equal(stale[2].state, 'fail'); + const failed = controlReceipt(row({evidence: 'none', reason: 'command_failed', status: 'no_contact'})); + assert.equal(failed[1].state, 'fail'); + const noMeter = controlReceipt(row({site_confirmation: 'independent_source_unknown'})); + assert.deepEqual([noMeter[3].value, noMeter[3].state], ['The grid meter is not a separate sensor', 'none']); + assert.ok(controlReceipt(row(), false).slice(2).every(s => s.value === 'Not current')); +}); + +test('numbers use words instead of signs and never invent zeros', () => { + const values = Object.fromEntries(controlNumbers(row({site_evidence: {grid_before_w: -5663, grid_after_w: -6471, + device_change_w: -836, grid_change_w: -808, other_change_w: -28, unexplained_change_w: 56, samples: 4, window_s: 15, + unmeasured_flows: ['easee:ev'], max_skew_ms: 0}}))); + assert.equal(values['Grid before'], '5.7 kW export'); + assert.equal(values['Device change'], '836 W toward discharge'); + assert.equal(values['Grid change'], '808 W toward export'); + assert.equal(values['Unexplained change'], '56 W more drawn'); + assert.doesNotMatch(Object.values(values).join(' '), /[−-]\d|\+\d/); + assert.equal(values['Left in the background'], 'easee (car charger)'); + const missing = Object.fromEntries(controlNumbers(row({site_evidence: {samples: 0}}))); + assert.equal(missing['Grid before'], undefined); + assert.equal(missing['Largest time gap'], undefined); + assert.deepEqual(controlNumbers(row(), false), []); +}); + +test('curve evidence expires with status and rejects invalid samples', () => { + const trace = [0, 5000, 10000].map(at_ms => ({at_ms, device_change_w: -800, adjusted_site_change_w: -780})); + assert.ok(controlCurve(row({site_evidence: {trace}}))); + assert.equal(controlCurve(row({site_evidence: {trace}}), false), null); + assert.equal(controlCurve(row({site_evidence: {trace: trace.slice(0, 2)}})), null); + assert.equal(controlCurve(row({site_evidence: {trace: [trace[0], trace[0], trace[1]]}})), null); +}); + +test('marks appear only when Core asks for attention', () => { + const planets = [{name: 'bat', role: 'battery'}, {name: 'car', role: 'ev'}]; + const calm = withControlMarks(planets, [row(), row({driver: 'car', kind: 'ev', status: 'waiting', reason: 'not_connected'})]); + assert.ok(calm.every(p => !p.controlMark)); + const marked = withControlMarks(planets, [row({status: 'not_following', reason: 'setpoint_changed', severity: 'warning'}), + row({driver: 'car', kind: 'device', status: 'not_following', reason: 'device_fault', severity: 'alarm'})]); + assert.deepEqual(marked.find(p => p.name === 'bat').controlMark, {tone: 'warning', label: 'Not following'}); + assert.deepEqual(marked.find(p => p.name === 'car').controlMark, {tone: 'alarm', label: 'Device fault'}); + assert.ok(withControlMarks(planets, [row({severity: 'alarm', status: 'no_contact', reason: 'readings_lost'})], false).every(p => !p.controlMark)); +}); + +test('a silent device keeps its bubble without old watts', () => { + const planets = withControlMarks([{name: 'a', role: 'battery', kw: 1}], [row({driver: 'a'}), + row({driver: 'b', status: 'no_contact', reason: 'readings_lost', severity: 'alarm', readings_fresh: false, actual_w: null})]); + const silent = planets.find(p => p.name === 'b'); + assert.equal(silent.placeholder, true); + assert.equal(silent.kw, 0); + assert.equal(silent.controlMark.tone, 'alarm'); + assert.equal(planets.find(p => p.name === 'a').kw, 1); + const monitored = withControlMarks([{name: 'm', role: 'battery', kw: 2}], [row({driver: 'm', status: 'not_controlled', reason: 'observe_only', readings_fresh: false})]); + assert.equal(monitored[0].kw, 2); + assert.notEqual(monitored[0].clickable, true); +}); + +test('a sheet gets its device rows and combined bubbles keep every device', () => { + const rows = [row(), row({driver: 'two'}), row({driver: 'car', kind: 'ev'}), row({driver: 'two', kind: 'device', reason: 'device_fault'})]; + assert.deepEqual(controlForPlanet(rows, {role: 'battery', name: 'two'}).map(r => r.kind), ['battery', 'device']); + assert.equal(controlForPlanet(rows, {role: 'battery', id: 'agg-battery', name: '2×'}).length, 3); + assert.equal(controlRows([{driver: 'x'}, null, row()]).length, 1); +}); + +test('the summary answers the question for the whole site', () => { + assert.deepEqual(controlSummary([row()]), {title: 'Yes, FTW is in control', tone: 'ok'}); + assert.equal(controlSummary([row(), row({severity: 'warning'})]).title, '1 device needs a look'); + assert.equal(controlSummary([row({severity: 'alarm'}), row({severity: 'alarm'})]).title, '2 devices need attention now'); + assert.equal(controlSummary([row({status: 'not_controlled', reason: 'observe_only'})]).tone, 'neutral'); + assert.equal(controlSummary([row()], false).tone, 'stale'); +}); diff --git a/web/energy-flow-readings.test.mjs b/web/energy-flow-readings.test.mjs index ba9769cc..c91ca55c 100644 --- a/web/energy-flow-readings.test.mjs +++ b/web/energy-flow-readings.test.mjs @@ -144,3 +144,18 @@ describe("fmtKwhShort", () => { assert.equal(fmtKwhShort(100.6), "101"); }); }); + +it('marks only devices that need attention and keeps an offline sibling visible without old watts', () => { + const status = {grid_w:1000,load_w:500,drivers:{a:{status:'ok',bat_w:1000},b:{status:'offline',bat_w:5000}},control_feedback:[ + {driver:'a',kind:'battery',status:'following',reason:'power_observed',severity:'info',evidence:'confirmed',readings_fresh:true}, + {driver:'b',kind:'battery',status:'no_contact',reason:'readings_lost',severity:'alarm',evidence:'accepted',readings_fresh:false,actual_w:null}, + ]}; + const planets=flowReadingsFromStatus(status).planets; + assert.equal(planets.find(p=>p.name==='a').controlMark,undefined); + const failed=planets.find(p=>p.name==='b'); + assert.deepEqual(failed.controlMark,{tone:'alarm',label:'Lost control'}); + assert.equal(failed.placeholder,true); + assert.equal(failed.kw,0); + assert.equal(failed.clickable,true); + assert.equal(flowReadingsFromStatus(status,{live:false}).planets.find(p=>p.name==='b').controlMark,undefined); +}); diff --git a/web/index.html b/web/index.html index 8165326d..a997f790 100644 --- a/web/index.html +++ b/web/index.html @@ -182,6 +182,7 @@

Power now