Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,8 @@ jobs:
run: fprime-util generate && fprime-util build -j4
- name: Deployment end to end (OPS-SAT run 3 package, counts against the demo)
run: ./scripts/fprime-deployment.sh
- name: Deployment end to end, version 2 (replicas as processes, a real kill)
run: VERSION=2 ./scripts/fprime-deployment.sh
- name: Unit tests and parity with the demo (MRU-SYS-001)
working-directory: fprime
run: fprime-util generate --ut && fprime-util check --all -j4
Expand Down
9 changes: 9 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,15 @@ jobs:
cp opssat/fprime/dist/* "pkg/$name/"
cp opssat/fprime/run opssat/fprime/shrink.seq opssat/fprime/tmr.seq opssat/fprime/restart.seq LICENSE "pkg/$name/"
tar -C pkg -czf "dist/$name.tar.gz" "$name"
- name: Build version 2 (replicas as processes) and compile its sequences
run: ./opssat/fprime2/build.sh
- name: Package version 2
run: |
name=mru-fprime-processes-${{ github.ref_name }}-armv7-linux-static
mkdir -p "pkg/$name"
cp opssat/fprime2/dist/* "pkg/$name/"
cp opssat/fprime2/run opssat/fprime2/shrink.seq opssat/fprime2/tmr.seq opssat/fprime2/restart.seq LICENSE "pkg/$name/"
tar -C pkg -czf "dist/$name.tar.gz" "$name"
- uses: actions/upload-artifact@v4
with:
name: fprime-armv7-linux-static
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,6 @@ opssat/fprime/dist-native/
opssat/fprime/toGround/
# F´ install folder written by fprime-util
fprime/build-artifacts/
opssat/fprime2/dist/
opssat/fprime2/dist-native/
opssat/fprime2/toGround/
13 changes: 13 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,19 @@
All notable changes to Mru Flight. Versions follow [Semantic Versioning](https://semver.org).
Release packages are on the [releases page](https://github.com/mruspace/flight/releases).

## 0.7.0 (2026-10-22)

### Added

- Run 3, version 2: each replica in a process of its own. `MruQuorum`, the manager process (RedundancyManager, HealthRegistry, FaultInjector, RunRecorder and F´'s CdhCore services), starts three `MruReplica` processes, one ReplicaHost each, linked through F´'s GenericHub. Requirements MRU-DEP-005 to 007.
- `SeqPacketDriver`: a byte-stream driver over one end of a `SOCK_SEQPACKET` Unix socket pair, which keeps message boundaries, so the hub needs no framing and the link no network. After it opens, it turns off reopening and ends its reconnect thread, so a send to a dead replica fails at once.
- Each replica process pins itself to a core (`-C`, default 0,1,1) and reports it. The kill is real: `RunRecorder` calls the deployment at the start of each cycle (MRU-RR-006, one unit test, 45 in all) and the manager sends SIGKILL to replica 0's process at cycle 240; the summary reports how each replica process ended and the experiment's CPU share.
- `opssat/fprime2/`: the sequences, `run`, `build.sh` and `package.sh` for version 2 (`exp4051-mru-flight-fprime-processes-VERSION.tar.gz`); the release adds `mru-fprime-processes-VERSION-armv7-linux-static.tar.gz`. `VERSION=2 scripts/fprime-deployment.sh` runs it in CI. Full length in ESA's image: 148 s, the same counts and events as version 1, replica 0 killed by SIGKILL at cycle 240, three processes pinned in every segment, 19.9 MB of address space, the experiment's CPU 2.9% to 3.75% under emulation.

### Changed

- The payload (the demo's hash over a 4 KiB block) is the module `Mru/Payload`, shared by ReplicaHost and RunRecorder; ReplicaHost keeps `compute` and `expectedKnownAnswer`.

## 0.6.0 (2026-10-22)

### Fixed
Expand Down
2 changes: 1 addition & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ authors:
license: Apache-2.0
repository-code: "https://github.com/mruspace/flight"
url: "https://mru.space"
version: 0.6.0
version: 0.7.0
date-released: 2026-10-22
keywords:
- fault tolerance
Expand Down
8 changes: 4 additions & 4 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ exclude = ["fprime"]
resolver = "2"

[workspace.package]
version = "0.6.0"
version = "0.7.0"
edition = "2021"
license = "Apache-2.0"
repository = "https://github.com/mruspace/flight"
Expand Down
11 changes: 7 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ F´ components have not run on flight hardware.
|---|---|
| [`quorum`](quorum/src/lib.rs) | The decision core: vote, compare, self-check, and the health record that decides when a replica must be diagnosed. `no_std`, no allocation, no dependencies. Unit-tested, checked with [Kani](https://github.com/model-checking/kani), and built in CI for a bare-metal ARM Cortex-M target. |
| [`quorum-ffi`](quorum-ffi/src/lib.rs) | The C interface to the core ([`quorum.h`](quorum-ffi/include/quorum.h)), as a `no_std` static library. Kani proves that its decide, mode, strike and clear functions give the core's answer. Used by the F´ components, and usable from any C or C++ framework. |
| [`fprime`](fprime) | F´ v4.4.1 components: `RedundancyManager`, `HealthRegistry`, `ReplicaHost`, and the test-only `FaultInjector` and `RunRecorder`, with spares, standby and state that survives resets. 44 unit tests, and a system test against the demo on the bench schedules: wrong results match in all 60 cases, and correct results match exactly in 36 ([requirements](docs/fprime-requirements.md), [design and results](docs/fprime-design-fpp.md)). `MruOpsSat`, the first F´ deployment: the components with F´'s command, event, telemetry, health and sequencing services, driven by a Linux timer, built as a static 32-bit ARM binary for OPS-SAT PRETTY's processor ([run 3 package](docs/opssat.md)). |
| [`fprime`](fprime) | F´ v4.4.1 components: `RedundancyManager`, `HealthRegistry`, `ReplicaHost`, and the test-only `FaultInjector` and `RunRecorder`, with spares, standby and state that survives resets. 45 unit tests, and a system test against the demo on the bench schedules: wrong results match in all 60 cases, and correct results match exactly in 36 ([requirements](docs/fprime-requirements.md), [design and results](docs/fprime-design-fpp.md)). `MruOpsSat`, the first F´ deployment: the components with F´'s command, event, telemetry, health and sequencing services, driven by a Linux timer, built as a static 32-bit ARM binary for OPS-SAT PRETTY's processor ([run 3 package](docs/opssat.md)). |
| [`bench`](bench/src/main.rs) | Times the decision core on the host: nanoseconds per call of `decide()` for each policy and mode, and of `Health::strike()`. No dependencies. |
| [`demo`](demo/src/main.rs) | The `quorum` program: three replica processes, a voter built on the core, fault injection, enforced resource limits, and a report of what was delivered and what it cost. Runs on Linux and macOS; release builds are fully static binaries for 64-bit and 32-bit ARM Linux and x86_64 Linux. Its only dependency is `libc`, for the system calls behind the limits. |

Expand Down Expand Up @@ -393,9 +393,12 @@ Changes are in [CHANGELOG.md](CHANGELOG.md).
1. **F´ components**: `RedundancyManager`, `HealthRegistry`, `ReplicaHost`, and
the test-only `FaultInjector` and `RunRecorder` are built
([design](docs/fprime-design.md)). The first deployment, `MruOpsSat`, runs
them with F´'s own services. It is built for OPS-SAT PRETTY's processor and
ran full length in ESA's experiment environment; a run on the SEPP is
planned, not yet proposed. Next: `PowerAccountant` and `ScrubScheduler`.
them with F´'s own services. A second, `MruQuorum` with three `MruReplica`
processes, puts each replica in a process of its own on the SEPP's two
cores, linked through F´'s GenericHub, with a real kill. Both are built for
OPS-SAT PRETTY's processor and ran full length in ESA's experiment
environment; a run on the SEPP is planned, not yet proposed. Next:
`PowerAccountant` and `ScrubScheduler`.
2. **Measurements on representative processors**: static ARM Linux binaries are
built and tested in CI under emulation. The 32-bit ARM binary ran on the
SEPP, the flight processor of OPS-SAT PRETTY, on the FlatSat and in orbit
Expand Down
42 changes: 41 additions & 1 deletion docs/fprime-design-fpp.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ topology that forgets `loadParameters()` does not test every cycle.
|---|---|
| Kani, `quorum` and `quorum-ffi` | 9 of 9 proofs verified |
| C test of `quorum.h` | passes |
| F´ unit tests | 44 (HealthRegistry 5, ReplicaHost 5, FaultInjector 8, RedundancyManager 23, RunRecorder 3), with address and undefined-behavior sanitizers |
| F´ unit tests | 45 (HealthRegistry 5, ReplicaHost 5, FaultInjector 8, RedundancyManager 23, RunRecorder 4), with address and undefined-behavior sanitizers |
| MRU-SYS-001, 60 cases (5 schedules, 2 policies, 3 upset rates, 2 seeds, 4,000 cycles) | 36 identical to the demo. In the other 24, a kill that leaves one replica costs the shrinking quorum 2 correct results and adds 3 caught disagreements, as predicted. Wrong results are identical in all 60. |

The expected counts come from the demo itself (`scripts/parity-cases.sh`),
Expand Down Expand Up @@ -218,6 +218,46 @@ RedundancyManager ─resultOut─▶ RunRecorder ─TlmGet─▶ telemetry datab
has (`--allow-multiple-definition`), and names a personality routine that
is never called with `panic=abort` (`MruOpsSat/RustSupport.cpp`).

### Version 2: `MruQuorum` and `MruReplica`

Added on 22 October 2026, option 2 of the [design note](fprime-design.md):
each replica in a process of its own, for the most representative setup the
OPS-SAT rules allow.

```
MruQuorum (manager process) MruReplica (x3)
manager -> injector -> hub_i -> adapter_i -> link_i ~~ link -> adapter -> hub -> host
manager <- injector <- hub_i <- adapter_i <- link_i ~~ link <- adapter <- hub <- host
```

- **The link**: `SeqPacketDriver`, a byte-stream driver with the shape of F´'s
`TcpClient`, over one end of a `SOCK_SEQPACKET` socket pair. Such a socket
keeps message boundaries, so each hub message arrives whole and the hub
needs no framing. The manager makes the pairs and starts the replica
processes before F´ starts (`posix_spawn`), handing each its end; there is
no address, no listening and no network. After the link opens, the driver
turns off F´'s automatic reopen and ends the reconnect thread: a send to a
replica that is gone fails at once instead of waiting a second on the
manager's thread (found in the first test, where the manager's queue
overflowed after the kill).
- **Cores**: each replica process pins itself (`sched_setaffinity`, allowed
for an unprivileged user) before its threads start; replica 0 on core 0 and
replicas 1 and 2 on core 1 by default, so the pair that compares after the
stuck replica is retired runs on different cores.
- **Real faults**: `RunRecorder` calls the deployment at the start of each
cycle, before the manager (MRU-RR-006); the manager process sends SIGKILL
(or SIGSTOP) to the replica process on the scheduled cycle. The stuck value
and the bit flip stay in `FaultInjector`. The summary reports how each
replica process ended (`0=signal 9 1=exit 0 2=exit 0`) and the CPU share of
all four processes.
- **The payload** moved to `Mru/Payload`, shared by `ReplicaHost` and
`RunRecorder`, so the manager process links the task without the replica
component.
- **A gap in F´'s autocoder** (v4.4.1, direct port calls): the hub's
serial-to-typed calls are written in namespace `Svc` and name the port
serializer without its module; the two topology headers make the Mru
serializers visible there.

## Change to MRU-SYS-001

The first MRU-SYS-001 asked for the same counts as the demo on the bench
Expand Down
4 changes: 3 additions & 1 deletion docs/fprime-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,9 @@ involved.
between replicas.
2. **Three processes.** One deployment per replica, linked through F´'s generic
hub over local sockets. Matches the demo, and an OPS-SAT experiment can run
this way on one Linux processor.
this way on one Linux processor. Built on 22 October 2026: `MruQuorum` and
`MruReplica`, over Unix socket pairs, the replicas pinned to the SEPP's two
cores ([design](fprime-design-fpp.md#version-2-mruquorum-and-mrureplica)).
3. **Separate processors.** The same deployments on separate boards of the
hardware-in-the-loop bench, then on flight hardware with independent
processors.
Expand Down
4 changes: 4 additions & 0 deletions docs/fprime-requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ Added on 22 October 2026 for the OPS-SAT deployment (run 3). It stands in for th
| MRU-RR-003 | The component shall give the run's counts with the manager's last policy, mode, live replicas, caught count and slot states, read from the telemetry database. | Unit test |
| MRU-RR-004 | The component shall record the end of the command sequence and whether every command in it succeeded. | Unit test |
| MRU-RR-005 | The component shall not be part of a flight build. | Inspection of the build |
| MRU-RR-006 | The component shall call a function the deployment gives it with the number of each cycle, at the start of the cycle, before the manager's work for it. | Unit test |

## Deployment `MruOpsSat`

Expand All @@ -117,6 +118,9 @@ Added on 22 October 2026: the first F´ deployment of the components, for OPS-SA
| MRU-DEP-002 | The policy and the faults shall be commanded by an F´ command sequence that ends before cycle 1. | Integration test |
| MRU-DEP-003 | The deployment shall stop at a time cap or on SIGTERM and write a summary of the run. | Integration test |
| MRU-DEP-004 | A static 32-bit ARM Linux build shall run in ESA's OPS-SAT experiment environment as user `exp`, within 128 MB of address space and the registered 5% CPU share. | Full-length run in ESA's environment image |
| MRU-DEP-005 | Version 2 (`MruQuorum` and `MruReplica`, added 22 October 2026) shall run each replica as a process of its own, linked to the manager process through F´'s GenericHub over a Unix socket pair that keeps message boundaries (no network, no framing). | Integration test (`VERSION=2 scripts/fprime-deployment.sh`) |
| MRU-DEP-006 | Each replica process shall pin itself to a configured core and report the core it runs on. | Integration test; full-length run in ESA's environment image |
| MRU-DEP-007 | Version 2 shall apply a kill or a hang as a real signal to a replica process (SIGKILL, SIGSTOP) at the start of a configured cycle, and the summary shall report how each replica process ended. | Integration test |

## System requirement

Expand Down
27 changes: 25 additions & 2 deletions docs/opssat.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ satellite, what it touches, its limits, how to start and stop it, what it sends
down, and what it can and cannot show. It describes the `quorum` program
that ran in runs 1 and 2. The package for a planned third run is the F´
deployment of the same decision core, with F´'s command, event and
telemetry interfaces ([run 3](#run-3-v060-the-f-deployment-planned)).
telemetry interfaces ([run 3](#run-3-v070-the-f-deployment-planned)).

## What runs

Expand Down Expand Up @@ -143,7 +143,7 @@ target, checks it against `SHA256SUMS`, adds `run`, the schedules, a
the required ownership and no hidden entries. `QUORUM_BIN=path ./package.sh`
packages a binary you built yourself.

### Run 3 (v0.6.0, the F´ deployment, planned)
### Run 3 (v0.7.0, the F´ deployment, planned)

Run 1 flew the demo, `quorum`, and run 2 uses it too. Run 3, planned and
not yet proposed to ESA, is the package for the F´ components ([design](fprime-design-fpp.md)) in a real F´ deployment,
Expand Down Expand Up @@ -224,6 +224,29 @@ and events as v0.5.1 (342 / 3 / 0 over 448 cycles, 239 / 109 / 0, the
restore at cycle 303), no cycle withheld, the timed run at 1.25%, 1.47% and
1.30%. The package is built by `package.sh` from the v0.6.0 release asset.

**Version 2, the package planned for run 3: each replica in a process of
its own** (v0.7.0, [`opssat/fprime2`](../opssat/fprime2)). Version 1 runs
the replicas as threads of one process and simulates the kill. Version 2 is
the most representative setup the OPS-SAT rules allow: `MruQuorum`, the
manager process, starts three `MruReplica` processes, one `ReplicaHost`
each, linked through F´'s GenericHub over Unix socket pairs (no network),
and pins replica 0 to core 0 and replicas 1 and 2 to core 1 of the SEPP. The
stuck value and the bit flip come from `FaultInjector` as before; the kill is
real: the manager sends SIGKILL to replica 0's process at the start of cycle
240. The segments, caps and sequences are the same, except that the kill is
no longer in the sequence. Eight files: `run`, `MruQuorum`, `MruReplica`,
the three sequences, `README.txt` and the license. The package built locally
by `package.sh` and run full length in ESA's image on 22 October 2026: 148
seconds, every exit as planned, the same counts and events as version 1
(342 / 3 / 0 over 448 cycles with replica 0 lost at 242; 239 then caught,
0 wrong; the restore at cycle 303), replica 0 ended by signal 9 and the
others cleanly, the three replica processes pinned in every segment, 19.9 MB
of address space for the manager. The experiment's CPU, all four processes,
was 2.9% to 3.75% of wall time under emulation (the manager 1.5% to 1.8%).
Most of the replicas' share is their start-up under the emulator, which
translates each binary; on the SEPP that cost is far lower, and the FlatSat
test measures it. Version 1 stays the fallback.

The CPU share comes from the cycle rate, not a throttle: the deployment is
idle between timer ticks. The optimized build matters: an unoptimized F´
build spent 84% of its time in the payload hash and ran at 8% at a 20 ms
Expand Down
4 changes: 4 additions & 0 deletions fprime/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -46,3 +46,7 @@ add_fprime_subdirectory("${CMAKE_CURRENT_LIST_DIR}/Mru")

# The OPS-SAT deployment (exp4051, run 3)
add_fprime_subdirectory("${CMAKE_CURRENT_LIST_DIR}/MruOpsSat")

# OPS-SAT run 3, version 2: the replicas as processes of their own
add_fprime_subdirectory("${CMAKE_CURRENT_LIST_DIR}/MruReplica")
add_fprime_subdirectory("${CMAKE_CURRENT_LIST_DIR}/MruQuorum")
1 change: 1 addition & 0 deletions fprime/Mru/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# Mru project-wide types, ports and components.
add_fprime_subdirectory("${CMAKE_CURRENT_LIST_DIR}/Types")
add_fprime_subdirectory("${CMAKE_CURRENT_LIST_DIR}/Ports")
add_fprime_subdirectory("${CMAKE_CURRENT_LIST_DIR}/Payload")
add_fprime_subdirectory("${CMAKE_CURRENT_LIST_DIR}/Components")
add_fprime_subdirectory("${CMAKE_CURRENT_LIST_DIR}/Test/Parity")
1 change: 1 addition & 0 deletions fprime/Mru/Components/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@ add_fprime_subdirectory("${CMAKE_CURRENT_LIST_DIR}/HealthRegistry")
add_fprime_subdirectory("${CMAKE_CURRENT_LIST_DIR}/ReplicaHost")
add_fprime_subdirectory("${CMAKE_CURRENT_LIST_DIR}/FaultInjector")
add_fprime_subdirectory("${CMAKE_CURRENT_LIST_DIR}/RunRecorder")
add_fprime_subdirectory("${CMAKE_CURRENT_LIST_DIR}/SeqPacketDriver")
2 changes: 2 additions & 0 deletions fprime/Mru/Components/ReplicaHost/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ register_fprime_module(
"${CMAKE_CURRENT_LIST_DIR}/ReplicaHost.cpp"
HEADERS
"${CMAKE_CURRENT_LIST_DIR}/ReplicaHost.hpp"
DEPENDS
Mru_Payload
)

register_fprime_ut(
Expand Down
Loading
Loading