Repository navigation
host-native: Initial support for host-native tasks - #2736
jamesmunns wants to merge 26 commits into
Conversation
This crate contains the protocol between a host-native `kern` impl and a host-native `userlib` impl.
Also updates ping and pong tasks, and adds stub task.
4b18bec to
2dba0fa
Compare
|
(sorry for the one force-push to bring this up to date with the |
Hmm, I will need to probably:
|
This was causing measurable code/stack bloat on some targets.
labbott
left a comment
labbott
left a comment
There was a problem hiding this comment.
I support getting more feedback on the architecture AND I also support merging this in.
I think it generally looks good and we're likely to make more progress trying to get it to do things if it's in tree vs not. I'm putting my approve bits on because I am a CODEOWNER.
| //! COBS encoding is used for framing, and messages are serialized using the | ||
| //! `postcard` crate. |
There was a problem hiding this comment.
Remind me why we chose COBS? (No objections, I also didn't see it documented in RFD753)
There was a problem hiding this comment.
I can add it there! COBS is my go-to for "turn a stream of bytes into a stream of frames". We use it for some other things as well (IPCC, iirc?). Since stdio is a "stream of bytes" medium, we want to turn that into discrete frames.
| //! Despite operating as multiple OS processes, the control flow between the | ||
| //! kernel and tasks is still lock-step (emulating single threaded behavior), | ||
| //! as all entities except for the currently running one will block waiting for | ||
| //! a reply from the currently running entity. |
There was a problem hiding this comment.
(Paragraph of thinking out loud) It took me a minute to think about this but the key words here are control flow . The interaction between (blocking) sys calls and interrupts is the most useful part of this. Techncially we could have multiple tasks "running" if they aren't making syscalls but I think that should be fine because the interesting work should come from syscalls.
There was a problem hiding this comment.
I think I agree with your thinking out loud, but yeah, it is intentional that we do "lockstep control flow across processes" to be more faithful to the system we are emulating. It also is already how userlib + the kernel behave.
| verbose, | ||
| release, | ||
| cfg, | ||
| tasks, |
There was a problem hiding this comment.
It's confusing that an empty tasks builds nothing, rather than everything. Make it an Option and default to building everything if None? It's also helpful to have a -l / --list option, which prints available tasks to build.
| .leases | ||
| .iter() | ||
| .map(|l| { | ||
| (l.attributes.0 & 2 != 0) |
There was a problem hiding this comment.
Can we use the bitfield values here instead of 2?
| base: 0usize, | ||
| size: 32usize, | ||
| attributes: RegionAttributes::empty(), | ||
| arch_data: crate::arch::host::compute_region_extension_data( |
There was a problem hiding this comment.
This is a pre-existing condition, but it seems like we should just have a crate::arch::$xyz::compute_region_desc(base, size, attributes) -> RegionDesc, instead of repeating the arguments as member variables and in this function call.
There was a problem hiding this comment.
...or an even more abstracted approach: hide it inconst fn RegionDesc::new, which calls a constructor for the ArchImpl::RegionDescExt associated type? This require another helper trait, but removes all of the "manually selecting the correct arch module".
While reviewing #2736 (which constructs a `TaskDesc` table in `host_config`), I realized we shouldn't have to rely on human memory to guarantee that regions are sorted.
mkeeter
left a comment
mkeeter
left a comment
There was a problem hiding this comment.
I'm generally feeling positive about the PR. I've got lots of little comments, and I'm mostly ambivalent about how many of them actually get changed before merging (versus after), but would like to hear your thoughts!
| // License, v. 2.0. If a copy of the MPL was not distributed with this | ||
| // file, You can obtain one at https://mozilla.org/MPL/2.0/. | ||
|
|
||
| //! # Configuration |
There was a problem hiding this comment.
This feels architecturally iffy. The RON file is serving two purposes:
- Defining the task table (task indices and executable paths). This is known at
xtasktime, and I don't think it should change for a given top-level host binary - Passing arguments to the executable (the tracing / realtime flags)
My intuition is to break this up into two pieces:
- We can build the task table at compile time with codegen. I'd guess this would be in a
build.rsscript reading an environment variable populated byxtask– very similar to what we have here, but done at build time instead of runtime. - We can pass arguments by actually parsing
std::env::argsright here! They're simple enough that we probably don't needclap, although we could use it for nicer ergonomics (the Big Computer is fast enough).
If these changes are made, then the top-level binary would be executable directly from the command line (right now, it complains about a missing HUBRIS_HOST_CONFIG, which is helpful but a bit of a speed bump).
There was a problem hiding this comment.
Removed, we now:
- Bake in task slots to the tasks
- Bake in a list of tasks to the kernel
- Pass all the args through to the kernel using
--
new invocation is:
cargo xtask host-run app/host/demo.toml -- --trace --stop-at 1000| // Safety: the pointer came from a `&mut Task` in the task table (see | ||
| // `set_current_task`), and this is only called between kernel entries, | ||
| // when no other reference into the table is live. | ||
| let index = unsafe { usize::from((*current).descriptor().index) }; |
There was a problem hiding this comment.
Nit: move the unsafe to have smaller scope, because I had trouble finding the unsafe part at first glance
| let index = unsafe { usize::from((*current).descriptor().index) }; | |
| let index = usize::from(unsafe { &*current }.descriptor().index); |
| procs.resize_with(index + 1, || None); | ||
| } | ||
| if let Some(mut old) = procs[index].take() { | ||
| old.child.kill().ok(); |
There was a problem hiding this comment.
Probably helpful to do some logging here.
This is an extract of the task-slot binding mechanism I was building in #2736. With today's `const` (and `xtask`), we can actually skip the post-linking patchup entirely, instead directly patching in via environment variables, as `xtask` is aware of all renames/remappings prior to building any tasks.
This has been replaced by the changes introduced by #2764
Kernel args are passed after the `--`.
fatal!/finish! take format arguments directly, removing format_args! at call sites.
Also avoids waiting for a response from the kernel if we are panicking.
Restores userlib target check.
Also removes `host-build`
This PR builds on #2704 and #2705 by implementing the respective
Archtraits on the native host architecture.Closes #2706.
From a high level, this compiles the kernel and each task as binaries for the host architecture, and then launching the kernel process. The kernel process then spawns each task as a child OS process, and communicates with it using stdio. The child tasks emit system calls to stdout, and block for a response on stdin. The kernel then processes system calls, and performs scheduling using
kern's actual scheduling logic, resuming one task at a time.The kernel can be configured to operate at "wall clock" time (so delays/scheduling is real, and should appear roughly at the same speed hardware would), or non-realtime, meaning that the system will execute as fast as possible, fast-forwarding the clock whenever the system would sit in the idle task.
Host apps are configured the same way firmware apps are: we define a folder in
app, and can add manifests in the TOML format. I've addedhostdefinitions for all of the necessary bits to make this work. You can now run the following command, and see the following output tracing raw syscalls that occur:$ cargo xtask host-run --trace --stop-at 1000 app/host/demo.toml Finished `dev` profile [optimized + debuginfo] target(s) in 0.16s Running `target/debug/xtask host-run --trace --stop-at 1000 app/host/demo.toml` building task-pong for the host Finished `dev` profile [optimized + debuginfo] target(s) in 0.10s building task-ping for the host Finished `dev` profile [optimized + debuginfo] target(s) in 0.10s building task-stub for the host Finished `dev` profile [optimized + debuginfo] target(s) in 0.09s building host-demo for the host Finished `dev` profile [optimized + debuginfo] target(s) in 0.09s wrote target/host/host-demo/host-config.ron running target/host/debug/host-demo [kernel t=0000000] task pong started: /Users/james/oxide/hubris/target/host/debug/task-pong (pid 21736) [kernel t=0000000] task ping started: /Users/james/oxide/hubris/target/host/debug/task-ping (pid 21737) [kernel t=0000000] task user_leds started: /Users/james/oxide/hubris/target/host/debug/task-stub (pid 21738) [kernel t=0000000] task user_leds: RECV mask 0x0 from None [kernel t=0000000] task pong: REFRESH_TASK_ID [kernel t=0000000] task pong: resumed from REFRESH_TASK_ID with [02, 00, 00, 00, 00, 00] [kernel t=0000000] task pong: SET_TIMER Some(500) bits 0x1 [kernel t=0000000] task pong: resumed from SET_TIMER with [00, 00, 00, 00, 00, 00] [kernel t=0000000] task pong: RECV mask 0x1 from None [kernel t=0000000] task ping: REFRESH_TASK_ID [kernel t=0000000] task ping: resumed from REFRESH_TASK_ID with [00, 00, 00, 00, 00, 00] [kernel t=0000000] task ping: SEND to pong: op 1 (5 bytes) [kernel t=0000000] task pong: resumed from RECV with [00, 01, 01, 05, 10, 00] [kernel t=0000000] task pong: REPLY to ping: code 0 (0 bytes) [kernel t=0000000] task pong: resumed from REPLY with [00, 00, 00, 00, 00, 00] [kernel t=0000000] task pong: RECV mask 0x1 from None [kernel t=0000000] task ping: resumed from SEND with [00, 00, 00, 00, 00, 00] [kernel t=0000000] task ping: process ended, MemoryAccess { address: None, source: User } [kernel t=0000000] task pong: resumed from RECV with [00, FFFF, 01, 00, 00, 00] [kernel t=0000000] task pong: SET_TIMER Some(1000) bits 0x1 [kernel t=0000000] task pong: resumed from SET_TIMER with [00, 00, 00, 00, 00, 00] [kernel t=0000000] task pong: SEND to user_leds: op 3 (8 bytes) [kernel t=0000000] task user_leds: resumed from RECV with [00, 00, 03, 08, 00, 00] [kernel t=0000000] task user_leds: REPLY to pong: code 0 (0 bytes) [kernel t=0000000] task user_leds: resumed from REPLY with [00, 00, 00, 00, 00, 00] [kernel t=0000000] task user_leds: RECV mask 0x0 from None [kernel t=0000000] task pong: resumed from SEND with [00, 00, 00, 00, 00, 00] [kernel t=0000000] task pong: RECV mask 0x1 from None [kernel t=0001000] task idle: waited for the next deadline [kernel t=0001000] task pong: resumed from RECV with [00, FFFF, 01, 00, 00, 00] [kernel t=0001000] task pong: SET_TIMER Some(1500) bits 0x1 [kernel t=0001000] task pong: resumed from SET_TIMER with [00, 00, 00, 00, 00, 00] [kernel t=0001000] task pong: SEND to user_leds: op 3 (8 bytes) [kernel t=0001000] task user_leds: resumed from RECV with [00, 00, 03, 08, 00, 00] [kernel t=0001000] task user_leds: REPLY to pong: code 0 (0 bytes) [kernel t=0001000] task user_leds: resumed from REPLY with [00, 00, 00, 00, 00, 00] [kernel t=0001000] task user_leds: RECV mask 0x0 from None [kernel t=0001000] task pong: resumed from SEND with [00, 00, 00, 00, 00, 00] [kernel t=0001000] task pong: RECV mask 0x1 from None kernel: stopping: time reached the configured stop at 1000 ticksThis is staged as separate commits adding each major "chunk" of changes, though I opened this as a single PR as we sort of need all the pieces existing to get the "tada" experience of it all working.
For the oxide folks, see https://rfd.shared.oxide.computer/rfd/0753 for more contextual details.