Repository navigation
Conversation
leftwo
force-pushed
the
alan/dsc-recreate-region
branch
from
November 6, 2026 18:09
78174b9 to
6e633ea
Compare
We now have the ability in dsc to recreate a region in place. "dsc cmd recreate -c N" replaces a stopped downstairs' region with a new, empty region with the same geometry and a new UUID. Tests that use dsc with reuse of a downstairs as a replacement can this ability to make the replacement "look like" what happens for real in the product where a replacement downstairs is always fresh and new. The new region is created next to the old one and renamed into place, so a failed create leaves the old region alone. dsc refuses if the downstairs is running or set to restart.
Creating a large region can take many minutes, longer than most
clients will wait. POST /recreate/cid/{cid} will return once the
recreate has started. The new GET /recreate/cid/{cid} returns
the creation status: idle, in progress, done with the new UUID,
or failed with the error.
The status is set to in progress before the POST returns, so a caller
polling after it can't see an earlier result. If the recreate is
refused, for example because the downstairs is running, the status is
failed with the reason. The status remains until the next recreate
starts.
The monitor creates the region in another task, so it keeps handling
commands. It refuses a second recreate and ignores start and enable
restart until the recreate finishes.
dsc cmd recreate -c N will start a recreation and then polls the
status until the recreation finishes or errors and prints the result.
The fake downstairs used by the tests can now hold a create in
progress, and a new test checks what the monitor does while a recreate
is running.
leftwo
force-pushed
the
alan/dsc-recreate-region
branch
from
November 6, 2026 20:50
6e633ea to
2e3fdee
Compare
Contributor
Author
|
@claude review |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
jmpesp
requested changes
Nov 9, 2026
Comment on lines
+1487
to
+1488
| res = async { (&mut recreate.as_mut().unwrap().handle).await }, | ||
| if recreate.is_some() => |
Contributor
There was a problem hiding this comment.
I think this is susceptible to futurelock, it's a &mut future in a tokio::select branch.
Contributor
Author
There was a problem hiding this comment.
I think this one was okay, as I don't believe we have other paths that need anything that this path has, but perhaps there are filesystem actions that could back things up.
Even so, it could be a setup that would trap future us where we did come in and update something in the loop that then allowed future lock to happen, so I've changed it around and we should be okay going forward.
Do a better job at reporting back errors when the region replacement runs into trouble.
Change the way we spawn a task inside a select to avoid any potential futurelock (PR comments)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With this change we now have the ability in dsc to recreate a region in place.
"dsc cmd recreate -c N" replaces a stopped downstairs' region with a new,
empty region with the same geometry and a new UUID. Tests that use dsc
with reuse of a downstairs as a replacement can use this ability to make the
replacement "look like" what happens for real in the product where a
replacement downstairs is always fresh and new.
The new region is created next to the old one and renamed into place, so a
failed create leaves the old region alone. dsc refuses this action if the
downstairs is running or set to auto restart.
Because creating a region can take several seconds, or even minutes depending
on the size and other factors, we had to switch from a create and wait
model (which would timeout) to a submit the request, then poll for
completion. This avoids future test flakes that we would see otherwise.
This is the second in a stack (of 4?) PRs to update the test infra in preparation
for the min-quorum work coming soon behind it.
Claude has written a bunch of tests here and while I may not have bothered
to write them myself, the do add some value overall so while I'm not attached
to them, I'm also not in opposition to them. The open API doc also eats up
a bunch of lines, so the meat of this PR is not as many lines as what it
looks like at first glance.