ALCF Reconstruction and Segmentation for bl832 - #114

Open

davramov wants to merge 55 commits into

als-computing:mainfrom

davramov:alcf_recon_segmentation

Commits on Jun 17, 2026

Updating ALCF endpoints to include the synaps-i allocation (to be set up)
davramov
committed
bc4f95b
Updating bl832 config.py to distinguish IRI and SYNAPS-I ALCF endpoints
davramov
committed
37a7921
Adding the config.yaml file for setting up the globus compute endpoint for reconstruction on ALCF
davramov
committed
ddd1884
Adding the config.yaml file for setting up the globus compute endpoint for segmentation on ALCF. Still needs to be configured for GPU and the environment with dependencies
davramov
committed
2233446
Adding segmentation Prefect task, and segmentation globus compute code for the TomographyController. Turning off TIFF to ZARR on ALCF for the demo
davramov
committed
233fdec
Making separate ALCF SYNAPS-I endpoint configs for raw, reconstructed, and segmented data
davramov
committed
c7c4e08
Refactoring ALCF reconstruction flow to use the prune_controller class
davramov
committed
89d9f38
Removing old commented out prune code
davramov
committed
9d26b09
linting and docstrings
davramov
committed
f70f856
Docstrings, linting, and type hints
davramov
committed
5e2dd44
Updating globus compute config for segmentation
davramov
committed
dda21bb
turning ALCF recon+segmentation into a separate flow from recon+zarr conversion
davramov
committed
af3631e
updating pytest for alcf reconstruction
davramov
committed
8b85bf6
Adjusting endpoint names for synaps
davramov
committed
d618187
adding the alcf_forge_recon_segment flow to prefect.yaml as a separate deployment
davramov
committed
dbdf877
updating bl832 dispatcher to include alcf_forge_recon_segment as a separate option
davramov
committed
70d2c87
adding transfer client uuid for ALCF SYNAPS-I
davramov
committed
255e2ab
this configuration worked for launching segmentation on 1 GPU
davramov
committed
62c7fc3
Updating segmentation compute endpoint config
davramov
committed
aafd341
Adding a separate wrapper for segmentation v2. At some point we may want to collapse these into the final version, but for testing purposes I'm leaving both codes
davramov
committed
b63b75b
adding globus compute configs for segmentation (single and multinode)
davramov
committed
582a330
removing old segment config
davramov
committed
ef79075
Segmentation wrapper v2 calls the segmentation inference version on polaris that scales well to multiple gpu nodes
davramov
committed
0a43b72
removing stale comments
davramov
committed
1d28eab
removing stale comments
davramov
committed
ec046aa
Cleaning, adding helpful comments
davramov
committed
0dcf022
typo, and cleaning file paths
davramov
committed
5267992
Adding ALCF recon multinode
davramov
committed
3e87678
More optimization. Went from 284s on 8 nodes to 77s
davramov
committed
9fd1ea2
docstring
davramov
committed
9f6d919
fixing typo
davramov
committed
d1ea6a7
Segmentation with inference_v4 working
davramov
committed
9026feb
Adding checkpoint v2
davramov
committed
fba137c
adding tasks/flows for segmentation with dino and cellpose on ALCF
davramov
committed
9e9f2f9
Adding combine segmentation results function to the ALCF controller
davramov
committed
d6a8cd0
Updating segmentation directory paths to put each models outputs into its own subfolder
davramov
committed
c17a707
Adding combine_segmentations to the multiseg flow
davramov
committed
5c5ba74
updating ALCF conda env for dino/cellpose
davramov
committed
021c0b4
Updating the combine step to use the dino/cellpose env
davramov
committed
df402d6
Adding globus compute configs for dino, cellpose, and combine steps
davramov
committed
190c0a3
linting
davramov
committed
ea9675f
Adding globus compute endpoint uuids
davramov
committed
264fe50
updating globus compute endpoint uuid and adding integration test functions for each segmentation step
davramov
committed
ce62d9f
linting
davramov
committed
910b120
adding new flow to prefect.yaml for deployment
davramov
committed
49f4923
updaing dispatcher flow to handle the multisegment option
davramov
committed
43073e3
Updating to match NERSC deployment
davramov
committed
7f72f04
removing commented code
davramov
committed
e304f48
removing cellpose flows/tasks
davramov
committed
3210537
renaming segmentation flow/tasks to segmentation_sam3 so it is obviously separated from dino segmentation
davramov
committed
a490f24
Docstrings
davramov
committed
37ec49f
Adding _tests/bl832/test_alcf.py to test: reconstruction, segmentation, multiresolution flows
davramov
committed
b07e2ed
Adding script used to generate latex segmentation timing table in the nature comms paper
davramov
committed
323b5f3
linting
davramov
committed
a756004
globus compute config
davramov
committed
8988087