OpenImpala: Strategic Development Roadmap (v3.0)

[jameslehoux]

OpenImpala: Strategic Development Roadmap (v4.0+)

This roadmap outlines the development trajectory of the OpenImpala framework,
prioritising scientific correctness, multi-physics extensibility, and
high-performance computing (HPC) scalability for porous media and transport
physics research.

---

Completed (v3.0 — v4.0)

Foundational CI/CD, Python packaging, architectural refactoring, GPU
acceleration, and solver infrastructure.

• Feature Request: Develop Python Bindings for OpenImpala #17 Feature Request: Develop Python Bindings for OpenImpala
• Refactor Build System Modernization to CMake #48 Refactor: Build System Modernisation to CMake
• Multi-phase computations #3 Multi-phase computations (Map phase IDs to unique transport coefficients)
• Electrical Conductivity Calculation #1 Electrical Conductivity Calculation
• Chore: Integrate Automated Code Formatting with clang-format #37, Chore: Integrate Static Analysis into CI Pipeline #38, CI: Implement Code Coverage Tracking and Reporting #81, CI: Implement Automated Documentation Generation (Doxygen/Sphinx) #110 CI: Formatting, Static Analysis, Codecov, and Doxygen
• Infrastructure: Integrate a C++ Unit Testing Framework (Catch2 or GTest) #79, QA: Implement Automated Regression Benchmarks with Synthetic Geometries #80, Refactor Separate Test Drivers from Core Library Source #49 QA: Catch2 Integration, Synthetic Benchmarks, and Test Separation
• Usability: Implement Enhanced Input File Validation #36, Feature: Add Pre-computation Sanity Checks and "Dry Run" Mode #39 Usability: Enhanced Input Validation and Sanity Checks
• Refactor Consolidate Duplicated SolverType Enums #50 Refactor: Consolidate Duplicated SolverType Enums
• Refactor Deconstruct the Monolithic Diffusion Driver #51 Refactor: Deconstruct the Monolithic Diffusion Driver
• Migrate Fortran Compute Kernels to native C++ AMReX Lambdas #52 Refactor: Migrate Fortran Compute Kernels to native C++ AMReX Lambdas
• Epic: Intelligent Solver Routing and Matrix-Free AMReX MLMG Integration #171 Epic: Intelligent Solver Routing and Matrix-Free AMReX MLMG Integration
• Feature Request: Expand OpenImpala into a comprehensive Microstructural Parameterization Engine for DFN models #170 Feature Request: Microstructural Parameterisation Engine (SSA, REV, PSD, Connected Components)
• Epic: Python Packaging and Distribution Strategy (PyPI & Conda-Forge) #147 Epic: Python Packaging — Phase 1 (pyproject.toml) & Phase 2 (cibuildwheel + PyPI)
• Enable and Optimize GPU Acceleration for Solve #25 Enable GPU Acceleration via CUDA (openimpala-cuda PyPI package) — initial enablement

Partially Addressed (significant progress, remaining work tracked below)

• [~] Refactor: Implement Component-Based Architecture for Multi-Physics Extensibility #15 Component-Based Architecture — base classes extracted (HypreStructSolver, TortuositySolverBase), full PhysicsModule/SolverStrategy abstraction remains
• [~] Refactor: Generalize Boundary Condition Setup for Tortuosity Calculation #16 Generalised Boundary Conditions — BC abstraction layer added, runtime-selectable BC types and bc.* input parameters remain
• [~] QA: Establish Scientific Verification & Validation (V&V) Pipeline #83 V&V Pipeline — synthetic analytical benchmarks added (uniform, series, parallel), experimental validation and theoretical bounds remain
• [~] Development of additional tutorials #9 Tutorials — 7-part series created, advanced topics and maintenance ongoing
• [~] Epic: Improve Code Coverage (Current: 27.31%) #109 Code Coverage — new modules added but test coverage still ~27%

---

Phase 0: Polish & Prove (v4.1.0)

Objective: Lock in the gains from v4.0 with automated release
infrastructure, expanded test coverage, a public documentation site,
performance baselines, and a JOSS publication before adding new features.

Priority	Issue	Description	Notes
High	#226	Sustainability: Publish OpenImpala in JOSS	Citable DOI, peer-reviewed software quality
High	#85	DevOps: Automate Semantic Versioning and Release Drafting	Prevent manual release pain
High	#109	Epic: Improve Code Coverage (Target: 50%+)	Many new v4.0 modules untested
High	#8	Usability: Sphinx/ReadTheDocs Documentation Site	Tutorials + Doxygen exist, need assembly
High	#31	Performance: Establish Profiling Baselines (CPU vs GPU)	Prerequisite for all solver tuning
High	#83	V&V: Experimental Validation & Theoretical Bounds	Need real datasets with known properties
High	#224	Usability: HPC "Pre-Flight" Checks & Memory Estimation	Prevent wasted HPC allocation hours
High	#205	Interactive Jupyter Notebook GUI via ipywidgets and PyVista	Frictionless notebook UX
High	#215	Jupyter Visualization: Native AMReX Plotting via yt	Quick win — out-of-core viz for large datasets
Medium	#217	Downstream Visualization: ParaView Streamline Tutorial & State File	Publication-quality 3D renders
Medium	#186	Strategy for an Interactive Profiling & Tuning Colab Notebook	GPU profiling sandbox
Medium	#84	CI: Automated Performance Benchmarking for PRs	Prevent regressions from future changes
Low	#10	Outreach: Record Webinar / Video Tutorial

---

Phase 1: Battery Integration & Usability (v4.2.0)

Objective: Make OpenImpala the definitive upstream parameterisation tool
for continuum battery models by enabling direct memory coupling with PyBaMM,
adding electrode-specific metrics, and improving user onboarding with
built-in data utilities.

Priority	Issue	Description	Notes
High	#65	Direct Memory-Coupling API for PyBaMM	Killer feature for battery community
High	#12	Electrode Tortuosity Factor	Well-scoped, high-value for users
High	#214	Usability: Built-in Otsu Thresholding Utility	Zero-dep, 3-line grayscale-to-solve workflow
High	#221	Usability: Native Digital Rocks Portal Data Fetcher	Reproducible tutorials with real datasets
Medium	#216	Python API: Interactive 3D Rendering Method (PyVista)	.plot_3d() on VoxelImage
Medium	#11	Non-Cubic Voxel Support	Important for real tomography data
Medium	#34	Advanced Post-Processing and Derived Quantities
Low	#147	Conda-Forge Recipe (Phase 3 of packaging epic)	Nice-to-have; HPC users have Apptainer

---

Phase 2: Architecture & Extensibility (v4.3.0)

Objective: Complete the component-based architecture to support pluggable
physics modules, runtime-configurable boundary conditions, and future
multi-physics coupling.

Priority	Issue	Description	Notes
High	#15	Complete Component-Based Architecture	PhysicsModule, SolverStrategy, SimulationManager interfaces
High	#16	Runtime-Selectable Boundary Conditions	bc.* input parameters, DirichletExternal vs InternalPhaseBoundary
Medium	#206	Develop openimpala-napari Plugin for Local Workstations	Native 3D viewer with zero-copy solver integration
Medium	#9	Tutorial Maintenance & Advanced Topics	Keep tutorials current with API changes

---

Phase 3: New Physics (v5.0.0)

Objective: Expand the physical formulations to support transient
processes, thermal transport, and new solver paradigms. Breaking changes to
the mathematical formulation warrant a major version bump.

Constraints: Changes must preserve the mathematical correctness of finite
difference stencils.

Priority	Issue	Description	Notes
High	#5	Generalised Boundary Conditions (physics-level)	Builds on #16 infrastructure
High	#35	Transient (Time-Dependent) Solver
High	#4	Heat Equation Computation
Medium	#70	Stochastic Random Walk Solver (AMReX ParticleContainer)	Grant-driven
Low	#72	Chemo-Mechanical Stress/Strain Solver	Grant-driven

---

Phase 4: HPC Scalability & I/O

Objective: Optimise I/O pipelines for massive out-of-core datasets,
improve parallel efficiency, add resilience for long-running HPC jobs,
and provide HPC-native packaging (Spack, AiiDA).

Priority	Issue	Description	Notes
High	#25	GPU Acceleration: Profiling & Optimisation	Initial enablement done; needs perf validation
High	#18	Parallel I/O for RawReader
High	#33	Parallel I/O for Writing Plotfiles
High	#32	Checkpoint/Restart Capability	Essential for long HPC runs
Medium	#220	HPC Infrastructure: Official Spack Package Recipe	Required for Tier-1 supercomputer deployment
Medium	#219	Reproducibility & Provenance: Develop aiida-openimpala Plugin	FAIR data principles, automated HPC orchestration
Medium	#218	Modern Data Ecosystem: Support Cloud-Native OME-Zarr I/O	Chunked, cloud-native format; pairs with napari
Medium	#26	Parallel Load Balancing
Medium	#27	OpenMP Threading Optimisation
Medium	#13	Memory Efficiency Improvements
Low	#67	In-Transit Coupling with Tomography Pipelines (Savu)	Superseded in part by #208

---

Phase 5: Solver Tuning

Objective: Systematic solver/preconditioner optimisation informed by
profiling data from Phase 0.

Prerequisites: #31 (profiling baselines) must be completed first.

Scope for AI Assistance: Strictly Human-Led. These require expert
numerical analysis to prevent functionally correct but slow implementations.

Priority	Issue	Description
High	#19	Implement and Evaluate BoomerAMG Preconditioner
High	#22	Optimise Krylov Solver Choice and Parameters
Medium	#20	Tune BoomerAMG Preconditioner Parameters
Medium	#21	Re-evaluate and Further Tune PFMG Preconditioner
Medium	#23	Investigate Matrix Scaling/Equilibration
Low	#28	Optimise Solver/Preconditioner Algorithmic Costs
Low	#30	Investigate Mixed-Precision Solves

---

Phase 6: Ecosystem Integration & Visualization

Objective: Embed OpenImpala into the broader tomography and porous media
ecosystem through upstream pipeline bridges, GUI plugins for non-coders, and
AI/ML workflow integration. Most items are documentation/tutorial-driven and
maintained in separate repositories to avoid bloating the core physics engine.

Upstream Pipeline Bridges

Priority	Issue	Description	Notes
High	#211	Upstream Pipeline Integration: TIGRE (Iterative GPU Reconstruction)	End-to-end GPU-accelerated metrology
High	#208	Upstream Pipeline Integration: TomoPy & HTTomo (Savu Successor)	APS + Diamond Light Source ecosystems
High	#210	Ecosystem Integration: CIL & PoreSpy Bridges	UK reconstruction + porous media communities
High	#223	Ecosystem Integration: Develop Orange Canvas Add-on (ESRF / Tomwer)	European synchrotron visual workflows

GUI & Outreach Plugins

Priority	Issue	Description	Notes
Medium	#209	Outreach: Develop a "Thin-Client" ImageJ / Fiji Plugin	Largest experimentalist userbase
Medium	#207	Epic: Enterprise HPC Web Dashboard (Trame)	Remote client-server rendering

AI / ML Workflows

Priority	Issue	Description	Notes
Medium	#225	Ecosystem: High-Throughput Data Generation for ML Surrogates	Ground-truth generator for AI battery community
Medium	#212	AI & Vision Workflows (SAM & ALS Ecosystems)	Bridge deep learning segmentation to physics

---

Phase 7: Future Research Directions

Objective: Advanced mathematical formulations and capabilities deferred
pending specific grant requirements or research needs.

Issue	Description
#6	Adaptive Mesh Refinement (AMR)
#66	Adjoint Formulations / Differentiable Physics for Microstructure Optimisation
#213	Epic: Direct-from-Grayscale Physics Solvers (Partial Volume Formulation)
#222	Epic: 4D Operando Transport via DVC (SPAM Integration)
#68	4D Operando Transport Mapping via DVC Ingestion
#69	Sub-Voxel Accuracy via AMReX Embedded Boundaries (EB)
#71	In-Situ ML Inference for AI-Driven Solver Preconditioning

---

Release Plan Summary

Release	Theme	Key Deliverables
v4.1.0	Polish & Prove	JOSS publication, 50%+ coverage, docs site, profiling baselines, HPC pre-flight checks, yt/ParaView viz guides
v4.2.0	Battery Integration & Usability	PyBaMM coupling, electrode tortuosity, Otsu thresholding, Digital Rocks data fetcher, non-cubic voxels
v4.3.0	Architecture	Complete component-based design, runtime BCs, napari plugin
v5.0.0	New Physics	Transient solver, heat equation, generalised BCs
v5.x	HPC & Tuning	Parallel I/O, checkpoint/restart, Spack recipe, AiiDA plugin, OME-Zarr, solver optimisation
v5.x	Ecosystem	TIGRE/TomoPy/HTTomo bridges, CIL/PoreSpy bridges, Orange/ImageJ plugins, ML data generation
v6.0+	Research Frontier	AMR, adjoint methods, embedded boundaries, grayscale solvers, 4D operando