[neumann-alexander]

Problem

Mesh refinement can run out of memory on large, densely sampled surfaces. We encountered this on a real mesh with approximately 960,000 triangles, where refinement exhausted roughly 128 GiB of available RAM.

The problem is in smooth_offset_field(): it obtains and retains radius-neighbor lists for every triangle at once. On a dense mesh, each triangle can have many neighbors, so these lists can consume far more memory than the mesh itself.

Why batching fixes this

Smoothing one triangle requires its neighbors and the previous iteration’s offsets—not the neighbor lists for every other triangle.

This change queries and processes consecutive batches of triangles, releasing each batch’s neighbor lists before requesting the next. Every triangle is still processed against the complete mesh. There is no subsampling, smaller smoothing radius, or approximate neighbor search.

The distance weights, neighbor ordering, and separation between previous and next iteration values are preserved. Batching changes how much temporary data is retained, not the intended calculation.

Configuration

Adds a positive-integer setting:

mesh_refinement:
  smoothing_batch_size: 128

Existing configurations inherit this default. Invalid values are rejected before expensive refinement work.

128 was selected using the large mesh because it reduced memory without an observed runtime penalty:

Batch size	Smoothing time	Process peak RAM
128	377 s	148 MiB
1000	392 s	288 MiB
4096	398 s	794 MiB

These are single-run, standalone smoothing measurements—not whole-workflow memory requirements. Memory still depends on mesh density and smoothing radius; batching is not an absolute RAM cap.

Validation

Validation was performed outside the repository; no repository test files were added or modified.

• All 960,641 smoothed values matched an independent reference using the original neighbor-query method bitwise.
• Outputs were bitwise identical across the three benchmarked batch sizes.
• 90 smaller numerical comparisons passed, including duplicate coordinates, radius boundaries, zero radius, empty input, multiple iterations, and NumPy integer arguments.
• Six synthetic surfaces completed three refinement iterations and output checks.
• The large workflow completed three iterations, final curvature, and thickness calculations in 4 h 14 min, with 51.6 GiB whole-job peak RAM, without OOM. Output checks passed.

Scope and limitations

The change is limited to batching, configuration, and input validation. It does not alter scientific parameters or introduce unrelated upstream changes.

Numerical equivalence was verified for smoothing; a complete old-versus-new large workflow comparison was not possible because the original operation exhausted available memory. A fresh Conda dependency resolution/build was not tested.

AI assistance: An AI coding agent proposed the batching approach, implemented the change, and executed the reported validation. The human contributor approved the approach and directed the work.