Skip to content

server: the soft open-file limit is raised to the hard one when the server listens - #760

Closed
Bizuayeu wants to merge 1 commit into
ashhart:mainfrom
Bizuayeu:pr/open-files
Closed

Bizuayeu wants to merge 1 commit into
ashhart:mainfrom
Bizuayeu:pr/open-files

Conversation

@Bizuayeu

@Bizuayeu Bizuayeu commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

What this changes

With a container's default 1,024 descriptors, idle connections can use up the rest, and the server then waits on
every accept. The listener already said so once a burst and backed off; it now also raises the soft
RLIMIT_NOFILE to the hard limit when it opens (65,536, then 10,240, when the hard limit is unlimited). A failure
to raise it leaves the limit as it was. This is #294's idea (by @plotarmordev) for the Zig server.

Receipt

  • Environment: TensorFold 1.0.9 (c36b536d) plus this change; one GB10 (DGX Spark, sm_121, CUDA 13.3).
  • Exactness, decode and prompt speed: not applicable (no change to generation).
  • Tests run:
    • zig build test: 294 passed, 5 skipped, 0 failed; the new "the soft open-file limit is raised to the hard one"
      needs this change.
    • zig build test-cuda-host: 430 passed, 0 failed. zig fmt --check: clean. lean_check: 538, as main.
  • Not run, and why: Metal and macOS (the call is POSIX setrlimit; the test runs on Linux here). The fallback for an unlimited hard limit (65,536, then 10,240) is not tested: a test can't raise its own hard limit to unlimited without privileges, and the test skips on a machine whose hard limit already is.

Checklist

  • Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
  • Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo.
  • No precision traded for speed. If the bits change, the description says which sums change and why.
  • Prompt processing is no slower than the last release.
  • The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
  • New tests fail before the change, pass after it, and skip cleanly without their dependency.
  • Comments and docstrings are one line. No measurements or history in the source.
  • No personal data, machine names, internal hosts or local paths. No AI attribution lines.

…erver listens

With a container's default 1024 descriptors, idle connections can use up the rest, and the
server then waits on every accept. The listener already said so once a burst and backed
off; it now also raises the soft RLIMIT_NOFILE to the hard limit when it opens (65536, then
10240, when the hard limit is unlimited), as ashhart#294 (by plotarmordev) did for the Python
engine. A failure to raise it leaves the limit as it was.
@Bizuayeu

Copy link
Copy Markdown
Contributor Author

Rebased onto 1.0.9 (c36b536d), the commit's lines unchanged. Host tests 430/430, zig build test 294 passed, 5 skipped.

@ashhart

ashhart commented Oct 14, 2026

Copy link
Copy Markdown
Owner

Thank you, @Bizuayeu, and thanks for carrying @plotarmordev's idea from #294 over to the Zig server.

The server already does this. Since 1.0.0, serve.zig raises the soft open-file limit to the hard one at start, before the listener opens, with the same fallbacks when the hard limit is unlimited (65,536, then 10,240), and logs the raise once (raiseOpenFileLimit, 267cd3c, with its test). A second raise in Listener.open would be another copy of that call, so I'm closing this one rather than merging it.

Your write-up of the container case, 1,024 descriptors with idle connections starving accept, is exactly the case that change was made for. Your server fixes in #759 and #761 shipped in 1.0.9, and #800 is in 1.0.10. More like them are very welcome.

@ashhart ashhart closed this Oct 14, 2026
@ashhart ashhart mentioned this pull request Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants