Skip to content

feat(port): serve a CWIST app over lwIP's raw TCP API (#447 Phase 3) - #452

Merged
gg582 merged 2 commits into
devfrom
feat/stm32-phase3-lwip
Oct 26, 2026
Merged

gg582 merged 2 commits into
devfrom
feat/stm32-phase3-lwip

Conversation

@gg582

@gg582 gg582 commented Oct 26, 2026

Copy link
Copy Markdown
Member

Phase 3 of the bare-metal STM32 epic (#447): the lwIP transport adapter. It builds on the Phase 1 platform layer (#449).

API (include/cwist/sys/port/lwip.h)

cwist_lwip_config cfg;
cwist_lwip_config_init(&cfg);            /* port 80, 4 conns, 4 KB head, 16 KB body, 30 s idle, keep-alive */
cwist_lwip_server *srv = cwist_lwip_listen(app, &cfg);
...
cwist_lwip_close(srv);

The firmware compiles src/sys/port/lwip.c next to its own lwIP and lwipopts.h, because the adapter depends on its TCP options. The adapter uses only the raw API, so it works under NO_SYS=1. It also uses only public CWIST API (cwist_http_parse_request_len → cwist_app_dispatch → cwist_http_response_serialize), so it is not part of libcwist, and no core code changes in this PR.

Behaviour

  • Framing: request bodies are framed by Content-Length, using the parser's own CL/TE rules (RFC 9112 §6.3), so the adapter never picks a different message boundary than the parser would. The adapter has to frame exactly: the parser takes every remaining byte as the body when there is no Content-Length, which would swallow a pipelined request.
  • Errors: framing failures close the connection after the reply:
    • 501 for Transfer-Encoding; chunked request bodies are not reassembled.
    • 400 for CL together with TE, or a bad or conflicting CL.
    • 431 or 413 above the configured caps.
  • Expect: Expect: 100-continue gets an interim 100 Continue before the body is read.
  • Keep-alive and pipelining: keep-alive is cfg.keep_alive && req->keep_alive && res->keep_alive, the same rule as the socket path. Pipelined requests are answered in order, one exchange at a time per connection. HEAD gets the head only.
  • Sending: responses go out zero-copy through tcp_write without COPY, and the buffer is freed only after the peer acknowledges it. Sending continues from tcp_sent; ERR_MEM is retried from tcp_poll.
  • Back-pressure: while a response is in flight, input beyond one request's worth is refused back to lwIP (ERR_MEM), so the TCP window throttles the peer.
  • Limits: extra accepts beyond max_conns are aborted. Idle connections time out through tcp_poll and sys_now().
  • Listener restart: with SO_REUSE=1 the listener sets SOF_REUSEADDR, so it can be re-created while connections it closed sit in TIME_WAIT. The tests hit exactly this case.
  • Not supported: deferred (cwist_async) and upgraded (WebSocket) responses. This is documented in the header.

Tests

lwIP STABLE-2_2_1_RELEASE is vendored as lib/lwip, for tests only. tests/lwip/ holds a NO_SYS lwipopts.h, arch/cc.h, and an in-process raw-API client. Client and server pcbs talk through lwIP's loopback netif on a fake clock, so every byte goes through real lwIP TCP.

  • test_lwip_adapter (in make test, host):

    • keep-alive and the connection count
    • two pipelined requests in one segment, answered in order
    • a body split across segments, with 100-continue
    • 501, CL+TE 400, bad CL 400, conflicting CL 400, 413, 431 with and without CRLFCRLF, and a parser-level 400 (missing Host); each closes the connection
    • a 20 KB response through a 2-segment send window, byte-exact
    • HEAD with no body
    • Connection: close from the handler, and HTTP/1.0
    • the connection cap: the third connect is aborted
    • the idle timeout: the idle connection closes while the active one survives
    • teardown, defaults, a NULL app, and a bind conflict

    I checked the assertions are live by changing one expectation and watching it fail. The test is also clean under -fsanitize=address,undefined with leak detection.

  • make baremetal-smoke now also runs hello-lwip.elf: the same adapter and lwIP on Cortex-M7 under qemu-arm, covering keep-alive, pipelining with a split body, an 8 KB response through the 2-segment window, a 413, and connection release. All pass.

Sizes (Cortex-M7, -Os, test lwipopts.h)

hello (no lwIP) hello-lwip
flash 44.1 KB 74.5 KB (CWIST + adapter + lwIP + test client)
static RAM (.data+.bss) 1.7 KB 62 KB: lwIP ram_heap 24 KB + pbuf pool 24 KB (test sizing), 8 KB test payload
heap high-water 22.3 KB 55.4 KB (includes the test client's receive buffer and the 8 KB response copy)
stack high-water 8.4 KB 8.3 KB

The lwIP pools in the test config are generous. A production lwipopts.h sizes them to the board, and Phase 6 docs will carry a sizing table.

Local verification

  • make test_lwip_adapter, make test_baremetal_port: pass
  • make baremetal-smoke: hello, hello-rtc, hello-lwip all pass
  • ASan/UBSan/LSan build of the adapter test: clean
  • check_test_wiring, check_api_baseline (4 functions + 2 types added), check_env_knobs, check_api_test_coverage, check_inline_exports, malloc_lint.sh: OK
  • arm-none-eabi-gcc -Wall -Wextra and clang -Wall -Wextra on the adapter: no warnings

Not in this PR

  • An RTOS/netconn variant and a W5500 adapter (follow-ups)
  • CI wiring for baremetal-smoke (Phase 5)
  • Docs (Phase 6)
  • Real hardware

Refs #447

gg582 added 2 commits October 26, 2026 20:06
- include/cwist/sys/port/lwip.h + src/sys/port/lwip.c: a listener on a
  raw-API tcp_pcb (NO_SYS-safe). Requests are framed by Content-Length with
  the parser's CL/TE rules (501 for Transfer-Encoding, 400 for CL+TE or a
  bad/conflicting CL, 431/413 above the configured caps), answered with
  100 Continue on Expect, parsed and run through cwist_app_dispatch, and
  serialized. Keep-alive follows request and response as on the socket
  path; pipelined requests are answered in order. Responses go out
  zero-copy via tcp_write and are freed once acknowledged; while one is
  in flight, excess input is refused back to lwIP so the window throttles
  the peer. Connection cap, idle timeout via tcp_poll, HEAD without body,
  SOF_REUSEADDR when SO_REUSE is on.
- The adapter is compiled by the firmware next to its own lwIP and
  lwipopts.h; it uses only public CWIST API and is not part of libcwist.
- lib/lwip: lwIP STABLE-2_2_1_RELEASE, vendored for the tests only.
- tests/test_lwip_adapter.c (make test): real lwIP TCP over the loopback
  netif in one process, covering keep-alive, pipelining, split bodies,
  100-continue, every framing error, Connection: close, HTTP/1.0, HEAD, a
  20 KB response through a 2-segment send window, the connection cap, the
  idle timeout and teardown. Clean under ASan/UBSan/LSan.
- tests/baremetal/hello_lwip.c joins make baremetal-smoke: the same stack
  on Cortex-M7 under qemu-arm.
cppcheck analyzes src/ without the firmware's lwIP tree, so LWIP_TCP was
undefined and the requirement check fired. lwip/opt.h always defines
LWIP_TCP and LWIP_CALLBACK_API, so check only once it has been seen; a
real build with LWIP_TCP=0 still hits the #error.
@gg582
gg582 merged commit ad7ecad into dev Oct 26, 2026
38 checks passed
gg582 added a commit that referenced this pull request Oct 26, 2026
Conflict: docs/api/v4.0-api-baseline.txt, where dev's lwIP adapter API
(#452) and this branch's Redis rate-limit API land in the same sections.
Regenerated with `make api-baseline` from the merged headers: against dev
it adds only this PR's entries, against the branch only dev's.
gg582 added a commit that referenced this pull request Oct 27, 2026
…in (#454)

* feat(port): serve a CWIST app over lwIP's raw TCP API (#447 Phase 3) (#452)

* feat(port): serve a CWIST app over lwIP's raw TCP API (#447 Phase 3)

- include/cwist/sys/port/lwip.h + src/sys/port/lwip.c: a listener on a
  raw-API tcp_pcb (NO_SYS-safe). Requests are framed by Content-Length with
  the parser's CL/TE rules (501 for Transfer-Encoding, 400 for CL+TE or a
  bad/conflicting CL, 431/413 above the configured caps), answered with
  100 Continue on Expect, parsed and run through cwist_app_dispatch, and
  serialized. Keep-alive follows request and response as on the socket
  path; pipelined requests are answered in order. Responses go out
  zero-copy via tcp_write and are freed once acknowledged; while one is
  in flight, excess input is refused back to lwIP so the window throttles
  the peer. Connection cap, idle timeout via tcp_poll, HEAD without body,
  SOF_REUSEADDR when SO_REUSE is on.
- The adapter is compiled by the firmware next to its own lwIP and
  lwipopts.h; it uses only public CWIST API and is not part of libcwist.
- lib/lwip: lwIP STABLE-2_2_1_RELEASE, vendored for the tests only.
- tests/test_lwip_adapter.c (make test): real lwIP TCP over the loopback
  netif in one process, covering keep-alive, pipelining, split bodies,
  100-continue, every framing error, Connection: close, HTTP/1.0, HEAD, a
  20 KB response through a 2-segment send window, the connection cap, the
  idle timeout and teardown. Clean under ASan/UBSan/LSan.
- tests/baremetal/hello_lwip.c joins make baremetal-smoke: the same stack
  on Cortex-M7 under qemu-arm.

* fix(port): keep cppcheck off the lwIP option #error

cppcheck analyzes src/ without the firmware's lwIP tree, so LWIP_TCP was
undefined and the requirement check fired. lwip/opt.h always defines
LWIP_TCP and LWIP_CALLBACK_API, so check only once it has been seen; a
real build with LWIP_TCP=0 still hits the #error.

(cherry picked from commit ad7ecad)

* feat(middleware): add shared Redis rate limiting (#453)

* feat(middleware): add shared Redis rate limiting

* test(redis): fix GCC buffer and pipe declarations

* fix(redis): Potential fix for pull request finding 'CodeQL / Potentially overflowing call to snprintf'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix(redis): validate frame encoding before pooled I/O

Preserve the upstream Autofix commit and supersede its void encoder with the independently reviewed checked implementation. Reject formatting failures and capacity mismatches before acquiring a connection; cover bounds and formatter failures without suppressing CodeQL.

---------

Co-authored-by: 이윤진(Lee Yunjin) <168180007+gg582@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Lee Yunjin <gzblues61@daum.net>
(cherry picked from commit 29df1ac)

* Potential fix for pull request finding 'Empty except'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>

---------

Co-authored-by: Jiho Lee <optional.int@kakao.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant