Skip to content

fix(ttx): fix localSession data race and unblock peers on close - #2340

Open
SurbhiAgarwal1 wants to merge 1 commit into
LFDT-Panurus:mainfrom
SurbhiAgarwal1:fix/2285-local-session-data-race
Open

SurbhiAgarwal1 wants to merge 1 commit into
LFDT-Panurus:mainfrom
SurbhiAgarwal1:fix/2285-local-session-data-race

Conversation

@SurbhiAgarwal1

Copy link
Copy Markdown
Contributor

Fixes #2285

Problem

  1. info.Closed in localSession was read and written without synchronization, causing a data race under -race during self-auditing flows (where an issuer is also an auditor).
  2. When a session is closed (e.g. via cleanupSessions), the peer sitting in <-session.Receive() was never unblocked and waited out the full DefaultReceiveTimeout (10 seconds).

Solution

  • Added sync.RWMutex, closedChan, inFlight tracking, and writeClosed boolean to localSession.
  • Info(), Receive(), Send(), and Close() are now synchronized and thread-safe.
  • Close() safely closes writeChannel when inFlight == 0, instantly releasing any peer waiting on <-peer.Receive().
  • Added unit tests in session_test.go covering peer unblocking on close, concurrent usage under -race, and double session closure.

@SurbhiAgarwal1
SurbhiAgarwal1 force-pushed the fix/2285-local-session-data-race branch 3 times, most recently from 4e2031d to 838c3a9 Compare August 31, 2026 18:46
@AkramBitar AkramBitar added the bug Something isn't working label Sep 3, 2026
Comment thread plan.md Outdated
@AkramBitar

Copy link
Copy Markdown
Contributor

In TestLocalBidirectionalChannel_ConcurrentCloseAndOperations, the send goroutine calls leftSession.Send(ctx, []byte{byte(i)}), but the Send method on view.Session takes only []byte — no context argument. This should be either leftSession.SendWithContext(ctx, []byte{byte(i)}) or leftSession.Send([]byte{byte(i)}). Please verify before merge.

@SurbhiAgarwal1
SurbhiAgarwal1 force-pushed the fix/2285-local-session-data-race branch from 80726c6 to d3f467d Compare September 3, 2026 09:14
@SurbhiAgarwal1
SurbhiAgarwal1 force-pushed the fix/2285-local-session-data-race branch from d3f467d to 9678e8d Compare September 3, 2026 09:20
@SurbhiAgarwal1

Copy link
Copy Markdown
Contributor Author

Hi @AkramBitar, thanks for the review!

I verified the view.Session interface definition in github.com/hyperledger-labs/fabric-smart-client/platform/view/view. In FSC, view.Session's Send method signature explicitly takes (ctx context.Context, payload []byte) error (with context as the first argument).

Omitting ctx causes a Go compilation error (*localSession does not implement view.Session: have Send([]byte) error, want Send(context.Context, []byte) error). So leftSession.Send(ctx, payload) matches the view.Session interface requirement.

@SurbhiAgarwal1
SurbhiAgarwal1 force-pushed the fix/2285-local-session-data-race branch from 9678e8d to 18897a6 Compare September 3, 2026 09:28
@Effi-S
Effi-S force-pushed the fix/2285-local-session-data-race branch from 18897a6 to f81559e Compare September 9, 2026 12:36
Copilot AI lite review requested due to automatic review settings September 9, 2026 12:36

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

@AkramBitar AkramBitar left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary: the core fix is correct — I could not find a send-on-closed-channel or double-close path (the closed check and inFlight++ share one critical section with Close(), and writeClosed guards the close), buffered messages survive the sender's Close(), and go test ./token/services/ttx/... ./token/services/utils/session/... -race is green at this head.

Two things are worth fixing before merge; the other three are nits that can be follow-ups.

  1. send became nondeterministic when the caller's context is already cancelled (inline comment on the select). This one has a real caller path in the self-audit responder.
  2. The close only unblocks the peer's receive; a peer blocked in send on a full buffer still stalls — the same class of hang #2285 is about, in the other direction.

Also, two test notes: TestLocalBidirectionalChannel_ConcurrentCloseAndOperations asserts nothing, and its receive goroutine's select { case <-ch: default: } almost always takes default, so it exercises the race detector but barely the receive path. The time.Sleep(10 * time.Millisecond) in TestLocalBidirectionalChannel_CloseReleasesPeerReceive is unnecessary — the peer gets the closed channel regardless of ordering.

}

return nil
select {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium: Send/SendError is now nondeterministic when the caller's context is already cancelled.

writeChannel is buffered (cap 10, set in NewLocalBidirectionalChannel), so with a cancelled ctx both case s.writeChannel <- msg and case <-ctx.Done() are ready, and Go picks pseudo-randomly. Measured on this head: 88 successes / 112 failures over 200 iterations with an already-cancelled context. Before this PR the buffered send always succeeded.

Concrete path: the self-audit responder runs via view3.RunView(logger, context, responderView, view.AsResponder(right)) on the initiator's view context (token/services/ttx/auditor.go:279). If the initiator returns first — its 1-minute signature timeout at auditor.go:189, or RunViewWithTimeout's defer cancel() — then the responder's signAndSendBack -> SendTyped(context.Context(), ...) delivers the signature about 44% of the time and returns context cancelled while sending message the rest, with no way for the caller to distinguish the two.

Suggested fix: test ctx.Err() before the select, or give the send case priority with a nested select { case ch <- msg: default: } before falling through to the three-way select.

select {
case s.writeChannel <- msg:
return nil
case <-s.closedChan:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium: the unblock-the-peer-on-close fix is one-directional — a peer blocked in send is never released.

send selects only on s.closedChan, which is its own close signal. When left.Close() runs it closes lr (right's read channel), but nothing touches rl, so a right.Send blocked on a full rl keeps waiting.

Verified: after filling the 10-slot buffer and blocking an 11th send, calling right.Close() on the peer left the send blocked past a 500 ms deadline. It only returns when the sender's own ctx is cancelled or its own session is closed — i.e. the same stall class #2285 is about, just on the send side.

Suggested fix: share a close signal between the two localSessions, or also select on the read channel of the writing side.

s.mu.RLock()
defer s.mu.RUnlock()

if s.closed {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium: Receive() still returns a nil channel after the local Close(), so the closing side still burns the full 10 s DefaultReceiveTimeout.

A nil channel blocks forever, so ReceiveRawWithTimeout (token/services/utils/session/session.go:83) falls through to case <-timeout.C and returns ErrTimeout only after DefaultReceiveTimeout. This PR now has a closedChan that would make it instant, but doesn't use it here — returning s.readChannel (which the peer closes) or a pre-closed channel would fail fast with ErrNilMessage.

This is pre-existing behaviour, so not a regression, but it is exactly the 10-second stall the PR is meant to remove, and as written the two sides of the same fake session end up with asymmetric close semantics.

return nil
case <-s.closedChan:
return errors.New("session is closed")
case <-ctx.Done():

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low: Send(nil, payload) now panics with a nil pointer dereference on ctx.Done(). Before this PR a nil context was only stored in msg.Ctx and was harmless.

No in-repo caller passes nil today, but NewLocalBidirectionalChannel / LeftSession / RightSession are exported, so this is a silent contract change. An if ctx == nil { ctx = context.Background() } guard — or dropping the ctx case per the fix suggested above — avoids it.

Comment thread token/services/ttx/session.go Outdated
s.closed = true
s.info.Closed = true
close(s.closedChan)
if s.inFlight == 0 && !s.writeClosed {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low: the !s.writeClosed term is dead here. writeClosed is only ever set inside send's deferred block, which requires s.closed == true; Close() has already returned early above in that case, so writeClosed is always false at this point.

Harmless, but it obscures the invariant — worth either dropping the term or adding a short comment stating why it cannot be true.

@SurbhiAgarwal1
SurbhiAgarwal1 force-pushed the fix/2285-local-session-data-race branch from ef0d463 to aff2cef Compare September 16, 2026 16:06
Fixes LFDT-Panurus#2285

Signed-off-by: Surbhi Agarwal <SurbhiAgarwal1@users.noreply.github.com>
@SurbhiAgarwal1
SurbhiAgarwal1 force-pushed the fix/2285-local-session-data-race branch from aff2cef to c946abe Compare September 16, 2026 16:16
@AkramBitar

Copy link
Copy Markdown
Contributor

@SurbhiAgarwal1

Could you please have a look at the failures?

Regards,
Akram

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

localSession: data race on info.Closed and peers left blocked on Receive after Close

3 participants