perf(metrics): count pool stats only while metrics are enabled [W1-07]

CHANGELOG.md

@@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Breaking Changes
+- **Pool statistics [W1-07]**: Get/Put operations are counted only while metrics are enabled. While disabled, `metrics.GetPoolStats` returns zero values for the three standard pools and omits custom pools. Counts collected earlier remain available when metrics are enabled again. The `gosqlx stats` command never enables metrics, so a fresh process prints zeros (as it effectively did before); call `metrics.Enable()` before the work you want counted.
+  - **Migration:** Call `metrics.Enable()` before the workload to be measured, for example at process start. Tests asserting pool balance through `GetPoolStats` deltas should enable metrics first and restore the previous state in cleanup. Counts are not retroactive.
+
 ### Added
 - **Benchmark and perf-gate infrastructure (performance overhaul, wave 0)**:
   pull requests are now judged by interleaved A/B measurements instead of
@@ -101,6 +105,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
     benchmarks that publish diagnostics no longer time a growing buffer.
 
 ### Performance
+- **Pools [W1-07]**: Skip atomic pool counter updates while metrics are disabled. Measured on the amd64 layout-randomized perf gate, `PoolGetPut/serial` is runner-dependent: -5.41% on an AMD EPYC 9V74 (Zen4, p<0.001), +3.99% and +3.93% (p<0.001, reported as REGRESSION) on an EPYC 7763 (Zen3) for an earlier head, and -37.06% (p<0.001) on an Intel Xeon 6973P-C for the final head, so the effect is CPU-dependent and no consistent improvement is claimed. The planned gain was <= -20%. allocs/op and B/op unchanged; the benefit is expected mainly in contended/parallel use, which the gate does not measure.
 - **Linter**: rule L007 (keyword case consistency) scans words without allocating (substring words, ASCII upper-casing into a stack buffer before the keyword lookup). Output is unchanged. On the canonical benchmarks (Apple M4, local), `BenchmarkLintRule/L007` allocates about 98% fewer objects (72565 to 1194 allocs/op) and about 97% fewer bytes, and `BenchmarkLintRW` allocates about 12.5% fewer objects. [W6-03]
 - **gosqlx**: errors returned by `Parse`, `Validate`, `Lint` and the tree helpers now wrap the sentinel and the underlying error in an unexported `wrapErr` that builds its message lazily, instead of `fmt.Errorf("%w: %w", ...)` on every failure. Error text, `errors.Is` and `errors.As` behaviour are unchanged; only the unexported concrete error type differs. The saving applies to callers that only check `errors.Is`/`errors.As` or drop the message: when the message is built (`err.Error()`, as the LSP and CLI do) the gain is about 2 allocs/op and 3-4% B/op per error (syntax_mid 21 -> 19 allocs). Because the message is now built on demand, `Error()` reflects any later mutation of the inner `*errors.Error` reached through `errors.As`; previously the text was fixed at creation. [W1-03]
 - **Linter L010**: scans bytes in a single pass instead of rebuilding
@@ -121,7 +126,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Tokenizer**: token Start/End locations are now computed from a cached current-line window (a range check and a subtraction) instead of two calls to the non-inlinable position lookup per token; out-of-window offsets take the existing lookup. Token positions are identical, including after a UTF-8 BOM and across pooled-tokenizer reuse (guarded by an oracle test). [W2-02]
 - **Tokenizer**: `@param` and SQL Server `@@var` token values are now slices of the input instead of `"@"+name` concatenations, removing one string allocation per parameter (SQL Server real-world allocs/op -90% to -100%, MariaDB/MySQL/Snowflake -50% to -67% on the large cells). Names that `readIdentifier` treats specially (`U&` escapes, a single letter before a quote, compound-keyword starts such as `@group by`, non-ASCII names) keep the previous path, so token types, values and spans are identical (pinned by a golden over 9 dialects). [W2-03]
 - **Tokenizer**: tokens are now built in place. The scanner and its token readers write each token's fields directly into its slot of the output slice instead of returning a 40-byte `models.Token` by value that was then copied into a 72-byte `models.TokenWithSpan` and appended. On amd64 those copies reloaded just-spilled structs with 16-byte loads that cannot be store-forwarded. Token output, spans and errors are identical (token goldens unchanged); no API or behaviour change. [W2-10]
-- **Pools**: the tokenizer, parser and AST pools now count Get/Put through precreated, cache-line padded atomic counters (`internal/poolstats`) instead of an RWMutex-guarded map keyed by pool name, removing a lock, map lookup and string hash from every pool operation. `metrics.GetPoolStats` and `metrics.ResetPoolStats` behave exactly as before; named custom pools keep the map path. [W1-01]
+- **Pools**: the tokenizer, parser and AST pools now count Get/Put through precreated, cache-line padded atomic counters (`internal/poolstats`) instead of an RWMutex-guarded map keyed by pool name, removing a lock, map lookup and string hash from every pool operation. `metrics.GetPoolStats` and `metrics.ResetPoolStats` retain their previous behaviour until W1-07 (see Breaking Changes); named custom pools keep the map path. [W1-01]
 - **Parser**: token preprocessing now takes a type-gated fast path. Only identifier, keyword, aggregate and compound tokens take the normalisation path; every other token is appended with a single copy. Output is identical (guarded by an old-versus-new differential test); no API or behaviour change.
 - **Parser**: the parser's current token is now a pointer into its token slice (with a per-parser zero EOF sentinel) instead of a 72-byte copy made on every `advance`, and `peekToken` returns a pointer, which removes most GC write-barrier work and struct copying from the parse loop. Parse results, errors and positions are byte-identical (behaviour goldens plus a recovery-path differential); no API change. A zero-value `&Parser{}` keeps working. [W3-02]
 - **Fingerprint, formatter, transform**: `fingerprint.Normalize`/`Fingerprint`, `Formatter.Format` and `transform.ParseSQL`/`ParseSQLWithDialect` tokenize through the internal pooled token scratch instead of the public `TokenizeString`. `Normalize` allocates 42-70% fewer bytes/op and `transform.ParseSQL` 49-78% fewer, with 2-3 fewer allocs/op and no output change. [W5-01]

internal/metricsgate/metricsgate.go

@@ -0,0 +1,32 @@
+// Copyright 2026 GoSQLX Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Package metricsgate holds the shared flag controlling metrics collection.
+package metricsgate
+
+import "sync/atomic"
+
+const cacheLine = 64
+
+var gate struct {
+	_  [cacheLine]byte
+	on atomic.Bool
+	_  [cacheLine]byte
+}
+
+// Enabled reports whether metrics collection is enabled.
+func Enabled() bool { return gate.on.Load() }
+
+// Set enables or disables metrics collection.
+func Set(on bool) { gate.on.Store(on) }

internal/metricsgate/metricsgate_test.go

@@ -0,0 +1,75 @@
+// Copyright 2026 GoSQLX Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package metricsgate
+
+import (
+	"sync"
+	"testing"
+	"unsafe"
+)
+
+func TestDefaultAndToggle(t *testing.T) {
+	if Enabled() {
+		t.Fatal("gate must default to disabled in a fresh test binary")
+	}
+	t.Cleanup(func() { Set(false) })
+	Set(true)
+	if !Enabled() {
+		t.Fatal("Set(true) did not enable gate")
+	}
+	Set(false)
+	if Enabled() {
+		t.Fatal("Set(false) did not disable gate")
+	}
+}
+
+func TestGatePadding(t *testing.T) {
+	size := unsafe.Sizeof(gate)
+	offset := unsafe.Offsetof(gate.on)
+	if size < 64+4+64 {
+		t.Errorf("gate size = %d, want at least 132", size)
+	}
+	if offset < 64 {
+		t.Errorf("flag offset = %d, want at least 64", offset)
+	}
+	if trailing := size - offset - unsafe.Sizeof(gate.on); trailing < 64 {
+		t.Errorf("trailing padding = %d, want at least 64", trailing)
+	}
+}
+
+func TestGateConcurrentToggle(t *testing.T) {
+	was := Enabled()
+	t.Cleanup(func() { Set(was) })
+	var wg sync.WaitGroup
+	for i := 0; i < 8; i++ {
+		wg.Add(1)
+		go func() {
+			defer wg.Done()
+			for j := 0; j < 5000; j++ {
+				_ = Enabled()
+			}
+		}()
+	}
+	for i := 0; i < 2; i++ {
+		wg.Add(1)
+		go func() {
+			defer wg.Done()
+			for j := 0; j < 5000; j++ {
+				Set(j%2 == 0)
+			}
+		}()
+	}
+	wg.Wait()
+}

internal/poolstats/poolstats.go

@@ -13,13 +13,17 @@
 // limitations under the License.
 
 // Package poolstats holds the precreated, cache-line padded counters for the
-// three standard pools (tokenizer, parser, ast). The typed API is a single
-// atomic add and inlines at every call site, with no lock, map lookup or
-// string hash. pkg/metrics exposes the counters through GetPoolStats and
-// ResetPoolStats.
+// three standard pools (tokenizer, parser, ast). Counters count only while
+// metrics are enabled. The typed API inlines at every call site, with no lock,
+// map lookup or string hash. pkg/metrics exposes the counters through
+// GetPoolStats and ResetPoolStats.
 package poolstats
 
-import "sync/atomic"
+import (
+	"sync/atomic"
+
+	"github.com/ajitpratap0/GoSQLX/internal/metricsgate"
+)
 
 // Counter is a Get/Put counter pair padded to its own 64-byte cache line so
 // the three standard pools never false-share.
@@ -29,11 +33,23 @@ type Counter struct {
 	_    [48]byte
 }
 
-// Get records one Get() and returns the new total.
-func (c *Counter) Get() int64 { return c.gets.Add(1) }
+// Get records one Get() only while metrics are enabled and returns the new total.
+// It returns 0 without recording anything when metrics are disabled.
+func (c *Counter) Get() int64 {
+	if !metricsgate.Enabled() {
+		return 0
+	}
+	return c.gets.Add(1)
+}
 
-// Put records one Put() and returns the new total.
-func (c *Counter) Put() int64 { return c.puts.Add(1) }
+// Put records one Put() only while metrics are enabled and returns the new total.
+// It returns 0 without recording anything when metrics are disabled.
+func (c *Counter) Put() int64 {
+	if !metricsgate.Enabled() {
+		return 0
+	}
+	return c.puts.Add(1)
+}
 
 // Load returns the current gets and puts.
 func (c *Counter) Load() (gets, puts int64) { return c.gets.Load(), c.puts.Load() }

internal/poolstats/poolstats_test.go

@@ -18,6 +18,8 @@ import (
 	"sync"
 	"testing"
 	"unsafe"
+
+	"github.com/ajitpratap0/GoSQLX/internal/metricsgate"
 )
 
 func TestCounterPaddedToCacheLine(t *testing.T) {
@@ -27,6 +29,7 @@ func TestCounterPaddedToCacheLine(t *testing.T) {
 }
 
 func TestCounterGetPutReset(t *testing.T) {
+	enablePoolStats(t)
 	var c Counter
 	if n := c.Get(); n != 1 {
 		t.Fatalf("first Get = %d, want 1", n)
@@ -57,6 +60,7 @@ func TestByName(t *testing.T) {
 }
 
 func TestCounterConcurrent(t *testing.T) {
+	enablePoolStats(t)
 	var c Counter
 	const goroutines, per = 8, 5000
 	var wg sync.WaitGroup
@@ -75,3 +79,69 @@ func TestCounterConcurrent(t *testing.T) {
 		t.Fatalf("Load = %d/%d, want %d each", g, p, goroutines*per)
 	}
 }
+
+func enablePoolStats(t testing.TB) {
+	was := metricsgate.Enabled()
+	metricsgate.Set(true)
+	t.Cleanup(func() { metricsgate.Set(was) })
+}
+
+func TestCounterDisabledIsNoop(t *testing.T) {
+	enablePoolStats(t)
+	metricsgate.Set(false)
+	var c Counter
+	if got := c.Get(); got != 0 {
+		t.Fatalf("disabled Get = %d, want 0", got)
+	}
+	if got := c.Put(); got != 0 {
+		t.Fatalf("disabled Put = %d, want 0", got)
+	}
+	if g, p := c.Load(); g != 0 || p != 0 {
+		t.Fatalf("disabled Load = %d/%d, want 0/0", g, p)
+	}
+	metricsgate.Set(true)
+	if got := c.Get(); got != 1 {
+		t.Fatalf("enabled Get = %d, want 1", got)
+	}
+	if got := c.Put(); got != 1 {
+		t.Fatalf("enabled Put = %d, want 1", got)
+	}
+	metricsgate.Set(false)
+	if got := c.Get(); got != 0 {
+		t.Fatalf("disabled Get after toggle = %d, want 0", got)
+	}
+	if got := c.Put(); got != 0 {
+		t.Fatalf("disabled Put after toggle = %d, want 0", got)
+	}
+	if g, p := c.Load(); g != 1 || p != 1 {
+		t.Fatalf("retained Load = %d/%d, want 1/1", g, p)
+	}
+}
+
+func TestCounterToggleRace(t *testing.T) {
+	enablePoolStats(t)
+	var c Counter
+	var wg sync.WaitGroup
+	for i := 0; i < 8; i++ {
+		wg.Add(1)
+		go func() {
+			defer wg.Done()
+			for j := 0; j < 5000; j++ {
+				c.Get()
+				c.Put()
+			}
+		}()
+	}
+	wg.Add(1)
+	go func() {
+		defer wg.Done()
+		for j := 0; j < 5000; j++ {
+			metricsgate.Set(j%2 == 0)
+		}
+	}()
+	wg.Wait()
+	metricsgate.Set(true)
+	if g, p := c.Load(); g < 0 || p < 0 {
+		t.Fatalf("negative counts: %d/%d", g, p)
+	}
+}

pkg/gosqlx/pool_hygiene_test.go

@@ -25,11 +25,24 @@ import (
 // These tests read process-global pool counters, so they use before/after
 // deltas, never call metrics.ResetPoolStats and never run in parallel.
 
+func enablePoolStats(t testing.TB) {
+	was := metrics.IsEnabled()
+	metrics.Enable()
+	t.Cleanup(func() {
+		if was {
+			metrics.Enable()
+		} else {
+			metrics.Disable()
+		}
+	})
+}
+
 func poolDelta(before, after map[string]metrics.PoolStat, name string) (gets, puts int64) {
 	return after[name].Gets - before[name].Gets, after[name].Puts - before[name].Puts
 }
 
 func TestPoolHygiene_Balanced(t *testing.T) {
+	enablePoolStats(t)
 	const n = 50
 	tests := []struct {
 		name    string
@@ -78,6 +91,7 @@ func TestPoolHygiene_Balanced(t *testing.T) {
 }
 
 func TestParseMultiple_SuccessCallerOwns(t *testing.T) {
+	enablePoolStats(t)
 	before := metrics.GetPoolStats()
 	results, err := ParseMultiple([]string{"SELECT 1", "SELECT 2", "SELECT 3"})
 	if err != nil {
@@ -195,3 +209,43 @@ func TestAnalyze_ResultIndependentOfPool(t *testing.T) {
 		t.Errorf("Score = %d, want 90", res.Score)
 	}
 }
+
+func TestPoolStats_RealPoolsGatedEndToEnd(t *testing.T) {
+	enablePoolStats(t)
+	const n = 20
+	run := func() {
+		t.Helper()
+		for i := 0; i < n; i++ {
+			tree, err := Parse("SELECT a FROM t WHERE b = 1")
+			if err != nil {
+				t.Fatalf("Parse: %v", err)
+			}
+			ast.ReleaseAST(tree)
+		}
+	}
+
+	metrics.Disable()
+	run()
+	metrics.Enable()
+	s0 := metrics.GetPoolStats()
+	metrics.Disable()
+	run()
+	metrics.Enable()
+	s1 := metrics.GetPoolStats()
+	for _, pool := range []string{"tokenizer", "parser", "ast"} {
+		if s1[pool] != s0[pool] {
+			t.Errorf("%s counted while disabled: before=%+v after=%+v", pool, s0[pool], s1[pool])
+		}
+	}
+
+	run()
+	s2 := metrics.GetPoolStats()
+	// On main, each Parse plus ReleaseAST makes exactly one Get and one Put
+	// for each of these three pools.
+	for _, pool := range []string{"tokenizer", "parser", "ast"} {
+		gets, puts := poolDelta(s1, s2, pool)
+		if gets != n || puts != n {
+			t.Errorf("%s enabled delta: gets=%d puts=%d, want %d each", pool, gets, puts, n)
+		}
+	}
+}

pkg/metrics/doc.go

@@ -92,6 +92,12 @@
 //	statementCount := len(ast.Statements)
 //	metrics.RecordParse(duration, statementCount, err)
 //
+// # Pool statistics
+//
+// Pool Get/Put statistics count only while metrics are enabled. Call Enable before
+// the workload to measure; counts are not retroactive. GetPoolStats returns only
+// the three standard pools with zero values while metrics are disabled.
+//
 // # Object Pool Metrics
 //
 // Track pool efficiency for all pool types: