[merlimat]

Motivation

#27337 lets integration test classes that ask for the same cluster share one. With it, #27338 and #27339, the
Messaging and Cli jobs get a few minutes shorter. CI - System - Pulsar Connectors - Process then becomes, with
CI - Unit - Brokers - Broker Group 1, the last test job of a PR run: it took 14.2, 15.5 and 10.6 min in the PR runs
41555633313, 41674571977 and 41675073847 of 3 Nov.
It also starts 1-3.5 min after the Integration jobs, as it waits for the pulsar-test-latest-version image.

On CI the Process job runs pulsar-process.xml with the function group. Three classes run:
PulsarFunctionsGoProcessTest, PulsarFunctionsJavaProcessTest and PulsarFunctionsPythonProcessTest. Each one
starts a cluster of the default spec plus two function workers, and stops them when it ends. On CI, 46-68 s pass
between the last test of one class and the first test of the next. The three classes of CI - System - Schema
(4.1-6.0 min) also ask for the default cluster, and lose 28-41 s between classes.

Modifications

Test code only.

• The three classes of pulsar-process.xml that run on CI share their cluster (isClusterShared()).
  • The function workers stay per class: PulsarFunctionsTestBase stops a class's workers in its teardown before the
    cluster is left running, and the next class starts two workers of its own. A class never finds another class's
    workers or what still runs on them, such as the localrun process that testFunctionLocalRun starts in a worker
    container.
  • testSerdeFunction deletes its function at the end, as every other test of these classes does (deleteFunction
    becomes protected). If it were left registered, the next class's leader would keep it assigned to the stopped
    worker. Its membership monitor would then move it onto one of the leader's own workers once it has been on a dead
    worker for longer than rescheduleTimeoutMs (60 s, checked every failureCheckFreqMs, 30 s).
• SchemaTest, JodaTimeTest and PythonSchemaTest share their cluster.

Each of these classes asks for the default cluster. It creates namespaces and topics with generated names, or with
names that no other class of its suite uses (SchemaTest's public/default/tp-ref, which it deletes, and its
<schema><schema> topics). None of them stops, pauses or reconfigures a container, or changes a broker or
cluster-wide setting.

In the other suites that CI runs, no two classes could share a cluster:

• pulsar-thread.xml: only PulsarFunctionsJavaThreadTest runs on CI. The job selects the function, source and
  sink groups, and PulsarWorkerRebalanceDrainThreadTest's tests are in none of them. The same goes for
  PulsarWorkerRebalanceDrainProcessTest in pulsar-process.xml, which also adds and stops workers.
• pulsar-function.xml: standalone containers (PulsarStandaloneTestSuite), not PulsarClusterTestBase.
• pulsar-plugin.xml: each class configures a broker plugin of its own.
• pulsar-transaction.xml: TcMetadataDiscoveryTest and V5ScalableTopicTransactionTest stop a broker.
  TransactionTest configures the brokers for the legacy coordinator in beforeStartCluster, and
  V5ScalableTopicTransactionOnOxiaTest runs on Oxia.
• pulsar-loadbalance.xml, pulsar-metrics.xml, tiered-*.xml and pulsar-k8s.xml have one class each.
  pulsar-standalone.xml, pulsar-upgrade.xml and pulsar-backwards-compatibility.xml run standalone, upgraded or
  per-version clusters, and in pulsar-websocket.xml and pulsar-tls.xml each class asks for a spec of its own.

Verifying this change

• Make sure that the change passes the CI checks.

This change is covered by the existing integration tests: every class of pulsar-process.xml (with
-PtestGroups=function, as CI runs it) and of pulsar-schema.xml passes.

Local runs, on a 10-CPU Docker VM with an image built from apache/master. Each suite ran in a Gradle run of its own,
test retries on as on CI, and before and after runs alternated, as the host was busy (load average 6-24). The time
from the end of one class to the start of the next class's first test covers the teardown, which no class time shows,
plus the next class's setup:

Between	#27337	this PR
PulsarFunctionsGoProcessTest -> PulsarFunctionsJavaProcessTest	23.7 s, 28.9 s	9.6 s, 11.9 s
PulsarFunctionsJavaProcessTest -> PulsarFunctionsPythonProcessTest	30.6 s, 27.4 s	8.3 s, 10.8 s
SchemaTest -> JodaTimeTest	16.1 s, 17.0 s	0.1 s, 0.1 s
JodaTimeTest -> PythonSchemaTest	19.3 s, 18.8 s	0.1 s, 0.1 s

For the function test classes, what remains is stopping a class's two workers and starting the next class's two
(their containers start 2.6-4.1 s apart, docker events). Test task times, which the busy host makes noisier:

Suite	#27337	this PR
pulsar-process.xml	388.3 s, 392.1 s	331.2 s, 344.7 s
pulsar-schema.xml	94.0 s, 94.2 s	50.0 s, 59.3 s

On CI, a cluster's setup takes 26-32 s (Develocity's class own time of the Schema classes, 2-3 Nov), and its stop
about 7 s. Each of the two jobs loses two setups and two stops, so the estimate is about 1 min less for each.

Checked as well, with mutations that are not committed:

• testSerdeFunction without the delete, followed by the Python class: for the 84 s that the Python class's workers
  ran, their leader listed the serde function's instance on the Java class's stopped worker ("alive" : false). With
  the delete, the Python class's workers only replay the function's deletion.
• setupFunctionWorkers() throwing in the Go class: the skipped tests mark the setup as failed, so the Go class's
  cluster and its retry's cluster are both stopped, and the Python class starts a cluster of its own.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

[codelipenghui]

Code Review approval for commit 5749c85.

Summary:

No actionable findings identified within the reviewed scope. The scoped cluster reuse preserves per-class function workers and removes the lingering serde registration. Compilation succeeded; runtime validation was blocked by Docker network address-pool exhaustion.