T8

BUILD.md

@@ -32,6 +32,12 @@ This will create two main files:
 * openaf.jar
 * openaf.jar.orig
 
+In an installed OpenAF directory, run `./oaf --repack` after replacing the JAR or
+changing the JDK to refresh the `.shared.oaf` archive used by generated launchers.
+Repacking trains and validates archive coverage while retaining the existing
+class path. See [Shared archive coverage](docs/cds.md) for fallback behavior and
+verification commands.
+
 ### Testing
 
 To test the project, you can use the following command:

LICENSES.txt

@@ -3611,12 +3611,14 @@ Apache License
 
 -----------------------
 Third-party name      : JLine
-Version               : 2.14.6
+Version               : 4.4.3 (legacy data adapters from 2.14.6)
 Changed from original : Yes
-Location in openaf.jar: lib/jline-2.14.6.jar
+Location in openaf.jar: lib/jline-{reader,terminal,terminal-jni,native}-4.4.3.jar
+Legacy compatibility data classes: src/jline (adapted from JLine 2.14.6; BSD license retained)
 License               : 
 
-Copyright (c) 2002-2016, the original author or authors.
+Copyright (c) 2002-2023, the original author or authors.
+Legacy data adapters: Copyright (c) 2002-2016, the original author or authors.
 All rights reserved.
 
 http://www.opensource.org/licenses/bsd-license.php

buildos.js

@@ -404,6 +404,7 @@ try {
 		"ow.oJob": OPENAF_BUILD_HOME + "/js/owrap.oJob.js",
 		"ow.sec": OPENAF_BUILD_HOME + "/js/owrap.sec.js",
 		"ow.metrics": OPENAF_BUILD_HOME + "/js/owrap.metrics.js",
+        "ow.instrumentation": OPENAF_BUILD_HOME + "/js/owrap.instrumentation.js",
 		"ow.python": OPENAF_BUILD_HOME + "/js/owrap.python.js",
 		"ow.debug": OPENAF_BUILD_HOME + "/js/owrap.debug.js",
 		"ow.obook": OPENAF_BUILD_HOME + "/js/owrap.oBook.js",

docs/cds.md

@@ -0,0 +1,55 @@
+# Shared archive coverage
+
+The generated OpenAF launchers already use `.shared.oaf` through
+`-XX:SharedArchiveFile`. Run `./oaf --repack` after updating the JAR or JDK to
+regenerate it with the launcher's JVM settings, including compact object headers
+and module options. Archive generation does not change the JAR's class path.
+
+Repacking now executes a local OpenAF/oJob warmup with
+`-XX:DumpLoadedClassList`, then creates and validates an archive. It selects the
+first usable coverage level:
+
+1. **Application**: classes recorded by the warmup, including OpenAF classes.
+2. **Platform**: recorded JDK-module classes plus the JDK's standard class list.
+3. **Default**: ordinary JVM archive generation if training or expanded coverage
+   is unavailable.
+
+The current repacked JAR has `Class-Path: .`. HotSpot rejects application archives
+when that entry points to a nonempty directory. The platform fallback improves
+coverage while preserving installation-directory Java classes and resources.
+Removing the manifest entry merely to enable application sharing would change
+existing behavior and is not part of this change.
+
+The warmup loads libraries and runs a small in-memory job. Training subprocesses
+set `openaf.cds.training=true` to skip both user and embedded startup profiles,
+and use default OpenAF flags. Ordinary executions continue loading profiles.
+The helper obtains effective VM options from the generated launcher; when no
+launcher exists yet, it uses the current JVM's settings. Repack again after
+generating launchers if those settings differ.
+
+Archive creation alone does not establish that it can be used. Each candidate is
+checked with `-Xshare:on -XX:+PrintSharedArchiveAndExit` against the actual JAR and
+VM settings. Only a validated candidate replaces `.shared.oaf`, by an atomic
+move. If every candidate fails, the previous archive is preserved and repacking
+reports the failure. Under the generated launcher's ordinary default sharing
+mode, a missing or incompatible archive permits normal JVM startup; explicitly
+requesting `-Xshare:on` instead makes an unusable archive fatal.
+
+To inspect the archive through a generated Unix launcher:
+
+```sh
+OAF_JARGS='-XX:+PrintSharedArchiveAndExit' ./oaf -c '1'
+```
+
+To inspect actual sharing during an oJob execution:
+
+```sh
+OAF_JARGS='-Xlog:class+load=info:file=cds-classes.log' ./ojob job.yaml
+```
+
+On macOS/JDK 26.0.2.1, the initial generated-launcher comparison increased the
+archive from 1,543 to 2,003 classes. Six retained samples per variant measured
+median first-instruction times of 458.5 → 446.5 ms for OpenAF and 951 → 942 ms for
+a minimal oJob. These are small local gains, not the roughly 35% application-CDS
+result obtained with an experimental manifest modification. Other JDK/OS
+combinations need separate measurement.

docs/cli.md

@@ -47,9 +47,9 @@ With no arguments, OpenAF launches the interactive GUI if a desktop is available
 
 | Flag | Argument | Description |
 |:-----|:---------|:-------------|
-| `--install` | – | Generate wrapper scripts (`oaf`, `opack`, `ojob`, `oafp`, `pyoaf`, etc.) in the current directory. |
+| `--install [args=<jvm-args>] [--noopacks]` | see description | Generate wrapper scripts (`oaf`, `opack`, `ojob`, `oafp`, `pyoaf`, etc.) in the current directory. `args=<jvm-args>` embeds extra JVM arguments (e.g. `-Xmx2g`) into every generated launcher and saves them to `.openaf-javaargs`. `--noopacks` skips auto-installing any `.opack` files found in the current directory during install. |
 | `--check` | – | Check if this is the latest released version. |
-| `--update` | – | Update OpenAF to the latest version. |
+| `--update [--force]` | – | Update OpenAF to the latest version (downloads a new `openaf.jar`, backs up the old one as `openaf.jar.old`, then re-`--repack`s). `--force` re-downloads even if already on the latest version. Disabled when the `noHomeComms` flag is set. See [Update process](#update-process) below. |
 | `--repack` | – | Repack `openaf.jar` for faster startup. |
 | `--console` | – | Launch the interactive OpenAF REPL console. |
 
@@ -61,7 +61,7 @@ With no arguments, OpenAF launches the interactive GUI if a desktop is available
 | `--opack` | – | Execute the opack package-manager CLI. Use with `-e` to pass sub-commands. |
 | `--py <file>` | script path | Run a Python script with OpenAF bridge support (loads `pyoaf.js`). |
 | `--oafpy` | – | Emit the `oaf.py` bridge module for use from standalone Python scripts. See [python.md](./python.md). |
-| `--sb <file>` | file path | Generate or prepend an OpenAF/oJob shebang to a JS or YAML file. |
+| `--sb <file>` | file path | Generate or prepend an OpenAF/oJob/oafp shebang line, chosen by the file's extension. See [Shebang scripts](#shebang-scripts) below. |
 | `--bashcompletion <arg>` | arg | Generate bash completion script. |
 | `--zshcompletion <arg>` | arg | Generate zsh completion script. |
 
@@ -76,10 +76,83 @@ After running `--install`, the following wrapper scripts are created in the inst
 | `opack` | `openaf --opack -e "$ARGS"` | oPack package manager CLI. |
 | `oafp` | `openaf -c "load(getOpenAFJar()+'::js/oafp.js')" -e "$ARGS"` | OpenAF data processor. See [oafp.md](./oafp.md). |
 | `pyoaf` | `openaf --py -e "$ARGS"` | Python runner. See [python.md](./python.md). |
-| `oafc` / `openaf-console` | `openaf --console` | Interactive REPL. |
-| `odoc` | `openaf -c "load(getOpenAFJar()+'::js/odoc.js')" -e "$ARGS"` | Documentation CLI. See [odoc.md](./odoc.md). |
-| `update` | `openaf --update` | Update shortcut. |
-| `oaf-sb` / `openaf-sb` / `ojob-sb` / `oafp-sb` | `openaf --sb` | Shebang generators. |
+| `oafc` / `openaf-console` | `openaf --console` | Interactive REPL. See [console.md](./console.md). |
+| `odoc` | `openaf -c "load(getOpenAFJar()+'::js/odoc.js')" -e "$ARGS"` | Documentation CLI. See [odoc.md](./odoc.md). Note: unlike the other rows in this table, `odoc` is not generated by `--install`/`genScripts.js` — it ships via the odoc tooling itself, so it may need to be aliased manually if not already on your `PATH`. |
+| `update` / `update.sh` / `update.bat` | `openaf --update` (plus a JRE refresh step) | Update shortcut. See [Update process](#update-process) below. |
+| `oaf-sb` / `openaf-sb` / `ojob-sb` / `oafp-sb` | `openaf -f "$SCRIPT" -e "$ARGS"` (resp. `--ojob`/oafp variants) | Shebang-line interpreters, invoked by the OS when a script starting with `#!/usr/bin/env oaf-sb` (etc.) is executed directly. See [Shebang scripts](#shebang-scripts) below. |
+
+## Shebang scripts
+
+`--sb <file>` turns a plain file into a self-executing script by prepending an interpreter line,
+similar to `#!/usr/bin/env python3` for Python. It picks the right interpreter from the file
+extension:
+
+| File type | Shebang line added | Runtime interpreter |
+|:----------|:--------------------|:---------------------|
+| `.js` | `#!/usr/bin/env <install-dir>/oaf-sb` | `openaf -f "$SCRIPT" -e "$ARGS"` |
+| `.yaml` / `.yml` / `.json` | `#!/usr/bin/env <install-dir>/ojob-sb` | `openaf --ojob -e "$SCRIPT $ARGS"` |
+| anything else (including no extension) | `#!/usr/bin/env -S <install-dir>/oafp-sb` | `openaf -c "load(...oafp.js)" -e "_shebang=true $OAFP_ARGS $ARGS"` |
+
+(As of this writing, the extension check only applied when prepending to an *existing* file —
+generating a brand-new `.yaml`/`.json`/non-`.js` file via `--sb` incorrectly fell back to the
+`.js`/`oaf-sb` template. This has been fixed in `js/genSB.js` so the extension is honored whether
+or not the file already exists.)
+
+If the target file doesn't exist, `--sb` creates it with just the shebang line (plus, for `.js`
+files, a starter `var params = processExpr(" ");`). If it exists and doesn't already start with
+`#!`, the shebang line is prepended to its current contents. Running `--sb` again on a file that
+already has a shebang is a no-op (it logs a warning and leaves the file untouched).
+
+```bash
+# Generate an OpenAF (.js) shebang script and make it executable
+openaf --sb hello.js
+chmod +x hello.js
+./hello.js abc=123 xyz=aaa
+```
+
+Inside `hello.js`, `oaf-sb` sets `__expr` from the trailing command-line tokens, so
+`var params = processExpr(" ");` yields `{ abc: "123", xyz: "aaa" }` — the same mechanism `ojob`
+and `oafp` use to parse `key=value` arguments.
+
+```bash
+# Generate an oJob shebang file and run it directly
+openaf --sb myjob.yaml
+chmod +x myjob.yaml
+./myjob.yaml env=prod
+```
+
+Once generated, these files no longer need the `openaf`/`ojob`/`oafp` command at all — the shebang
+line resolves `oaf-sb`/`ojob-sb`/`oafp-sb` from the same directory as the other generated wrapper
+scripts (via `/usr/bin/env`), so the install directory must be on `PATH` (or referenced with an
+absolute path) for this to work.
+
+## Update process
+
+`--update` (and the generated `update`/`update.sh`/`update.bat` wrapper) checks the OpenAF home
+servers for a newer release, and if one is found (or `--force` is given):
+
+1. Backs up the *current* `openaf.jar` to `openaf.jar.old` next to the install.
+2. Downloads the new release in two forms: a plain build and a pre-`--repack`ed one.
+3. Installs the pre-repacked build as the new `openaf.jar` (keeping the plain build as
+   `openaf.jar.orig`, which a later `--repack` needs if modules are ever added/excluded), then
+   restarts with `--repack` for fast startup.
+
+The `update.sh`/`update.bat` wrapper additionally refreshes the bundled JRE before touching the
+jar: it runs `./ojob ojob.io/oaf/javaUpdate`, and if that produces a `jre.tgz` (Unix) or `jre.zip`
+(Windows), the old `jre/` directory is moved to `jre.old` and the archive is extracted as the new
+`jre/`. This step — and `--update`/`--check` in general — is skipped entirely when the `noHomeComms`
+flag is set (e.g. restricted/offline installs), which is also why `--install` won't generate
+`update.sh`/`update.bat` at all in that mode.
+
+```bash
+# Update in place
+update
+# or
+openaf --update
+
+# Force re-download even if already current
+openaf --update --force
+```
 
 ## Common usage examples
 
@@ -118,4 +191,5 @@ openaf -v
 - [oafp.md](./oafp.md) — OpenAF data processor.
 - [opacks.md](./opacks.md) — oPack package manager.
 - [python.md](./python.md) — Python integration.
-- [odoc.md](./odoc.md) — documentation engine.
+- [odoc.md](./odoc.md) — documentation engine.
+- [console.md](./console.md) — interactive REPL commands, aliases, and profile/history files.

docs/console.md

@@ -0,0 +1,99 @@
+# OpenAF Interactive Console (oafc / openaf-console)
+
+[Index](./index.md) | [CLI Reference](./cli.md) | [OpenAF Reference](./openaf.md) | [Flags](./openaf-flags.md)
+
+The interactive console is OpenAF's REPL: a line editor (with history, completion and multi-line
+input) on top of the same JavaScript engine used to run scripts. Anything that isn't a built-in
+console command is evaluated as OpenAF/JavaScript and, unless output has been turned off, the
+result is printed.
+
+## Starting the console
+
+```bash
+openaf --console      # raw flag
+oafc                   # generated wrapper (recommended)
+openaf-console         # generated wrapper (same as oafc)
+```
+
+All three are equivalent; `oafc`/`openaf-console` are thin wrapper scripts created by
+`--install` (see [cli.md](./cli.md#generated-wrapper-commands)) that just run
+`openaf --console "$@"`.
+
+On startup the console:
+- Loads `.openaf-console_profile` from the user's home/OpenAF directory, if present, and runs its
+  contents as console commands (one per line) — useful for predefining aliases or DB connections.
+- Loads/saves command history to `.openaf-console_history` in the same directory.
+- Prints a version banner and, unless `noHomeComms` is set, checks for a newer OpenAF release.
+
+## Basic usage
+
+```
+$ oafc
+openaf:1> print("Hello World!");
+Hello World!
+openaf:2> 1 + 1
+2
+openaf:3> var db = new DB("org.h2.Driver", "jdbc:h2:mem:", "sa", "")
+openaf:4> sql db select 1 as a, 2 as b
+```
+
+- Expressions that evaluate to a value (not just statements) have their result printed automatically.
+- Multi-line input is supported by default (`multi` command toggles it) — an unbalanced brace/paren
+  keeps the prompt open for the next line.
+- `Ctrl-C` / `Ctrl-D` behave as usual for exiting or interrupting.
+
+## Built-in commands
+
+These are reserved words handled directly by the console (anything else is treated as a script
+command). Run `help` with no arguments inside the console to see this list live; `help <term>`
+searches the same offline help database used by `<odoc>`-documented functions (see
+[odoc.md](./odoc.md)).
+
+| Command | Description | Example |
+|:--------|:-------------|:--------|
+| `help` | Show the help screen, or look up a term in the offline help database. | `help`, `help AF`, `help scope` |
+| `exit` | Exit the console. | `exit` |
+| `time` | Toggle timing of every command executed (default off). | `time` |
+| `output` | Toggle printing of command results (default on). | `output` |
+| `beautify` | Toggle beautified/pretty-printed output (default on). | `beautify` |
+| `color` | Toggle ANSI colorized output (default on when the terminal supports ANSI); `color on`/`color off` set it explicitly. | `color`, `color off` |
+| `desc` | Describe the available methods/properties of a class. | `desc AF` |
+| `scope` | List the current JS scope, optionally filtered by a regexp. | `scope sha` |
+| `alias` | Create (or list) an alias for a console command line. With no argument, lists current aliases. | `alias ola=print("hi");` |
+| `watch` | With no argument, reports whether watch is active. `watch on` / `watch off` toggle it without changing the expression. `watch <expr>` re-evaluates `<expr>` before every prompt. `watch <N> <expr>` live-refreshes `<expr>`'s output every `N` seconds until `q` is pressed. | `watch new Date();`, `watch 5 ow.server.getThreadsInfo()` |
+| `pause` | Toggle pausing long output at the terminal height (default on). | `pause` |
+| `table <expr>` | Evaluate `<expr>` and render an array-of-maps result as an ASCII table. | `table listFiles(".")` |
+| `tree` | With no argument, reports whether persistent tree rendering is active (default on). `tree on` / `tree off` switch all subsequent map/array output between tree and flat table view. `tree <expr>` renders `<expr>`'s result as a tree once, without changing the persistent mode. | `tree off`, `tree myObj` |
+| `view <expr>` | Evaluate `<expr>` and render a map/array result as a tree (default) or flat table, depending on the `tree` setting. With no argument (or `on`/`off`), reports/toggles a separate persistent "view" mode instead. | `view myObj` |
+| `sql <db> <stmt>` | Run a `SELECT` over a `DB` object variable and print a table. | `sql db select * from t` |
+| `dsql <db> <stmt>` | Describe the columns a query would produce, without fetching rows. | `dsql db select * from t` |
+| `esql <db> <stmt>` | Execute a non-SELECT statement (INSERT/UPDATE/DDL) over a `DB` object. | `esql db update t set a=1` |
+| `diff <A> with[New\|Changes\|Full] <B>` | Show differences between two objects/variables; `with` shows just the diff, `withChanges` just changed keys, `withNew` diff+changes, `withFull` the complete merged view. | `diff a with b`, `diff a withChanges b` |
+| `pin <prefix>` | Prepend `<prefix>` to every subsequent command until an empty line is entered (or `pin` is invoked again). | `pin sql db` |
+| `multi` | Toggle multi-line expression entry (default on). | `multi` |
+| `edit` | Compose a command in `$EDITOR` (or `vi`); `edit last` / `edit history` reopen a previous command. | `edit last` |
+| `clear` | Clear the screen. | `clear` |
+| `reset` | Reset the terminal (Unix consoles only), useful after a program leaves it in a bad state. | `reset` |
+| `purge` | Purge the entire command history. | `purge` |
+
+## Built-in aliases
+
+A few convenience aliases ship enabled by default (see `alias` with no arguments to list all
+currently defined aliases, including any you add):
+
+| Alias | Expands to | Example |
+|:------|:-----------|:--------|
+| `opack <args>` | `oPack(<args>)` — run the opack manager without leaving the console. | `opack list` |
+| `ojob <file> [k=v ...]` | Runs the given oJob file, passing remaining tokens as arguments (via `processExpr`). | `ojob myjob.yaml env=prod` |
+| `ojobio` | Shortcut for `oJobRunFile("ojob.io")`. | `ojobio` |
+| `sh [command]` | Drop into (or run one command in) the underlying system shell. | `sh ls -la` |
+| `encryptText` | Prompts for text and prints its encrypted form (see `askEncrypt`). | `encryptText` |
+
+Custom aliases are defined with `alias name=<console command line>` and cannot reuse a reserved
+word. Aliases persist only for the session unless recreated from `.openaf-console_profile`.
+
+## See also
+
+- [cli.md](./cli.md) — full `openaf`/`oaf` flag reference and generated wrapper scripts.
+- [odoc.md](./odoc.md) — the offline help database used by `help <term>`.
+- [openaf-flags.md](./openaf-flags.md) — `noHomeComms` and other runtime flags.

docs/highlight-js.md

@@ -0,0 +1,13 @@
+# Bundled highlight.js
+
+Updated on 2026-09-20 from 11.4.0 to 11.12.0.
+
+`js/highlight.js` is a single minified browser bundle: the upstream common build followed by these upstream language modules: prolog, powershell, pgsql, awk, handlebars, asciidoc, dos, nginx and dockerfile. This preserves all 43 previously registered languages and adds the common build's GraphQL and WebAssembly grammars (45 total). No runtime CDN requests or module downloads are required. Existing `hljs.highlightAll()` integration and the locally maintained light/dark CSS remain compatible.
+
+Source archive: https://registry.npmjs.org/@highlightjs/cdn-assets/-/cdn-assets-11.12.0.tgz
+
+Verified npm integrity: `sha512-KvOKXODaiFmId9xaq3xc5xCL66wVLUuOngDbO9B/kewbFTqdGbn2nJxNhN3H5R1cgDTVj6R8vH0zgiNDEGjpDw==`
+
+The upstream BSD-3-Clause notices remain in the bundle. To reproduce, extract the release archive and concatenate `highlight.min.js` followed by `languages/<name>.min.js` for the modules listed above, separated by newlines.
+
+Validation: loaded the combined bundle, checked preservation of the previous language list, and invoked highlighting for every registered grammar. The opack browser regression suite also checks this source asset when `OPENAF_SOURCE` points to this checkout. Rebuild OpenAF using its normal build process to include the asset in a distributable JAR; editing this file does not update an already installed runtime.