vortex-java

Pure-Java reader/writer for the Vortex columnar file format. 100% Java, no JNI, no sun.misc.Unsafe. Uses the FFM API (MemorySegment/Arena, Java 25+) for zero-copy memory-mapped reads — good performance out of the box, without native dependencies.

Benchmarked against vortex-jni (the Vortex Rust reference's JNI bindings) on read throughput, Top-N read latency, and compression ratio — see docs/explanation.md#benchmarks for the full tables and methodology.

Who is this for

• JVM analytics engines and OLAP systems
• Anyone who wants mmap-backed, zero-copy columnar reads without native-library management
• Windows JVM users. The Rust reference's JNI bindings (vortex-jni) ship Linux + macOS binaries only — vortex-java is the only Vortex implementation that runs on Windows JVMs out of the box. CI builds the full reactor on Linux + macOS + Windows × JDK 25 + 26.

Quickstart

<dependency>
  <groupId>io.github.dfa1.vortex</groupId>
  <artifactId>vortex-reader</artifactId>
  <version>0.13.2</version>
</dependency>

Minimal read example

try (VortexReader vf = VortexReader.open(Path.of("data/example.vortex"));
     var iter = vf.scan(ScanOptions.all())) {
    while (iter.hasNext()) {
        try (Chunk chunk = iter.next()) {
            LongArray ts = chunk.column("timestamp");
            for (long i = 0; i < ts.length(); i++) {
                System.out.println(ts.getLong(i));
            }
        }
    }
}

Lifecycle. ScanIterator implements Iterator<Chunk> and Chunk implements AutoCloseable. Each chunk owns a confined Arena; closing it releases the decoded buffers. Calling iter.next() while a prior chunk is still open throws IllegalStateException. Use try-with-resources, or iter.forEachRemaining(c -> ...) which closes each chunk for you. See docs/explanation.md#memory-model.

Minimal write example

<dependency>
  <groupId>io.github.dfa1.vortex</groupId>
  <artifactId>vortex-writer</artifactId>
  <version>0.13.2</version>
</dependency>

DType.Struct schema = DType.structBuilder()
        .field("timestamp", DType.I64)
        .field("symbol",    DType.UTF8)
        .field("price",     DType.F64)
        .field("volume",    DType.I64.asNullable())     // boxed Long[] → nullable
        .build();

try (var ch = FileChannel.open(Path.of("data/example.vortex"),
                               StandardOpenOption.CREATE, StandardOpenOption.WRITE);
     var writer = VortexWriter.create(ch, schema, WriteOptions.cascading(3))) {
    writer.writeChunk(c -> c
            .put(ColumnName.of("timestamp"), new long[]   {1_700_000_000_000L, 1_700_000_001_000L})
            .put(ColumnName.of("symbol"),    new String[] {"AAPL", "AAPL"})
            .put(ColumnName.of("price"),     new double[] {189.95, 190.10})
            .put(ColumnName.of("volume"),    new Long[]   {100L, null}));  // null in nullable col
}

Each .put is validated at the call site: unknown column names, mismatched array types, and boxed arrays for non-nullable columns throw IllegalArgumentException immediately. Missing columns surface as IllegalStateException when the lambda returns.

For more examples — projection, filtering, custom encodings, and the CLI — see the tutorial.

Documentation

Docs follow the Diátaxis framework.

Document	Mode	Contents
docs/tutorial.md	Tutorial	Step-by-step: write and read your first Vortex file
docs/how-to.md	How-to	Recipes: count rows, convert Parquet, filter, project, custom encodings
docs/reference.md	Reference	API surface, CLI subcommands, operator tables, file-format trailer
docs/compatibility.md	Reference	Encoding support table, S3 fixture status
docs/explanation.md	Explanation	Design rationale, memory model, benchmarks
docs/testing.md	Explanation	Test strategy: layers, counts per module, what each layer verifies

Vortex implementations

Project	Language	Notes
vortex-data/vortex	Rust	Reference implementation + JNI bindings
LaurieRhodes/vortex-go	Go	Pure-language port
dfa1/vortex-java	Java	This library

Contributing

Requirements: Java 25+. Build: ./mvnw verify.

Forks welcome. See CONTRIBUTING.md for full build reference, coding conventions, and how to add a new encoding.

This project uses Claude Code for implementation work. Architecture, API design, and all decisions are human-driven.