gpu4j
Matrix Maths and Half Precision, in Plain Java
Version 0.4.0 · Java 17+ · Apache-2.0 · io.github.craton-co:gpu4j-core
The operations a model actually runs
A transformer spends its life in matrix multiplication. gpu4j puts that operation — plus half precision and resident weights — behind a typed Java call, so inference and data-prep can run inside your JVM application rather than beside it in a separate service.
Model weights dominate GPU memory, so half precision is not a micro-optimisation: it decides whether a model fits on the hardware you already own. gpu4j stores operands in binary16 while accumulating in full precision, halving the memory a weight tensor occupies without surrendering the accuracy that matters at the depths a transformer uses.
And the integration stays small. No kernel to author, no CUDA toolchain on your build machines, no Python bridge, and no second codebase to keep in sync — the numerical core lives in the same application, the same process and the same deployment as the data it operates on.
| Compute kernel (16.7M+ elements) | CPU (HotSpot C2) | TornadoVM GPU | CratonVM GPU | vs HotSpot | vs TornadoVM |
|---|---|---|---|---|---|
| Integer division chain | 2,179 ms | 27 ms | 7 ms | 311x faster | 3.9x faster |
| Double division chain | 1,784 ms | 135 ms | 82 ms | 21.8x faster | 1.6x faster |
| 128 multiply-adds per element | 1,298 ms | 26 ms | 7 ms | 185x faster | 3.7x faster |
| Dot-product reduction | 1,168 ms | unimplemented | 2 ms | 584x faster | n/a |
| Ray tracer kernel, 7680x4320 | 837.1 ms | 24.29 ms | 12.29 ms | 68x faster | 2.0x faster |
Measured on an RTX 2060 with a full host-to-device-and-back round trip, every result checksum-verified bit-for-bit against HotSpot. These are CratonVM's figures — the runtime gpu4j executes on. Full methodology is published with CratonVM.
The pipeline you already run, accelerated in place
Your Spark jobs, Flink operators, ETL transforms and scoring functions are already Java. The numerical stages that dominate your wall clock can move to the accelerator without becoming a separate service.
No new service in the architecture diagram
gpu4j is a library on your classpath, not a sidecar to operate. The accelerated work runs in the same process as the application that owns the data, which removes the RPC hop, the serialization tax and the second on-call rotation that a separate compute service brings with it.
Data movement that does not eat the win
Getting data onto an accelerator and results back is where many GPU integrations quietly lose their advantage. gpu4j moves array payloads in bulk at hardware transfer rates, so the arithmetic you paid for is the arithmetic you get.
Reductions the alternatives skip
Dot products and reductions are the backbone of similarity search, scoring and analytics. The engine accelerates them 98x over a production JVM — a workload the leading alternative Java GPU framework does not implement at all.
Testable in the pipeline you already have
A GPU library that only works on GPU hardware is a library your CI cannot test. A stand-in backend installs in one line, so the code your team writes against this API is unit-testable on any build agent, with no device attached.
The Technical Edge
Why experts choose gpu4j
Data that stays on the device
GpuArray is a device-resident buffer with a real lifecycle. Weights uploaded once stay resident across calls, so a decode step multiplying by the same weights on every token pays the transfer cost once rather than per token — the difference between a demo and a deployment. Device-only allocation is available for intermediates the host never reads.
Ordering you already understand
GpuStream and GpuFuture are the concurrency model Java developers already know. Work on one stream runs in order and independent streams overlap, so you can chain a long sequence of operations and wait once at the end, or await each step individually when you need its outcome. Transposition is expressed as a read, not a copy: either operand can be read transposed in place, with no duplicate tensor and no extra pass.
Correct before fast
Every accelerated result is checked bit-for-bit against the standard JVM. Where the runtime cannot guarantee an identical answer it runs your method on the CPU instead of guessing, and tells you why. Acceleration that changes your numbers is a liability, not a feature.
Teams this was built for
AI/ML platform teams
Teams serving models from a JVM stack who want matrix maths and half precision without standing up a Python service beside their application.
Data engineering teams
Teams whose Spark or Flink pipelines are bounded by numerical stages, and who would rather accelerate them in place than split the pipeline across two runtimes.
Java platform teams
Teams who own a large Java estate and need GPU throughput without adding a CUDA hiring profile or a second codebase to maintain.
Quantitative and scientific teams
Teams doing dense linear algebra on the JVM who need predictable, bit-checked results rather than a fast answer of uncertain provenance.
Add one dependency
<dependency>
<groupId>io.github.craton-co</groupId>
<artifactId>gpu4j-core</artifactId>
<version>0.4.0</version>
</dependency>gpu4j: the GPU, from Java you would actually write. Built by Craton Software Company.
gpu4j is under active development. GPU figures were measured on an NVIDIA RTX 2060 and vary with hardware, problem size and data shape; kernel-selection figures are device-side and launch-overhead figures are host-side, and neither is end-to-end call latency, which additionally includes host-to-device transfer. Comparative benchmarks hold the hardware constant and vary only the software. The device side requires CratonVM built with GPU support and an NVIDIA GPU. Java is a trademark of Oracle and/or its affiliates.