Great effort and what an achievement! #LLM inference implemented in #Java, automatically accelerated on #GPUs with #TornadoVM & CUDA native support! #opensource #AI #Java #GPU
GPULlama3 v1.0.0 is out - now #CUDA-native ⚡ ✅ TornadoVM CUDA backend w/ tensor-core (MMA) batch prefill ✅ FP16 & Q8_0 · Llama, Qwen, Devstral & more models ✅ An OpenAI-compatible server (llama-tornado --server) Pure #Java. No JNI. github.com/beehive-lab/GPULlama3.java #opensource #AI #LLM #GPU