Run AI Models for Edge Application Development with Just a Few Commands
NVIDIA, Friday, October 30th, 2026
NVIDIA shows how TensorRT Edge-LLM and the trtedge CLI let embedded developers run models like Nemotron-3.5 Lightning on DGX Spark in a few commands.
DGX Spark 64GB serves as a prototyping, fine-tuning and testing platform for embedded AI, while TensorRT Edge-LLM is a lightweight C++ inference runtime spanning development on Spark and deployment on Jetson Thor, DRIVE AGX Thor and IGX Thor.
The trtedge CLI downloads models and builds optimized TensorRT engines, and an OpenAI-compatible API server can serve the model locally for prototyping.
With DSpark speculative decoding using three draft tokens, mean generation speed rose from 73.4 to 126.9 tokens per second on HumanEval prompts, with peak memory at 27.9GB.