Introduction
The DJX Spark is on the horizon, expected to be available in a few weeks. While we wait, GMK Tech has released the Evo X2, a mini PC with 128 gigabytes of shared RAM with the GPU. This makes it ready for local Large Language Models (LLMs) and machine learning tasks, and it comes at less than half the price of the DJX Spark.
The Evo X2 Specifications
-
Processor: AMD's new Ryzen AI Max Plus 395 with 16 cores, 32 threads, and a clock speed of up to 5.1 gigahertz.
-
GPU: Radeon 8060S, which is supposed to rival an RTX 4060.
-
Memory: 128 gigabytes of LPDDR5X mobile memory at 8,000 megahertz.
-
Storage: Dual PCIe Gen 4 SSD slots, upgradable to 16 terabytes.
-
Connectivity: Two USB 4 ports, five USB A ports, an SD card reader, HDMI, DisplayPort, two headset ports, and a five gigabit ethernet port.
-
Wi-Fi: Wi-Fi 7.
Testing the Evo X2 for LLMs
The Evo X2 was tested with various LLM models using LM Studio. The results showed that it can handle large models with decent speed. For example, a 32 billion parameter model was processed at 10.58 tokens per second, and a 70 billion parameter model at 5.1 tokens per second. The GPU was fully utilized during these tests, and the power consumption was around 170 watts.
Comparing the Evo X2 to Other Machines
The Evo X2 was compared to the M4 Mac Mini and the M4 Pro Mac Mini. The M4 Mac Mini has a memory bandwidth of 120 gigabytes per second, while the M4 Pro Mac Mini has a memory bandwidth of 273 gigabytes per second. The Evo X2 has a published memory bandwidth of 256 gigabytes per second. The tests showed that the Evo X2 is much faster than the M4 Mac Mini, but slightly slower than the M4 Pro Mac Mini.
Llama CPP and Optimizations
Llama CPP is the backbone of LM Studio. It can be compiled for different platforms and with different optimizations. The Evo X2 was tested with Llama CPP compiled for Vulkan, which gave good results. The Llama Bench tool was used to measure the prompt processing and token generation speeds. The results showed that the Evo X2 is faster than the M4 Mac Mini and comparable to the M4 Pro Mac Mini.
Memory Bandwidth and Performance
Memory bandwidth is a crucial factor in the performance of LLMs. The higher the memory bandwidth, the faster the model can be processed. The Evo X2 has a decent memory bandwidth, but the measurements using the Stream Benchmark were lower than expected. This could indicate that the chip can be pushed further for better performance.
Comparing Desktop and Laptop Versions
The Evo X2 was compared to the laptop version of the same chip, the Flow Z13. The results showed that the desktop version is generally faster, except for one model where the laptop version performed better.
Different Chip Architectures
There are different chip architectures available for running LLMs, including the AMD Ryzen AI Max Plus 395, the Apple Unified Memory Architecture (UMA), and the NVIDIA Blackwell chip. Each architecture has its own advantages and disadvantages, and the performance depends on various factors such as memory bandwidth and memory partitioning.
Static Partitioning and Memory Management
The Evo X2 uses static partitioning for memory management, which divides the memory into set blocks before the computer starts up. This makes it easier to set up and more stable, but it also wastes memory if a program doesn't use all the space in a block.
Future Improvements and Support
AMD is working on improving the performance of the Ryzen AI Max Plus 395 for LLMs. They are also working on supporting Rackham, which could further improve the performance. However, Rackham support is still limited and not very stable at the moment.
Price and Value
The Evo X2 is not a cheap mini PC. The 64 gigabyte version costs $1,499, and the 128 gigabyte version costs $2,000. However, considering its performance and capabilities, it offers good value for money compared to other machines in its class.
Conclusion
The GMK Tech Evo X2 is a powerful mini PC that is well-suited for local LLMs and machine learning tasks. It offers good performance, a wide range of connectivity options, and the potential for future improvements. While it is not the cheapest option on the market, it provides a lot of value for its price. If you are looking for a mini PC for AI-related work, the Evo X2 is definitely worth considering.
{# 
{#