在旧的Mali Vulkan驱动上成功运行Llama.cpp

Jetson Tan | 12 Jul 2026

Warning: 自2026/8/20起, 本文已无法复现!

大多数人在旧手机上运行Llama.cpp时都选择使用CPU后端, 因为它最稳定, 且性能也不错. 然而GPU更适合大模型推理, 还有Vulkan这样的高效推理API, GPU在安卓的特殊环境中却难以利用. 有了Llama.cpp-For-AArch64这个项目, 在旧手机上利用GPU成为了可能.

本文将带领读者走完碰壁->调试/反汇编->跑通的全流程, 并将成果发布于上述的Github仓库中.

Output 1.22.559.594 I common_memory_breakdown_print: | memory breakdown [MiB] | total free self model context compute unaccounted | 1.22.559.637 I common_memory_breakdown_print: | - Vulkan0 (Mali-G77 MC9) | 7385 = 7385 + ( 544 = 456 + 28 + 59) + -544 | 1.22.559.645 I common_memory_breakdown_print: | - Host | 80 = 73 + 0 + 7 | 1.22.559.653 I common_memory_breakdown_print: | - CPU_REPACK | 182 = 182 + 0 + 0 | 1.22.560.337 D ~llama_context: Vulkan0 compute buffer size is 59.5098 MiB, matches expectation of 59.5098 MiB 1.22.560.376 D ~llama_context: Vulkan_Host compute buffer size is 7.0118 MiB, matches expectation of 7.0118 MiB
Control

一些准备

首先, 下载 Termux. 然后运行以下命令(换源和setup storage自己处理):

Command apt update && apt upgrade -y apt install git ccache cmake clang ninja vulkan-headers vulkan-loader-android vulkan-tools glslang spirv-headers spirv-tools vulkan-extension-layer -y git clone https://github.com/ggml-org/llama.cpp.git mv llama.cpp llama cd llama vulkaninfo --summary

现在应该能看见GPU型号.

初见错误

如果你按照官方文档编译Llama.cpp并运行, 可能会遇到这样的报错:

Output Vulkan 1.2 required. "llama-cli" received signal SIGSEGV, Segmentation fault.

即程序访问了非法内存且api版本低, 由于CPU后端没有此类问题, 所以猜测Vulkan驱动还有未实现函数, 即"骗子驱动".

现在有几种解决办法:

由于我没有root权限, 而且不爱哭, 所以选择第二种办法.

Command nano ggml/src/ggml-vulkan/ggml-vulkan.cpp

把6725行的if逻辑删掉即可, 重新编译

再次报错

重新运行之后, 确实没有apiVersion的错误了, 但依旧报出Segmentation fault. 尝试用GDB调试一下, 结果如下:

Output #0 0x0000000000000000 in ?? () <-- 空指针调用导致段错误 #1 0x0000007fabb6a250 in ggml_vk_create_buffer(...) #2 0x0000007faba31f94 in ggml_backend_vk_host_buffer_type_alloc_buffer(...) #3 0x0000007faf748c28 in llama_context::output_reserve(int)()

不难看出问题出在ggml_vk_create_buffer(), 所以就回到ggml/src/ggml-vulkan/ggml-vulkan.cpp看看情况.

深入源码

搜到了下面的代码片段:

Code if (device->buffer_device_address) { const vk::BufferDeviceAddressInfo addressInfo(buf->buffer); buf->bda_addr = device->device.getBufferAddress(addressInfo); } device->memory_logger->log_allocation(buf, size); return buf;

由此可以猜测bda_addr就是解决问题的关键, 先去查查驱动

反汇编vulkan驱动

在反汇编之前, 可以先用nm查一下导出表:

Output ~/storage/downloads $ nm -D libGLES_mali.so | grep -i "GetBuffer" 00000000006e5ba8 T glGetBufferParameteri64v 00000000006ef35c T glGetBufferParameteriv 00000000006ef3c8 T glGetBufferPointerv 00000000006ef434 T glGetBufferPointervOES ~/storage/downloads $ nm -D libvulkan.so | grep -i "GetBuffer" 00000000000195dc T vkGetBufferDeviceAddress 0000000000018710 T vkGetBufferMemoryRequirements 0000000000019414 T vkGetBufferMemoryRequirements2 00000000000195c4 T vkGetBufferOpaqueCaptureAddress

大一统驱动libGLES_mali.so里没有vkGetBufferDeviceAddress, 而马甲驱动libvulkan.so里有. 骗子驱动实锤了. 再反汇编libvulkan.so看看:

Output objdump -d libvulkan.so >> libvulkan.asm ... 00000000000195dc : ; 1. 拿到 VkDevice 句柄（x0）指向的第一个内存区域（通常是设备的分层上下文/调度表） 195dc: ldr x8, [x0] ; 2. 从这个上下文的 0x4F0 偏移处，读取一个函数指针 195e0: ldr x2, [x8, #0x4f0] ; 3. 直接跳转到那个函数指针去执行！ 195e4: br x2 ; 下面三行是重复的，可能是编译器对齐或者另一个变体函数的尾部，逻辑完全一样 195e8: ldr x8, [x0] 195ec: ldr x2, [x8, #0x4f0] 195f0: br x2

可以看到libvulkan.so没有任何防御措施, 再去libGLES_mali.so里查证一下

Output ~/storage/downloads $ objdump -d libGLES_mali.so | grep "#0x4f0" ... 742ca8: f9027a7f str xzr, [x19, #0x4f0] 742e08: f9027a7f str xzr, [x19, #0x4f0] 742f88: f9027a7f str xzr, [x19, #0x4f0] 743204: f9027a7f str xzr, [x19, #0x4f0] 74391c: f9027a7f str xzr, [x19, #0x4f0] 7439ec: f9027a7f str xzr, [x19, #0x4f0] 744d1c: f9027a7f str xzr, [x19, #0x4f0] 7454a4: f9027a7f str xzr, [x19, #0x4f0] 745630: f9027a7f str xzr, [x19, #0x4f0] 745a18: f9027a7f str xzr, [x19, #0x4f0] 745b58: f9027a7f str xzr, [x19, #0x4f0] 745da8: f9027a7f str xzr, [x19, #0x4f0] 745fd4: f9027a7f str xzr, [x19, #0x4f0] 7465a0: f9027a7f str xzr, [x19, #0x4f0] 746c78: f9027a7f str xzr, [x19, #0x4f0] 747574: f9027a9f str xzr, [x20, #0x4f0]

libGLES_mali.so不语, 只是一味写入NULL. 一大堆没实现的函数, 只能考虑用低速方法来实现CPU和GPU之间的数据传输.

放弃Vulkan拓展

把buf->bda_addr设为0, 再次编译运行

Output 1.13.567.995 I common_memory_breakdown_print: | - Vulkan0 (Mali-G77 MC9) | 7385 = 7385 + ( 478 = 456 + 7 + 14) + -478 |

成功!

一些结果及反思

llama-cli测试

模型体量 后端类型 Prompt 速度 Generation 速度
0.5b gpu 3.5 t/s 15.8 t/s
cpu 37.2 t/s 19.3 t/s
1.5b gpu 4.1 t/s 6.8 t/s
cpu 12.2 t/s 8.5 t/s
4b gpu 1.1 t/s 2.9 t/s
cpu 2.5 t/s 1.8 t/s
8b
(已撞 Swap 墙)		 gpu 0.4 t/s 1.6 t/s
cpu 2.3 t/s 1.6 t/s

注: 根据进一步测试, CPU后端的prompt速度可以远超GPU后端. 但generation速度则会在多轮对话后被GPU反超(2倍).

llama-bench测试

Output ~/.../build/bin $ taskset -c 4,5,6,7 ./llama-bench -m 0.5b.gguf -t 4 -ngl 100 ggml_vulkan: WARNING: Instance extension VK_EXT_debug_utils not found. ggml_vulkan: Found 1 Vulkan devices: ggml_vulkan: 0 = Mali-G77 MC9 (Mali-G77 MC9) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 16 | shared memory: 32768 | int dot: 0 | matrix cores: none | model | size | params | backend | ngl | threads | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | ------: | --------------: | -------------------: | | qwen2 0.5B Q2_K - Medium | 278.92 MiB | 619.57 M | Vulkan | 100 | 4 | pp512 | 59.72 ± 0.50 | | qwen2 0.5B Q2_K - Medium | 278.92 MiB | 619.57 M | Vulkan | 100 | 4 | tg128 | 17.08 ± 0.78 | build: 1d1d9a9ed (9968) ~/.../build/bin $ taskset -c 4,5,6,7 ./llama-bench -m 0.5b.gguf -t 4 -ngl 0 ggml_vulkan: WARNING: Instance extension VK_EXT_debug_utils not found. ggml_vulkan: Found 1 Vulkan devices: ggml_vulkan: 0 = Mali-G77 MC9 (Mali-G77 MC9) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 16 | shared memory: 32768 | int dot: 0 | matrix cores: none | model | size | params | backend | ngl | threads | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | ------: | --------------: | -------------------: | | qwen2 0.5B Q2_K - Medium | 278.92 MiB | 619.57 M | Vulkan | 0 | 4 | pp512 | 54.44 ± 0.38 | | qwen2 0.5B Q2_K - Medium | 278.92 MiB | 619.57 M | Vulkan | 0 | 4 | tg128 | 18.44 ± 3.49 | build: 1d1d9a9ed (9968) ~/.../build/bin $ taskset -c 4,5,6,7 ./llama-bench -m 0.5b.gguf -t 4 -ngl 10 ggml_vulkan: WARNING: Instance extension VK_EXT_debug_utils not found. ggml_vulkan: Found 1 Vulkan devices: ggml_vulkan: 0 = Mali-G77 MC9 (Mali-G77 MC9) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 16 | shared memory: 32768 | int dot: 0 | matrix cores: none | model | size | params | backend | ngl | threads | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | ------: | --------------: | -------------------: | | qwen2 0.5B Q2_K - Medium | 278.92 MiB | 619.57 M | Vulkan | 10 | 4 | pp512 | 53.55 ± 1.12 | | qwen2 0.5B Q2_K - Medium | 278.92 MiB | 619.57 M | Vulkan | 10 | 4 | tg128 | 11.56 ± 0.72 | build: 1d1d9a9ed (9968) ~/.../build/bin $ taskset -c 4,5,6,7 ./llama-bench -m 0.5b.gguf -t 4 -ngl 20 ggml_vulkan: WARNING: Instance extension VK_EXT_debug_utils not found. ggml_vulkan: Found 1 Vulkan devices: ggml_vulkan: 0 = Mali-G77 MC9 (Mali-G77 MC9) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 16 | shared memory: 32768 | int dot: 0 | matrix cores: none | model | size | params | backend | ngl | threads | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | ------: | --------------: | -------------------: | | qwen2 0.5B Q2_K - Medium | 278.92 MiB | 619.57 M | Vulkan | 20 | 4 | pp512 | 56.07 ± 1.30 | | qwen2 0.5B Q2_K - Medium | 278.92 MiB | 619.57 M | Vulkan | 20 | 4 | tg128 | 15.76 ± 0.04 | build: 1d1d9a9ed (9968)

由于I/O瓶颈和兼容操作, GPU性能并未得到充分发挥. 但至少与CPU相差不大. 若在旗舰机上运行, Mali GPU也许能超越CPU. 说实话, ARM最近放出来的新一代GPU "G1 Ultra" 还是很令人期待的.

我实际上有一个构想, 用mmap划一块RAM给GPU, 让GPU访问以降低I/O瓶颈.

值得一提的是, 用同样的方法修改OpenCL后端并不能成功, 因为没有适配Mali的OpenCL算子.

日后将尝试Adreno上的Vulkan/OpenCL后端, 以及MTK的APU和Qualcomm的Hexagon NPU.

回到主页