Bug: abort on Android (pixel 8 pro)

[nivibilla]

What happened?

Getting GGML_ASSERT: ggml.c:21763: svcntb() == QK8_0 when trying to build and run llama.cpp with llama 3 8b q4 on Android (pixel 8 pro)

Name and Version

version: 3220 (f702a90)
built with clang version 18.1.7 for aarch64-unknown-linux-android28

What operating system are you seeing the problem on?

Other? (Please let us know in description)

Relevant log output

./llama-server -m ./models/Meta-Llama-3-8B-Instruct.Q4_K_M.gguf --chat-template llama3 --no-mmap
INFO [                    main] build info | tid="542774631704" timestamp=1719314594 build=3220 commit="f702a90e"
GGML_ASSERT: ggml.c:21763: svcntb() == QK8_0
Aborted```

[nivibilla]

Seems to be an issue caused by the SVE integration PR

it works fine after testing with an older release before that PR.

[yh-yao]

Have we fixed it?

[slaren]

You need to build without SVE.

[nivibilla]

@slaren I didn't explicitly enable it. I assume it's enabled by default. Is there a flag to put on to disable while building?

[slaren]

@msy-kato can you take a look? It seems that SVE is being enabled by default, which then causes llama.cpp to crash in ggml_cpu_has_sve if the CPU does not have the correct vector size.

[msy-kato]

@slaren @nivibilla the flags for SVE are set here and default is off.

https://github.com/ggerganov/llama.cpp/blob/f619024764e72261f14d7c31d892b8fb976603b4/ggml/src/CMakeLists.txt#L965-L967

https://github.com/ggerganov/llama.cpp/blob/f619024764e72261f14d7c31d892b8fb976603b4/ggml/CMakeLists.txt#L93

However, SVE may have been enabled by GGML_NATIVE.
So could you try with cmake -DGGML_NATIVE=OFF ...?

[sqzhang-jeremy]

@slaren @nivibilla the flags for SVE are set here and default is off.

https://github.com/ggerganov/llama.cpp/blob/f619024764e72261f14d7c31d892b8fb976603b4/ggml/src/CMakeLists.txt#L965-L967

https://github.com/ggerganov/llama.cpp/blob/f619024764e72261f14d7c31d892b8fb976603b4/ggml/CMakeLists.txt#L93

However, SVE may have been enabled by GGML_NATIVE. So could you try with cmake -DGGML_NATIVE=OFF ...?

It worked for my Pixel 8 Pro.

cmake -DGGML_NATIVE=OFF . # first Run CMake with the GGML_NATIVE option explicitly set to OFF
make # then make

[slaren]

It is still a problem that SVE is getting enabled with GGML_NATIVE, which is enabled by default. Either the SVE code should work with all vector sizes, or it must be disabled unless explicitly enabled with GGML_SVE.

[msy-kato]

Yes. I'll submit PR for the fix.