Skip to content

Commit 6d16169

Browse files
authored
ggml : prevent builds with -ffinite-math-only (ggml-org#7726)
This enforces a check that -fno-finite-math-only was set and that the operating compiling mode is not in finite maths mode. This is because during rewriting of silu and softmax for cpu ggml-org#7154 there emerged an issue where the result that was observed when >1 slot was nondeterministic as found by @JohannesGaessler. @LostRuins narrowed the problem down to -ffinite-math-only which was theorised to be due to SiLU, instead of flushing small values to 0, returns NaN or some other garbage. @jart proposed a fix that @ggerganov then implemented in this fix ref ggml-org#7154 (comment)
1 parent bde7cd3 commit 6d16169

File tree

2 files changed

+6
-1
lines changed

2 files changed

+6
-1
lines changed

cmake/arm64-windows-llvm.cmake

+1-1
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ set( CMAKE_CXX_COMPILER clang++ )
99
set( CMAKE_C_COMPILER_TARGET ${target} )
1010
set( CMAKE_CXX_COMPILER_TARGET ${target} )
1111

12-
set( arch_c_flags "-march=armv8.7-a -fvectorize -ffp-model=fast" )
12+
set( arch_c_flags "-march=armv8.7-a -fvectorize -ffp-model=fast -fno-finite-math-only" )
1313
set( warn_c_flags "-Wno-format -Wno-unused-variable -Wno-unused-function -Wno-gnu-zero-variadic-macro-arguments" )
1414

1515
set( CMAKE_C_FLAGS_INIT "${arch_c_flags} ${warn_c_flags}" )

ggml.c

+5
Original file line numberDiff line numberDiff line change
@@ -2272,6 +2272,11 @@ inline static float ggml_silu_f32(float x) {
22722272
return x/(1.0f + expf(-x));
22732273
}
22742274

2275+
#if __FINITE_MATH_ONLY__
2276+
#error "some routines in ggml.c require non-finite math arithmetics -- pass -fno-finite-math-only to the compiler to fix"
2277+
#error "ref: https://github.com/ggerganov/llama.cpp/pull/7154#issuecomment-2143844461"
2278+
#endif
2279+
22752280
#if defined(__ARM_NEON) && defined(__aarch64__)
22762281

22772282
// adapted from arm limited optimized routine

0 commit comments

Comments
 (0)