Define magic numbers as integer constants (#1518)

[imaami]

The underlying representation of multibyte character literals is
implementation-defined. This could, at least in principle, cause
cross-build data export/import issues independent of endianness.

Define magic numbers as integer literals to be on the safe side.

[ice0]

@imaami
Can you make a short example showing the problem?
I've read the whole discussion but still haven't got proof that this could be the problem.

[imaami]

@imaami Can you make a short example showing the problem? I've read the whole discussion but still haven't got proof that this could be the problem.

It isn't currently a practical problem as far as compiler behavior is concerned. My first conclusion was incorrect, all three major compilers (GCC, Clang, MSVC) generate the same 4-byte sequence from the relevant multibyte character literals ('ggjt' etc.). But they don't agree with one another because of any standardized language feature, they just happen to agree for whatever reason. They could all behave in different ways and would nevertheless conform to the C and C++ standards.

The only "problem" left on the table is the implementation-defined nature of multibyte character literals and how a compiler is allowed to represent one in memory. Nothing in the C or C++ standards mandates a specific order for the constituent bytes. So although in practice this issue is only a theoretical one as of now, defining the magic numbers as integer literals would still be a more robust alternative.

[imaami]

Oh, and there's also another thing: the language standards define the type of a multibyte character literal as int. If llama.cpp assumes that multibyte character literals are guaranteed to have an underlying representation of at least 4 bytes, then let me tell you, exporting a weights file on an 8-bit MCU is going to fail miserably due to int being only 16 bits wide. ;) (Although I suspect trying to do that would be unpleasant for other reasons too.)

[ggerganov]

I guess these also have to be updated:

https://github.com/ggerganov/llama.cpp/blob/2d5db48371052087a83974abda3767d1aedec598/llama.cpp#L430-L447

[imaami]

I guess these also have to be updated:

https://github.com/ggerganov/llama.cpp/blob/2d5db48371052087a83974abda3767d1aedec598/llama.cpp#L430-L447

How did I miss these? :D I'll get on it and rebase to the current master too.

[slaren]

There is also one for the lora files:
https://github.com/ggerganov/llama.cpp/blob/2d5db48371052087a83974abda3767d1aedec598/llama.cpp#L2238-L2240

[imaami]

Wow, thanks for the quick approval. :)