Cannot include cpython/Include/internal/pycore_optimizer.h in C++ extension due to use of protected C++ keyword "not"

[tim-stephenson]

Bug report

Bug description:

The code in cpython/Include/internal/pycore_optimizer.h causing problems

typedef struct {
    uint8_t tag;
    bool not;
    uint16_t value;
} JitOptTruthiness;

• PR which created the issue: GH-130415: Use boolean guards to narrow types to values in the JIT #130659

CPython versions tested on:

3.14

Operating systems tested on:

macOS

Linked PRs

• gh-131331: rename not field to invert for C++ extension modules #131334

[tim-stephenson]

My understanding is that there is no good way around this for C++ extensions: https://stackoverflow.com/questions/72177535/how-can-i-include-a-c-header-that-uses-a-c-keyword-as-an-identifier-in-c

[picnixz]

AFAIK, cpython/Include/internal/pycore_optimizer.h is not meant to be included directly, nor from an extension module as it's internal. Is there an issue because of some indirect includes if one uses #include "Python.h"? or is this only caused when one tries to include that specific header?

EDIT: I'm marking it as a release blocker if this implies that C++ extensions cannot be built anymore but if it's because of internal includes, then we can remove the label.

[picnixz]

cc @brandtbucher

[tim-stephenson]

The use is here: https://github.com/numba/numba/blob/main/numba/_dispatcher.cpp. The explanation to why they are using internal includes is https://github.com/numba/numba/blob/d650f3057839f99a0ab77262f27e80f4b9ae6739/numba/_dispatcher.cpp#L811

[picnixz]

I think it doesn't hurt on our side if just use something else than not. We can use not_. I'll make a PR for that.

EDIT: However, I would like to mention that using internal headers is at one's own risk, though, in this case, the fix on our side is easy.

[markshannon]

I would argue that allowing internal headers to be accessed in third-party code does hurt us.
Accessing the internals without a clear understanding of the necessary invariants can leave the VM in a broken state.
The resulting issues can be hard to debug and can consume of a lot of our time.

Regarding the numba issue. If there is a feature that is needed but does not have a public API, then we need to tell extension developers to submit a feature request, like this one from the Cython developers.

[picnixz]

I would argue that allowing internal headers to be accessed in third-party code does hurt us.

Yes, in general, that's what I would say. But if the fix is simple enough and if it's easier than to make a feature request, then I'd say let's do it on our side. But I agree that I would prefer not to worry about compatibilty with projects including internal headers.

Would you be willing to consider accepting my fix for this specific issue though?