Building with Homebrew llvm/clang fails with error: 'TARGET_OS_IPHONE' is not defined #5622

dimpase · 2021-07-19T15:39:16Z

What did you do?

On latest Homebrew on macOS 11.4 (x86_64) installed llvm package, and tried to build Pillow with
clang/clang++ it provides.

What did you expect to happen?

that it just works (many Python packages build just fine in such a setup)

What actually happened?

error: 'TARGET_OS_IPHONE' is not defined quite early in the build, compiling _imaging.c
In more detail,

clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch x86_64 -g -DHAVE_LIBJPEG -DHAVE_OPENJPEG -DHAVE_LIBZ -DHAVE_LIBTIFF -DHAVE_XCB -DPILLOW_
VERSION="8.4.0.dev0" -I/usr/local/Cellar/openjpeg/2.4.0/include/openjpeg-2.4 -I/usr/local/Cellar/jpeg/9d/include -I/usr/local/Cellar/libtiff/4.3.0/include -I/Users/dima/software/Pillow -I/usr/local/Cellar
/freetype/2.10.4/include/freetype2 -I/usr/local/Cellar/fribidi/1.0.10/include/fribidi -I/usr/local/Cellar/little-cms2/2.12/include -I/usr/local/Cellar/ntl/11.5.1/include -I/usr/local/Cellar/readline/8.1/i
nclude -I/usr/local/include -I/Library/Frameworks/Python.framework/Versions/3.9/include -I/usr/local/Cellar/freetype/2.10.4/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/D
eveloper/SDKs/MacOSX.sdk/usr/include -I/Library/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c src/_imaging.c -o build/temp.macosx-10.9-x86_64-3.9/src/_imaging.o
In file included from src/_imagingft.c:22:
In file included from /Library/Frameworks/Python.framework/Versions/3.9/include/python3.9/Python.h:25:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/stdio.h:64:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/_stdio.h:93:16: warning: pointer is missing a nullability type specifier (_Nonnull, _Nullable, or
 _Null_unspecified) [-Wnullability-completeness]
        unsigned char   *_base;
                        ^
... [many similar warning]

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/_stdio.h:144:18: warning: pointer is missing a nullability type specifier (_Nonnull, _Nullable, o
r _Null_unspecified) [-Wnullability-completeness]
        struct __sFILEX *_extra; /* additions to FILE to not break ABI */
In file included from src/_webp.c:2:
In file included from /Library/Frameworks/Python.framework/Versions/3.9/include/python3.9/Python.h:25:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/stdio.h:64:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/_stdio.hIn file included from src/_imagingft.c:22:
In file included from /Library/Frameworks/Python.framework/Versions/3.9/include/python3.9/Python.h:25:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/stdio.h:220::5: error: 'TARGET_OS_IPHONE' is not defined, evaluates to 0 [-Werror,-Wundef-prefix=TARGET_OS_]
                        ^
93:16: warning: #if TARGET_OS_IPHONE
    ^

... [more similar warnings and errorrs]
...

What are your OS, Python and Pillow versions?

OS: macOS 11.4
Python: python3: 3.9.6 , python2 2.7.16
Pillow: current master 426b730, and also with 8.1.2

brew install llvm
export PATH=/usr/local/opt/llvm/bin:$PATH
CC=clang CXX=clang++ make install

The text was updated successfully, but these errors were encountered:

radarhere · 2021-07-20T02:58:26Z

The way I read that log, the problem comes from _imagingft.c, not _imaging.c.

Pillow/src/_imagingft.c

Line 22 in 426b730

#include "Python.h"

The problem is not caused by our code directly, rather it is stdio.h being called from Python.h.

Could we get a full copy of the log? What version of Xcode are you using?

dimpase · 2021-07-20T08:28:16Z

indeed, it's _imagingft.c, sorry.

% xcodebuild -version
Xcode 12.4
Build version 12D4e

Apparently pillow is too aggressive in appending all sorts of include directories for the compiler to search, and manages to break stdio.h inclusion in this case:

% clang -v
Homebrew clang version 12.0.1
Target: x86_64-apple-darwin20.5.0
Thread model: posix
InstalledDir: /usr/local/opt/llvm/bin

macOS defines TARGET_OS_IPHONE in TargetConditionals.h which is normally included 1st thing (no need to explicitly doing it).

Please see the full log at https://users.ox.ac.uk/~coml0531/tmp/pillow-8.1.2.log

radarhere · 2021-07-20T12:58:38Z

If you open your setup.py, and remove the code added at https://github.com/python-pillow/Pillow/pull/4974/files, does that fix the error?

dimpase · 2021-07-20T13:40:01Z

this leads to a problem finding zlib

…

On Tue, 20 Jul 2021, 13:59 Andrew Murray, ***@***.***> wrote: If you open your setup.py, and remove the code added at https://github.com/python-pillow/Pillow/pull/4974/files, does that fix the error? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#5622 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJXYHAOYNTEUGPODI3YWILTYVXJTANCNFSM5AT6U6KA> .

radarhere · 2021-07-21T07:56:53Z

So the goal here is to detect zlib.h, but not stdio.h.

Ok, so what if we avoid /usr/ in favour of System/Library/Frameworks/Kernel.framework/Versions/A/Headers/libkern. Does https://github.com/radarhere/Pillow/tree/zlib / radarhere@d514686 work for you?

dimpase · 2021-07-21T09:20:07Z

With this patch as is, zlib library cannot be found. If I leave the line

_add_directory(library_dirs, os.path.join(sdk_path, "usr", "lib"))

in, then I get basically the same errors as before.

dimpase · 2021-07-21T09:35:14Z

hmm, no, wait, it works (at least CC=clang make inplace completes).

dimpase · 2021-07-21T10:10:34Z

OK, indeed, your patch works, unamended (CC=clang make install builds and passes tests).

What's the essential difference between these two include locations? It's quite weird...

radarhere · 2021-07-21T10:22:55Z

xcrun --show-sdk-path is /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk (on my machine).

Your problem is happening because stdio.h, from
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/stdio.h
is being included.

But, as we've found, if we ignore that directory, then we're not including zlib.h from
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/zlib.h

So I wondered if zlib.h was anywhere else inside /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk.

$ pwd
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
$ find . -iname zlib.h
./usr/include/zlib.h
./System/Library/Frameworks/Kernel.framework/Versions/A/Headers/libkern/zlib.h
$ ls ./System/Library/Frameworks/Kernel.framework/Versions/A/Headers/libkern/
Block.h		OSByteOrder.h	OSMalloc.h	_OSByteOrder.h	copyio.h	i386		machine		version.h
OSAtomic.h	OSDebug.h	OSReturn.h	arm		crc.h		libkern.h	sysctl.h	zconf.h
OSBase.h	OSKextLib.h	OSTypes.h	c++		crypto		locks.h		tree.h		zlib.h

So at that location, we have zlib.h, but no stdio.h

dimpase · 2021-07-21T10:41:55Z

By the way, libkern/zlib.h corresponds to an older version of zlib. (1.2.3), while the other, "bad" one, corresponds to 1.2.11.

radarhere · 2021-07-21T13:11:36Z

Ok, how about https://github.com/radarhere/Pillow/tree/macos_zlib / radarhere@e7270eb ? Does that work for you too?

It uses /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/zlib.h instead, which is 1.2.11 on my machine.

dimpase · 2021-07-21T13:30:01Z

OK, this works.

radarhere · 2021-07-21T14:12:49Z

Cool. Thanks for checking. I've created PR #5624 to resolve this.

radarhere added the macOS label Jul 19, 2021

radarhere changed the title ~~building with Hombrew llvm/clang fails with error: 'TARGET_OS_IPHONE' is not defined~~ Building with Homebrew llvm/clang fails with error: 'TARGET_OS_IPHONE' is not defined Jul 19, 2021

radarhere added Installation Usually a problem with … Build and removed Installation Usually a problem with … labels Jul 20, 2021

radarhere mentioned this issue Jul 21, 2021

Prefer command line tools SDK on macOS #5624

Merged

hugovk closed this as completed in #5624 Jul 29, 2021

h-vetinari mentioned this issue Nov 10, 2021

Allow setting specific SDK on macOS #5827

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Building with Homebrew llvm/clang fails with error: 'TARGET_OS_IPHONE' is not defined #5622

Building with Homebrew llvm/clang fails with error: 'TARGET_OS_IPHONE' is not defined #5622

dimpase commented Jul 19, 2021

radarhere commented Jul 20, 2021

dimpase commented Jul 20, 2021

radarhere commented Jul 20, 2021

dimpase commented Jul 20, 2021 via email

radarhere commented Jul 21, 2021

dimpase commented Jul 21, 2021 •

edited by radarhere

Loading

dimpase commented Jul 21, 2021

dimpase commented Jul 21, 2021

radarhere commented Jul 21, 2021

dimpase commented Jul 21, 2021

radarhere commented Jul 21, 2021

dimpase commented Jul 21, 2021

radarhere commented Jul 21, 2021

Building with Homebrew llvm/clang fails with error: 'TARGET_OS_IPHONE' is not defined #5622

Building with Homebrew llvm/clang fails with error: 'TARGET_OS_IPHONE' is not defined #5622

Comments

dimpase commented Jul 19, 2021

What did you do?

What did you expect to happen?

What actually happened?

What are your OS, Python and Pillow versions?

radarhere commented Jul 20, 2021

dimpase commented Jul 20, 2021

radarhere commented Jul 20, 2021

dimpase commented Jul 20, 2021 via email

radarhere commented Jul 21, 2021

dimpase commented Jul 21, 2021 • edited by radarhere Loading

dimpase commented Jul 21, 2021

dimpase commented Jul 21, 2021

radarhere commented Jul 21, 2021

dimpase commented Jul 21, 2021

radarhere commented Jul 21, 2021

dimpase commented Jul 21, 2021

radarhere commented Jul 21, 2021

dimpase commented Jul 21, 2021 •

edited by radarhere

Loading