external/gazelle~~go_deps~com_github_dougthor42_go_tree_sitter/python/scanner.c:1:10: fatal error: '../array.h' file not found with gazelle 0.38+

[remiphilippe]

🐞 bug report

Affected Rule

The issue is caused by the rule: rules_python_gazelle_plugin (version 1.1.0) when running with gazelle 0.38+

Is this a regression?

Maybe, the issue starts with gazelle 0.38

Description

A clear and concise description of the problem...

When running gazelle bazel run //python:gazelle update go_tree_sitter fails to build

This is using the config from https://github.com/bazelbuild/rules_python/tree/main/examples/bzlmod_build_file_generation

When downgrading gazelle to 0.37 it runs (at the expense of features in go, namely the build_file_generation = "clean" which was introduced in 0.38)

The issue seems linked to smacker/go-tree-sitter#175

🔬 Minimal Reproduction

https://github.com/bazelbuild/rules_python/tree/main/examples/bzlmod_build_file_generation with gazelle 0.38+

🔥 Exception or Error

INFO: Analyzed target //python:gazelle (51 packages loaded, 282 targets configured).
ERROR: /private/var/tmp/_bazel_ripper/5d3e93b0f5b6e19252cd0dca794d49db/external/gazelle~~go_deps~com_github_dougthor42_go_tree_sitter/python/BUILD.bazel:3:11: GoCompilePkg external/gazelle~~go_deps~com_github_dougthor42_go_tree_sitter/python/python.a failed: (Exit 1): builder failed: error executing GoCompilePkg command (from target @@gazelle~~go_deps~com_github_dougthor42_go_tree_sitter//python:python) bazel-out/darwin_x86_64-opt-exec-ST-d57f47055a04/bin/external/rules_go~~go_sdk~myproject__download_0/builder_reset/builder compilepkg -sdk external/rules_go~~go_sdk~myproject__download_0 -goroot ... (remaining 43 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
external/gazelle~~go_deps~com_github_dougthor42_go_tree_sitter/python/scanner.c:1:10: fatal error: '../array.h' file not found
#include "../array.h"
         ^~~~~~~~~~~~
1 error generated.
compilepkg: error running subcommand external/toolchains_llvm~~llvm~llvm_toolchain/bin/cc_wrapper.sh: exit status 1
Target //python:gazelle failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 29.799s, Critical Path: 5.75s
INFO: 33 processes: 9 internal, 24 darwin-sandbox.
ERROR: Build did NOT complete successfully
ERROR: Build failed. Not running target

🌍 Your Environment

Operating System:

  
Mac OS 15.1.1 (24B91) Intel
  

Output of bazel version:

  
Build label: 7.4.1-homebrew
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Tue Jan 1 00:00:00 1980 (315532800)
Build timestamp: 315532800
Build timestamp as int: 315532800
  

Rules_python version:

  
bazel_dep(name = "rules_python", version = "1.1.0")
bazel_dep(name = "rules_python_gazelle_plugin", version = "1.1.0")
  

Anything else relevant?

[aignas]

I think this may be duplicate of #1913. Does the error go away when using https://github.com/bazel-contrib/musl-toolchain or similar?

[remiphilippe]

here is a repro branch https://github.com/remiphilippe/bazel-rules_python_array_c/tree/not-working I added musl to it (already had it on my repo) same issue.

[remiphilippe]

ok I think I just found, the issue is not linked to gazelle version, but the default build_file_generation we use. If I add go_deps.gazelle_override( path = "github.com/dougthor42/go-tree-sitter", build_file_generation = "auto", )

to my MODULES.bazel it works (example branch https://github.com/remiphilippe/bazel-rules_python_array_c/tree/working)

[tokongs]

We also see a similar issue. The change of build_file_generation didn't fix it.

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
ld.lld: error: undefined symbol: ts_current_free
>>> referenced by array.h:164 (./external/gazelle++go_deps+com_github_dougthor42_go_tree_sitter/python/../array.h:164)
>>>               /tmp/rules_go_work-3424267487/cgo/github.com/dougthor42/go-tree-sitter/python/_x3.o:(_array__delete)

ld.lld: error: undefined symbol: ts_current_realloc
>>> referenced by array.h:185 (./external/gazelle++go_deps+com_github_dougthor42_go_tree_sitter/python/../array.h:185)
>>>               /tmp/rules_go_work-3424267487/cgo/github.com/dougthor42/go-tree-sitter/python/_x3.o:(_array__reserve)

ld.lld: error: undefined symbol: ts_current_malloc
>>> referenced by array.h:187 (./external/gazelle++go_deps+com_github_dougthor42_go_tree_sitter/python/../array.h:187)
>>>               /tmp/rules_go_work-3424267487/cgo/github.com/dougthor42/go-tree-sitter/python/_x3.o:(_array__reserve)
compilepkg: error running subcommand external/hermetic_cc_toolchain++toolchains+zig_sdk/tools/x86_64-linux-gnu.2.19/c++: exit status 1

[jbedard]

FWIW I patched this when using tree-sitter: https://github.com/aspect-build/aspect-cli/blob/main/third_party/github.com/smacker/go-tree-sitter/cc_library.patch

Is this at all related to the reasons for using a fork of go-tree-sitter?