YARN-11794. hadoop-yarn-csi does not build on included Centos 7 docker image

[stoty]

also fix Centos 7 docker image build

Description of PR

Revert io.grpc:protoc-gen-grpc-java to 1.65.0 to enable building on Centos 7.
Also backported Centos 7 image build fix from trunk.

How was this patch tested?

Started Centos7 container.
ran 'mvn clean package -DskipTests'

For code changes:

• Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
• Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
• If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	7m 6s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ branch-3.4 Compile Tests _
+1 💚	mvninstall	22m 11s		branch-3.4 passed
+1 💚	compile	0m 22s		branch-3.4 passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	0m 20s		branch-3.4 passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	mvnsite	0m 22s		branch-3.4 passed
+1 💚	javadoc	0m 24s		branch-3.4 passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 20s		branch-3.4 passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	shadedclient	19m 19s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 16s		the patch passed
+1 💚	compile	0m 15s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	0m 15s		the patch passed
+1 💚	compile	0m 14s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	javac	0m 14s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 17s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	0m 14s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 14s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	shadedclient	19m 8s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 28s		hadoop-yarn-csi in the patch passed.
+1 💚	asflicense	0m 25s		The patch does not generate ASF License warnings.
		72m 24s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7499/1/artifact/out/Dockerfile
GITHUB PR	#7499
Optional Tests	dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs compile javac javadoc mvninstall shadedclient xmllint
uname	Linux e364faa10d57 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.4 / 8a80d59
Default Java	Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7499/1/testReport/
Max. process+thread count	552 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7499/1/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[pan3793]

It looks risky to use a different gRPC version for codegen and runtime, an alternative I can imagine -track the generated source code in Git, and only regenerate the code on each gRPC upgrading

[stoty]

There are no code changes in the compiler between those versions , @pan3793

Only the build OS is changed, event the netty version change is reverted:
https://github.com/grpc/grpc-java/commits/master/compiler

The incompatibility is caused by
grpc/grpc-java@71eb5fb

committing generated code to git is troublesome, I would very much like to avoid that.

[pan3793]

There are no code changes in the compiler between those versions

it's fine for these specific version pair, but may not be good for future upgrading

committing generated code to git is troublesome, I would very much like to avoid that.

I tend to agree with you in most cases, but I think for this case, it's a most safe and relatively cheap way.

[stoty]

it's fine for these specific version pair, but may not be good for future upgrading

We can cross that rover when we get there.

If you REALLY don't want to use different versions, then I'd preder recompiling the plugin from source on Centos 7, though I'd hate to put in that much effort to support EOL distros.

[pan3793]

Another way used by Spark, allow user to configure pre-installed protoc/grpc commands to generate the code, then we can compile and install those commands from the source code on the CentOS 7 build image (I'm not familiar with the native world, so have no idea how much maintenance effort will be paid for this way)

[stoty]

Yes, that's what I meant by recompiling the grpc compiler from source.

[pan3793]

BTW, Hadoop actually has committed a few generated code into Git repo
https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/arm-java

[slfan1989]

Thanks to both of you for the discussion! In the current situation, I lean towards supporting Story's approach, as long as it doesn't block the progress of Hadoop 3.4.2. In fact, we don't actually need this CentOS7 script during the release process. If a similar issue arises in a future version of Ubuntu, I believe we can follow @pan3793 method to fix it.