Cannot benefit from Gradle configuration cache due to cache problems

[steve-todorov]

Description of the bug

With new Gradle 8.6 and the latest setup-gradle@v3 it is now possible to safely and securely store Gradle configuration cache which should improve the Gradle daemon startup and build times.

Unfortunately when using the Vaadin Gradle Plugin the configuration cache fails with the following error:

FAILURE: Build failed with an exception.

* What went wrong:
Configuration cache problems found in this build.

15 problems were found storing the configuration cache, 13 of which seem unique.
- Class `com.vaadin.flow.server.frontend.FrontendToolsLocator`: external process started '/opt/hostedtoolcache/node/18.19.0/x64/bin/node -v'
  See https://docs.gradle.org/8.6/userguide/configuration_cache.html#config_cache:requirements:external_processes
- Class `com.vaadin.flow.server.frontend.FrontendToolsLocator`: external process started 'which node'
  See https://docs.gradle.org/8.6/userguide/configuration_cache.html#config_cache:requirements:external_processes
- Class `com.vaadin.flow.server.frontend.FrontendUtils`: external process started '/opt/hostedtoolcache/node/18.19.0/x64/bin/node --version'
  See https://docs.gradle.org/8.6/userguide/configuration_cache.html#config_cache:requirements:external_processes
- Class `gradlegitproperties.org.eclipse.jgit.util.FS`: external process started '/usr/bin/git --version'
  See https://docs.gradle.org/8.6/userguide/configuration_cache.html#config_cache:requirements:external_processes
- Class `gradlegitproperties.org.eclipse.jgit.util.FS`: external process started '/usr/bin/git config --system --edit'
  See https://docs.gradle.org/8.6/userguide/configuration_cache.html#config_cache:requirements:external_processes
- Task `:data:generateJooqSources` of type `org.gradle.api.tasks.JavaExec`: cannot serialize object of type 'org.gradle.api.internal.tasks.DefaultSourceSet', a subtype of 'org.gradle.api.tasks.SourceSet', as these are not supported with the configuration cache.
  See https://docs.gradle.org/8.6/userguide/configuration_cache.html#config_cache:requirements:disallowed_types
- Task `:webapp:copyClasspathResourcesForDockerImage` of type `org.gradle.api.DefaultTask`: cannot serialize Gradle script object references as these are not supported with the configuration cache.
  See https://docs.gradle.org/8.6/userguide/configuration_cache.html#config_cache:requirements:disallowed_types
- Task `:webapp:copyClasspathResourcesForDockerImage` of type `org.gradle.api.DefaultTask`: cannot serialize object of type 'org.gradle.api.internal.project.DefaultProject', a subtype of 'org.gradle.api.Project', as these are not supported with the configuration cache.
  See https://docs.gradle.org/8.6/userguide/configuration_cache.html#config_cache:requirements:disallowed_types
- Task `:webapp:generateGitProperties` of type `com.gorylenko.GenerateGitPropertiesTask`: invocation of 'Task.project' at execution time is unsupported.
  See https://docs.gradle.org/8.6/userguide/configuration_cache.html#config_cache:requirements:use_project_during_execution
- Task `:webapp:vaadinBuildFrontend` of type `com.vaadin.gradle.VaadinBuildFrontendTask`: cannot serialize object of type 'org.gradle.api.internal.project.DefaultProject', a subtype of 'org.gradle.api.Project', as these are not supported with the configuration cache.
  See https://docs.gradle.org/8.6/userguide/configuration_cache.html#config_cache:requirements:disallowed_types
- Task `:webapp:vaadinBuildFrontend` of type `com.vaadin.gradle.VaadinBuildFrontendTask`: invocation of 'Task.project' at execution time is unsupported.
  See https://docs.gradle.org/8.6/userguide/configuration_cache.html#config_cache:requirements:use_project_during_execution
- Task `:webapp:vaadinPrepareFrontend` of type `com.vaadin.gradle.VaadinPrepareFrontendTask`: cannot serialize object of type 'org.gradle.api.internal.project.DefaultProject', a subtype of 'org.gradle.api.Project', as these are not supported with the configuration cache.
  See https://docs.gradle.org/8.6/userguide/configuration_cache.html#config_cache:requirements:disallowed_types
- Task `:webapp:vaadinPrepareFrontend` of type `com.vaadin.gradle.VaadinPrepareFrontendTask`: invocation of 'Task.project' at execution time is unsupported.
  See https://docs.gradle.org/8.6/userguide/configuration_cache.html#config_cache:requirements:use_project_during_execution

See the complete report at file:///home/runner/work/project/build/reports/configuration-cache/1jq3mq1vr8s4v3i6l9aeumbf8/96wlf1meeyl5jhjnfvpm1p2sy/configuration-cache-report.html
> Starting an external process '/usr/bin/git --version' during configuration time is unsupported.
> Starting an external process '/usr/bin/git config --system --edit' during configuration time is unsupported.
> Starting an external process 'which node' during configuration time is unsupported.
> Starting an external process '/opt/hostedtoolcache/node/18.19.0/x64/bin/node -v' during configuration time is unsupported.
> Starting an external process '/opt/hostedtoolcache/node/18.19.0/x64/bin/node --version' during configuration time is unsupported.

The full error log with stacktrace can be found in the attached file:

error.log

Expected behavior

Gradle build should pass successfully with configuration cache enabled.

Minimal reproducible example

• Add this to your projectRoot/gradle.properties:

# Kotlin
kotlin.caching.enabled=true
kotlin.incremental=true
kotlin.incremental.useClasspathSnapshot=true
kotlin.parallel.tasks.in.project=true
# Gradle
org.gradle.parallel=true
org.gradle.caching=true
org.gradle.daemon.idletimeout=3600000
org.gradle.configuration-cache=true
org.gradle.configuration-cache.max-problems=100

• Generate a "secret phrase" using openssl rand -base64 16 and store it as a secret GRADLE_ENCRYPTION_KEY
• In your workflow add the following:

      - name: Set up JDK
        uses: actions/setup-java@v4
        with:
          java-version: '21'
          distribution: 'temurin'

      - name: Setup Gradle
        uses: gradle/actions/setup-gradle@v3
        with:
          # The Gradle wrapper's version (already the default, putting it here to clarity)
          gradle-version: wrapper
          # Removing unused files from Gradle User Home before saving to cache (i.e. older versions of gradle)
          gradle-home-cache-cleanup: true
          # Cache downloaded JDKs in addition to the default directories.
          gradle-home-cache-includes: |
            caches
            notifications
            jdks
          cache-encryption-key: ${{ secrets.GRADLE_ENCRYPTION_KEY }}
          # Only write to the cache for builds on the 'main' and 'release' branches. (Default is 'main' only.)
          # Builds on other branches will only read existing entries from the cache.
          #cache-read-only: ${{ github.ref != 'refs/heads/main'  }}
          # Allow writing to cache
          cache-read-only: true

      - name: Build project
        env:
          GRADLE_ENCRYPTION_KEY: ${{ secrets.GRADLE_ENCRYPTION_KEY }}
        run: ./gradlew build --scan --stacktrace

Versions

• Vaadin / Flow version: 24.3.3
• Java version: 21-tem
• OS version: ubuntu

[mshabarov]

Gradle's configuration cache seems to be a new feature in latest versions and, thus, I'd consider this as an enhancement in Vaadin, despite current Gradle plugin fails with it.
Let us read about this feature in Gradle and figure out how we can fix the above errors.

[steve-todorov]

@mshabarov Any updates on this?

[mshabarov]

@steve-todorov no updates unfortunately, this isn't a priority for Vaadin Flow team at the moment. But seems this ticket gets more votes, thus I'm not loosing a hope that we'd start doing it at some point, though cannot give any promise.

[steve-todorov]

Thanks for the update @mshabarov. This is very unfortunate considering the latest Gradle 8.10 has additional configuration cache improvements (>=30%)

[benstpierre]

Gradle 8.11 came out today with further improvements to configuration cache performance. Sadly we cannot take advantage of it due to this bug. https://docs.gradle.org/8.11/release-notes.html

[mcollovati]

@steve-todorov thanks for your comment.
--configuration-cache should not be required, since the same configuration is specified in gradle.properties. Actually, looking at the logs the configuration cache is correctly written.
I'm currently trying to test with the Spring gradle starter, to get rid of gretty plugin

[mcollovati]

Tested on the Spring gradle starter: configuration cache is written, but now on the second run (and all other runs) it says

Calculating task graph as configuration cache cannot be reused because file '../../../.gradle/caches/modules-2/files-2.1/org.jspecify/jspecify/1.0.0/fae4a252aeee8a649653b2f239067bcead8f80cb/jspecify-1.0.0.pom' has changed.

Locally it seems to work fine; I always see Reusing configuration cache when building without applying any change.

After a couple of attempts, it now seems to work. If I get correctly, the missing part was to prevent the GH action to write the cache for the PR, but only write it on the target branch
Basically, the following config, given the PR branch is test/spikes/20950-gradle_cache.

cache-read-only: ${{ github.ref != 'refs/heads/main' && github.ref != 'refs/heads/spikes/20950-gradle_cache' }}

https://github.com/vaadin/base-starter-spring-gradle/actions/runs/13460501965/job/37614748186?pr=258#step:5:29

Reusing configuration cache.
....
> Task :vaadinPrepareFrontend FROM-CACHE
Build cache key for task ':vaadinPrepareFrontend' is bc6e8134eb2687a12d72d414b185887c
Task ':vaadinPrepareFrontend' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':vaadinPrepareFrontend' with cache key bc6e8134eb2687a12d72d414b185887c
Resolve mutations for :processResources (Thread[#[55](https://github.com/vaadin/base-starter-spring-gradle/actions/runs/13460501965/job/37614748186?pr=258#step:5:56),Execution worker Thread 3,5,main]) started.
:processResources (Thread[#55,Execution worker Thread 3,5,main]) started.

[mcollovati]

Support for Gradle cache has been merged also in Hilla and Platform

[vaadin-bot]

This ticket/PR has been released with Vaadin 24.8.0.alpha1 and is also targeting the upcoming stable 24.8.0 version.

[mshabarov]

@steve-todorov and all who interested in Gradle configuration cache support, could you please guys try using Vaadin 24.8.0.alpha1 pre-release that has this support and let us know about the result?

[drse]

@steve-todorov and all who interested in Gradle configuration cache support, could you please guys try using Vaadin 24.8.0.alpha1 pre-release that has this support and let us know about the result?

After one week of use, seems to be working.