Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent infinite recursion when migrating secrets to credentials while loading GitLab server configuration #450

Merged
merged 2 commits into from
Oct 15, 2024

Conversation

dwnusbaum
Copy link
Member

@dwnusbaum dwnusbaum commented Oct 4, 2024

I recently ran into a case where the migration code in #267 was causing an infinite loop while loading extensions during Jenkins startup, leading to various issues.

The problem is that readResolve on a class that is part of a @Extension PersistentDescriptor will be called while extensions are being loaded because PersistentDescriptor.load is annotated with @PostConstruct which gets called here when the extension is loaded. This means that it is not safe to do anything that requires other extensions to be loaded from within load and any other methods that loading may trigger. In this case the problem was the call to CredentialsProvider.lookupCredentials in GitLabServer.readResolve. To avoid this issue, I moved the migration to a static @Initializer method that runs after EXTENSIONS_AUGMENTED, so all extensions have already been loaded at that point.

Here is what the OldDataMonitor warning looked like, filtered to just the repeating stack frames:
INFO    hudson.diagnosis.OldDataMonitor#report: Trouble loading io.jenkins.plugins.gitlabserverconfig.servers.GitLabServers@4a65323
java.lang.StackOverflowError
    ... everything below repeats after this ...
    at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at hudson.ExtensionFinder$GuiceFinder$SezpozModule.onProvision(ExtensionFinder.java:634)
    at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:117)
    at com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:66)
    at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:93)
    at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:300)
    at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
    at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:169)
    at hudson.ExtensionFinder$GuiceFinder$FaultTolerantScope$1.get(ExtensionFinder.java:445)
    at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:45)
    at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1148)
    at hudson.ExtensionFinder$GuiceFinder._find(ExtensionFinder.java:403)
    at hudson.ExtensionFinder$GuiceFinder.find(ExtensionFinder.java:394)
    at hudson.ClassicPluginStrategy.findComponents(ClassicPluginStrategy.java:335)
    at hudson.ExtensionList.load(ExtensionList.java:384)
    at hudson.ExtensionList.ensureLoaded(ExtensionList.java:320)
    at hudson.ExtensionList.iterator(ExtensionList.java:172)
    at jenkins.model.Jenkins.getDescriptor(Jenkins.java:1649)
    at io.jenkins.plugins.gitlabserverconfig.servers.GitLabServersTest$CredentialsProviderThatRequiresDescriptorLookup.getCredentials(GitLabServersTest.java:71)
    at com.cloudbees.plugins.credentials.CredentialsProvider.getCredentialsInItemGroup(CredentialsProvider.java:1191)
    at com.cloudbees.plugins.credentials.CredentialsProvider.lookupCredentialsInItemGroup(CredentialsProvider.java:389)
    at com.cloudbees.plugins.credentials.CredentialsProvider.lookupCredentials(CredentialsProvider.java:348)
    at io.jenkins.plugins.gitlabserverconfig.servers.GitLabServer.migrateWebhookSecretCredentials(GitLabServer.java:421)
    at io.jenkins.plugins.gitlabserverconfig.servers.GitLabServer.readResolve(GitLabServer.java:412)
    at jdk.internal.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at com.thoughtworks.xstream.core.util.SerializationMembers.callReadResolve(SerializationMembers.java:78)
    at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:290)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:74)
    at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:72)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:68)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:52)
    at com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.readBareItem(AbstractCollectionConverter.java:132)
    at hudson.util.RobustCollectionConverter.populateCollection(RobustCollectionConverter.java:87)
    at com.thoughtworks.xstream.converters.collections.CollectionConverter.unmarshal(CollectionConverter.java:81)
    at hudson.util.RobustCollectionConverter.unmarshal(RobustCollectionConverter.java:78)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:74)
    at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:72)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:68)
    at hudson.util.RobustReflectionConverter.unmarshalField(RobustReflectionConverter.java:454)
    at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:350)
    at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:289)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:74)
    at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:72)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:68)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:52)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:136)
    at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
    at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1464)
    at hudson.util.XStream2.unmarshal(XStream2.java:230)
    at hudson.util.XStream2.unmarshal(XStream2.java:201)
    at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1441)
    at hudson.XmlFile.unmarshal(XmlFile.java:196)
    at hudson.XmlFile.unmarshal(XmlFile.java:179)
    at hudson.model.Descriptor.load(Descriptor.java:935)
    ...

There was also this Guice exception in the logs:

WARNING h.ExtensionFinder$GuiceFinder$FaultTolerantScope$1#error: Failed to instantiate Key[type=io.jenkins.plugins.gitlabserverconfig.servers.GitLabServers, annotation=[none]]; skipping this component
java.lang.IllegalStateException: Singleton is called recursively returning different results

Testing done

See new automated test.

Submitter checklist

  • Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
  • Ensure that the pull request title represents the desired changelog entry
  • Please describe what you did
  • Link to relevant issues in GitHub or Jira
  • Link to relevant pull requests, esp. upstream and downstream changes
  • Ensure you have provided tests - that demonstrates feature works or fixes the issue

@dwnusbaum dwnusbaum requested a review from jetersen as a code owner October 4, 2024 16:22
Comment on lines +70 to +74
// Prior to fix, this caused the GitLabServer migration code to recurse infinitely, causing problems when
// starting Jenkins.
// In practice this was caused by a lookup of another descriptor type, but I am using GitLabServers for
// clarity.
Jenkins.get().getDescriptor(GitLabServers.class);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just a minimal reproducer. In practice the real issue was with a CloudBees-specific CredentialProvider implementation which included a call to Jenkins.getDescriptor.

Comment on lines +58 to +59
assertThat(
logger.getMessages(), not(hasItem(containsString("Trouble loading " + GitLabServers.class.getName()))));
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't sure about the best way to test the regression, but this assertion does fail if you revert the changes to src/main, and the test log output will be full of exceptions and error messages.

Copy link

@jeromepochat jeromepochat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jetersen jetersen added the bug Something isn't working label Oct 15, 2024
@jetersen jetersen merged commit 41fbc8b into jenkinsci:master Oct 15, 2024
18 checks passed
@dwnusbaum dwnusbaum deleted the migration-recursion branch October 16, 2024 13:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants