Dependencies and Class Loading
Jenkins has a complex, modular architecture. To enable plugins to use the rich array of libraries available in the Java ecosystem, and build on one another using plugin-to-plugin APIs, the Jenkins plugin extension mechanism goes beyond simple plugin manifests.
The de facto build tool for Jenkins plugins is Apache Maven. All examples here will use Maven. Building plugins using Gradle is also possible but particular capabilities may lag behind.
Jenkins class loading
Before diving into how to refer to libraries and other plugins from your plugin, it is useful to understand the elements of Java class loading and how that applies to Jenkins. If you are not a Java developer, you can get away with ignoring all this for simple plugins, but will need to understand class loading to troubleshoot strange errors or perform advanced packaging.
The plugin class loader hierarchy
□ Java Platform
↖
□ “application classpath” (servlet container): java -jar jenkins.war
↖
□ Jenkins core: jenkins.war!/WEB-INF/lib/*.jar
↖
□ plugin A: $JENKINS_HOME/plugins/a.jpi!/WEB-INF/lib/*.jar
↖ ↖
□ plugin C: $JENKINS_HOME/plugins/c.jpi!/WEB-INF/lib/*.jar ← □ UberClassLoader
↖ ↙ ↙
□ plugin B: $JENKINS_HOME/plugins/b.jpi!/WEB-INF/lib/*.jar
⋮
Java class loaders have parents to which they delegate. For purposes of Jenkins, what matters most is that “Jenkins core” delegates to the Java Platform, and Jenkins plugins all delegate to Jenkins core.
Plugins may also delegate to one another.
In the above example, C declares dependencies on A and B.
In its c.jpi!/META-INF/MANIFEST.MF
this would look something like:
Plugin-Dependencies: a:1.13,b:1.6
Thus C can directly refer to classes defined in either A or B, as well as to Jenkins and the Java Platform:
package org.jenkinsci.plugins.c;
import java.util.Date;
import jenkins.model.Jenkins;
import org.jenkinsci.plugins.a.Alpha;
import org.jenkinsci.plugins.b.Beta;
public class Charlie {/* … */}
Each (enabled) plugin gets its own ClassLoader
.
It is possible for two distinct plugins to define a class of the same name, so long as neither depends on the other.
There is also a single special UberClassLoader
which delegates to all enabled plugins.
This never loads any additional resources but it can be used to look up any plugin class or resource.
Initiating vs. defining loaders
When you initiate loading of a class
Class<?> alpha = Charlie.class.getClassLoader().loadClass("org.jenkinsci.plugins.a.Alpha");
you are asking a particular ClassLoader
(here, C’s) to look up a class by name.
(Normally this will start by asking all of its parents to find that class,
and only look in c.jpi!/WEB-INF/lib/*.jar
as a last resort.)
What happens if Alpha.class
refers to various other classes, for example by using them in import
statements?
Java considers the defining loader of Alpha.class
(A’s), and only asks that class loader.
The fact that you started loading in C is irrelevant.
This means that C cannot provide some class A might need to link against.
A needs to be able to “see” any such class on its own.
Otherwise you will get a NoClassDefFoundError
at runtime.
Context class loaders
Java defines a Thread.getContextClassLoader()
.
Jenkins does not use this much; it will normally be set by the servlet container to the Jenkins core loader.
Some Java libraries have a fundamental design flaw, originating in premodular systems with a “flat classpath”, whereby they expect classes defined by clients of the library to be available by name without any qualification:
package org.somelib;
public class LibClass {
public static Object use(String name) throws Exception {
return getUserClass(name).newInstance();
}
private Class<?> getUserClass(String name) throws ClassNotFoundException {
return Class.forName(name);
}
}
This does not work properly in Jenkins because the library org.somelib
might be defined in plugin A,
while the user class is defined in plugin B depending on A.
Class.forName(String)
uses the calling class (LibClass
) to determine the initiating ClassLoader
;
since plugin A cannot load classes from plugin B, this will result in a ClassNotFoundException
.
Other libraries try to work around the issue as follows:
package org.somelib;
public class LibClass {
public static Object use(String name) throws Exception {
return getUserClass(name).newInstance();
}
private Class<?> getUserClass(String name) throws ClassNotFoundException {
return Class.forName(name, true, Thread.currentThread().getContextClassLoader());
}
}
This also fails by default in Jenkins, since the contextClassLoader
can see only Jenkins core (not even plugin A, much less B).
Jenkins plugin code can work around the issue in some cases:
package org.jenkinsci.plugins.b;
import org.somelib.LibClass;
class UsesThatLib {
Object someMethod() {
Thread t = Thread.currentThread();
ClassLoader orig = t.getContextClassLoader();
t.setContextClassLoader(UsesThatLib.class.getClassLoader());
try {
return LibClass.use(UsesThatLib.class.getName());
} finally {
t.setContextClassLoader(orig);
}
}
}
(When the particular class loader needed is unclear, UberClassLoader
can be used as a fallback,
though this is not as safe since lookups could be ambiguous in case two unrelated plugins both bundled the same library.)
Ultimately however it is a design flaw in the library if it fails to allow clients to directly specify a ClassLoader
to use for lookups
(or preregister Class
instances for particular names).
Consider patching the library or looking harder for appropriate APIs that already exist.
As an example, java.io.ObjectInputStream
(used for deserializing Java objects) by default uses a complicated algorithm to guess at a ClassLoader
,
but you can override resolveClass
to remove the need for guessing (as hudson.remoting.ObjectInputStreamEx
in fact does).
Views and other resources
Jenkins plugins normally contain “Stapler views” like config.jelly
as well as other resources in src/main/resources/
.
While you can explicitly load these from Java code:
package org.jenkinsci.plugins.a;
public class Alpha {
/** loads {@code /org/jenkinsci/plugins/a/config.txt} from {@code a.jpi!/WEB-INF/lib/a.jar} */
static URL config() throws IOException {
return Alpha.class.getResource("config.txt");
}
}
normally such resources would be loaded on your behalf, for example by the convention of Jenkins looking for a view.
In such cases the lookup passes through UberClassLoader
, so your resource path (/org/jenkinsci/plugins/a/config.txt
)
had better be globally unique.
Messages.properties
used for localization is a little different,
since this is actually compiled to Messages.class
during the build,
and thus behaves like any other Java class referred to statically from your plugin code:
package org.jenkinsci.plugins.a;
public class Alpha {
/** compiled from {@code /org/jenkinsci/plugins/a/Messages.properties#Alpha.message} */
static String message() throws IOException {
return Messages.Alpha_message();
}
}
Depending on other plugins
Making your plugin depend on other plugins is easy: just declare dependencies in your POM, by hand or using your favorite IDE.
<dependencies>
<dependency>
<groupId>org.jenkins-ci.plugins</groupId>
<artifactId>a</artifactId>
<version>1.13</version>
</dependency>
<dependency>
<groupId>org.jenkins-ci.plugins</groupId>
<artifactId>b</artifactId>
<version>1.6</version>
</dependency>
</dependencies>
The Maven packaging type for Jenkins plugins understands to translate this to the Plugin-Dependencies
manifest header,
which will be understood by the Jenkins plugin manager, as well as the update center and other tools.
The Maven compiler plugin similarly understands that a-1.13.jar
and b-1.6.jar
should be added to your classpath when building your plugin.
Extensions and inversion of control
A “service locator” pattern is used throughout Jenkins for modularity and extensibility. For example, if a plugin (or core) defines an API
package org.jenkinsci.plugins.someapi;
import hudson.ExtensionPoint;
public interface Checker extends ExtensionPoint {
boolean doesThisSeemOK(String input);
}
then another plugin may declare a dependency on that API
<dependency>
<groupId>org.jenkins-ci.plugins</groupId>
<artifactId>someapi</artifactId>
<version>1.0</version>
</dependency>
and add an extension:
package org.jenkinsci.plugins.somethingelse;
import hudson.Extension;
import org.jenkinsci.plugins.someapi.Checker;
@Extension
public class MyChecker implements Checker {
@Override
public boolean doesThisSeemOK(String input) {
return !input.contains("/");
}
}
Now any code able to link against someapi
can use those implementations;
most commonly this is done inside the same API plugin:
package org.jenkinsci.plugins.someapi;
import hudson.ExtensionList;
class RunsChecks {
static boolean allFine(String input) {
for (Checker c : ExtensionList.lookup(Checker.class)) {
if (!c.doesThisSeemOK(input)) {
return false;
}
}
return true;
}
}
It is important to understand that while MyChecker
needs to link against Checker
, mandating that dependency
,
RunsChecks
does not need to be able to link against MyChecker
(or any of the other implementations).
While the local variable c
’s implementation class might be in the somethingelse
plugin,
it need only care about the declared type Checker
.
Bundling third-party libraries
Sometimes plugins need to use Java libraries beyond what is available in the Java Platform and Jenkins itself. For example, a plugin connecting to a particular service might use a Java SDK provided by the vendor.
Doing this is very easy—in principle. Simply declare a Maven dependency on that library:
<dependency>
<groupId>com.yoyodyne.cloud</groupId>
<artifactId>cloud-access-sdk</artifactId>
<version>1.0</version>
</dependency>
(This assumes that the library is available in Maven Central. If not, it is possible to upload artifacts to the Jenkins Artifactory repository for use from plugins. Ask on the developer list for help. Do not attempt to keep such binaries in source control.)
Besides making SDK classes (say, com.yoyodyne.cloud.*
) available during compilation,
the maven-hpi-plugin
used to create Jenkins plugins will notice that this dependency is not itself a Jenkins plugin,
and instead bundle it inside yourplugin.hpi
as WEB-INF/lib/cloud-access-sdk-1.0.jar
.
At runtime, the plugin class loader will load classes from WEB-INF/lib/cloud-access-sdk-1.0.jar
,
just as it would from WEB-INF/lib/yourplugin.jar
(your plugin’s own code, from src/main/java/
and src/main/resources/
).
Thus your plugin’s classes can refer to classes in that library.
Other plugins depending on your plugin can, too.
Checking WEB-INF/lib/*.jar
for junk
Beware that Maven dependencies include all transitive dependencies.
This can lead to unexpected results when bundling libraries.
For example, the POM for com.yoyodyne.cloud:cloud-access-sdk
might declare that it needs commons-net:commons-net:3.5
.
Your plugin will thus wind up bundling commons-net-3.5.jar
as well.
If you are not careful, WEB-INF/lib/
may fill up with megabytes of stuff which is not actually used.
Using library wrapper plugins
In practice it is undesirable for Jenkins feature plugins to bundle assorted third-party libraries.
Typically other plugins will need some of the same libraries, so multiple plugins will wind up bundling copies.
Even if all of these copies happen to be the exact same version, “downstream” plugins can wind up getting linkage errors.
And users will wind up downloading multiple copies of the same code,
increasing product and $JENKINS_HOME/plugins/
sizes, update center load, HotSpot compiler delays, etc.
Another problem is that updating library versions becomes harder to centralize when they are bundled independently in multiple plugins. While having each plugin define an exact version of a library does reduce the risk of API compatibility errors, this is outweighed by the need to update a library with assurance when new security vulnerabilities (or other critical fixes) are announced.
To centralize library management, you can instead define a wrapper plugin, or find one someone else has already defined.
This is a Jenkins plugin which contains no sources of its own and simply includes a dependency
on some library.
After being published on the update center, other “feature” plugins can declare plain plugin-to-plugin dependencies on it and thus use the library.
Occasionally it can make sense to include a few API classes in the wrapper plugin itself, where any Jenkins code using the library would reasonably need some boilerplate adapters to standard Jenkins facilities.
To mitigate the risk of library plugin updates with incompatible changes breaking plugin functionality,
it is suggested to include the major version of the library in the wrapper plugin’s artifactId
: for example, commons-lang3
.
(This assumes that the library follows something like Semantic Versioning.)
Thus there could be multiple major releases loaded simultaneously.
Of course this means that critical fixes must be issued as updates to all release lines, so only do this for supported versions of the library.
pluginFirstClassLoader
and its discontents
The default behavior of Java class loaders, and of Jenkins plugin class loaders as well,
is to service loadClass
requests first by asking the parent loader(s) for the class of that name,
and only if that fails to check whether the class could be defined among JARs present in the initiating loader.
This is generally sensible since it ensures that types mentioned in bytecode from different plugins
are resolved at runtime to the same Class
objects and thus allowing APIs to work using shared type signatures.
Sometimes, however, it is not wanted for the parent to be searched first. For example, Jenkins core currently bundles a plethora of third-party libraries and exposes all of their packages to plugins. For that matter, some plugins bundle third-party libraries that are then incidentally exposed to other plugins needing a dependency on APIs defined in the bundling plugins. If a plugin also needs some variant of the same library for its own use, it will surprisingly not be able to load it, because the parent class loader will find it first.
To avoid that behavior, it is possible to set a flag pluginFirstClassLoader
in the plugin’s Maven POM.
(This simply produces a JAR manifest entry that is interpreted by the Jenkins plugin system at runtime.)
This flag instructs the plugin class loader to check its own JARs first for any mentioned class names.
You must be very careful when using this mode, since any type normally defined in core (or an “upstream” plugin)
which is mentioned in any part of the Java signature of a method you are calling must not be duplicated in your JARs, or linkage errors will result.
Thus it is best reserved to libraries used purely by internal implementation.
A more limited flag is maskClasses
, which blocks only selected classes or packages from the parent loader, rather than everything.
You must manually verify that the masked classes are complete under the transitive closure of the Java linker:
for example, masking one package but not another from a library bundled in core could make classes in the masked package unresolvable.
There is a related flag globalMaskClasses
which adjusts the behavior of every loaded plugin to essentially override a component of the Java Platform.
If you did not understand any part of this section, do not use these options. Even if you did, think twice.
Shading
Another approach to taming the complexity of library dependencies and class loading is loosely referred to as shading,
exemplified by the Maven Shade plugin (though multiple techniques are possible).
In this case, rather than trying to control where a given classLoader.loadClass("org.apache.commons.lang.StringUtils")
call finds the defining loader,
the idea is to bundle the entire library under a distinct package prefix, rewriting all static and reflective class (or resource) loads,
both within the library and from code defined in the plugin doing the bundling.
Thus your-plugin.hpi!/WEB-INF/lib/commons-lang-shaded.jar
might contain entries like org/jenkinsci/plugins/yourplugin/commonslang/StringUtils.class
;
and plugin source code like
import org.apache.commons.lang.StringUtils;
// …
if (StringUtils.isEmpty(arg)) {/* … */}
would result in bytecode in your-plugin.hpi!/WEB-INF/lib/classes.jar
referring in its constant pool to the type org.jenkinsci.plugins.yourplugin.commonslang.StringUtils
.
Since that type name is unique in the whole Jenkins JVM, there is no risk of it being loaded from the wrong place;
from the perspective of Jenkins, it is as if you copied and pasted the whole Commons Lang library into some sources in your plugin.
While this system does address the risk of linkage errors, it does nothing to reduce the profusion of library versions in Jenkins, as described in the section on library wrappers. In some cases, however, library wrappers themselves will use this same trick for official purposes, so that the wrapped library will be present under a package name indicating the major release version. Plugins using the wrapped library therefore would refer to the repackaged name:
import org.jenkinsci.commons_lang2.StringUtils;
@Restricted
annotations
The public
modifier in Java allows types, methods, constructors, and fields to be accessed from any other class in the system capable of linking against the defining type.
Thus it forms part of the API, as do protected
members.
However in some cases use of these access modifiers are forced for technical reasons rather than out of an intent to define an API:
a method might need to be accessed from a class in another package in the same plugin, preventing use of default package access;
an @Extension
needs to be public
to allow the Jenkins service loader to instantiate it;
a FormValidation doCheckName(@QueryParameter String value)
method must be public
to expose it as a Stapler web method and thus to JavaScript on a form.
In such cases you should block the member from being used by outside code: nonessential API additions are at risk of being used in unintended ways and forcing your plugin to maintain the member more or less forever lest backward compatibility be broken. This can be accomplished by use of a special annotation available in Jenkins code:
@Restricted(NoExternalUse.class)
@Extension
public class MyListener extends ItemListener {/* … */}
Other plugins attempting to refer to MyListener
will receive a build-time error.
Therefore you are free to rename, move, delete, or otherwise modify MyListener
at any time.
Several kinds of restriction are available; consult Javadoc for details.
This system has no effect on accesses using java.lang.reflect
.
JenkinsRule
vs. acceptance-test-harness
class loading
There are three main categories of automated test used in Jenkins:
-
Unit tests, including mocking frameworks such as PowerMock.
-
Functional tests based on the
JenkinsRule
API defined injenkins-test-harness
. -
Acceptance tests located in the
acceptance-test-harness
repository.
These have different levels of fidelity to the class loading behavior of plugin code running in production Jenkins servers.
Unit tests simply pick up the Java classpath (java.class.path) defined by Maven’s test
scope.
Acceptance tests run a full Jenkins server and install plugins (including the plugin(s) being tested) using Jenkins’ normal mechanisms. Since the test does not compile or link against any types defined in the Jenkins runtime (only against the Selenium web driver), and does not even run in the same JVM as Jenkins, it has no interaction with the class loading of Jenkins. Thus the class loading behavior of plugins running in an acceptance test can be assumed to be the same as in production.
Class loading in functional tests is intermediate in behavior, but closer to that of unit tests.
Test code does link against Jenkins core and (test
-scoped) plugin types,
and everything in the Java classpath is in fact loaded in Java’s application class loader—including plugins in test
scope.
This means that certain mistakes in plugin metadata (for example, misuse of pluginFirstClassLoader
) may go unnoticed.
(JenkinsRule
does start a real Jenkins service, and in some cases other plugins can get installed which were not in the Java classpath.
These get their own class loaders.)
Whenever there is any doubt about whether class loading behavior could affect plugin or core code,
use RealJenkinsRule
which launches Jenkins in a separate JVM with the regular class loader hierarchy.
Jenkins modules
Historical versions of jenkins.war
included a few components called modules which are built and packaged just like plugins, and can refer to types defined in Jenkins core,
but which are bundled alongside core and cannot be managed by users in the plugin manager.
The usual reason for this design is to include features which either cannot be disabled,
or must be present early in the Jenkins startup sequence, before plugins have been fully initialized.
For purposes of class loading, these components behave like anything else bundled inside core: modules do not get their own class loaders.
For Maven dependency management purposes, plugins can declare provided
-scope dependencies on these modules if they wish to use their APIs,
taking care to select the same version as is bundled in the baseline version of core.
https://github.com/jenkinsci/jep/tree/master/jep/230/README.adoc removed this system at least from the default Jenkins distribution.