Using Java’s Security Policy to Block Internet Access for JUnit Tests
We have a relatively large Java code base (387K LOC as reported by our SonarQube installation), written by a team of 82 engineers (as of this writing). As part of our workflow, we ensure that all new code contributed by the usual pull request workflow is covered with 85% of coverage.
Recently, one of our teams introduced a test case which relied on an external resource (on the internet) to pass. Out of the blue, our build engine, Bamboo, started reporting test failures. We dove right into it, since we require all tests to pass for a pull request to be merged. After a brief investigation, we narrowed it down to this particular test case, which started failing.
What was wrong? It turns out that this particular service recently implemented a throttling mechanism, and due to concurrent builds, some used to fail. This wasn’t handled cleanly in that particular test case, and once fixed, everything turned green.
The Dilemma
Although we addressed the problem initially, it led us to ask ourselves, how do we prevent this from happening in the future? At CleverTap, once we hit a problem, we not only patch it, but bulletproof the solution so it never occurs again.
The immediate question was: should we allow unit tests to hit resources on the internet? Turns out, the answer to this was no.
At no point should an engineer without access to the internet be unable to run test cases in their development environment (think of cases such as developing during an eight hour flight).
So how do we prevent unit tests from hitting the internet?
Potential Solutions
Does JUnit Support This Natively?
Surprisingly no, as is evident from this GitHub issue.
Use A Custom SecurityManager
As instructed by that particular issue linked above, we started forming ideas on how to prevent this. While using a custom security manager worked well, we unfortunately didn’t have a base class from which all tests across all our modules extended.
Run Tests In A Docker Container
This one could possibly be an easy option, but we passed on it quickly since it would require significant changes to our build infrastructure.
Use A Custom Java Security Policy File
Instead of using a custom SecurityManager, we could use a custom Java Security Policy configuration. This solution was feasible for the following reasons:
No Changes In The Codebase
Since we use Maven + JUnit, we could simply set our new policy file in the argLine parameter:
<!--The double == below is required, since it prevents the default policy from loading.-->
<argLine>
-Djava.security.manager
-Djava.security.policy==${maven.multiModuleProjectDirectory}/java_policy_for_tests.policy
</argLine>
Controlled By Engineers
Engineers should own the entire build pipeline, and there shouldn’t be any changes to the build infrastructure.
The Policy File
So what does a security policy file look like? None of us had ventured into this aspect of the JVM before, and it took some quick learning. Java has a default policy file, which is always loaded. By default, it grants all permissions to the entire codebase.
After spending an hour or so, we were able to build a security policy which was fairly permissible, while still restricting access to the internet. This is what we came up with:
// A Java security policy file which prohibits access to the outside network.
// This is useful for force failing tests which hit the internet, since the
// development workflow MUST never be blocked/throttled due to an external API.
grant codeBase "file:${java.ext.dirs}/*" {
permission java.security.AllPermission;
};
// Inspired from https://developer.jboss.org/docs/DOC-17431
grant codeBase "file:${java.home}/lib/*" {
permission java.security.AllPermission;
};
// For java.home pointing to the JDK jre directory
grant codeBase "file:${java.home}/../lib/*" {
permission java.security.AllPermission;
};
grant {
// Allow resolution for *, since it helps create URL entries with actual domain names.
permission java.net.SocketPermission "*:*", "resolve";
permission java.net.SocketPermission "localhost:0", "accept, listen, connect, resolve";
permission java.net.SocketPermission "127.0.0.1:*", "accept, listen, connect, resolve";
permission java.net.SocketPermission "[0:0:0:0:0:0:0:1]:*", "accept, listen, connect, resolve";
permission java.util.PropertyPermission "*", "read, write";
permission java.io.FilePermission "<<ALL FILES>>", "read, write, execute, delete, readlink";
permission java.io.FilePermission "/*", "read";
permission java.lang.reflect.ReflectPermission "*", "*";
permission java.lang.RuntimePermission "*";
permission java.security.SecurityPermission "*";
permission java.security.LoggingPermission "control";
permission java.lang.management.ManagementPermission "monitor";
permission java.lang.management.ManagementPermission "control";
permission com.sun.tools.attach.AttachPermission "createAttachProvider";
permission com.sun.tools.attach.AttachPermission "attachVirtualMachine";
permission javax.management.MBeanServerPermission "*";
permission javax.management.MBeanPermission "*", "*";
permission javax.management.MBeanTrustPermission "*";
permission java.net.NetPermission "*";
permission javax.security.auth.AuthPermission "*";
permission javax.xml.bind.JAXBPermission "*";
};
Tracing Some Unusual Permissions
Some of the permissions there are a bit unusual, and were hard to track down. We use PowerMockito, which plays with the classloader and even bytecode that’s generated. In order to track what permissions it required, we found a handy debug flag:
-Djava.security.debug=access,failure
This debug flag was useful in tracking down some mysterious permissions that PowerMockito used. One can also use that tool with the default security policy file to understand the minimum set of permissions required for a particular application.
Conclusion
We managed to leverage Java’s built-in security policy to stop tests from accessing the internet, and prevent this from happening again. Feel free to adapt the policy above to your liking, and let us know what your experience with it is!
