When builds fail for no reason: Feeding Maven memory

Posted by: Lincoln Baxter III on 04/25/2012
Ever experience a wonderfully fantastic “green bar!” in Eclipse, NetBeans, or IntelliJ, only to find that when you run your ANT or Maven build, you get an equally catastrophic build failure? If you have, you’ve probably tried what most of us tried, and you’ve attempted to increase Maven’s heap capacity using MVN_OPTS:
export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=512m"
That would normally work fine if the error we received is something like:
java.lang.OutOfMemoryError: PermGen space
Unfortunately, though, this doesn’t solve the problem because Maven actually uses a separate JVM for each JUnit test execution! So while we have successfully enabled Maven to be a hog, our tests are still running in a constrained environment. What we need to do instead is increase the memory capacity of the Maven Surefire launcher:
Increasing JUnit Test Memory in Maven
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <argLine>-Xms256m -Xmx512m -XX:MaxPermSize=512m</argLine>
  </configuration>
</plugin>

Our solution

This will expand your test memory capacity significantly, and if it’s still not enough, you can play with these numbers for your environment; however, to all you Maven users out there who’ve run into these problems, may the Force be with you, because sometimes it’s not obvious when a memory issue is the root of your problems. However, if you are using Arquillian in your test suite, chances are that you are going to run into a memory issue at some point, because the memory required to start up a container like Jetty or Tomcat in your test suite is significant. If you are using JBoss AS7 or GlassFish, then your server container will run in a different JVM and this will not typically be an issue.

A very confusing example

Here is an interesting JVM core dump in the Rewrite and PrettyFaces test suites that was actually being caused by an out-of-memory error leading to native code execution failure, but nothing in the error trace would lead us to think it! (Unimportant bits omitted:)
Core dump caused by Out-of-memory error
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGBUS (0x7) at pc=0x00007fac3c24e3bf, pid=32027, tid=140377731290880
#
# JRE version: 7.0_03-b04
# Java VM: Java HotSpot(TM) 64-Bit Server VM (22.1-b02 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libc.so.6+0x1313bf]  __nss_hosts_lookup+0x1117f
#
# Core dump written. Default location: ~/Projects/ocpsoft/prettyfaces/annotations/core or core.32027
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.sun.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

---------------  T H R E A D  ---------------

Current thread (0x0000000000bef800):  JavaThread "main" [_thread_in_native, id=32030, stack(0x00007fac3cbed000,0x00007fac3ccee000)]

siginfo:si_signo=SIGBUS: si_errno=0, si_code=2 (BUS_ADRERR), si_addr=0x00007fac3534c000

Registers:
...

Top of Stack: (sp=0x00007fac3cce7b68)
...

Instructions: (pc=0x00007fac3c24e3bf)
...

Register to memory mapping:
...


Stack: [0x00007fac3cbed000,0x00007fac3ccee000],  sp=0x00007fac3cce7b68,  free space=1002k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libc.so.6+0x1313bf]  __nss_hosts_lookup+0x1117f
C  [libzip.so+0x5057]  ZIP_GetNextEntry+0x57
j  java.util.zip.ZipFile.getNextEntry(JI)J+0
j  java.util.zip.ZipFile.access$500(JI)J+2
j  java.util.zip.ZipFile$1.nextElement()Ljava/util/zip/ZipEntry;+54
j  java.util.zip.ZipFile$1.nextElement()Ljava/lang/Object;+1
j  java.util.jar.JarFile$1.nextElement()Ljava/util/jar/JarFile$JarFileEntry;+4
j  java.util.jar.JarFile$1.nextElement()Ljava/lang/Object;+1
j  sun.misc.URLClassPath$JarLoader.validIndex(Ljava/lang/String;)Z+42

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  java.util.zip.ZipFile.getNextEntry(JI)J+0
j  java.util.zip.ZipFile.access$500(JI)J+2
j  java.util.zip.ZipFile$1.nextElement()Ljava/util/zip/ZipEntry;+54
j  java.util.zip.ZipFile$1.nextElement()Ljava/lang/Object;+1
j  java.util.jar.JarFile$1.nextElement()Ljava/util/jar/JarFile$JarFileEntry;+4
j  java.util.jar.JarFile$1.nextElement()Ljava/lang/Object;+1
j  sun.misc.URLClassPath$JarLoader.validIndex(Ljava/lang/String;)Z+42
J  sun.misc.URLClassPath$JarLoader.getResource(Ljava/lang/String;ZLjava/util/Set;)Lsun/misc/Resource;
J  sun.misc.URLClassPath$JarLoader.getResource(Ljava/lang/String;Z)Lsun/misc/Resource;
j  sun.misc.URLClassPath$JarLoader.findResource(Ljava/lang/String;Z)Ljava/net/URL;+3
j  sun.misc.URLClassPath.findResource(Ljava/lang/String;Z)Ljava/net/URL;+17
j  java.net.URLClassLoader$2.run()Ljava/net/URL;+12
j  java.net.URLClassLoader$2.run()Ljava/lang/Object;+1
v  ~StubRoutines::call_stub
j  java.security.AccessController.doPrivileged(Ljava/security/PrivilegedAction;Ljava/security/AccessControlContext;)Ljava/lang/Object;+0
j  java.net.URLClassLoader.findResource(Ljava/lang/String;)Ljava/net/URL;+13
j  java.lang.ClassLoader.getResource(Ljava/lang/String;)Ljava/net/URL;+30
j  java.net.URLClassLoader.getResourceAsStream(Ljava/lang/String;)Ljava/io/InputStream;+2

Conclusion

It takes a lot of work to crash the JVM like this, so here’s to more memory, more memory for Maven, more memory for all!

About Lincoln Baxter III

Lincoln Baxter III

Creator of PrettyFaces - Url Rewriting for JSF, PrettyTime - Social-style date & time formatting for Java, and a community contributor to the JSF2 Expert Group; his latest project is ScrumShark, an open-source agile project management tool.

Beginning his career in C, C++ development for hardware signal testing automation, Lincoln soon moved on to Perl, dynamic programming languages, artificial intelligence and, more recently - web application frameworks such as Java Server Faces and Groovy on Grails for financial and small business solutions.

When he is not swimming, running, or playing Ultimate Frisbee, Lincoln is focused on improving the openness of Java, the Java Community Process(JCP), and bringing the J2EE platform to small businesses and freelancers.

More About Lincoln »

NFJS, the Magazine

May Issue Now Available
  • On the road to learning

    by Raju Gandhi
  • Refactoring to Modularity

    by Kirk Knoernschild
  • RESTful Groovy

    by Kenneth Kousen
  • Getting Started with D3.js

    by Brian Sletten
Learn More »