Brief Introduction to troubleshooting memory in Java

Last edited: Weds. April 7, 2010

Corrections or questions: bryanesmith at gmail dot com

Contents

• Purpose
• Introduction
• How to generate the heap dump
  • JVM flag to generate heap on OOM error
  • Using jmap to generate heap on running process
• How to view the contents of the heap dump
• Scenario with large heap dump
• Miscellaneous
• Terms

Purpose

The purpose of this guide is to help a new Java developer troubleshoot memory problems, such as performance degradation or OutOfMemoryError issues in Java 6. (Some of these tools might be available for previous and subsequent versions, but I am basing this document on my experience with Java 6.)

I also intend for this document to be used as a quick reference guide.

Introduction

In a long-living Java process, performance degradation is often due to a memory leak. This happens because the JVM spends increasingly more time performing garbage collection.

Eventually, an OutOfMemoryError occurs. I have seen these in two scenarios:

1. The application cannot allocate more heap space.
2. The application spends more than 98% of its time in garbage collection.*

(* There are JVM options to tune this sort of behavior; however, this default value is reasonable in most cases considering that the process is mostly unproductive.)

The keys to successfully troubleshooting a memory issue is to generate a heap dump and seeing what is taking up so much memory.

How to generate the heap dump

JVM flag to generate heap on OOM error

As of Java 6, you can set a runtime JVM flag to generate a heap dump on an OutOfMemoryError: -XX:+HeapDumpOnOutOfMemoryError. If you are running the application from command-line:

If you are running from an IDE, you should add this flag as a JVM parameter.

If an OutOfMemoryError occurs, a file will be generated with the name java_pid[pid].hprof in the working directory for the process. E.g., if the process has a pid of 4432, then the file would be called java_pid4432.hprof.

Using jmap to generate heap on running process

This is useful when a Java process does not have the JVM flag set to generate a heap on OOM error or an OOM error never occurs but performance is degrading. Note that the jmap utility comes with the JRE 6 installation. (jmap is available for Java 5, but has different command-line parameters. Type the command without any parameters for usage information.)

The command to generate the heap dump:

$ jmap -dump:file=dump.hprof [pid]

You need to know your process id. In Windows, you can find it using the task manager. In Unix, I use ps -Af | grep -i java from the command-line:

In this example, the pid is 3565. So the command would be:

How to view the contents of the heap dump

You can use jhat, which comes with the JRE 6 installation. To run it from command line:

Note that jhat requires a lot of heap space, so you offer it as much as you can when working with large heap dumps. E.g., to offer 1 GB of heap space to jhat:

Once jhat has loaded your heap dump, you can browse the heap histogram, as well as object references, in a web browser. By default, the process listens on port 7000, so you would simply visit http://localhost:7000.

Two other options:

• Eclipse Memory Analyzer (MAT)
• IBM HeapAnalyzer

In the next session, I will discuss why you might need an alternative to jhat such as MAT when working with large heap dumps.

Scenario with large heap dump

One of our servers ran out of heap space, java.lang.OutOfMemoryError: Java heap space. I used jmap to generate the output before restarting the process, and I downloaded the heap dump from the server to my laptop.

The heap dump is 982MB. My laptop has a little more than 2GB of free memory, and so I first attempted to open the heap dump using jhat. But even when I allocated 2GB of space to jhat, it would eventually fail with an OOM error.

I then tried IBM HeapAnalyzer with the same result as jhat.

Next, I tried MAT. The first attempt failed after a long time. So I opened MemoryAnalyzer.ini in the MAT directory and set the maximum heap space to 2GB, and the tool succeeded in opening the heap dump in under a minute.

MAT generates indices to help manage the heap information. Apparently, it buffers less in memory, allowing it to succeed where the other tools did not.

Once the tool was running, I generated a heap histograph and found the problem immediately.

Two classes of objects were using well over 90% of all memory. One of the classes was only for troubleshooting and wasn't being used, so I quickly removed references to it in the code an the problem was solved.

Usually OOM errors take more troubleshooting than this, but the process is similar. Note that jhat is very useful, but if you have problems with a large heap dump, then the Eclipse Memory Analyzer (MAT) might be the tool you need.

Miscellaneous

• If you anticipate that your heap dump will be large and difficult to work with, then you can use jmap to generate a heap histogram directly: jmap -histo [pid]
  • Note, however, that this will only give you an overview by class; in the case that you have 500 MB of heap used for binary arrays, for example, this will not help you determine what is creating these objects or where the references are being held. This is the limitation of a heap histogram; being able to track object references is one of the main benefits of a heap dump.

Terms

• garbage collection: in Java, dynamically allocated memory for objects is managed by the JVM. Memory used by objects is reclaimed so long as there are no references to the objects. (See: memory leak)
• heap: memory available for runtime storage. Practically speaking, in Java this is the memory used by the creation of objects.
• heap dump: store contents of heap space for a process in a file to inspect later.
• heap histogram: list the classes with object in memory at a given time with instance counts and total heap space used.
• JVM: Java virtual machine. It is the abstracted environment in which Java runs so that the same code can be used on any machine regardless of the operating system.
• memory leak: in Java, happens when an object is no longer used but is still referenced by another object, so it is never garbage collected.
• object references: objects in Java has references to other objects. E.g., a Person object might have a name of type String, in which case the Person object has a reference to string. Once that reference is removed (by either [a] setting the reference to null or another string, or [b] garbage collecting the Person object), then the String can be garbage collected.