Monday, June 3, 2013

Eclipse Memory Analyzer

Analyzing Memory Structure in Java is always Challenging. As a Application Server admin , I use certain tools to find the memory usage of a Server.

Eclipse Memory Analyzer is one such tool which provides a clear understanding of the memory usage in a Java process. In this article we will see the basics of the Eclipse Memory Analyzer and its uses. Eclipse Memory Analyzer is useful for both tracking memory leaks and for periodically reviewing the state of your system.

Getting the Memory Dump

A JVM heap dump is basically a “snapshot” of the Java heap memory at a given time. It is quite different than a JVM thread dump which is a snapshot of the threads.

Generating a Heap Dump can be done by using the “jmap” command available in the java software.

jmap -dump:format=b,file=heap.hprof JAVA_PID

For Oracle Jrocket we can use
jrcmd <JAVA_PID> hprofdump filename=abc.hprof

So Consider generating a heap Dump Files for the Process ID 22979.
Dev:vx1111:jbs002-jas $ jmap -dump:file=my_stack.bin 22979
Dumping heap to /config/jboss/ews/1.0/domains/jas/my_stack.bin ...
Heap dump file created

It’s easy to get an OutOfMememory exception when opening the java heap. The dump file can be very memory consuming if you application was in the moment it was taken. If you experience the problem you should give to the JVM as much memory as you can:
Most of the times the heap dumps are generated with the .hprof extension.

What is HProf?
HProf is a tool built into JDK for profiling the CPU and heap usage within a JVM. A Java process crash may produce an hprof file containing a heap dump of the process at the time of the failure.
So Once the Heap Dump is generated we need to analyze the file

Typical information which can be found in heap dumps (once more - depending on the heap dump type) is:

All Objects
Class, fields, primitive values and references

All Classes
Classloader, name, super class, static fields

Garbage Collection Roots
Objects defined to be reachable by the JVM

Thread Stacks and Local Variables
The call-stacks of threads at the moment of the snapshot, and per-frame information about local objects

A heap dump does not contain allocation information so it cannot resolve questions like who had created the objects and where they have been created.

Details
















When We open the heap dump file using the Memory Analyzer . the Pie Chart shows you the biggest objects. When we click on the areas in Pie Chart , we can see the classes which are taking more usage. This means that if we were able to eliminate the high usage of “java.lang.ref.Finalizer” , we can eliminate 5.4mb of the usage.

Click on the “Leak Suspects” to obtain information about the leak suspects and also about the System


The “Leak Suspects” tells you about the classes that can cause issues and are considered as Suspects for leaking memory. The Memory analyzer displays the couple of Problem Suspects along with the amount of memory they occupy.

Click on the “Table of Contents” below to get more information about the System. The Details include




From this , we can obtain various levels of information including , Heap Dump Overview, thread details, System properties and Top Consumers of Memory e.t.c.

Histogram : Using Histogram we can list the number of instances per class, the shallow size and the retained size .






















The Histogram also shows the number of instances of a particular class and how much memory each one uses.

We can also use the histogram in grouping .We can group by Class,Class loader , Super class and also package.









The Histogram has also a filtered Option by which we can filter the classes accordingly. we can search for specific class like








We can see how many of the classes are loaded from the “com” package structure along with the number of them loaded and also classes.

The amount of memory each object is using can also be viewed.

As you can see there are 2 calculations Shallow Heap and Retained heap.

Shallow heap is the memory consumed by one object. An object needs 32 or 64 bits (depending on the OS architecture) per reference, 4 bytes per Integer, 8 bytes per Long, etc. Depending on the heap dump format the size may be adjusted (e.g. aligned to 8, etc...) to model better the real consumption of the VM.

Shallow size of an object is the amount of allocated memory to store the object itself, not taking into account the referenced objects

Retained size of an object is its shallow size plus the shallow sizes of the objects that are accessible, directly or indirectly, only from this object. In other words, the retained size represents the amount of memory that will be freed by the garbage collector when this object is collected

Dead objects are shown only with shallow size, as they do not actually retain any other objects.

Generally speaking, shallow size of an object is its "flat" size in the heap whereas the retained size of the same object is the amount of heap memory that will be freed when the object is garbage collected.

For example the shallow size of an instance of java.lang.String will not include the memory needed for the underlying char[].

From Eclipse Documentation
The following diagram represents objects in the Java heap. Objects A and B are garbage collection roots, for example method parameters, locally created objects, or objects that are used for wait(), notify(), or synchronized() methods. The set of objects A and B has a retained set consisting of objects A, B, C, D, E, F, G, and H.












If, for example, object G was referenced by a garbage collection root other than A or B, then object G would remain if objects A and B were removed during garbage collection. Object G would therefore not be in the retained set of objects A and B.

Retained heap, or retained size

The total heap size of all the objects in the retained set. This value is the amount of memory that is consumed by all the objects that are kept alive by the objects at the root of the retained set.
In general terms, the shallow heap of an object is the size of the object in the heap. The retained size of the same object is the amount of heap memory that is freed when the object is garbage collected.

As an Example , if an ArrayList held 100 items, and each item required 16 bytes, then removing the ArrayList would free 16 x 100 + X, where X is the shallow size of the ArrayList.

 


We can see that “java.lang.Class” has more usage with each object taking 57.664 bytes.

This can also be read as , the java.lang.Class in heap dump has exactly 6036 times in memory and has more than >=8004,320 bytes occupied in memory.

Find Duplicate Classes:We can Find the Duplicate Classes being loaded by following ,










And We can see the Duplicate classes like




















The dominator tree

The dominator tree is a data structure that allows you to answer the question about the biggest objects in almost no time

jmap -dump:live,format=b,file=dump.hprof <PID>

To determine who is creating these objects, or find out what the purpose of some structures is, the actual instances with their incoming and outgoing references are required. To get them, choose List Objects with incoming References from the context menu. Now a tree structure is displayed, showing all instances with all incoming references. Those references keep the object alive and prevented them from being garbage collected

Outgoing References are interesting as well, because they show the actual contents of the instances

The dominator tree allows you to identify the largest memory graphs.

Top Consumers

Click on the Top Consumers  from which we can get information about the top Consumers which are  expensive objects grouped by class and by package like












We can Get the Same thing using 













Top Components
Click on the Top Components from which we can get information about the top Consumers which are bigger than 1% of the heap like











When You  Click the “Table Of Contents” below this ,we can find various other Information like

Duplicate String






Information about Empty Collections,Collection Fill Ratios,Soft References,Weak References,Finalizer Statistics and Map Collision Ratios Can be Obtianed by using memory Analyzer.













Immediate Dominators
The Immediate Dominators are very useful to quickly find out who is responsible for a set of objects,as it directly answers the question "who Keeps the Objects alive"





















Once you select the Immediate Dominators , we can see this screen which allows us to enter a pattern which can be ignored while searching . IN the below case , iam skipping java.* and some other making sure only the application related classes are seen





















Querying Heap Objects

Mat Provides a way to Query Heap Objects using QQL Like



















You can see the Rectangle from where you can start the OOL Query Browser

Class Loader Information

Class Loader information is very important since they are the pieces which load objects.
From the Histogram we can 












Next to the class loader name, the table contains the defined classes and the number of live instances. If one and the same component is loaded multiple times, the number of live instances can indicate which class loaders is more alive and which one should be garbage collected.

Thread Details

















You can enter the Query or Thread ID that you need to see.Now from the below image the Thread Ids that belong to com.* will be shown.


















GC Roots : The so-called GC (Garbage Collector) roots are objects special for garbage collector. Garbage collector collects those objects that are not GC roots and are not accessible by references from GC roots.

One object can belong to more than one kind of root. The root kinds are:

Class - class loaded by system class loader. Such classes can never be unloaded. They can hold objects via static fields. Please note that classes loaded by custom class loaders are not roots, unless corresponding instances of java.lang.Class happen to be roots of other kind(s).

Thread - live thread
Stack Local - local variable or parameter of Java method
JNI Local - local variable or parameter of JNI method
JNI Global - global JNI reference
Monitor Used - objects used as a monitor for synchronization

Sample Class For generating OOM 
package com.sample;

import java.util.ArrayList;
import java.util.List;

public class Main {
 public static void main(final String... args) {
 System.out.println("start");
 List<Integer> numbers = new ArrayList<Integer>();
 for (int i = 0; i <= Integer.MAX_VALUE; i++) { // going to fail
 numbers.add(i);
 }
 System.out.println("stop");
 }
}

java -Xmx10M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/heap.bin -jar outofmemory.jar start

Much More To Come , Happy learning :-)