Java Memory Model

Every developer at some point will face the Out Of memory Issues. As we know that every java program runs inside the JVM, this article will explain the memory arrangements inside the JVM. 

There are common areas which are used the Java programs, Stack, Heap and Perm Space 

Let’s starts with

Heap

When a Java program is started the Java Virtual Machine gets some memory from Operating System. Java Virtual Machine or JVM uses this memory for all its need and part of this memory is call java heap memory. Objects that are created by the Java Program are stored inside this heap. No matter, where object is created in code e.g. as member variable, local variable or class variable, they are always created inside heap space in Java.

So when ever objects are created inside the heap, they need to be deleted manually once their part is completed. We need to keep track of all the objects in order to clean them.

Java Heap is divided into

The Young generation divided into 3 partitions Eden, S1 and S2 spaces and this is sometimes called as New Space. The Old generation is sometimes called tenured space. These S1 and S2 are called survivor spaces and sometimes referred as From Space and To Space. At any point of time one Space will be empty.

What is Eden and how do these work?

Objects are initially allocated in Eden area. Eden area is set up as Stack; an Object allocation is implemented as a Pointer increment. When this area is full the GC performs a reachablilty test and copies all live objects from Eden to one of the Survivor spaces. At any point of time one Survivor space is empty. So the Survivor space contains objects that have passed at least one GC.

Why 2 Survivor Space? And why not one?

The main reason is need to deal with fragmentation. Since objects are initially allocated in Eden Area, when this area is full a GC is performed and objects that still hold the reference (alive) are copied to the one of the Survivor Space.

Once a Survivor space is full, surviving objects are moved to the other Survivor space. Then, the Survivor space that is full will be changed to a state where there is no data at all.

The objects that survived these steps that have been repeated a number of times are moved to the old generation. These Survivor spaces servers as a destination of the next copying collection of any living Objects in Eden Space.

Old Generation

The Old Generations contains objects that have existed for some time in the survivor spaces. It means the objects that are in Old generation still have references from application code or from some other parts.

Why Eden and Old Generations?

The separation of heap is to allocate new objects into the young generation thus providing a better way to clean them when not needed using different GC strategies.

As we know that we create objects, use them and delete them .These objects often die young. We mostly don’t refer those objects throughout our code. So we create objects initially in the Eden Space which are removed during GC.

If the objects survives until a certain number of young generation GCs has passed, it will finally be moved into the old generation and stay there with all other long-lived objects

Permanent Space

Permanent Space is the area of the JVM that is used to store data Structures and Class Information. Java objects are from Java classes. JVM has the internal representation of Java objects which are stored in heap where as internal representation of Java classes are stored inside the perm Space. The Perm Space is independent from Heap Area.

Why this? Since Java Class is internal representation of Java VM and we should not fill Heap with these .This makes developer hard to understand whether the live objects belong to application of the Java itself.

The Perm Space is special because it holds the data needed by the JVM to describe the Objects that do not have an equivalent representation in Java Level.

Permanent Space GC

Garbage collection happens in this area too. The Garbage Collection in this area is called “class GC”. We can enable or disable this feature using the JVM argument “-noclassgc”.

Native Area or Code Cache

Native Memory is an area which is used by the JVM for its internal operations, to execute JNI operations etc. This is the memory area separate from the JVM heap that contains all the JVM byte code for a method compiled down to native code. The Size of the Native Memory depends on the Architecture of the Operating System and the amount of memory which is already committed to the Java Heap.

There is no way to find the exact size of a native area, but by using simple formula can we can find  Native Memory = (Process Size – Max Heap Size – Max Perm Size)

Stack 

Stack area is a memory area set aside for the thread of execution. So whenever a method is called a block is arranged on the stack, with the local variables and some book keeping data. When the execution of method is completed, the block becomes un used and can be removed. This is done when a next method is called. The blocks on the stack are always removed in the LIFO (last in First Out). The most recently used memory block is the one that is always removed first.

The thread stack memory space is not allocated from the general heap (whose size is controlled via -Xmx), but typically from the native memory that the JVM uses.

The Stack Size will limit the number of threads that our application can create. The utilities available find the Stack Information are jps and jstack.

Java allows us to define stack Size for Threads. We explicitly set the Stack Size using

-Xss ( eg : -Xss128k)

The memory Equation will be

heap Size + (no. Of Threads x Thread Stack Size) – total Ram Used by JVM

NOTE : Linux

There is an ulimit setting that defines the upper limit of the stack Size at the OS level.

To see the Upper limit in Kilo bytes

ulimit -s

The default Stack Sizes are

Platform	Java 1.6	Java 1.5 & 1.4
Linux/Solaris x86 32 bit	320k	512k
Linux/Solaris x86 64 bit	1024k	1024k
Windows 32 bit	32k	256k
Windows 64 bit	1024k	256k

Now the important part

Setting the Sizes

Heap

heap size can be set using '-Xms'  and '-Xmx'.

-Xms : initial Heap Size

-Xmx : max Heap Size

If the JVM tried to reclaim more memory than the Max heap size, a

“java.lang.OutOfMemory: Java Heap Space” is thrown.

If we don’t specify any heap settings, the default values are (for J2Se 5)

-Xms : 4MB

-Xmx : 64MB 

NOTE: The default values differ depending on the Hardware configurations. Consider a Server Class machine (2 or more processors and a 2GB Ram), the default value of the initial heap size is “1/64 of physical memory” and the maximum heap Size is “1/4 of Physical Memory”. If we use Linux and have a Server class machine then the Initial heap Size = 32MB and Max Heap Size=512MB.

Generator Sizing

The other most important setting that we can do is the Generator sizing. The young generation can be adjusted using

-XX:NewSize

-XX:MaxNewSize

When you set

java  myProgram.java -Xmx=1024mb -Xms=1024mb -XX:NewSize=400m  -XX:MaxNewSize=400m

In this case, the jvm starts with 400 reserved to Young generation from 1024mb of heap.

NewSize is split between the Eden space and the survivor spaces.

Survivor Space can be tuned by using, -XX:SurvivorRatio=6

This specifies how much space can be granted to survivor space compared to the whole young generation. What this does is sets the ratio between each of the survivor space and Eden space to be 1:6 means the Eden receives 6 units of space where are each survivor space receives 1 unit of space means 1/8 of young generation.

By default 1/8 of the NewSize is reserved for survivor space

How do we size things?

It is said that we should provide plenty of memory to young generation for applications that creates a lot of short lived objects and more memory for Old generation which creates lot of long lived objects like pools or caches.

Suggested Size for a Young Generation would be 1/3 of total heap up to ½ of the heap.

Increasing the young generation more can cause issues.

The Survivor space settings should be shaped for better performance. If you don't provide a value for them they will be allocated with 1/34 of the young generation. Allocating fewer sizes to survivor space can cause allocated large objects to go old Space in few GC Cycles.

So consider if you have the young generation too small, there will be no space left and short lived objects too will be moved to Old generation which are then cleaned by major GC.While a size of young generation is more than half of heap will cause the old generation to be out of space causing major GC again.

Let's say you set a Heap size of 1.5 GB in a Linux environment, you need to consider the following:

1.5GB (Heap) + 256mb (PermGen) + (Number of thread * 512kb) = the result might not be larger than 3 GB for Linux or 2 GB for Windows

Do Objects create in stack or Heap?

An object will be created in heap. If we define

int[] array = new int[100];

The only variables that are stored on the stack are primitives and object references. In the above line, the array reference is stored on the stack, but it references data is stored on the heap.

But consider a case, where we create an object which is created on heap and references are stored on stack. At this point when the method is completed all data on the stack should be deleted. The references holding the object will also be deleted causing no references to the object that is stored in heap. To eliminate this Java uses something called “Escape analysis”

If an object created on heap from a method with a local scope, it does not have to be allocated on the heap at all; it can simply be allocated on the thread’s local stack.  Everything allocated on the local stack is effectively de-allocated when the stack is popped at method exit.  If the object were allocated on the heap, it would have to be collected later by the garbage collector.

How to find the initial and Max heap Sizes for an application?

Finding the initial and max heap sizes for an application is somewhat different. The max Heap size can be identified by load testing our application.

Consider an application which will go to a maximum memory while we are doing a load test. Consider the max memory while load testing was 768mb.Now a 20-30% of additional memory can be added to the memory that we obtained during load testing. So that would be 768+ (20-30) % of 768 which would be around 1GB.

Finding the initial heap memory is much similar. Whenever we load test our application there will be some stage where there will be bounce in the heap graph at a specific point which will be our initial memory Setting.

But as experts say to keep the max and min memory setting to same fixed level .That is if we have the max memory to 1024mb, set as

java  myProgram.java  -Xmx=1024mb -Xms=1024mb.

Why do we need to set the both Xmx and Xms same to fixed level?

Contiguous memory plays an important role in heap. According to docs, the memory for the heap does not need to be contiguous. So your code shouldn't make assumptions about the continuity of the heap. When we define both Xms and Xmx with same values, a fixed contiguous memory block is allocated for JVM which will reduce the overhead of computing a new heap block when ever memory is needed by JVM but this would lead to waste of memory in the early stages of application as it would take more time to start but setting the initial and max to same fixed value will provide performance boost when application is running.

How is the default Java Size determined?

The Client JVM default maximum heap size is half of the physical memory up to physical memory size of 194mb and otherwise one fourth of the physical memory up to a physical memory size of 1 gigabyte.

If your machine has 128 megabytes of physical memory, then the maximum heap size is 64 megabytes, and greater than or equal to 1 gigabyte of physical memory results in a maximum heap size of 256 megabytes.

The Server JVM Heap size is almost same as the Client JVM except that the default maximum heap size for 32-bit JVMs is 1 gigabyte, corresponding to a physical memory size of 4 gigabytes, and for 64-bit JVMs is 32 gigabytes, corresponding to a physical memory size of 128 gigabytes."

How do I find out the default memory provided?

In order to find out the default memory limitations provided to the java, we can use a JVM parameter like

java -XX:+PrintFlagsFinal -version | grep HeapSize

    uintx ErgoHeapSizeLimit                         = 0               {product}

    uintx HeapSizePerGCThread                       = 87241520        {product}

    uintx InitialHeapSize                          := 30763456        {product}

    uintx LargePageHeapSizeThreshold                = 134217728       {product}

    uintx MaxHeapSize                              := 492830720       {product}

java version "1.7.0_60"

Java(TM) SE Runtime Environment (build 1.7.0_60-b19)

Java HotSpot(TM) 64-Bit Server VM (build 24.60-b09, mixed mode)

This will give you all the default values.

How do I find how much of memory is being used by my application?

Java provides certain utilities which help us to extract information about the jvm. We can jmap available in java software to get the amount of memory being used. Get the Process ID of the JVM and use with jmap as

$ jmap -heap 16528

Attaching to process ID 16528, please wait...

Debugger attached successfully.

Server compiler detected.

JVM version is 24.60-b09

Using thread-local object allocation.

Mark Sweep Compact GC

Heap Configuration:

   MinHeapFreeRatio = 40

   MaxHeapFreeRatio = 70

   MaxHeapSize      = 536870912 (512.0MB)

   NewSize          = 1310720 (1.25MB)

   MaxNewSize       = 17592186044415 MB

   OldSize          = 5439488 (5.1875MB)

   NewRatio         = 2

   SurvivorRatio    = 8

   PermSize         = 21757952 (20.75MB)

   MaxPermSize      = 268435456 (256.0MB)

   G1HeapRegionSize = 0 (0.0MB)

Heap Usage:

New Generation (Eden + 1 Survivor Space):

   capacity = 161021952 (153.5625MB)

   used     = 33940360 (32.36804962158203MB)

   free     = 127081592 (121.19445037841797MB)

   21.07809499166921% used

Eden Space:

   capacity = 143130624 (136.5MB)

   used     = 32622888 (31.111610412597656MB)

   free     = 110507736 (105.38838958740234MB)

   22.792388580657622% used

From Space:

   capacity = 17891328 (17.0625MB)

   used     = 1317472 (1.256439208984375MB)

   free     = 16573856 (15.806060791015625MB)

   7.363746279761905% used

To Space:

   capacity = 17891328 (17.0625MB)

   used     = 0 (0.0MB)

   free     = 17891328 (17.0625MB)

   0.0% used

tenured generation:

   capacity = 357957632 (341.375MB)

   used     = 87783688 (83.71704864501953MB)

   free     = 270173944 (257.65795135498047MB)

   24.523485505681297% used

Perm Generation:

   capacity = 128057344 (122.125MB)

   used     = 128023792 (122.09300231933594MB)

   free     = 33552 (0.0319976806640625MB)

   99.9737992379414% used

28081 interned Strings occupying 3366312 bytes.

More to Come , Happy learning :-)