Garbage Collection: ConcMarkSweep vs. RMI

Abstract

Use of the ConcMarkSweep garbage collector for the "tenured" generation is undesirable when RMI (Remote Method Invocation) garbage collection is occurring.

Introduction

Java doesn''t require that you explicitly release memory allocations in your program, which greatly reduces the time to build useful applications and eliminates most memory leaks. However, a significant part of any application is spent doing memory management, and in the case of Java, all application threads must be suspended for some period of time during any "garbage collection", or "GC". Garbage collection is the operation performed automatically by the JVM that recycles memory that had been allocated for an object, but now is not referenced by any part of the application, and therefore can be reclaimed and reused. The application pauses caused by GC are known as "stop-the-world pauses", because all application threads must be suspended during those pauses.

The challenge with user-facing applications (like Web application servers) is to make those pauses as brief as possible, with the goal that no user will notice them.

Since most memory allocations are released almost immediately after they''re allocated, the Sun JVM uses a "generational" memory allocation and collection scheme to improve efficiency by focusing collection efforts on recently allocated memory. The main memory allocation heap is split into two parts: the "young" (aka "new") generation, and the "old" (aka "tenured") generation. The young generation is collected every time it fills up; objects that survive a few collections in the young gen are promoted to the old gen. The old generation is collected more infrequently.

There are three main types of collections:

1. Young generation collections (using the currently selected young gen collector)
2. Old generation collections (using the currently selected old gen collector)
3. Full collections, which stop the JVM and clean and compact all memory.

Full collections are expensive and tend to produce long pauses, so it''d be nice to be able to avoid them.

You can change the selection of garbage collector by editing the "jvm.args" line of your "jvm.config" file. (Note that "-server" must always be the first argument on that line.)

Young Generation Collection

The default collector for the young gen is either the "Default New" or "Parallel New" collector, depending on the platform and number of processors in the machine. ColdFusion explicitly selects the old "Parallel" collector using the JVM argument "-XX:+UseParallelGC", as opposed to the "new" parallel collector selected using "-XX:+UseParNewGC". Unfortunately, the "old" parallel collector is incompatible with both JVM debugging and the "concurrent" tenured generation collector.

A larger young generation means that collections will occur less often, but take more time. Also, it''s fairly important to ensure that the young generation isn''t too large to fit in the available memory in the tenured generation, or full collections may be forced. This is a requirement called the "young generation guarantee".

Young generation collections, even with a large young generation size selected, tend to be very brief, so long as the young generation guarantee is met. If the young generation guarantee isn''t met, then a Full collection is forced.

Tenured Generation Collection

The default collector for the tenured gen uses a "stop-the-world" pause for the duration of its collection. If you have many objects to be collected in the tenured generation, this can cause excessive pauses.

An alternative, the Concurrent Mark Sweep (CMS) collector, uses a few brief stop-the-world pauses during collection, but the bulk of collection occurs while application threads are running. While less efficient than the default tenured gen collector, the reduced pauses are often desirable for quasi-realtime application like Web applications. On the face, it appears that the CMS collector is ideal for Web applications.

The CMS collector is enabled using the JVM arg "-XX:+UseConcMarkSweepGC". But read on before rushing out to enable it.

Enter RMI

When you''re using RMI, you may be holding references to memory on non-local machines, so RMI will schedule Full collections at some interval. By default for Java 1.5 and below, that interval is every minute, which can cause significant overhead. However, there is no way to eliminate RMI scheduled collections; even "-XX:+DisableExplicitGC" will not prevent these collections. You can change the interval of collections, but setting too long of an interval may create very long Full GC pauses when it finally does run. (Pauses in excess of 60 seconds are not rare with a one hour interval, if the old generation is large and close to full.)

RMI collections and ConcMarkSweep

Normally, the Young generation space is split into a few areas: Eden, where new allocations are made, and two Survivor spaces, called "from space" and "to space". Measurements of these spaces are visible in the stdout log when you have "-XX:+PrintHeapAtGC" set.

When the Young gen is collected, the objects in Eden that are still allocated are copied to a Survivor space. Those objects will be bounced back and forth between the two Survivor spaces for a few collections, and eventually promoted to the tenured gen if they live long enough. Most allocations should not live long enough to make it to tenured.

It appears that (at least on Windows) enabling ConcMarkSweepGC actually disables the Survivor spaces, and so when Full GCs are finally called (which are unavoidably called by RMI), the Full GCs wind up throwing out all of the objects that were promoted early to Tenured, which adds significant time to those stop-the-world collections.

So, the goal is to give the Full collector as little work to do as possible, and since ConcMarkSweep constantly bleeds memory into Tenured, the Full GC pauses are end-user noticeable, often taking more than four seconds from every minute.

What seems to work really well is this set of options (split into separate lines for clarity):

-Dsun.rmi.dgc.client.gcInterval=600000
-Dsun.rmi.dgc.server.gcInterval=600000
-XX:SurvivorRatio=4
-XX:NewSize=64m
-XX:+UseParNewGC
[no ConcMarkSweepGC]

The RMI collection intervals control how often RMI triggers a Full GC. The default is one minute (60,000ms) which is too frequent, but the most commonly suggested replacement is one hour (3,600,000ms), which can cause the "[Unloading class....]" phase of a Full GC (collection of the "perm" gen, where classes are stored) to take a noticeably long time.

The survivor ratio sets the Survivor spaces a bit larger than normal (1/4 the size of Eden), so that there''s no overflow into Tenured. The NewSize parameter actually sets the Eden size, so the adjusted size of the whole Young gen (Eden + two survivor spaces) is 64mb + the two survivor spaces (16mb each) = 96mb.

We''re finding that the Full GCs are taking about 1.2-1.5s when called every 10 minutes by RMI, and the Tenured collector never needs to run, because the size of the tenured gen (after startup) is almost totally static, since practically all new objects die somewhere in the Young gen.

Young gen collections are occurring about every 10-20sec, and usually take about 0.15s, with an occasional 0.6s.

The only problem we''re having now, is that it appears that sometimes the RMI collections don''t start until after the first tenured generation collection, which in our case means 19+ hours of slow memory creep after startup before the first major collection, which takes about a minute (stop-the-world) to complete. If we notice that occurring, we use the SeeFusion interface to force a Full GC once, shortly after startup.