Garbage Collection tuning - G1 is now a good option

[Shawn Heisey]

I've been working with Oracle employees to find better GC tuning
options. The results are good enough to share with the community:

https://wiki.apache.org/solr/ShawnHeisey#GC_Tuning

With the latest Java 7 or Java 8 version, and a couple of tuning
options, G1GC has grown up enough to be a viable choice. Two of the
settings on that list were critical for making the performance
acceptable with my testing: ParallelRefProcEnabled and G1HeapRegionSize.

I've included some notes on the wiki about how you can size the G1 heap
regions appropriately for your own index.

Thanks,
Shawn

[William Bell]

But tons of people on this mailing list do not recommend AggressiveOpts

Why do you recommend it?

Post by Shawn Heisey
I've been working with Oracle employees to find better GC tuning
https://wiki.apache.org/solr/ShawnHeisey#GC_Tuning
With the latest Java 7 or Java 8 version, and a couple of tuning
options, G1GC has grown up enough to be a viable choice. Two of the
settings on that list were critical for making the performance
acceptable with my testing: ParallelRefProcEnabled and G1HeapRegionSize.
I've included some notes on the wiki about how you can size the G1 heap
regions appropriately for your own index.
Thanks,
Shawn

[Shawn Heisey]

Post by William Bell
But tons of people on this mailing list do not recommend AggressiveOpts
Why do you recommend it?

I haven't done any comparisons with and without it. To call it a
"recommendation" is a little bit strong. I use it, and I am seeing good
results.

My reading indicates that AggressiveOpts basically enables settings that
are being considered for defaults in a later Java version. If they are
being seriously considered for new defaults, they are probably something
that I want to be using. I have also heard that there are sometimes
bugs with that option, but I have not personally had any problems.

If you don't want to use that option, feel free to leave it out. I will
update my wiki page with a note about AggressiveOpts.

Thanks,
Shawn

[Mark Miller]

bq. But tons of people on this mailing list do not recommend AggressiveOpts

It's up to you to decide - that is why it's an option. It will enable more
aggressive options that will tend to perform better. On the other hand,
these more aggressive options and optimizations have a history of being
more buggy. Depending on your needs and risk tolerance, you might make a
different choice.

You probably want to research the specific issues it has been found to
cause and whether those issues are important to you or affect the JVM you
are using. You also have to keep in mind that probably all of the issues
are not known or documented and that updates may introduce new issues. You
face some risk like that no matter what though.

- Mark

Post by Shawn Heisey

Post by William Bell
But tons of people on this mailing list do not recommend AggressiveOpts
Why do you recommend it?

I haven't done any comparisons with and without it. To call it a
"recommendation" is a little bit strong. I use it, and I am seeing good
results.
My reading indicates that AggressiveOpts basically enables settings that
are being considered for defaults in a later Java version. If they are
being seriously considered for new defaults, they are probably something
that I want to be using. I have also heard that there are sometimes
bugs with that option, but I have not personally had any problems.
If you don't want to use that option, feel free to leave it out. I will
update my wiki page with a note about AggressiveOpts.
Thanks,
Shawn

[Otis Gospodnetic]

Not sure about AggressiveOpts, but G1 has been working for us nicely.
We've successfully used it with HBase, Hadoop, Elasticsearch, and other
custom Java apps (all still Java 7, but Java 8 should be even better). Not
sure if we are using in on our Solr instances.

e.g. see http://blog.sematext.com/2013/06/24/g1-cms-java-garbage-collector/

Otis
--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/

Post by William Bell
But tons of people on this mailing list do not recommend AggressiveOpts
Why do you recommend it?

Post by Shawn Heisey
I've been working with Oracle employees to find better GC tuning
https://wiki.apache.org/solr/ShawnHeisey#GC_Tuning
With the latest Java 7 or Java 8 version, and a couple of tuning
options, G1GC has grown up enough to be a viable choice. Two of the
settings on that list were critical for making the performance
acceptable with my testing: ParallelRefProcEnabled and G1HeapRegionSize.
I've included some notes on the wiki about how you can size the G1 heap
regions appropriately for your own index.
Thanks,
Shawn

--
Bill Bell
cell 720-256-8076

[Shawn Heisey]

Post by Shawn Heisey
I've been working with Oracle employees to find better GC tuning
https://wiki.apache.org/solr/ShawnHeisey#GC_Tuning
With the latest Java 7 or Java 8 version, and a couple of tuning
options, G1GC has grown up enough to be a viable choice. Two of the
settings on that list were critical for making the performance
acceptable with my testing: ParallelRefProcEnabled and G1HeapRegionSize.
I've included some notes on the wiki about how you can size the G1 heap
regions appropriately for your own index.

A note was just recently added to the Lucene wiki, saying that the G1
collector should never be used with Lucene, because there are bugs that
might cause index corruption. Solr uses Lucene, so this would apply.

https://wiki.apache.org/lucene-java/JavaBugs#Oracle_Java_.2F_Sun_Java_.2F_OpenJDK_Bugs

I have never had any problems with it, but perhaps I spoke too soon when
recommending G1. Does anyone else have anything to add?

Thanks,
Shawn

[William Bell]

It looks like 32 bit is affected.

Post by Shawn Heisey

Hi everyone,
I am a committer to the Lucene/Solr project. We've recently hit what
we believe is a JIT/GC bug -- it manifests itself only when G1GC is
I've been working with Oracle employees to find better GC tuning
https://wiki.apache.org/solr/ShawnHeisey#GC_Tuning
With the latest Java 7 or Java 8 version, and a couple of tuning
options, G1GC has grown up enough to be a viable choice. Two of the
settings on that list were critical for making the performance
acceptable with my testing: ParallelRefProcEnabled and G1HeapRegionSize.
I've included some notes on the wiki about how you can size the G1 heap
regions appropriately for your own index.

A note was just recently added to the Lucene wiki, saying that the G1
collector should never be used with Lucene, because there are bugs that
might cause index corruption. Solr uses Lucene, so this would apply.
https://wiki.apache.org/lucene-java/JavaBugs#Oracle_Java_.2F_Sun_Java_.2F_OpenJDK_Bugs
I have never had any problems with it, but perhaps I spoke too soon when
recommending G1. Does anyone else have anything to add?
Thanks,
Shawn