newbie questions about cache stats & query perf

[AJ Weber]

Sorry, I did search for an answer, but didn't find an applicable one.
I'm currently stuck on 1.4.1 (running in Tomcat 6 on 64bit Linux) for
the time being...

When I see stats like this:
name: documentCache
class: org.apache.solr.search.LRUCache
version: 1.0
description: LRU Cache(maxSize=512, initialSize=512)
lookups : 0
hits : 0
hitratio : 0.00
inserts : 0
evictions : 0
size : 0
warmupTime : 0
cumulative_lookups : 8158
cumulative_hits : 685
cumulative_hitratio : 0.08
cumulative_inserts : 7473
cumulative_evictions : 3023

I don't understand "lookups" vs. "cumulative_lookups", etc. I _do_
understand that a hit-ratio of 0.08 isn't a very good one.

Something I definitely find strange is that I've allocated 4G of RAM to
the java heap, but solr consistently remains around 1.7G. I'm trying to
give it all the RAM I can spare (I could go higher, but it's not even
using what I'm giving it) to make it faster.

The index takes-up roughly 25GB on disk, and indexing is very fast
(well, nothing we're complaining about anyway). We're trying to figure
out why queries against the default, document content are slow (15-30
seconds for only a few mm total documents). Mergefactor=3, if that helps.

So if anyone could point me to someplace that defines what these stats
mean, and if anyone has any immediate tips/tricks/recommendations as to
increasing query performance (and whether this documentCache is a good
candidate to be increased substantially), I would very much appreciate it.

-AJ

[Otis Gospodnetic]

Hi,

In your Solr version there is a notion of Searcher being opened and
reopened. Every time that happens those non-cumulative stats reset. The
cumulative_ stats just don't refresh, so you have numbers from when the
whole Solr started, not just from the last time Searcher opened.

Your cache is small, which is why you have evictions, which is partially
why you have low hit rate, which is partially why your queries are slow.
But 15-30 seconds is crazy high, so I am sure there are other issues.

Note that you should *not* give Solr/Tomcat all the RAM you can spare -
leave it to the OS to use for index caching. If you don't have issues with
full heap (OOMs or crazy GCing) with say -Xmx=2g, then use that.

Plug: http://sematext.com/spm/solr-performance-monitoring/index.html

Otis
--
Solr & ElasticSearch Support
http://sematext.com/

Sorry, I did search for an answer, but didn't find an applicable one. I'm
currently stuck on 1.4.1 (running in Tomcat 6 on 64bit Linux) for the time
being...
name: documentCache
class: org.apache.solr.search.**LRUCache
version: 1.0
description: LRU Cache(maxSize=512, initialSize=512)
lookups : 0
hits : 0
hitratio : 0.00
inserts : 0
evictions : 0
size : 0
warmupTime : 0
cumulative_lookups : 8158
cumulative_hits : 685
cumulative_hitratio : 0.08
cumulative_inserts : 7473
cumulative_evictions : 3023
I don't understand "lookups" vs. "cumulative_lookups", etc. I _do_
understand that a hit-ratio of 0.08 isn't a very good one.
Something I definitely find strange is that I've allocated 4G of RAM to
the java heap, but solr consistently remains around 1.7G. I'm trying to
give it all the RAM I can spare (I could go higher, but it's not even using
what I'm giving it) to make it faster.
The index takes-up roughly 25GB on disk, and indexing is very fast (well,
nothing we're complaining about anyway). We're trying to figure out why
queries against the default, document content are slow (15-30 seconds for
only a few mm total documents). Mergefactor=3, if that helps.
So if anyone could point me to someplace that defines what these stats
mean, and if anyone has any immediate tips/tricks/recommendations as to
increasing query performance (and whether this documentCache is a good
candidate to be increased substantially), I would very much appreciate it.
-AJ