Some issues concerning SOLR 1.4.1

[Bastian Spitzer]

Hi,

We just migrated from SOLR 1.4 to 1.4.1. We are observing some new
Errors in the logs that didnt occured
before the migration, so we want to share them with you and are hoping
to get some help solving them.

We are using 1Master and 1Slave with replication on 2 different machines
running only the solr instances.

The first one is this Warning, and maybe ist the reason for the
following Errors:

SEVERE: ConcurrentLRUCache was not destroyed prior to finalize(),
indicates a bug -- POSSIBLE RESOURCE LEAK!!!
Jun 29, 2010 9:39:13 AM org.apache.solr.common.util.ConcurrentLRUCache
finalize
SEVERE: ConcurrentLRUCache was not destroyed prior to finalize(),
indicates a bug -- POSSIBLE RESOURCE LEAK!!!
Jun 29, 2010 9:39:13 AM org.apache.solr.request.UnInvertedField uninvert

Then every now and then those 2 pop up:

SEVERE: Error during auto-warming of key:+mandant_id:1000
+partner_id:1000 +active_1000:T +int(priority_1000) +language_id:1004
+provider:"han kr?ll":java.io.
IOException: read past EOF
at
org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.jav
a:151)
at
org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.j
ava:38)
at
org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:78)
at
org.apache.lucene.index.SegmentTermPositions.readDeltaPosition(SegmentTe
rmPositions.java:73)
at
org.apache.lucene.index.SegmentTermPositions.nextPosition(SegmentTermPos
itions.java:69)
at
org.apache.lucene.search.PhrasePositions.nextPosition(PhrasePositions.ja
va:76)
at
org.apache.lucene.search.PhrasePositions.firstPosition(PhrasePositions.j
ava:65)
at
org.apache.lucene.search.ExactPhraseScorer.phraseFreq(ExactPhraseScorer.
java:34)
at
org.apache.lucene.search.PhraseScorer.doNext(PhraseScorer.java:105)
at
org.apache.lucene.search.PhraseScorer.nextDoc(PhraseScorer.java:89)
at
org.apache.lucene.search.ConjunctionScorer.<init>(ConjunctionScorer.java
:42)
at
org.apache.lucene.search.ConjunctionScorer.<init>(ConjunctionScorer.java
:33)
at
org.apache.lucene.search.BooleanScorer2$2.<init>(BooleanScorer2.java:185
)
at
org.apache.lucene.search.BooleanScorer2.countingConjunctionSumScorer(Boo
leanScorer2.java:180)
at
org.apache.lucene.search.BooleanScorer2.makeCountingSumScorerSomeReq(Boo
leanScorer2.java:237)
at
org.apache.lucene.search.BooleanScorer2.makeCountingSumScorer(BooleanSco
rer2.java:215)
at
org.apache.lucene.search.BooleanScorer2.<init>(BooleanScorer2.java:101)
at
org.apache.lucene.search.BooleanQuery$BooleanWeight.scorer(BooleanQuery.
java:328)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:250)
at org.apache.lucene.search.Searcher.search(Searcher.java:171)
at
org.apache.solr.search.SolrIndexSearcher.getDocSetNC(SolrIndexSearcher.j
ava:651)
at
org.apache.solr.search.SolrIndexSearcher.getDocSet(SolrIndexSearcher.jav
a:545)
at
org.apache.solr.search.SolrIndexSearcher.cacheDocSet(SolrIndexSearcher.j
ava:520)
at
org.apache.solr.search.SolrIndexSearcher$2.regenerateItem(SolrIndexSearc
her.java:296)
at
org.apache.solr.search.FastLRUCache.warm(FastLRUCache.java:168)
at
org.apache.solr.search.SolrIndexSearcher.warm(SolrIndexSearcher.java:148
1)
at org.apache.solr.core.SolrCore$2.call(SolrCore.java:1130)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecuto
r.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
va:908)
at java.lang.Thread.run(Thread.java:619)

Jun 29, 2010 9:39:00 AM org.apache.solr.common.SolrException log
SEVERE: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at
java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.
java:439)
at
org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:467)
at
org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:319)
at
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.ja
va:264)
at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:31
7)
at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acc
ess$101(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run
Periodic(ScheduledThreadPoolExecutor.java:181)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run
(ScheduledThreadPoolExecutor.java:205)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecuto
r.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
va:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NullPointerException
at
org.apache.lucene.index.SegmentReader$Norm.decRef(SegmentReader.java:407
)
at
org.apache.lucene.index.SegmentReader.doClose(SegmentReader.java:890)
at
org.apache.lucene.index.IndexReader.decRef(IndexReader.java:170)
at
org.apache.lucene.index.DirectoryReader.doClose(DirectoryReader.java:809
)
at
org.apache.lucene.index.IndexReader.decRef(IndexReader.java:170)
at
org.apache.solr.search.SolrIndexReader.decRef(SolrIndexReader.java:429)
at
org.apache.solr.search.SolrIndexSearcher.close(SolrIndexSearcher.java:23
4)
at org.apache.solr.core.SolrCore$6.close(SolrCore.java:1246)
at org.apache.solr.util.RefCounted.decref(RefCounted.java:57)
at org.apache.solr.core.SolrCore$5.call(SolrCore.java:1192)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more

And finaly i get this Error when a lot updates reach the master and the
slave replication has to do alot updates:

SEVERE: SnapPull failed
org.apache.solr.common.SolrException: Index fetch failed :
at
org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:329)
at
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.ja
va:264)
at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:31
7)
at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acc
ess$101(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run
Periodic(ScheduledThreadPoolExecutor.java:181)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run
(ScheduledThreadPoolExecutor.java:205)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecuto
r.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
va:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.FileNotFoundException: File does not exist
./solr/data/index.20100629081400/_12fy.prx
at org.apache.solr.common.util.FileUtils.sync(FileUtils.java:55)
at
org.apache.solr.handler.SnapPuller$FileFetcher$1.run(SnapPuller.java:911
)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more

Any ideas/hints would be appreciated

Kind regards
B.Spitzer

[Bastian Spitzer]

What i forgot to mention is that those Errors only occur on the Slave, the Master is working just fine.
Ram/Hardware/Java Version/Config/Startup parameters etc. are exactly the same on both Machines.

-----Ursprüngliche Nachricht-----
Von: Bastian Spitzer [mailto:***@magix.net]
Gesendet: Dienstag, 29. Juni 2010 09:56
An: solr-***@lucene.apache.org
Betreff: Some issues concerning SOLR 1.4.1

Hi,

We just migrated from SOLR 1.4 to 1.4.1. We are observing some new Errors in the logs that didnt occured before the migration, so we want to share them with you and are hoping to get some help solving them.

We are using 1Master and 1Slave with replication on 2 different machines running only the solr instances.

The first one is this Warning, and maybe ist the reason for the following Errors:

SEVERE: ConcurrentLRUCache was not destroyed prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
Jun 29, 2010 9:39:13 AM org.apache.solr.common.util.ConcurrentLRUCache
finalize
SEVERE: ConcurrentLRUCache was not destroyed prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
Jun 29, 2010 9:39:13 AM org.apache.solr.request.UnInvertedField uninvert

Then every now and then those 2 pop up:

SEVERE: Error during auto-warming of key:+mandant_id:1000
+partner_id:1000 +active_1000:T +int(priority_1000) +language_id:1004
+provider:"han kr?ll":java.io.
IOException: read past EOF
at
org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.jav
a:151)
at
org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.j
ava:38)
at
org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:78)
at
org.apache.lucene.index.SegmentTermPositions.readDeltaPosition(SegmentTe
rmPositions.java:73)
at
org.apache.lucene.index.SegmentTermPositions.nextPosition(SegmentTermPos
itions.java:69)
at
org.apache.lucene.search.PhrasePositions.nextPosition(PhrasePositions.ja
va:76)
at
org.apache.lucene.search.PhrasePositions.firstPosition(PhrasePositions.j
ava:65)
at
org.apache.lucene.search.ExactPhraseScorer.phraseFreq(ExactPhraseScorer.
java:34)
at
org.apache.lucene.search.PhraseScorer.doNext(PhraseScorer.java:105)
at
org.apache.lucene.search.PhraseScorer.nextDoc(PhraseScorer.java:89)
at
org.apache.lucene.search.ConjunctionScorer.<init>(ConjunctionScorer.java
:42)
at
org.apache.lucene.search.ConjunctionScorer.<init>(ConjunctionScorer.java
:33)
at
org.apache.lucene.search.BooleanScorer2$2.<init>(BooleanScorer2.java:185
)
at
org.apache.lucene.search.BooleanScorer2.countingConjunctionSumScorer(Boo
leanScorer2.java:180)
at
org.apache.lucene.search.BooleanScorer2.makeCountingSumScorerSomeReq(Boo
leanScorer2.java:237)
at
org.apache.lucene.search.BooleanScorer2.makeCountingSumScorer(BooleanSco
rer2.java:215)
at
org.apache.lucene.search.BooleanScorer2.<init>(BooleanScorer2.java:101)
at
org.apache.lucene.search.BooleanQuery$BooleanWeight.scorer(BooleanQuery.
java:328)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:250)
at org.apache.lucene.search.Searcher.search(Searcher.java:171)
at
org.apache.solr.search.SolrIndexSearcher.getDocSetNC(SolrIndexSearcher.j
ava:651)
at
org.apache.solr.search.SolrIndexSearcher.getDocSet(SolrIndexSearcher.jav
a:545)
at
org.apache.solr.search.SolrIndexSearcher.cacheDocSet(SolrIndexSearcher.j
ava:520)
at
org.apache.solr.search.SolrIndexSearcher$2.regenerateItem(SolrIndexSearc
her.java:296)
at
org.apache.solr.search.FastLRUCache.warm(FastLRUCache.java:168)
at
org.apache.solr.search.SolrIndexSearcher.warm(SolrIndexSearcher.java:148
1)
at org.apache.solr.core.SolrCore$2.call(SolrCore.java:1130)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecuto
r.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
va:908)
at java.lang.Thread.run(Thread.java:619)

Jun 29, 2010 9:39:00 AM org.apache.solr.common.SolrException log
SEVERE: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at
java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.
java:439)
at
org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:467)
at
org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:319)
at
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.ja
va:264)
at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:31
7)
at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acc
ess$101(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run
Periodic(ScheduledThreadPoolExecutor.java:181)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run
(ScheduledThreadPoolExecutor.java:205)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecuto
r.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
va:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NullPointerException
at
org.apache.lucene.index.SegmentReader$Norm.decRef(SegmentReader.java:407
)
at
org.apache.lucene.index.SegmentReader.doClose(SegmentReader.java:890)
at
org.apache.lucene.index.IndexReader.decRef(IndexReader.java:170)
at
org.apache.lucene.index.DirectoryReader.doClose(DirectoryReader.java:809
)
at
org.apache.lucene.index.IndexReader.decRef(IndexReader.java:170)
at
org.apache.solr.search.SolrIndexReader.decRef(SolrIndexReader.java:429)
at
org.apache.solr.search.SolrIndexSearcher.close(SolrIndexSearcher.java:23
4)
at org.apache.solr.core.SolrCore$6.close(SolrCore.java:1246)
at org.apache.solr.util.RefCounted.decref(RefCounted.java:57)
at org.apache.solr.core.SolrCore$5.call(SolrCore.java:1192)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more

And finaly i get this Error when a lot updates reach the master and the slave replication has to do alot updates:

SEVERE: SnapPull failed
org.apache.solr.common.SolrException: Index fetch failed :
at
org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:329)
at
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.ja
va:264)
at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:31
7)
at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acc
ess$101(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run
Periodic(ScheduledThreadPoolExecutor.java:181)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run
(ScheduledThreadPoolExecutor.java:205)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecuto
r.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
va:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.FileNotFoundException: File does not exist ./solr/data/index.20100629081400/_12fy.prx
at org.apache.solr.common.util.FileUtils.sync(FileUtils.java:55)
at
org.apache.solr.handler.SnapPuller$FileFetcher$1.run(SnapPuller.java:911
)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more

Any ideas/hints would be appreciated

Kind regards
B.Spitzer

[debtaylor]

I know this post was a while ago, but it's the only one I've found which
exactly matches what we're seeing with our solr application. We recently
upgraded to 1.4.1 and all of the issues you have listed are happening to us.
Did you find a solution? Thanks.