incubator-blur-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravikumar Govindarajan <ravikumar.govindara...@gmail.com>
Subject Re: IndexReaderClosedException...
Date Tue, 10 May 2016 13:46:01 GMT
Thanks a lot Aaron.

I guess we took a commit of 0.2.2 that doesn't have the
IndexRefreshWriteLock (IRWL). It looks like it co-ordinates between
searches & incoming mutation commits. If so, then it will likely solve the
first issue for us (AlreadyClosedException)


Can you recollect if that was the reason IRWL was introduced?

On Tue, May 10, 2016 at 6:40 PM, Aaron McCurry <amccurry@gmail.com> wrote:

> On Tue, May 10, 2016 at 2:30 AM, Ravikumar Govindarajan <
> ravikumar.govindarajan@gmail.com> wrote:
>
> > Actually there are 2 issues...
> >
> > 1. IndexReaderClosedException
> > 2. HDFS Stream Closed
> >
>
> Likely when the index is closed it closes the underlying indexinputs as
> well causing the HDFS Stream closed exception.
>
>
> >
> > Merge completion results in File Deletion & ultimately HDFS Stream Closed
> > during Search....
> >
> > I use IndexFileDeleter with KeepOnlyLastCommitDeletionPolicy. This
> blindly
> > deletes the file, without bothering to cross-check IndexReader.RefCount >
> > 0.
> >
>
> Hmm.  You can see here:
>
>
> https://github.com/apache/incubator-blur/blob/release-0.2.2-incubating/blur-core/src/main/java/org/apache/blur/manager/writer/BlurIndexSimpleWriter.java#L303
>
> That once the new index is available it is swapped into the index ref
> object and the old one is sent to the index closer.  Once the ref to the
> index are low enough it closes the index.  Or at least it should.
>
> I will continue looking into the problem but I don't have a solution for
> you yet.
>
> Aaron
>
>
>
> >
> >
> > *Exception(message:Unknown error during rewrite,
> > stackTraceStr:java.io.IOException: Stream closed*
> > at org.apache.hadoop.hdfs.DFSInputStream.pread(DFSInputStream.java:1385)
> > at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:1374)
> > at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:89)
> > at
> >
> >
> org.apache.blur.store.hdfs.HdfsIndexInput.readInternal(HdfsIndexInput.java:62)
> > at
> >
> >
> org.apache.blur.store.buffer.ReusedBufferedIndexInput.readBytes(ReusedBufferedIndexInput.java:167)
> > at
> >
> >
> org.apache.blur.store.buffer.ReusedBufferedIndexInput.readBytes(ReusedBufferedIndexInput.java:122)
> > at
> >
> >
> org.apache.blur.store.hdfs.MmapCacheIndexInput.readAndcache(MmapCacheIndexInput.java:24)
> > at
> >
> >
> org.apache.blur.store.blockcache_v2.CacheIndexInput.fillNormally(CacheIndexInput.java:354)
> > at
> >
> >
> org.apache.blur.store.blockcache_v2.CacheIndexInput.fill(CacheIndexInput.java:379)
> > at
> >
> >
> org.apache.blur.store.blockcache_v2.CacheIndexInput.tryToFill(CacheIndexInput.java:297)
> > at
> >
> >
> org.apache.blur.store.blockcache_v2.CacheIndexInput.readByte(CacheIndexInput.java:151)
> > at
> >
> >
> org.apache.blur.lucene.warmup.TraceableIndexInput.readByte(TraceableIndexInput.java:62)
> > at org.apache.lucene.store.DataInput.readVInt(DataInput.java:108)
> > at
> >
> >
> org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.loadBlock(BlockTreeTermsReader.java:2366)
> > at
> >
> >
> org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum.seekCeil(BlockTreeTermsReader.java:1949)
> > at
> >
> >
> org.apache.blur.index.ExitableReader$ExitableTermsEnum.seekCeil(ExitableReader.java:250)
> > at
> >
> org.apache.lucene.index.FilteredTermsEnum.next(FilteredTermsEnum.java:225)
> > at
> >
> >
> org.apache.lucene.search.TermCollectingRewrite.collectTerms(TermCollectingRewrite.java:78)
> > at
> >
> >
> org.apache.lucene.search.ConstantScoreAutoRewrite.rewrite(ConstantScoreAutoRewrite.java:95)
> > at
> >
> >
> org.apache.lucene.search.MultiTermQuery$ConstantScoreAutoRewrite.rewrite(MultiTermQuery.java:220)
> > at
> org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:288)
> > at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:412)
> > at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:412)
> > at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:412)
> > at
> >
> > On Mon, May 9, 2016 at 4:42 PM, Ravikumar Govindarajan <
> > ravikumar.govindarajan@gmail.com> wrote:
> >
> > > One extra info we gleaned from the logs...
> > >
> > > 1. Merge Starts & is about to complete
> > > 2. Searcher is opened
> > > 3. Merge Completes
> > > 4. Ref-count drops to 0 in IndexReader
> > > 5. IndexReader closed while Searcher is still open
> > >
> > > This seems to be the main pattern for causing the Exception
> > >
> > > --
> > > Ravi
> > >
> > > On Mon, May 9, 2016 at 3:08 PM, Ravikumar Govindarajan <
> > > ravikumar.govindarajan@gmail.com> wrote:
> > >
> > >> Thanks Aaron...
> > >>
> > >> Just a quick question. Lucene itself has ref-counting to close it's
> > >> readers no? Or Blur has it's own logic to handle it?
> > >>
> > >> --
> > >> Ravi
> > >>
> > >> On Fri, May 6, 2016 at 7:56 PM, Aaron McCurry <amccurry@gmail.com>
> > wrote:
> > >>
> > >>> Likely yes.  If have a few minutes this weekend I can look through
> that
> > >>> version and see if I can point you in the right direction.
> > >>>
> > >>> On Fri, May 6, 2016 at 8:46 AM, Ravikumar Govindarajan <
> > >>> ravikumar.govindarajan@gmail.com> wrote:
> > >>>
> > >>> > Sometimes during an ongoing search we receive an
> > >>> > IndexReaderClosedException...
> > >>> >
> > >>> > We are on an older version of Blur (0.2.2). Has this been fixed
in
> > >>> newer
> > >>> > versions or we have been using it wrongly?
> > >>> >
> > >>> > *stackTraceStr:org.apache.lucene.store.AlreadyClosedException:
this
> > >>> > IndexReader cannot be used anymore as one of its child readers
was
> > >>> closed*
> > >>> > at
> > org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:257)
> > >>> > at
> > >>> >
> > >>> >
> > >>>
> >
> org.apache.lucene.index.FilterAtomicReader.fields(FilterAtomicReader.java:380)
> > >>> > at
> > >>> >
> > >>> >
> > >>>
> >
> org.apache.blur.index.ExitableReader$ExitableFilterAtomicReader.fields(ExitableReader.java:81)
> > >>> > at
> > >>> >
> > >>> >
> > >>>
> >
> org.apache.lucene.search.TermCollectingRewrite.collectTerms(TermCollectingRewrite.java:52)
> > >>> > at
> > >>> >
> > >>> >
> > >>>
> >
> org.apache.lucene.search.ConstantScoreAutoRewrite.rewrite(ConstantScoreAutoRewrite.java:95)
> > >>> > at
> > >>> >
> > >>> >
> > >>>
> >
> org.apache.lucene.search.MultiTermQuery$ConstantScoreAutoRewrite.rewrite(MultiTermQuery.java:220)
> > >>> > at
> > >>>
> > org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:288)
> > >>> > at
> > org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:412)
> > >>> >
> > >>>
> > >>
> > >>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message