Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of leonidms@gmail.com designates
 209.85.200.169 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:to:subject:in-reply-to:mime-version
         :content-type:references;
        b=C62MHL5vyRm3BMCs7kXa+cNuqfSrOhF9aLo0ZLyq2urXAQITHp2L/fygo9KgYwZSEg
         NSwW2xaIe+vmlleo9A8s9QdsG+6wCMQZScV29wcOFYcEt4g8rqa07m6/vbDjSuGfhQt/
         wCLu14ocYedN5I9XMrDqgjzyEgmeV873NVBEI=
Message-ID: <dcd377ca0809041537u6aed064ctaa41cb4fec9cfa62@mail.gmail.com>
Date: Fri, 5 Sep 2008 01:37:16 +0300
From: "Leonid M." <leonidms@gmail.com>
To: java-user@lucene.apache.org
Subject: Re: Problem with lucene search starting to return 0 hits when a few
 seconds earlier it was returning hundreds
In-Reply-To: <62568.57815.qm@web45215.mail.sp1.yahoo.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_64338_16521290.1220567836257"
References: <62568.57815.qm@web45215.mail.sp1.yahoo.com>

------=_Part_64338_16521290.1220567836257
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Anyway it is worth trying (to ensure docs aren't removed between searches).What
if running MatchAllDocsQuery or smth similar? Still getting different hits
count on query rerun?

PS. I'm kinda newbie with Lucene and Lucene API. So don't take my notes too
seriously :)

On Fri, Sep 5, 2008 at 12:46 AM, Justin Grunau <jjmg01@yahoo.com> wrote:

> Sorry, I forgot to include the visibility filters:
>
>                final BooleanQuery visibilityFilter = new BooleanQuery();
>                visibilityFilter.add(new TermQuery(new Term("isPublic",
> "true")),
>                        Occur.SHOULD);
>                visibilityFilter.add(new TermQuery(new Term("reader",
> user.getId())),
>                        Occur.SHOULD);
>
>
> These visibility filters ensure that a user only sees files which he or she
> has access to see.
>
> I am pretty certain nobody else has modified the index in the meantime, but
> why is that important?  We have several other servers -- whose only
> difference is a smaller data set -- with dozens of concurrent users, and the
> index on those servers gets modified and read concurrently all the time, but
> none of these other servers have ever exhibited this bug.
>
>
>
> ----- Original Message ----
> From: Leonid M. <leonidms@gmail.com>
> To: java-user@lucene.apache.org
> Sent: Thursday, September 4, 2008 5:35:47 PM
> Subject: Re: Problem with lucene search starting to return 0 hits when a
> few seconds earlier it was returning hundreds
>
> * And what's about visibility filter? * Are you sure no one else accesses
> IndexReader and modifies index? See reader.maxDocs() to be confident.
>
> On Fri, Sep 5, 2008 at 12:19 AM, Justin Grunau <jjmg01@yahoo.com> wrote:
>
> > We have some code that uses lucene which has been working perfectly well
> > for several months.
> >
> > Recently, a QA team in our organization has set up a server with a much
> > larger data set than we have ever tested with in the past:  the resulting
> > lucene index is about 3G in size.
> >
> > On this particular server, the same lucene code which has been reliable
> in
> > the past is now exhibiting erratic behavior.  The first time you do a
> > search, it returns the correct number of hits.  The second time you do a
> > search, it may or may not return the correct set.  By the third time you
> do
> > a search, it will return 0 hits even for a search that was returning
> > hundreds of hits only a few seconds earlier.  All subsequent searches
> will
> > return 0 hits until you stop and restart the java process.
> >
> > A snippet of the relevant code follows:
> >
> >                    // getReader() returns the singleton IndexReader
> object
> >                final IndexReader reader = getReader();
> >
> >                    // ANALYZER is another singleton
> >                final QueryParser queryParser = new QueryParser("text",
> > ANALYZER);
> >                queryParser.setDefaultOperator(spec.getDefaultOp());
> >                final Query query =
> > queryParser.parse(spec.getSearchText()).rewrite(
> >                        reader);
> >                final IndexSearcher searcher = new IndexSearcher(reader);
> >
> >                final Hits hits = searcher.search(query, new
> > CachingWrapperFilter(
> >                        new QueryWrapperFilter(visibilityFilter)));
> >                total = hits.length();
> >
> >
> >
> > I understand that Lucene should be able to handle very large datasets, so
> > I'd be surprised if this were an actual Lucene bug.  I'm hoping it's just
> > that I'm doing something "wrong" which has gone unnoticed so far for
> several
> > months because we've never had an index this large.
> >
> > We're using lucene verison 2.2.0.
> >
> > Thanks!
> >
> > Justin Grunau
> >
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>
>
> --
> Bests regards,
> Leonid Maslov!
> Personal blog: http://leonardinius.blogspot.com/
>
> Random thought:
> Princess Margaret  - "I have as much privacy as a goldfish in a bowl."
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>


-- 
Bests regards,
Leonid Maslov!
Personal blog: http://leonardinius.blogspot.com/

Random thought:
John Belushi  - "I owe it all to little chocolate donuts."

------=_Part_64338_16521290.1220567836257--