lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From arun k <arunk...@gmail.com>
Subject Re: CompressingStoredFieldsFormat doesn't show improvement
Date Fri, 01 Feb 2013 05:48:22 GMT
Hi,

Random data was indexed. I wanted to see the worst case where little data
is same across documents and  which in most of my cases is.
So, i guess in these scenarios compression becomes an overhead.

Arun




On Thu, Jan 31, 2013 at 8:00 PM, Robert Muir <rcmuir@gmail.com> wrote:

> The top method here is your random string generation.
>
> are you indexing random data?
>
> On Thu, Jan 31, 2013 at 12:46 AM, arun k <arunk786@gmail.com> wrote:
> > Hi,
> >
> > Please find the snapshots here.
> > http://picpaste.com/Lucene3.0.2-G00Z5FfX.png
> > http://picpaste.com/Lucene4.1-LsxpcQk0.png
> >
> > Arun
> >
> >
> > On Wed, Jan 30, 2013 at 5:30 PM, Uwe Schindler <uwe@thetaphi.de> wrote:
> >
> >> Hi,
> >>
> >>
> >>
> >> there is nothing attached to your mail; maybe the mailing list software
> >> removed it. Can you place it somewhere on the web (e.g. pastebin,…)?
> >>
> >>
> >>
> >> Uwe
> >>
> >>
> >>
> >> -----
> >>
> >> Uwe Schindler
> >>
> >> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>
> >>  <http://www.thetaphi.de/> http://www.thetaphi.de
> >>
> >> eMail: uwe@thetaphi.de
> >>
> >>
> >>
> >> From: arun k [mailto:arunk786@gmail.com]
> >> Sent: Wednesday, January 30, 2013 12:04 PM
> >> To: java-user
> >> Subject: Re: CompressingStoredFieldsFormat doesn't show improvement
> >>
> >>
> >>
> >> Adrein,
> >> Please find the attached profilers report.
> >>
> >>
> >>
> >> On Wed, Jan 30, 2013 at 3:35 PM, Adrien Grand <jpountz@gmail.com>
> wrote:
> >>
> >> On Wed, Jan 30, 2013 at 8:08 AM, arun k <arunk786@gmail.com> wrote:
> >> > Adrein,
> >> >
> >> > I have created an index of size 370M of 1 million docs of 40 fields
> of 40
> >> > chars and did the profiling.
> >> > I see that the indexing and in particular the addDocument &
> >> > ConcurrentMergeScheduler in 4.1 takes double the time compared to
> 3.0.2.
> >>
> >> Can you provide me with the detailed profiles?
> >>
> >>
> >> > Looks like CompressionStoredFieldsFormat is of little use in my
> scenario.
> >>
> >> You can can disable stored fields compression and use another
> >> StoredFieldsFormat by defining a custom codec
> >> (
> >>
> http://lucene.apache.org/core/4_1_0/core/org/apache/lucene/codecs/package-summary.html#package_description
> >> ).
> >>
> >>
> >> --
> >> Adrien
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
> >>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message