Mailing-List: contact general-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@lucene.apache.org
Received-SPF: pass (nike.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=Received:X-Yahoo-SMTP:X-YMail-OSG:X-Yahoo-Newman-Property:Message-Id:From:To:In-Reply-To:Content-Type:Content-Transfer-Encoding:Mime-Version:Subject:Date:References:X-Mailer;
  b=JhZkbb0ncqQZZPnhmsCMBe9dJfBaargA3nJMQf/3lvhk4qrFLdkCmwARWtQckOHQKtAnIRmUyS1hZJWQCXdOYJjow/Y++MKbu8cyYrvKCNmKWAqwsajRk8V+V/CN98NjuVUGo7QVNKR7CDp8nFhFgEx92nOg2n0LW6KYqySIrvA=
  ;
Message-Id: <C7B1B0F9-6C50-4F0D-962C-9FA169FF0698@yahoo.com>
From: Chris Collins <chris_j_collins@yahoo.com>
To: general@lucene.apache.org
In-Reply-To: <24195272.post@talk.nabble.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v935.3)
Subject: Re: Index Ratio
Date: Wed, 24 Jun 2009 19:47:52 -0700
References: <24195272.post@talk.nabble.com>

You mention documents of various file types.  It really depends on  
what those types are.  For example the amount of text found in a  
powerpoint file is slim pickins.  Ratios with office type apps tend to  
be pretty fluffy.  I have seen considerably better than 20-30% when  
extracting text from such formats, some down to the ratio your talking  
of.

C
On Jun 24, 2009, at 5:47 PM, pof wrote:

>
> Hi, I just completed a batch test index of ~1100 documents of  
> various file
> types and I noticed that the original documents take up about 145MB  
> but my
> index is only 1.7MB?? I remember reading somewhere that the typical
> compression rate is about 20-30% or something, but mine is a little  
> over 1%!
> I'm not complaining or anything It just struck me a odd especially  
> as I have
> a lot of archive files and emails with attachments that I parse as  
> well. Has
> anyone else experienced something like this, I'm just curious.
>
> Cheers. Brett.
> -- 
> View this message in context: http://www.nabble.com/Index-Ratio-tp24195272p24195272.html
> Sent from the Lucene - General mailing list archive at Nabble.com.
>