lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ritesh Ambastha <group.rit...@gmail.com>
Subject Re: Index structuring
Date Wed, 04 Jun 2008 10:47:32 GMT

The number of docs I have indexed till now is : 1,633,570
I am bit afraid as the number of indexed docs will grow atleast 5-10 times
in very near future. 

Regards,
Ritesh Ambastha 



Shalin Shekhar Mangar wrote:
> 
> A lot of this also depends on the number of documents. But we have
> successfully used Solr with upto 10-12 million documents.
> 
> On Wed, Jun 4, 2008 at 4:10 PM, Ritesh Ambastha <group.ritesh@gmail.com>
> wrote:
> 
>>
>> Thanks Noble,
>>
>> That means, I can go ahead with single Index for long.
>> :)
>>
>> Regards,
>> Ritesh Ambastha
>>
>> Noble Paul നോബിള്‍ नोब्ळ् wrote:
>> >
>> > For the datasize you are proposing , single index should be fine .Just
>> > give the m/c enough RAM
>> >
>> > Distributed search involves multiple requests made between shards
>> > which may be an unncessary overhead.
>> > --Noble
>> >
>> > On Wed, Jun 4, 2008 at 4:02 PM, Ritesh Ambastha
>> <group.ritesh@gmail.com>
>> > wrote:
>> >>
>> >> Thanks Noble,
>> >>
>> >> I maintain two separate indexes on my disk for two different search
>> >> services.
>> >> The index size of two are: 91MB and 615MB. I am pretty sure that these
>> >> index
>> >> size will grow in future, and may reach 10GB.
>> >>
>> >> My doubts :
>> >>
>> >> 1. When should I start partitioning my index?
>> >> 2. Is there any performance issue with partitioning? For eg: A query
>> on
>> >> 1GB
>> >> and 500MB indexed data will take same time to give the result? Or
>> lesser
>> >> the
>> >> index size, lesser the response time?
>> >>
>> >>
>> >> Regards,
>> >> Ritesh Ambastha
>> >>
>> >> Noble Paul നോബിള്‍ नोब्ळ् wrote:
>> >>>
>> >>> You could have been more specific on the dataset size.
>> >>>
>> >>> If your data volumes are growing you can partition your index into
>> >>> multiple shards.
>> >>> http://wiki.apache.org/solr/DistributedSearch
>> >>> --Noble
>> >>>
>> >>> On Sat, May 31, 2008 at 9:02 PM, Ritesh Ambastha
>> >>> <group.ritesh@gmail.com>
>> >>> wrote:
>> >>>>
>> >>>> Dear Readers,
>> >>>>
>> >>>> I am a newbie in solr world. I have successfully deployed solr on
my
>> >>>> machine, and I am able to index a large DB table. I am pretty sure
>> that
>> >>>> internal index structure of solr is much capable to handle large
>> data
>> >>>> sets.
>> >>>>
>> >>>> But, say my data size keeps growing at jet speed, then what should
>> be
>> >>>> the
>> >>>> index structure? Do I need to follow some specific index structuring
>> >>>> patterns/algos for handling such massive data?
>> >>>>
>> >>>> I am sorry as I may be sounding novice in this area. I would
>> appreciate
>> >>>> your
>> >>>> thoughts/suggestions.
>> >>>>
>> >>>> Regards,
>> >>>> Ritesh Ambastha
>> >>>> --
>> >>>> View this message in context:
>> >>>> http://www.nabble.com/Index-structuring-tp17576449p17576449.html
>> >>>> Sent from the Solr - User mailing list archive at Nabble.com.
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> --Noble Paul
>> >>>
>> >>>
>> >>
>> >> --
>> >> View this message in context:
>> >> http://www.nabble.com/Index-structuring-tp17576449p17643690.html
>> >> Sent from the Solr - User mailing list archive at Nabble.com.
>> >>
>> >>
>> >
>> >
>> >
>> > --
>> > --Noble Paul
>> >
>> >
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Index-structuring-tp17576449p17643798.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>>
> 
> 
> -- 
> Regards,
> Shalin Shekhar Mangar.
> 
> 

-- 
View this message in context: http://www.nabble.com/Index-structuring-tp17576449p17643909.html
Sent from the Solr - User mailing list archive at Nabble.com.


Mime
View raw message