lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Klaas <mike.kl...@gmail.com>
Subject Re: Multiple indexes
Date Thu, 30 Aug 2007 18:25:24 GMT

On 30-Aug-07, at 10:57 AM, Nathaniel E. Powell wrote:

> Is there functionality for partitioning Solr indexes onto multiple  
> machines? For this to work, I suppose that Solr would have to  
> combine the results from the various machines. I think Nutch does  
> this with the distributed searcher functionality.

Not currently developed.  See

http://wiki.apache.org/solr/FederatedSearch and
http://issues.apache.org/jira/browse/SOLR-303

-Mike


> -Nathan
>
> -----Original Message-----
> From: Mike Klaas [mailto:mike.klaas@gmail.com]
> Sent: Thursday, August 30, 2007 11:44 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Multiple indexes
>
>
> On 29-Aug-07, at 10:21 PM, James liu wrote:
>
>> Does it affect with doc sizeļ¼Ÿ
>>
>> for example 2 billion docs, 10k doc,,,,2 billion docs, but doc size
>> is 10m.
>
> There might be other places that have 2G limit (see lucene index
> format docs), but many things are vints and can grow larger.
>
> Of course you will hit physical limits of your machine long before
> you can achieve your hypothetical situation: that's 20,000 Tb, which
> is many, many times the size of a complete internet crawl.
>
> -Mike
>>
>> 2007/8/30, Mike Klaas <mike.klaas@gmail.com>:
>>>
>>> 2 billion docs (signed int).
>>>
>>> On 29-Aug-07, at 6:24 PM, James liu wrote:
>>>
>>>> what is the limits for Lucene and Solr.
>>>>
>>>> 100m, 1000m, 5000m or other number docs?
>>>>
>>>> 2007/8/24, Walter Underwood <wunderwood@netflix.com>:
>>>>>
>>>>> It should work fine to index them and search them. 13 million
>>>>> docs is
>>>>> not even close to the limits for Lucene and Solr. Have you had
>>>>> problems?
>>>>>
>>>>> wunder
>>>>>
>>>>> On 8/23/07 7:30 AM, "Jae Joo" <jaejoo@gmail.com> wrote:
>>>>>
>>>>>> Is there any solution to handle 13 millions document shown as
>>>>>> below?
>>>>>> Each document is not big, but the number of ones is 13 million.
>>>>>> Any way to utilize the multiple indexes?
>>>>>>
>>>>>> Thanks,
>>>>>> Jae Joo
>>>>>>
>>>>>>
>>>>>> <doc><field name="trade2"></field>
>>>>>> <field name="company_name">Unlimi-Tech Software Inc</field>
>>>>>> <field name="phys_stabrv">ON</field>
>>>>>> <field name="trade4"></field>
>>>>>> <field name="status_id_descr">Single Location</field>
>>>>>> <field name="trade3"></field>
>>>>>> <field name="phys_country">Canada</field>
>>>>>> <field name="phys_zip">K1C 4R1</field>
>>>>>> <field name="phys_city">Ottawa</field>
>>>>>> <field name="phys_state">Ontario</field>
>>>>>> <field name="sic2">G2_Computer Software</field>
>>>>>> <field name="phys_address">1447a Youville Dr</field>
>>>>>> <field name="sic1">G_Technology &amp; Communications</field>
>>>>>> <field name="duns_number">203439018</field>
>>>>>> <field name="trade1"></field>
>>>>>> <field name="phys_county">Carleton</field>
>>>>>> <field name="trade5"></field>
>>>>>> <field name="status_id_rank">30</field>
>>>>>> <field name="sic4">G2173_Computer Programming Services</field>
>>>>>> <field name="sic8">G217308D_Computer software development</field>
>>>>>> </doc>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> regards
>>>> jl
>>>
>>>
>>
>>
>> -- 
>> regards
>> jl
>


Mime
View raw message