Return-Path: Delivered-To: apmail-lucene-general-archive@www.apache.org Received: (qmail 3623 invoked from network); 7 Feb 2008 01:31:56 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 7 Feb 2008 01:31:56 -0000 Received: (qmail 63506 invoked by uid 500); 7 Feb 2008 01:31:47 -0000 Delivered-To: apmail-lucene-general-archive@lucene.apache.org Received: (qmail 63479 invoked by uid 500); 7 Feb 2008 01:31:47 -0000 Mailing-List: contact general-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: general@lucene.apache.org Delivered-To: mailing list general@lucene.apache.org Received: (qmail 63468 invoked by uid 99); 7 Feb 2008 01:31:47 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 06 Feb 2008 17:31:47 -0800 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [209.85.198.186] (HELO rv-out-0910.google.com) (209.85.198.186) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Feb 2008 01:31:17 +0000 Received: by rv-out-0910.google.com with SMTP id k20so2241262rvb.5 for ; Wed, 06 Feb 2008 17:31:24 -0800 (PST) Received: by 10.141.115.6 with SMTP id s6mr7196586rvm.4.1202347884245; Wed, 06 Feb 2008 17:31:24 -0800 (PST) Received: from Perse-2.local ( [220.233.189.99]) by mx.google.com with ESMTPS id l31sm8045732rvb.27.2008.02.06.17.31.22 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 06 Feb 2008 17:31:23 -0800 (PST) Message-ID: <47AA5F68.5050002@holsman.net> Date: Thu, 07 Feb 2008 12:31:20 +1100 From: Ian Holsman User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: general@lucene.apache.org Subject: Re: Lucene-based Distributed Index Leveraging Hadoop References: <3B5AF75FB405D64B8089C6863C52AB2E2A939D@NJ-E2K3-MBOX01.cnet.cnwk> <47AA1F1E.5030603@holsman.net> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org Ning Li wrote: > One main focus is to provide fault-tolerance in this distributed index > system. Correct me if I'm wrong, I think SOLR-303 is focusing on merging > results from multiple shards right now. We'd like to start an open source > project for a fault-tolerant distributed index system (or join if one > already exists) if there is enough interest. Making Solr work on top of such > a system could be an important goal and SOLR-303 is a big part of it in that > case. > I guess it depends on how you set up your shards in 303. We plan on having a master/slave relationship on each shard, so that each shard would sync the same way solr does currently. regards Ian > I should have made it clear that disjoint data sets are not a requirement of > the system. > > > On Feb 6, 2008 12:57 PM, Ian Holsman wrote: > > >> Hi. >> AOL has a couple of projects going on in the lucene/hadoop/solr space, >> and we will be pushing more stuff out as we can. We don't have anything >> going with solr over hadoop at the moment. >> >> I'm not sure if this would be better than what SOLR-303 does, but you >> should have a look at the work being done there. >> >> One of the things you mentioned is that the data sets are disjoint. >> SOLR-303 doesn't require this, and allows us to have a document stored >> in multiple shards (with different caching/update characteristics). >> >> >> > >