Return-Path: Delivered-To: apmail-lucene-java-dev-archive@www.apache.org Received: (qmail 6557 invoked from network); 15 Apr 2010 12:33:06 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 15 Apr 2010 12:33:06 -0000 Received: (qmail 84773 invoked by uid 500); 15 Apr 2010 12:33:05 -0000 Delivered-To: apmail-lucene-java-dev-archive@lucene.apache.org Received: (qmail 84729 invoked by uid 500); 15 Apr 2010 12:33:05 -0000 Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: java-dev@lucene.apache.org Delivered-To: mailing list java-dev@lucene.apache.org Received: (qmail 84722 invoked by uid 99); 15 Apr 2010 12:33:05 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 15 Apr 2010 12:33:05 +0000 X-ASF-Spam-Status: No, hits=4.7 required=10.0 tests=FREEMAIL_FROM,FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of torindan@gmail.com designates 74.125.92.24 as permitted sender) Received: from [74.125.92.24] (HELO qw-out-2122.google.com) (74.125.92.24) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 15 Apr 2010 12:32:57 +0000 Received: by qw-out-2122.google.com with SMTP id 8so364865qwh.53 for ; Thu, 15 Apr 2010 05:32:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:received:message-id:subject:from:to:content-type; bh=5DHxuKxRJjbnZTJFxvshmu50/gIkLQ3xpylRKjnkn9Q=; b=StJMt4lNYix2CcgzlUstNixniueFykmTi6S6rMbDo55/FUDp/pCo4y7FsrPjNi9d42 EOp+o3G1Ht4NBTWd1/+oNCCAW4rX6EdlKgAJSDQTHWm5nVQzzAdPujObr029kVI5Z8d4 Z7XlSnLV6px6vyTq9t+0sB+6q2ZS4C2wHTy+M= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=TP8lSq9/gYGpmfPhebcNEkPaCdmlVMycsOLJt7BQ+9Fp9KUYuA4phadYnbUiKUmogU 6h9MA5FLC7HmTQ1y+oBptI0+IhIg7CJHHGX0G3iu+fPivTW8jsYeZzdlhNfFYHpoGQSN JohqHxStr4SCGhVsFYE2PkoQmSNfexAst5AlQ= MIME-Version: 1.0 Received: by 10.229.101.195 with HTTP; Thu, 15 Apr 2010 05:32:35 -0700 (PDT) In-Reply-To: References: <6A31E90C-34F8-4510-94C8-FA67ED511E27@gmail.com> Date: Thu, 15 Apr 2010 15:32:35 +0300 Received: by 10.229.97.207 with SMTP id m15mr4648209qcn.6.1271334756027; Thu, 15 Apr 2010 05:32:36 -0700 (PDT) Message-ID: Subject: Re: Proposal about Version API "relaxation" From: =?UTF-8?B?RGFuaWwgxaJPUklO?= To: java-dev@lucene.apache.org Content-Type: multipart/alternative; boundary=0016364ee60e8611a1048445b039 X-Virus-Checked: Checked by ClamAV on apache.org --0016364ee60e8611a1048445b039 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable All I ask is a way to migrate existing indexes to newer format. On Thu, Apr 15, 2010 at 15:21, Robert Muir wrote: > its open source, if you feel this way, you can put the work to add featur= es > to some version branch from trunk in a backwards compatible way. > > Then this branch can have a backwards-compatible minor release with new > features, but nothing ground-breaking. > > but this kinda stuff shouldnt hinder development on trunk. > > > On Thu, Apr 15, 2010 at 8:17 AM, Danil =C5=A2ORIN wr= ote: > >> Sometimes it's REALLY impossible to reindex, or has absolutely prohibiti= ve >> cost to do in a running production system (i can't shut it down for >> maintainance, so i need a lot of hardware to reindex ~5 billion document= s, i >> have no idea what are the costs to retrieve that data all over again, bu= t i >> estimate it to be quite a lot) >> >> And providing a way to migrate existing indexes to new lucene is crucial >> from my point of view. >> >> I don't care what this way is: calling optimize() with newer lucene or >> running some tool that takes 5 days, it's ok with me. >> >> Just don't put me through full reindexing as I really don't have all tha= t >> data anymore. >> It's not my data, i just receive it from clients, and provide a search >> interface. >> >> It took years to build those indexes, rebuilding is not an option, and >> staying with old lucene forever just sucks. >> >> Danil. >> >> On Thu, Apr 15, 2010 at 14:57, Robert Muir wrote: >> >>> >>> >>> On Thu, Apr 15, 2010 at 7:52 AM, Shai Erera wrote: >>> >>>> Well ... I must say that I completely disagree w/ dropping index >>>> structure back-support. Our customers will simply not hear of reindexi= ng 10s >>>> of TBs of content because of version upgrades. Such a decision is key = to >>>> Lucene adoption in large-scale projects. It's entirely not about wheth= er >>>> Lucene is a content store or not - content is stored on other systems,= I >>>> agree. But that doesn't mean reindexing it is tolerable. >>>> >>>> >>> I don't understand how its helpful to do a MAJOR version upgrade withou= t >>> reindexing... what in the world do you stand to gain from that? >>> >>> The idea here, is that development can be free of such hassles. >>> Development should be this way. >>> >>> If you, Shai, need some feature X.Y.Z from Version 4 and don't want to >>> reindex, and are willing to do the work to port it back to Version 3 in= a >>> completely backwards compatible way, then under this new scheme it can >>> happen. >>> >>> >>> -- >>> Robert Muir >>> rcmuir@gmail.com >>> >> >> > > > -- > Robert Muir > rcmuir@gmail.com > --0016364ee60e8611a1048445b039 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
All I ask is a way to migrate existing indexes to newer format.


On Thu, Apr 15, 2010 at 15= :21, Robert Muir <= rcmuir@gmail.com> wrote:
its open source, if you feel this way,= you can put the work to add features to some version branch from trunk in = a backwards compatible way.

Then this branch can have a backwards-compatible minor = release with new features, but nothing ground-breaking.

but this kinda stuff shouldnt hinder development on tru= nk.


On Thu, Apr 15, 2010 at 8:17 AM, Danil =C5=A2ORIN <torindan@gmail= .com> wrote:
Sometimes it's REALLY impossible to= reindex, or has absolutely prohibitive cost to do in a running production = system (i can't shut it down for maintainance, so i need a lot of hardw= are to reindex ~5 billion documents, i have no idea what are the costs to r= etrieve that data all over again, but i estimate it to be quite a lot)

And providing a way to migrate existing indexes to new = lucene is crucial from my point of view.

I don'= ;t care what this way is: calling optimize() with newer lucene or running s= ome tool that takes 5 days, it's ok with me.

Just don't put me through full reindexing as I real= ly don't have all that data anymore.
It's not my data, i = just receive it from clients, and provide a search interface.

It took years to build those indexes, rebuilding is not an option, and = staying with old lucene forever just sucks.

Danil.

On Thu, Apr 15, 2010 at 14:57, Robert Muir <= span dir=3D"ltr"><= rcmuir@gmail.com> wrote:


On T= hu, Apr 15, 2010 at 7:52 AM, Shai Erera <serera@gmail.com> wr= ote:
Well ... I must say that I completely disagree w/ dropping= index structure back-support. Our customers will simply not hear of reinde= xing 10s of TBs of content because of version upgrades. Such a decision is = key to Lucene adoption in large-scale projects. It's entirely not about= whether Lucene is a content store or not - content is stored on other syst= ems, I agree. But that doesn't mean reindexing it is tolerable.


I don't understand how= its helpful to do a MAJOR version upgrade without reindexing... what in th= e world do you stand to gain from that?=C2=A0

The = idea here, is that development can be free of such hassles. Development sho= uld be this way.

If you, Shai, need some feature X.Y.Z from Version 4 an= d don't want to reindex, and are willing to do the work to port it back= to Version 3 in a completely backwards compatible way, then under this new= scheme it can happen.


--
Robert Muir
rcmuir@gmail.com




--
Robert Muir
rcmuir@gmail.com

--0016364ee60e8611a1048445b039--