Return-Path: Delivered-To: apmail-lucene-solr-dev-archive@minotaur.apache.org Received: (qmail 81112 invoked from network); 11 Mar 2009 14:27:12 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 11 Mar 2009 14:27:12 -0000 Received: (qmail 68785 invoked by uid 500); 11 Mar 2009 14:27:10 -0000 Delivered-To: apmail-lucene-solr-dev-archive@lucene.apache.org Received: (qmail 68756 invoked by uid 500); 11 Mar 2009 14:27:10 -0000 Mailing-List: contact solr-dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: solr-dev@lucene.apache.org Delivered-To: mailing list solr-dev@lucene.apache.org Received: (qmail 68745 invoked by uid 99); 11 Mar 2009 14:27:10 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Mar 2009 07:27:10 -0700 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=FS_REPLICA,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of yseeley@gmail.com designates 209.85.221.108 as permitted sender) Received: from [209.85.221.108] (HELO mail-qy0-f108.google.com) (209.85.221.108) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Mar 2009 14:27:01 +0000 Received: by qyk6 with SMTP id 6so41883qyk.29 for ; Wed, 11 Mar 2009 07:26:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:reply-to:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type:content-transfer-encoding; bh=shW4o80mgJ4nzNlhWyRQZWAqnuGQu6k9/LoQazk2ki8=; b=XkTsQ6OPQWCP93jtjypINPE4Vu0DfewwtDv/yJRBbuCbFp+V46QYTLECCrAHsVp1Xw JDvkT+GABw4dO+35bg2ELKMOipRcW6jqwJClHKhEq5iBkFmauRQiTvnbbuNK4qfuZOy3 yucF414YVHsiKU3dD8lofGV3QKjYf1MIitKec= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:reply-to:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; b=o0U13lYVTW/LfFQp6cRCdWwJmnnAzrBOyuUOMWMdVZVvmMXyIp2RUV7/mXLa/DiHPk etWyYYhXXeaWvoQvaNf5HHCYYgK+urcZNwOfYm96UT0GN222OdFUs+uw6LVj5M2PouRD /AjBj41ADE6AQbEp98XLdy19GgR+OlklbV++E= MIME-Version: 1.0 Sender: yseeley@gmail.com Reply-To: yonik@lucidimagination.com Received: by 10.229.98.202 with SMTP id r10mr4302807qcn.68.1236781594639; Wed, 11 Mar 2009 07:26:34 -0700 (PDT) In-Reply-To: References: Date: Wed, 11 Mar 2009 10:26:34 -0400 X-Google-Sender-Auth: 214feeb7fb32de6d Message-ID: Subject: Re: fsync, potential data loss, java replication From: Yonik Seeley To: solr-dev@lucene.apache.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org On Wed, Mar 11, 2009 at 12:25 AM, Noble Paul =E0=B4=A8=E0=B5=8B=E0=B4=AC=E0= =B4=BF=E0=B4=B3=E0=B5=8D=E2=80=8D =E0=A4=A8=E0=A5=8B=E0=A4=AC=E0=A5=8D=E0= =A4=B3=E0=A5=8D > hi yonik , if you can point to the place (code) where lucene does this > we can copy the same code to Solr Replication as well FSDirectory.sync(String name) is the lowest level. That's called from IndexWriter.startCommit() and then SegmentInfos.finishCommit() to flush the segment file. That's hairy code... I imagine we may want to just use the code from FSDirectory.sync() and make our own sync service based on an executor. It seems like we would want to throw a file to the sync executor as soon as we've downloaded it. Then wait until all files are sync'd before moving everything out of the temp directory. -Yonik http://www.lucidimagination.com