Return-Path: Delivered-To: apmail-hadoop-mapreduce-dev-archive@minotaur.apache.org Received: (qmail 61842 invoked from network); 12 Jul 2010 22:36:43 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 12 Jul 2010 22:36:43 -0000 Received: (qmail 15935 invoked by uid 500); 12 Jul 2010 22:36:43 -0000 Delivered-To: apmail-hadoop-mapreduce-dev-archive@hadoop.apache.org Received: (qmail 15887 invoked by uid 500); 12 Jul 2010 22:36:42 -0000 Mailing-List: contact mapreduce-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-dev@hadoop.apache.org Delivered-To: mailing list mapreduce-dev@hadoop.apache.org Received: (qmail 15879 invoked by uid 99); 12 Jul 2010 22:36:42 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 12 Jul 2010 22:36:42 +0000 X-ASF-Spam-Status: No, hits=2.9 required=10.0 tests=HTML_MESSAGE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [209.85.214.176] (HELO mail-iw0-f176.google.com) (209.85.214.176) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 12 Jul 2010 22:36:33 +0000 Received: by iwn37 with SMTP id 37so7655532iwn.35 for ; Mon, 12 Jul 2010 15:35:12 -0700 (PDT) Received: by 10.231.176.16 with SMTP id bc16mr15816699ibb.4.1278974112377; Mon, 12 Jul 2010 15:35:12 -0700 (PDT) MIME-Version: 1.0 Received: by 10.231.162.69 with HTTP; Mon, 12 Jul 2010 15:34:52 -0700 (PDT) In-Reply-To: References: From: Todd Lipcon Date: Mon, 12 Jul 2010 15:34:52 -0700 Message-ID: Subject: Re: SequenceFile and hadoop sync To: mapreduce-dev@hadoop.apache.org Content-Type: multipart/alternative; boundary=00163628532aa52b34048b385dc6 X-Virus-Checked: Checked by ClamAV on apache.org --00163628532aa52b34048b385dc6 Content-Type: text/plain; charset=ISO-8859-1 Hi Ted, You'll find that SequenceFile.Writer has a syncFs() method in the append branch. This syncs the underlying FSDataOutputStream. -Todd On Mon, Jul 12, 2010 at 3:29 PM, Ted Yu wrote: > We are using cdh3b2 which contains append/sync functionality. > Our code uses SequenceFile.BlockCompressWriter.append() to write to > sequence > file. Since FSDataOutputStream out doesn't have public visibility (in > SequenceFile.Writer), I am wondering how we can utilize the new sync > functionality. > > Your comment is appreciated. > -- Todd Lipcon Software Engineer, Cloudera --00163628532aa52b34048b385dc6--