Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of harsh@cloudera.com designates
 209.85.219.42 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <556325346CA26341B6F0530E07F90D96017084360539@GBGH-EXCH-CMS.sig.ads>
References: 
 <556325346CA26341B6F0530E07F90D96017084360539@GBGH-EXCH-CMS.sig.ads>
From: Harsh J <harsh@cloudera.com>
Date: Tue, 19 Mar 2013 05:06:04 +0530
Message-ID: 
 <CAOcnVr06LjMApj=v8+urSHBMx5Yq=7zOPLd-bzFTsWuSbHDKqg@mail.gmail.com>
Subject: Re: hadoop file append
To: "<user@hadoop.apache.org>" <user@hadoop.apache.org>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

Appending on 1.x releases is available but not tested/supported and
can be toggled to be disabled completely. Appending works better on
2.x releases.

On Mon, Mar 18, 2013 at 9:14 PM, Tony Burton <TBurton@sportingindex.com> wr=
ote:
> Hi list,
>
>
>
> I=92m using Hadoop 1.0.3 for a MapReduce task and I thought it might be a
> simple job to append a Counter value and some text to the end of a file
> (which ultimately will be in AWS S3). How wrong I was :)
>
>
>
> I=92ve been reading about o.a.h.fs.FileSystem.append and whether it does =
or
> doesn=92t work in my version of hadoop, and I'm not sure if it does or no=
t.
> Can anyone say for definite either way? I'm going with the conclusion tha=
t
> it can't be trusted to perform as expected.
>
>
>
> What I'd like to do is this: Perform a daily MapReduce job and add up som=
e
> some values along the way, and as well as the output of the MR job, add t=
hat
> day's date with the Counter value to the end of a file (in a different
> location), so the file grows by one line daily. Assuming
> o.a.h.fs.FileSystem.append is currently out of favour, do you have any
> suggestions for achieving this?
>
>
>
> Thanks!
>
>
>
> Tony
>
>
>
>
> *************************************************************************=
****
> P Please consider the environment before printing this email or attachmen=
ts
>
>
> This email and any attachments are confidential, protected by copyright a=
nd
> may be legally privileged. If you are not the intended recipient, then th=
e
> dissemination or copying of this email is prohibited. If you have receive=
d
> this in error, please notify the sender by replying by email and then del=
ete
> the email completely from your system. Neither Sporting Index nor the sen=
der
> accepts responsibility for any virus, or any other defect which might aff=
ect
> any computer or IT system into which the email is received and/or opened.=
 It
> is the responsibility of the recipient to scan the email and no
> responsibility is accepted for any loss or damage arising in any way from
> receipt or use of this email. Sporting Index Ltd is a company registered =
in
> England and Wales with company number 2636842, whose registered office is=
 at
> Gateway House, Milverton Street, London, SE11 4AP. Sporting Index Ltd is
> authorised and regulated by the UK Financial Services Authority (reg. no.
> 150404) and Gambling Commission (reg. no. 000-027343-R-308898-001). Any
> financial promotion contained herein has been issued and approved by
> Sporting Index Ltd.
>
>
> Outbound email has been scanned for viruses and SPAM


--
Harsh J