hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Collins <...@cloudera.com>
Subject Re: [DISCUSS] Remove append?
Date Thu, 22 Mar 2012 22:22:10 GMT
On Thu, Mar 22, 2012 at 3:11 PM, Konstantin Boudnik <cos@apache.org> wrote:
> On Thu, Mar 22, 2012 at 10:25AM, Eli Collins wrote:
>> On Thu, Mar 22, 2012 at 1:26 AM, Konstantin Shvachko
>> <shv.hadoop@gmail.com> wrote:
>> > Eli,
>> >
>> > I went over the entire discussion on the topic, and did not get it. Is
>> > there a problem with append? We know it does not work in hadoop-1,
>> > only flush() does. Is there anything wrong with the new append
>> > (HDFS-265)? If so please file a bug.
>> > I tested it in Hadoop-0.22 branch it works fine.
>> >
>> > I agree with people who were involved with the implementation of the
>> > new append that the complexity is mainly in
>> > 1. pipeline recovery
>> > 2. consistent client reading while writing, and
>> > 3. hflush()
>> > Once it is done the append itself, which is reopening of previously
>> > closed files for adding data, is not complex.
>> >
>> > You mentioned it and I agree you indeed should be more involved with
>> > your customer base. As for eBay, append was of the motivations to work
>> > on stabilizing 0.22 branch. And there is a lot of use cases which
>> > require append for our customers.
>> > Some of them were mentioned in this discussion.
>> >
>>
>> From what I've seen 0.22 isn't ready for production use. Aside from
>> not supporting critical features like security, it doesn't have a
>> size-able user-base behind it testing and fixing bugs, etc. All things
>> I'd imagine an org like eBay would want.  I've never gotten a request
>> to support 0.22 from a customer.
>
> This statement looks like FUD to me, because eBay (and a coupla other shops,
> as has been stated elsewhere) are using 0.22 in the production and are
> seemingly happy with that.
>

That's my experience, take it for what it's worth.

Not having important features like security, having very few commits,
etc is not FUD, you can check that via svn.

Thanks,
Eli

Mime
View raw message