hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arpit Agarwal <aagar...@hortonworks.com>
Subject Re: Is hdfs "Append to file" command ready for production?
Date Tue, 29 Jul 2014 22:06:58 GMT
By tested I meant tested for performance. It is fine functionally.

On Tue, Jul 29, 2014 at 3:04 PM, Arpit Agarwal <aagarwal@hortonworks.com>

> Most of those Jiras are for the append feature in general and not the
> appendToFile CLI.
> The append feature used via the FileSystem API is stable in Apache Hadoop
> 2.2 and later. I added the appendToFile CLI as a convenience and it has not
> been tested/tuned for performance so YMMV.
> On Tue, Jul 29, 2014 at 9:29 AM, Manikanda Prabhu <gmkprabhu1983@gmail.com
> > wrote:
>> Hi,
>> We are planning to use one of the hdfs commands "appendToFile" in our
>> file process, would someone confirm it's production ready or any open
>> issues thats still in discussion.
>> In my research, I found the following JIRA's directly or related to this
>> command and its all closed (except HDFS 1060). please let me know if i
>> missed anything
>>   JIRA Id Description Status  HDFS-1060 Append/flush should support
>> concurrent "tailer" use case Open  HADOOP-6239
>> HDFS-4905 Command-line for append Fixed  HDFS-744 Support hsync in HDFS
>> Fixed  HDFS-222 Support for concatenating of files into a single file
>> Fixed  HDFS-265 Revisit append- This jira revisits append, aiming for a
>> design and implementation supporting a semantics that are acceptable to its
>> users. Fixed  HADOOP-5224 Disable append Fixed  HADOOP-5332 Make support
>> for file append API configurable Fixed  HDFS-200 In HDFS, sync() not yet
>> guarantees data available to the new readers Fixed   HADOOP-1708 Make
>> files visible in the namespace as soon as they are created Fixed
>> HADOOP-1700 Append to files in HDFS Fixed
>> Regards,
>> Mani

NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

View raw message