hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9272) Implement a unix-like cat utility
Date Tue, 20 Oct 2015 21:22:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14965797#comment-14965797
] 

Allen Wittenauer commented on HDFS-9272:
----------------------------------------

bq.  Is the issue that if/when the RPC protocol changes older clients wouldn't be aware?

Yes, this is exactly the problem.  You want higher levels of client compatibility than what
Hadoop RPC provides since those are typically outside of the control of the teams that run
the servers.

bq. I'm going to put up a patch for now just because we don't have anything capable of testing
libhdfs++ against a real cluster at the moment; Haohui's working on fixing that. A couple
people are ramping up on this project so in the short term I think being able to say cat ran
fine under valgrind or asan after changes offers some protection against regressions.

I think that's reasonable.  I just wouldn't want it exposed to users given the fragility of
RPC.  (We already have these problems with 'hadoop fs' in its HDFS form.)

> Implement a unix-like cat utility
> ---------------------------------
>
>                 Key: HDFS-9272
>                 URL: https://issues.apache.org/jira/browse/HDFS-9272
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs-client
>            Reporter: James Clampffer
>            Assignee: James Clampffer
>            Priority: Minor
>
> Implement the basic functionality of "cat" and have it build as a separate executable.
> 2 Reasons for this:
> We don't have any real integration tests at the moment so something simple to verify
that the library actually works against a real cluster is useful.
> Eventually I'll make more utilities like stat, mkdir etc.  Once there are enough of them
it will be simple to make a C++ implementation of the hadoop fs command line interface that
doesn't take the latency hit of spinning up a JVM.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message