hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 张鹏 <peng.zh...@xiaomi.com>
Subject Re: Compatibility Yarn 2.4 with HDFS 2.0
Date Fri, 06 Jun 2014 09:47:15 GMT
Also I tested using hdfs client 2.0 access hdfs server 2.4, this also got
no compatibility:
"Incorrect header or version mismatch from 10.2.201.245:59310 got version
7 expected version 9"

Does this mean if we update our production cluster from 2.0 to 2.4, all
clients must be re-build?

Any suggestions on updating like this?

--
Thanks,
Peng


在 6/6/14 5:06 PM, "张鹏" <peng.zhang@xiaomi.com> 写入:

>You mean YARN will call some interfaces of FileSystem that not existed in
>2.0?
>Because I think YARN and MR depend on FileSystem interface of
>hadoop-common.
>
>I want to hack as below:
>
>                     -------> HDFS 2.0 --------> Hadoop-common 2.0
>                   /
>        YARN and MR
>                   \
>                     ------->Hadoop-common 2.4
>
>So Yarn and MR can use new interfaces added in hadoop-common 2.4, while
>hdfs client uses old IPC implementation.
>
>--
>Thanks,
>Peng
>
>在 6/6/14 4:45 PM, "Yu Azuryy" <azuryy.yfd@gmail.com> 写入:
>
>>Even if you done this, it's unstable, because Yarn2.4 added some IPC
>>interfaces. then HDFS MUST has related interfaces.
>>
>>
>>On Fri, Jun 6, 2014 at 4:22 PM, 张鹏 <peng.zhang@xiaomi.com> wrote:
>>
>>> Hi all,
>>>
>>> I want to upgrade to Yarn 2.4 only. But when it access Hdfs, IPC Server
>>> version is mismached.
>>>
>>> HDFS (IPC v7) cant respond to YARN (IPC v9) with below error log:
>>> WARN org.apache.hadoop.ipc.Server: Incorrect header or version mismatch
>>> from 127.0.0.1:47957 got version 9 expected version 7
>>>
>>> I want to know whether it is possible to do this?
>>>
>>> Maybe I can change pom to depend on hdfs 2.0, and use maven shade to
>>>make
>>> hadoop common(2.4 and 2.0) work together
>>>
>>> Any suggestions?
>>>
>>> --
>>> Thanks,
>>> Peng
>>>
>>>
>

Mime
View raw message