Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of harsh@cloudera.com designates
 209.85.219.51 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAJv_xg8EjHDXHFuGDxiXPeTUf4dT3Kn4rz5p2zfeCxHs36k83g@mail.gmail.com>
References: 
 <CAJv_xg8pFgWCa01BDrM=YYZYpALEitd6KQfQ0PZ0-z-2wXVkkQ@mail.gmail.com>
 <CAOcnVr3L+dooYHUbYuBtFtkx9PzgEG8VDrG-7xHMb-UREFepTw@mail.gmail.com>
 <CAOcnVr3+OWRC-FZWepqO5s2EWcDh47R_PD+jdj5y4UREkyEHyw@mail.gmail.com>
 <CAJv_xg8EjHDXHFuGDxiXPeTUf4dT3Kn4rz5p2zfeCxHs36k83g@mail.gmail.com>
From: Harsh J <harsh@cloudera.com>
Date: Sat, 7 Sep 2013 20:47:22 +0530
Message-ID: 
 <CAOcnVr1onNMQ9C3_KhUtB925WV1tg_OtHtQbZXOZXBVY32_RSw@mail.gmail.com>
Subject: Re: How to support the (HDFS) FileSystem API of various Hadoop
 Distributions?
To: "<user@hadoop.apache.org>" <user@hadoop.apache.org>
Content-Type: text/plain; charset=ISO-8859-1

Hi,

I think the simpler way is to distribute multiple pre-built packages,
targeting each version/distribution, instead of trying to detect which
one to load?

On Sat, Sep 7, 2013 at 4:14 PM, Christian Schneider
<cschneiderpublic@gmail.com> wrote:
> Hi Harsh,
> Thanks for the suggestion. And yes, the .isRoot() lead to this question :).
> Also I wasn't sure about the ideal way of packaging this utility.
>
> My "dream" is to have 1 binary that is able to deal with different versions
> (IPC 3, 5, 7, ...).
> The users will download the binary package and it is compatible with a wide
> range of versions.
>
> From your suggestions I tried to run 0.20.2 against CDH4.3, - but it is not
> compatible with IPC V7.
>
> So I think I need to do the same then HBase Compat does. "Somehow" the tool
> needs to check
> the version of the cluster and then load the correct implementation for
> that.
>
> But how to check the IPC version?
>
> Best Regards,
> Christian.
>
>
> P.S.: Thanks, that motivates me to continue :)
>
>
> 2013/9/6 Harsh J <harsh@cloudera.com>
>>
>> Oh and btw, nice utility! :)
>>
>> On Fri, Sep 6, 2013 at 7:50 AM, Harsh J <harsh@cloudera.com> wrote:
>> > Hello,
>> >
>> > There are a few additions to the FileSystem that may bite you across
>> > versions, but if you pick an old stable version such as Apache Hadoop
>> > 0.20.2, and stick to only its offered APIs, it would work better
>> > across different version dependencies as we try to maintain FileSystem
>> > as a stable interface as much as we can (there was also more recent
>> > work to ensure the stabilization). I looked over your current code
>> > state and it seemed to have pretty stable calls that I think have
>> > existed across several versions and exists today, but I did notice you
>> > had to remove an isRoot as part of a previous commit, which may have
>> > lead to this question?
>> >
>> > If that doesn't work for you, you can also switch out to using
>> > sub-modules carrying code specific to a build version type (such as
>> > what HBase does at https://github.com/apache/hbase/tree/trunk/ (see
>> > the hbase-hadoop-compat directories)).
>> >
>> > On Fri, Sep 6, 2013 at 2:59 AM, Christian Schneider
>> > <cschneiderpublic@gmail.com> wrote:
>> >> Hi,
>> >> I start to write a small ncdu clone to browse HDFS on the CLI
>> >> (http://nchadoop.org/). Currently i'm testing it against CDH4, - but I
>> >> like
>> >> to make it available for a wider group of users (Hortonworks, ..).
>> >>
>> >> Is it enough to pick different vanilla Versions (for IPC 5, 7)?
>> >>
>> >> Best Regards,
>> >> Christian.
>> >>
>> >
>> >
>> >
>> > --
>> > Harsh J
>>
>>
>>
>> --
>> Harsh J
>
>


-- 
Harsh J