hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kumar Vavilapalli <vino...@apache.org>
Subject Re: [DISCUSS] Looking to a 2.8.0 release
Date Wed, 25 Nov 2015 19:11:38 GMT
Haohui,

It’ll help to document this whole line of discussion about hdfs jar change and its impact/non-impact
for existing users so there is less confusion.

Thanks
+Vinod


> On Nov 11, 2015, at 3:26 PM, Haohui Mai <ricetons@gmail.com> wrote:
> 
> bq. If and only if they take the Hadoop class path at face value.
> Many applications don’t because of conflicting dependencies and
> instead import specific jars.
> 
> We do make the assumptions that applications need to pick up all the
> dependency (either automatically or manually). The situation is
> similar with adding a new dependency into hdfs in a minor release.
> 
> Maven / gradle obviously help, but I'd love to hear more about it how
> you get it to work. In trunk hadoop-env.sh adds 118 jars into the
> class path. Are you manually importing 118 jars for every single
> applications?
> 
> 
> 
> On Wed, Nov 11, 2015 at 3:09 PM, Haohui Mai <ricetons@gmail.com> wrote:
>> bq. currently pulling in hadoop-client gives downstream apps
>> hadoop-hdfs-client, but not hadoop-hdfs server side, right?
>> 
>> Right now hadoop-client pulls in hadoop-hdfs directly to ensure a
>> smooth transition. Maybe we can revisit the decision in the 2.9 / 3.x?
>> 
>> On Wed, Nov 11, 2015 at 3:00 PM, Steve Loughran <stevel@hortonworks.com> wrote:
>>> 
>>>> On 11 Nov 2015, at 22:15, Haohui Mai <ricetons@gmail.com> wrote:
>>>> 
>>>> bq.  it basically makes the assumption that everyone recompiles for
>>>> every minor release.
>>>> 
>>>> I don't think that the statement holds. HDFS-6200 keeps classes in the
>>>> same package. hdfs-client becomes a transitive dependency of the
>>>> original hdfs jar.
>>>> 
>>>> Applications continue to work without recompilation as the classes
>>>> will be in the same name and will be available in the classpath. They
>>>> have the option of switching to depending only on hdfs-client to
>>>> minimize the dependency when they are comfortable.
>>>> 
>>>> I'm not claiming that there are no bugs in HDFS-6200, but just like
>>>> other features we discover bugs and fix them continuously.
>>>> 
>>>> ~Haohui
>>>> 
>>> 
>>> currently pulling in hadoop-client gives downstream apps hadoop-hdfs-client,
but not hadoop-hdfs server side, right?
> 


Mime
View raw message