hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@hortonworks.com>
Subject Re: Putting the hdfs client as a separate jar
Date Thu, 03 Apr 2014 11:59:47 GMT
It's not an issue with hdfs/hadoop JARs itself, but the POMs -and the same
problem exists with the hadoop core JAR - too much stuff you don't need
client side.

We can address this -without changing the packaging into an hdfs-client.jar
(and so complicating everything related to HDFS code).

All we need to do is create an hdfs-client POM which you add as a
dependency when you want the client and nothing server side.





On 2 April 2014 23:27, Haohui Mai <hmai@hortonworks.com> wrote:

> Hi,
>
> Many downstream projects needs the ability to access hdfs. In order to do
> this, currently downstream projects are forced to bring in the whole hdfs
> jar and its dependency, since both the hdfs server and the hdfs client
> reside in the same jar.
>
> To integrate with hdfs, the downstream projects are forced to manage the
> excess dependency from the hdfs server side (e.g., jersey, servlet, netty,
> and jsp-runtime, just to name a few). In my own experience, I ended up
> spending quite a bit of time on tweaking the POMs to work around collisions
> of dependent jars.
>
> To solve this problem, I propose to reorganize the code to put hdfs client
> into a separate jar. That way the client jar no longer depends on the jars
> that are required by the server side, therefore it is easier for the
> downstream projects to integrate with hdfs.
>
> Your feedbacks are appreciated.
>
> Regards,
> Haohui
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message