hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@hortonworks.com>
Subject Re: HDFS Abstract Filesystem and hadoop-client
Date Tue, 16 Feb 2016 11:39:47 GMT

> On 16 Feb 2016, at 05:56, Haohui Mai <ricetons@gmail.com> wrote:
> 
> Thanks for catching them. I think it should be moved as well. Filed
> HDFS-9815 to track the effort.
> 
> ~Haohui
> 
> 

I worry about how much is going to break with the split

1/ we should consider making the full hadoop-hdfs JAR the client dependency, with a note saying
"you should be able to strip this out"
2/ or ... this split needs to be tagged as potentially incompatible. As those of us who have
been using 2.8+ downstream are hitting incompatibilities

> On Mon, Feb 15, 2016 at 4:01 AM, Steve Loughran <stevel@hortonworks.com> wrote:
>> 
>> Is it my imagination or does org.apache.hadoop.fs.Hdfs is in the hadoop-hdfs JAR,
and not hadoop-hdfs-client JAR?
>> 
>> Because someone has been complaining on the spark-dev list, that if they try to instantiate
>> 
>> FiileContext.getFileContext("hdfs://namenode:8020"), they get to see a stack trace
starting with
>> 
>> java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.Hdfs
not found"
>> 
>> 
>> 1. Is this really the case —the HDFS and webhdfs AFS impls are not in hadoop-client?
>> 2. If so, why not?
>> 3. Can it be corrected before 2.8.0 goes into beta release?
>> 4. Otherwise, the dependencies of hadoop-client are going to have to reinstate hadoop-hdfs
as a dependency
>> 
>> -Steve
> 

Mime
View raw message