hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Wang <andrew.w...@cloudera.com>
Subject Re: Running HDFS from source broken since HDFS-11596
Date Tue, 25 Jul 2017 23:51:33 GMT
I looked into this more and filed HDFS-12197 with a possible solution.
Thanks for the report, Lars!

On Fri, Jul 21, 2017 at 12:51 AM, Lars Francke <lars.francke@gmail.com>
wrote:

> Thanks John, that was helpful. I see that you're using the hadoop-dist
> directory while the wiki points directly to the project folders (e.g.
> hadoop-hdfs-project etc.).
>
> The former works the latter doesn't. So I guess it's a matter of updating
> the wiki.
>
>
> On Thu, Jul 20, 2017 at 9:09 AM, John Zhuge <john.zhuge@gmail.com> wrote:
>
> > Hi Lars,
> >
> > I am able to run pseudo-distributed mode from a dev tree. Here is the
> > wiki: https://hadoop.apache.org/docs/current/hadoop-
> > project-dist/hadoop-common/SingleCluster.html#Pseudo-
> Distributed_Operation
> > .
> >
> > Check out my script pseudo_dist
> > <https://github.com/jzhuge/hadoop-sanity-tests/blob/
> master/bin/pseudo_dist> to
> > start/stop a pseudo-distributed cluster.
> >
> > Here are the steps:
> >
> >    1. mvn install -DskipTests -DskipShade -Dmaven.javadoc.skip -Pdist
> >    -Dtar
> >    2. pseudo_dist start ~/hadoop-sanity-tests/config/insecure/
> >    3. test_env hdfs dfs -ls /tmp
> >
> > Thanks,
> >
> > On Wed, Jul 19, 2017 at 11:49 PM, Lars Francke <lars.francke@gmail.com>
> > wrote:
> >
> >> I've already asked in <https://issues.apache.org/jira/browse/HDFS-11596
> >
> >> but haven't gotten a reply so far so I thought I'd bump it here.
> >>
> >> The issue replaces the compile time dependency of the various HDFS
> >> projects
> >> to hdfs-client with a "provided" dependency.
> >>
> >> Unfortunately that means that HDFS cannot be run anymore from source as
> is
> >> documented in the Wiki (<
> >> https://wiki.apache.org/hadoop/HowToSetupYourDevelopmentEnvironment>)
> and
> >> as used to be possible before the patch. This is because the hdfs client
> >> classes (e.g. ClientProtocol is the first one that HDFS complains about
> >> during startup) are not in the classpath anymore.
> >>
> >> I wonder how all of you are running Hadoop these days from source? I've
> >> always followed the Wiki instructions but maybe they are out of date and
> >> there's a better way?
> >>
> >> Thanks,
> >> Lars
> >>
> >
> >
> >
> > --
> > John
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message