Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@accumulo.apache.org
MIME-Version: 1.0
Reply-To: vines@apache.org
In-Reply-To: 
 <CA+zmODjzJf+QYaQbcuUCyWdeM42MuS4T_fxiTt7QxgMFMprKCw@mail.gmail.com>
References: 
 <CA+zmODjzJf+QYaQbcuUCyWdeM42MuS4T_fxiTt7QxgMFMprKCw@mail.gmail.com>
From: John Vines <vines@apache.org>
Date: Thu, 9 May 2013 13:28:27 -0400
Message-ID: 
 <CADczPYTutOs7i-iF-B3R1U-4Hu0HbirF4Cq0p=F+Mu1RW10RNA@mail.gmail.com>
Subject: Re: Accumulo and Bigtop Hadoop
To: "user@accumulo.apache.org" <user@accumulo.apache.org>
Content-Type: multipart/alternative; boundary=047d7b3441fe3ccab704dc4c6376

--047d7b3441fe3ccab704dc4c6376
Content-Type: text/plain; charset=ISO-8859-1

Accumulo 1.4 does not work against hadoop 2 without changing the poms and
recompiling. Accumulo 1.5, not yet released, will support both with
recompilation. One option is to use Bigtop-0.3.x, which uses hadoop-1.0.1
if I remember correctly.


On Thu, May 9, 2013 at 1:22 PM, Roshan Punnoose <roshanp@gmail.com> wrote:

> I have a feeling this has already been asked, but I could not find the
> answer on google.
>
> Bigtop is installing hadoop 2.0.2-alpha into /usr/lib/hadoop,
> /usr/lib/hadoop-hdfs, /usr/lib/hadoop-yarn, etc. However, Accumulo is setup
> to have the "general.classpaths" of accumulo-site to point
> /usr/lib/hadoop/(share)/(hdfs|yarn)/*.jar.
>
> As a quick workaround I had to edit the accumulo-env.sh and
> accumulo-site.sh:
>
> accumulo-env.sh
> ###for bigtop
> export HADOOP_COMMON_DIR="./"
> export HADOOP_COMMON_LIB_JARS_DIR="lib"
> export HADOOP_COMMON_LIB_NATIVE_DIR="lib/native"
> export HDFS_DIR="./"
> export HDFS_LIB_JARS_DIR="lib"
> export YARN_DIR="./"
> export YARN_LIB_JARS_DIR="lib"
> export MAPRED_DIR="./"
> export MAPRED_LIB_JARS_DIR="lib"
>
> export HADOOP_LIBEXEC_DIR=${HADOOP_LIBEXEC_DIR:-"/usr/lib/hadoop/libexec"}
> export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/etc/hadoop/conf"}
> export HADOOP_COMMON_HOME=${HADOOP_COMMON_HOME:-"/usr/lib/hadoop"}
> export HADOOP_HDFS_HOME=${HADOOP_HDFS_HOME:-"/usr/lib/hadoop-hdfs"}
> export
> HADOOP_MAPRED_HOME=${HADOOP_MAPRED_HOME:-"/usr/lib/hadoop-mapreduce"}
> export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-"/usr/lib/hadoop-yarn"}
> ###for bigtop
>
> accumulo-site.sh (added to the general.classpaths)
>       $HADOOP_MAPRED_HOME/.*.jar,
>       $HADOOP_MAPRED_HOME/$MAPRED_LIB_JARS_DIR/.*.jar,
>       $HADOOP_YARN_HOME/.*.jar,
>       $HADOOP_YARN_HOME/$YARN_LIB_JARS_DIR/.*.jar,
>       $HADOOP_HDFS_HOME/[^.].*.jar,
>       $HADOOP_HDFS_HOME/$HDFS_LIB_JARS_DIR/.*.jar,
>
> This above is working. Has anyone ran into this and found a better
> solution?
>
> Thanks!
>
> Roshan
>

--047d7b3441fe3ccab704dc4c6376
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Accumulo 1.4 does not work against hadoop 2 without changi=
ng the poms and recompiling. Accumulo 1.5, not yet released, will support b=
oth with recompilation. One option is to use Bigtop-0.3.x, which uses hadoo=
p-1.0.1 if I remember correctly.<br>

</div><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On Thu,=
 May 9, 2013 at 1:22 PM, Roshan Punnoose <span dir=3D"ltr">&lt;<a href=3D"m=
ailto:roshanp@gmail.com" target=3D"_blank">roshanp@gmail.com</a>&gt;</span>=
 wrote:<br>

<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr">I have a feeling this has a=
lready been asked, but I could not find the answer on google.<div><br></div=
>
<div>
Bigtop is installing hadoop 2.0.2-alpha into /usr/lib/hadoop, /usr/lib/hado=
op-hdfs, /usr/lib/hadoop-yarn, etc. However, Accumulo is setup to have the =
&quot;general.classpaths&quot; of accumulo-site to point /usr/lib/hadoop/(s=
hare)/(hdfs|yarn)/*.jar.=A0</div>


<div><br></div><div>As a quick workaround I had to edit the accumulo-env.sh=
 and accumulo-site.sh:</div><div><br></div><div>accumulo-env.sh</div><div><=
div>###for bigtop</div><div>export HADOOP_COMMON_DIR=3D&quot;./&quot;</div>


<div>export HADOOP_COMMON_LIB_JARS_DIR=3D&quot;lib&quot;</div><div>export H=
ADOOP_COMMON_LIB_NATIVE_DIR=3D&quot;lib/native&quot;</div><div>export HDFS_=
DIR=3D&quot;./&quot;</div><div>export HDFS_LIB_JARS_DIR=3D&quot;lib&quot;</=
div>


<div>export YARN_DIR=3D&quot;./&quot;</div><div>export YARN_LIB_JARS_DIR=3D=
&quot;lib&quot;</div><div>export MAPRED_DIR=3D&quot;./&quot;</div><div>expo=
rt MAPRED_LIB_JARS_DIR=3D&quot;lib&quot;</div><div><br></div><div>export HA=
DOOP_LIBEXEC_DIR=3D${HADOOP_LIBEXEC_DIR:-&quot;/usr/lib/hadoop/libexec&quot=
;}</div>


<div>export HADOOP_CONF_DIR=3D${HADOOP_CONF_DIR:-&quot;/etc/hadoop/conf&quo=
t;}</div><div>export HADOOP_COMMON_HOME=3D${HADOOP_COMMON_HOME:-&quot;/usr/=
lib/hadoop&quot;}</div><div>export HADOOP_HDFS_HOME=3D${HADOOP_HDFS_HOME:-&=
quot;/usr/lib/hadoop-hdfs&quot;}</div>


<div>export HADOOP_MAPRED_HOME=3D${HADOOP_MAPRED_HOME:-&quot;/usr/lib/hadoo=
p-mapreduce&quot;}</div><div>export HADOOP_YARN_HOME=3D${HADOOP_YARN_HOME:-=
&quot;/usr/lib/hadoop-yarn&quot;}</div><div>###for bigtop</div><div><br></d=
iv>


<div>accumulo-site.sh (added to the general.classpaths)</div><div><div>=A0 =
=A0 =A0 $HADOOP_MAPRED_HOME/.*.jar,</div><div>=A0 =A0 =A0 $HADOOP_MAPRED_HO=
ME/$MAPRED_LIB_JARS_DIR/.*.jar,</div><div>=A0 =A0 =A0 $HADOOP_YARN_HOME/.*.=
jar,</div>
<div>=A0 =A0 =A0 $HADOOP_YARN_HOME/$YARN_LIB_JARS_DIR/.*.jar,</div><div>=A0=
 =A0 =A0 $HADOOP_HDFS_HOME/[^.].*.jar,</div><div>=A0 =A0 =A0 $HADOOP_HDFS_H=
OME/$HDFS_LIB_JARS_DIR/.*.jar,</div><div><br></div><div>This above is worki=
ng. Has anyone ran into this and found a better solution?</div>


<div><br></div><div>Thanks!</div><span class=3D"HOEnZb"><font color=3D"#888=
888"><div><br></div><div>Roshan</div></font></span></div></div></div>
</blockquote></div><br></div>

--047d7b3441fe3ccab704dc4c6376--