Return-Path: X-Original-To: apmail-accumulo-user-archive@www.apache.org Delivered-To: apmail-accumulo-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 34BE410F9B for ; Thu, 16 Jan 2014 20:01:09 +0000 (UTC) Received: (qmail 25632 invoked by uid 500); 16 Jan 2014 20:01:08 -0000 Delivered-To: apmail-accumulo-user-archive@accumulo.apache.org Received: (qmail 25585 invoked by uid 500); 16 Jan 2014 20:01:08 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 25577 invoked by uid 99); 16 Jan 2014 20:01:07 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 16 Jan 2014 20:01:07 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of busbey@cloudera.com designates 209.85.216.179 as permitted sender) Received: from [209.85.216.179] (HELO mail-qc0-f179.google.com) (209.85.216.179) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 16 Jan 2014 20:01:02 +0000 Received: by mail-qc0-f179.google.com with SMTP id e16so2728920qcx.38 for ; Thu, 16 Jan 2014 12:00:41 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=k1Wnd5TcAGy5muLkbffdvuyXfklRWfhEwMYO7ljeqBI=; b=A0DRUsXIdLj9+lFNnQ2z2cTJCaTy+K0X086CaxlH7RxQoU/k7W44bAHGjBgZJ1JRT2 Adt8TMFNjtLeD95fLjlhbbdFAH3V+ur/1qIpVxdLZfXRcszi7Sh0HaogXZaxHA1FTtU+ M8oQruPRg9eCZvh20QJQkiwArT/qbgG8WEx9U0Zy8smb3DhUHvz/Il0ilqzjLx7jixID Fs6CvvUlUnzqmWCi0Q7rKabLcyQJPfQjs0sHAy2ywPOi58EVcXGrR4sgX7g+uBfdm8o5 fEbZ/fZekcGLDQxzyJZHXQoEhxIRMZTpJJZ/wjMxvmPKqGH7QNGnNIBMpNF+oZsILtF8 hFfg== X-Gm-Message-State: ALoCoQmt8S9gQsWIniVsJbzqSnDgJS/KdXdKOop8wbHGu/9LKk5/bcZatq01AuP8iOPRn4hvMBgt X-Received: by 10.224.16.72 with SMTP id n8mr19508812qaa.76.1389902440915; Thu, 16 Jan 2014 12:00:40 -0800 (PST) MIME-Version: 1.0 Received: by 10.229.232.67 with HTTP; Thu, 16 Jan 2014 12:00:20 -0800 (PST) In-Reply-To: References: <393DCF49EDE8714CA6D5F4E436424EA87B417BC549@mailva.adaptivemethods.com> <393DCF49EDE8714CA6D5F4E436424EA87B417BC557@mailva.adaptivemethods.com> <393DCF49EDE8714CA6D5F4E436424EA87B417BC561@mailva.adaptivemethods.com> <393DCF49EDE8714CA6D5F4E436424EA87B417BC581@mailva.adaptivemethods.com> <393DCF49EDE8714CA6D5F4E436424EA87B417BC5BD@mailva.adaptivemethods.com> <393DCF49EDE8714CA6D5F4E436424EA87B417BC72D@mailva.adaptivemethods.com> <393DCF49EDE8714CA6D5F4E436424EA87B417BC73B@mailva.adaptivemethods.com> <04920BD67C651C469D0387704CD7692A8F046284AD@21ct-exg07.21technologies.com> From: Sean Busbey Date: Thu, 16 Jan 2014 14:00:20 -0600 Message-ID: Subject: Re: accumulo startup issue: Accumulo not initialized, there is no instance id at /accumulo/instance_id To: Accumulo User List Content-Type: multipart/alternative; boundary=047d7bea363643730e04f01be1f6 X-Virus-Checked: Checked by ClamAV on apache.org --047d7bea363643730e04f01be1f6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Thanks for the overview Charles! Any chance I can convince you to polish this up a little and add it to our user documentation as a quickstart guide= ? (Notes inline below) On Thu, Jan 16, 2014 at 1:50 PM, Ott, Charles H. wrote: > Disclaimer, > > > > Not advocating this is the best approach, just what I=E2=80=99m currently= doing, > put this together pretty quick, but it should be mostly complete for > settting up accumulo on cdh hdfs/zk > > > > > > > > echo "Increasing *Swapiness* Factor to limit use of swap space." > > echo "# *swappiness* for *accumulo*" >> /etc/sysctl.conf > > echo "vm.swappiness =3D 10" >> /etc/sysctl.conf > > > You should set vm.swappiness to 0. Both Hadoop and Accumulo recommend this. > > > > > #accumulo-site.xml > > > > general.classpaths > > > > $ACCUMULO_HOME/server/target/classes/, > > $ACCUMULO_HOME/lib/accumulo-server.jar, > > $ACCUMULO_HOME/core/target/classes/, > > $ACCUMULO_HOME/lib/accumulo-core.jar, > > $ACCUMULO_HOME/start/target/classes/, > > $ACCUMULO_HOME/lib/accumulo-start.jar, > > $ACCUMULO_HOME/fate/target/classes/, > > $ACCUMULO_HOME/lib/accumulo-fate.jar, > > $ACCUMULO_HOME/proxy/target/classes/, > > $ACCUMULO_HOME/lib/accumulo-proxy.jar, > > $ACCUMULO_HOME/lib/[^.].*.jar, > > $ZOOKEEPER_HOME/zookeeper[^.].*.jar, > > $HADOOP_CONF_DIR, > > $HADOOP_PREFIX/[^.].*.jar, > > $HADOOP_PREFIX/lib/[^.].*.jar, > > $HADOOP_HDFS_HOME/.*.jar, > > $HADOOP_HDFS_HOME/lib/.*.jar, > > $HADOOP_MAPREDUCE_HOME/.*.jar, > > $HADOOP_MAPREDUCE_HOME/lib/.*.jar > > > > Classpaths that accumulo checks for updates and class > files. > > When using the Security Manager, please remove the > ".../target/classes/" values. > > > > > > > This will give you some warnings about multiple slf4j binaries on CDH4.4.0+ . You can fix it by making sure you only match jars with a version number. See an example[1]. > then of course, always run your Accumulo binaries/scripts using the HDFS > account. I=E2=80=99m sure I=E2=80=99m missing a few steps here and there= =E2=80=A6 > > > You should avoid running Accumulo as the hdfs super user, both as a matter of proper separation of concerns and as a practical need if you are going to run Accumulo in a production environment with multiple HDFS users. -Sean [1]: https://gist.github.com/busbey/8285352#file-accumulo-site-cdh4-xml-L99 --047d7bea363643730e04f01be1f6 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Than= ks for the overview Charles! =C2=A0Any chance I can convince you to polish = this up a little and add it to our user documentation as a quickstart guide= ?

(Notes inline below)

On Thu, Jan 16, 2014 = at 1:50 PM, Ott, Charles H. <CHARLES.H.OTT@leidos.com> wrote:
Disclaimer,

=C2=A0

Not advocating this is the best approach, ju= st what I=E2=80=99m currently doing, put this together pretty quick, but it= should be mostly complete for settting up accumulo on cdh hdfs/zk

=C2=A0

=C2=A0<snip>

=C2=A0

echo "Increasing Swapiness Factor = to limit use of swap space."

echo "# swappiness= for accumulo" >> /etc/sysctl.conf=

echo "vm.swappiness =3D 10" >>= ; /etc/sysctl.conf

<= u>=C2=A0


You should set vm.swappiness t= o 0. Both Hadoop and Accumulo recommend this.

=C2= =A0

<= span style=3D"font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73= ,125)">

=C2=A0

<snip><= /p>

#accumulo-site.xml

<property>

=C2=A0=C2=A0=C2=A0 <name>general.classpat= hs</name>

=C2=A0=C2=A0=C2=A0 <value>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = $ACCUMULO_HOME/server/target/classes/,

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 $ACCUMULO_HOM= E/lib/accumulo-server.jar,

<= span style=3D"font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73= ,125)">=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 $ACCUMULO_HOME/core/target/classes/,<= u>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 $ACCUMULO_HOM= E/lib/accumulo-core.jar,

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 $ACCUMULO_HOME/start/target/classes/,

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 $ACCUMULO_HOM= E/lib/accumulo-start.jar,

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 $ACCUMULO_HOME/fate/target/classes/,

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 $ACCUMULO_HOM= E/lib/accumulo-fate.jar,

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 $ACCUMULO_HOME/proxy/target/classes/,

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 $ACCUMULO_HOM= E/lib/accumulo-proxy.jar,

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 $ACCUMULO_HOME/lib/[^.].*.jar,<= u>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 $ZOOKEEPER_HO= ME/zookeeper[^.].*.jar,

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 $HADOOP_CONF_DIR,

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 $HADOOP_PREFI= X/[^.].*.jar,

=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 $HADOOP_PREFIX/lib/[^.].*.jar,

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 $HADOOP_HDFS_= HOME/.*.jar,

=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 $HADOOP_HDFS_HOME/lib/.*.jar,=

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 $HADOOP_MAPRE= DUCE_HOME/.*.jar,

= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 $HADOOP_MAPREDUCE_HOME/lib/.*.jar<= /u>

=C2=A0=C2=A0=C2=A0 </value><= /u>

=C2=A0=C2=A0=C2=A0 <descrip= tion>Classpaths that accumulo checks for updates and class files.=

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 When using th= e Security Manager, please remove the ".../target/classes/" value= s.

=C2=A0=C2=A0=C2=A0 </description>

=C2=A0 </property>=

=C2=A0

<= div>
This will give you some warnings about multiple slf4j bi= naries on CDH4.4.0+ . You can fix it by making sure you only match jars wit= h a version number.

See an example[1].

=C2=A0

<= span style=3D"font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73= ,125)">

then of course, alw= ays run your Accumulo binaries/scripts using the HDFS account.=C2=A0 I=E2= =80=99m sure I=E2=80=99m missing a few steps here and there=E2=80=A6

=C2=A0

<= div>
You should avoid running Accumulo as the hdfs super user= , both as a matter of proper separation of concerns and as a practical need= if you are going to run Accumulo in a production environment with multiple= HDFS users.

-Sean

=
--047d7bea363643730e04f01be1f6--