accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Trost <jason.tr...@gmail.com>
Subject Re: Accumulo On MapR
Date Mon, 26 Mar 2012 12:02:25 GMT
Todd,

Did you gather any performance numbers?  If so, how did it compare to HDFS?
 And are there any plans to try it on a larger cluster (I'm assuming you
were working with one VM).

Thanks,

--Jason

On Mon, Mar 26, 2012 at 12:11 AM, Todd Stavish <toddstavish@gmail.com>wrote:

> Final update, I tested 1.4.0rc5 against MapR v1.2.3. Everything works
> well. The only necessary install steps are the port mapping ones. Very
> clean.
>
> -Todd
>
>
> On Tue, Mar 20, 2012 at 9:39 PM, Todd Stavish <toddstavish@gmail.com>
> wrote:
> > Here are the complete instructions for running Accumulo on MapR M5 VM ->
> >
> > http://t.co/RJJ8Ht4B
> >
> > Thanks for all of the help,
> > -Todd
> >
> >
> >
> >
> > On Tue, Mar 20, 2012 at 1:10 PM, Todd Stavish <toddstavish@gmail.com>
> wrote:
> >>> I looked at the code and its just simply cast to a
> >>> DistributedFileSystem inorder to check if HDFS is in safe mode.  It
> >>> should do an instanceof operation and then do the case.  Would you
> >>> mind opening a ticket about this?
> >>
> >> Thanks Keith. Done.
> >>
> >> https://issues.apache.org/jira/browse/ACCUMULO-476
> >>
> >>
> >>
> >>>
> >>>>
> >>>>
> >>>> On Fri, Mar 16, 2012 at 9:19 AM, Eric Newton <eric.newton@gmail.com>
> wrote:
> >>>>> The monitor and gc do run in their own processes, which is kinda
> hinted at
> >>>>> the JVM memory sizes in conf/accumulo-env.sh
> >>>>>
> >>>>> The monitor may not be working because your jvm/os security profile
> is
> >>>>> different from those we've seen to date.
> >>>>> Replace conf/monitor.security.policy with:
> >>>>>
> >>>>>  grant{
> >>>>>   permission java.security.AllPermission;
> >>>>> };
> >>>>>
> >>>>> Or just chase down all the security violations you are getting in
> your
> >>>>> jvm/OS.  A quick look at the monitor logs and they seem ok.
> >>>>>
> >>>>> Your tserver isn't starting because some other process is listening
> on the
> >>>>> port.  The monitor can't talk to the master because it's waiting
for
> some
> >>>>> tablet servers to show up.  The monitor may not display until it
> talks to
> >>>>> the master.  Good news: the logger started up fine.
> >>>>>
> >>>>> Do this:
> >>>>>
> >>>>> echo localhost > conf/gc
> >>>>>
> >>>>> That will get the gc to start up.
> >>>>>
> >>>>> Then this:
> >>>>>
> >>>>> # kill everything accumulo related
> >>>>> pkill -9 -f accumulo.start
> >>>>>
> >>>>> # start everything again
> >>>>> ./bin/start-all.sh
> >>>>>
> >>>>> Most of these issues, as well as some example configs for specific
> memory
> >>>>> sizes are fixed in the accumule-1.4.0 release candidate.
> >>>>>
> >>>>> -Eric
> >>>>>
> >>>>>
> >>>>> On Fri, Mar 16, 2012 at 12:00 PM, Todd Stavish <
> toddstavish@gmail.com>
> >>>>> wrote:
> >>>>>>
> >>>>>> accumulo init seemed to work. Full disclosure, I am running
accumlo
> in
> >>>>>> the MapR M5 vm. I cranked down the memory allocations in
> >>>>>> accumulo-env.sh to compensate (maybe too much?). I'm not seeing
all
> of
> >>>>>> the process launch. Does the monitor and GC run in their own
> >>>>>> processes? Neither the monitoring web page nor accumulo shell
are
> >>>>>> working. Accumulo shell output below, in addition, ... running
> >>>>>> processes, logs, and confs attached.
> >>>>>>
> >>>>>> Running the whole MapR/Accumulo stack all in a single VM is
flirting
> >>>>>> with oxymoron. My goal was just to go through the install sequence
> and
> >>>>>> run a few samples. If there is little hope in getting this to
work
> in
> >>>>>> a vm, please let me now.
> >>>>>>
> >>>>>>
> >>>>>> mapr@mapr-desktop:/opt/accumulo-1.3.5-incubating/bin$ ./accumulo
> shell
> >>>>>> Enter current password for 'mapr'@'myinstance': *******
> >>>>>> 16 08:41:15,971 [impl.ServerClient] WARN : Failed to find an
> available
> >>>>>> server in the list of servers: [mapr-desktop:9999:9999 (120000)]
> >>>>>> 16 08:41:15,974 [shell.Shell] ERROR:
> >>>>>> org.apache.accumulo.core.client.AccumuloException:
> >>>>>> org.apache.thrift.transport.TTransportException: Failed to connect
> to
> >>>>>> a server
> >>>>>>
> >>>>>>
> >>>>>> Here's what's running:
> >>>>>>
> >>>>>> mapr     31166  0.2  3.8 708684 78332 ?        Sl   08:08  
0:04
> >>>>>> /usr/lib/jvm/java-6-sun-1.6.0.26/bin/java -Dapp=master -classpath
> >>>>>>
> >>>>>>
> /opt/accumulo-1.3.5-incubating//conf:/opt/accumulo-1.3.5-incubating//src/start/target/classes:/opt/accumulo-1.3.5-incubating//lib/accumulo-start-1.3.5-incubating.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-core-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-fam-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/log4j-1.2.16.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-1.0.4.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-api-1.0.4.jar
> >>>>>> -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75
> -Xmx256m
> >>>>>> -Xms256m -XX:OnOutOfMemoryError=kill -9 %p
> >>>>>>
> >>>>>>
> -Djava.library.path=/opt/mapr/hadoop/hadoop-0.20.2//lib/native/Linux-amd64-64
> >>>>>> org.apache.accumulo.start.Main master --address mapr-desktop
> >>>>>>
> >>>>>> mapr     31674  0.2  3.8 703408 78112 ?        Sl   08:08  
0:03
> >>>>>> /usr/lib/jvm/java-6-sun-1.6.0.26/bin/java -Dapp=logger -classpath
> >>>>>>
> >>>>>>
> /opt/accumulo-1.3.5-incubating//conf:/opt/accumulo-1.3.5-incubating//src/start/target/classes:/opt/accumulo-1.3.5-incubating//lib/accumulo-start-1.3.5-incubating.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-core-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-fam-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/log4j-1.2.16.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-1.0.4.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-api-1.0.4.jar
> >>>>>> -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75
> -Xmx256m
> >>>>>> -Xms256m -XX:OnOutOfMemoryError=kill -9 %p
> >>>>>>
> >>>>>>
> -Djava.library.path=/opt/mapr/hadoop/hadoop-0.20.2//lib/native/Linux-amd64-64
> >>>>>> org.apache.accumulo.start.Main logger --address mapr-desktop
> >>>>>>
> >>>>>> mapr     31853  0.3  4.4 752476 91420 ?        Sl   08:08  
0:04
> >>>>>> /usr/lib/jvm/java-6-sun-1.6.0.26/bin/java -Dapp=tserver -classpath
> >>>>>>
> >>>>>>
> /opt/accumulo-1.3.5-incubating//conf:/opt/accumulo-1.3.5-incubating//src/start/target/classes:/opt/accumulo-1.3.5-incubating//lib/accumulo-start-1.3.5-incubating.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-core-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-fam-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/log4j-1.2.16.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-1.0.4.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-api-1.0.4.jar
> >>>>>> -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75
> -Xmx256m
> >>>>>> -Xms256m -Xss128k -XX:OnOutOfMemoryError=kill -9 %p
> >>>>>>
> >>>>>>
> -Djava.library.path=/opt/mapr/hadoop/hadoop-0.20.2//lib/native/Linux-amd64-64
> >>>>>> org.apache.accumulo.start.Main tserver --address mapr-desktop
> >>>>>>
> >>>>>> -Todd
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Thu, Mar 15, 2012 at 4:38 PM, Todd Stavish <
> toddstavish@gmail.com>
> >>>>>> wrote:
> >>>>>> > Domo, Takuya-san.  For posterity's sake ->
> >>>>>> >
> >>>>>> > In accumulo-site.xml:
> >>>>>> >
> >>>>>> > <property>
> >>>>>> >  <name>instance.zookeeper.host</name>
> >>>>>> >  <value>localhost:5181</value>
> >>>>>> >  <description>comma separated list of zookeeper
> servers</description>
> >>>>>> > </property>
> >>>>>> >
> >>>>>> >
> >>>>>> >
> >>>>>> >
> >>>>>> >
> >>>>>> > On Wed, Mar 14, 2012 at 9:33 PM, Takuya UESHIN <
> ueshin@happy-camper.st>
> >>>>>> > wrote:
> >>>>>> >> Hi,
> >>>>>> >>
> >>>>>> >> You need to specify ZooKeeper client port to 5181 when
you use
> MapR.
> >>>>>> >> MapR's ZooKeeper client port is 5181 by default.
> >>>>>> >>
> >>>>>> >> Thanks.
> >>>>>> >>
> >>>>>> >>
> >>>>>> >> 2012/3/15 Keith Turner <keith@deenlo.com>
> >>>>>> >>>
> >>>>>> >>> Is zookeeper up and running?
> >>>>>> >>>
> >>>>>> >>> If so can you jstack the init java process and
send the stack
> trace of
> >>>>>> >>> the main thread?
> >>>>>> >>>
> >>>>>> >>> Keith
> >>>>>> >>>
> >>>>>> >>> On Wed, Mar 14, 2012 at 5:31 PM, Todd Stavish <
> toddstavish@gmail.com>
> >>>>>> >>> wrote:
> >>>>>> >>> > Hello,
> >>>>>> >>> >
> >>>>>> >>> > I followed all of the setup / initialization
steps. However,
> when I
> >>>>>> >>> > ran accumulo init, it hangs on naming the
instance.
> >>>>>> >>> >
> >>>>>> >>> > mapr@mapr-desktop:/opt/accumulo-1.3.5-incubating/bin$
> ./accumulo
> >>>>>> >>> > init
> >>>>>> >>> > 14 14:15:24,536 [util.Initialize] INFO : Hadoop
Filesystem is
> >>>>>> >>> > maprfs:///
> >>>>>> >>> > 14 14:15:24,584 [util.Initialize] INFO : Accumulo
data dir is
> >>>>>> >>> > /accumulo
> >>>>>> >>> > 14 14:15:24,585 [util.Initialize] INFO : Zookeeper
server is
> >>>>>> >>> > localhost:2181
> >>>>>> >>> > Instance name : test
> >>>>>> >>> >
> >>>>>> >>> > Init doesn't seem to produce any logs (at
least not in the
> >>>>>> >>> > accumolo/logs dir), is there a verbose mode
or something that
> can
> >>>>>> >>> > help
> >>>>>> >>> > me figure out where it is failing?
> >>>>>> >>> >
> >>>>>> >>> > Thanks,
> >>>>>> >>> > -Todd
> >>>>>> >>
> >>>>>> >>
> >>>>>> >>
> >>>>>> >>
> >>>>>> >> --
> >>>>>> >> Takuya UESHIN
> >>>>>> >> Tokyo, Japan
> >>>>>> >>
> >>>>>> >> http://twitter.com/ueshin
> >>>>>
> >>>>>
>

Mime
View raw message