accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: Can't get Accumulo 1.5 monitor interface to start in my browser
Date Wed, 11 Sep 2013 23:52:57 GMT
Pete,

I assume you did some leg-work before hitting up the mailing list...  
any recommendations on where you think might make sense on encapsulating 
some of this information?

FAQ format? Rolled into documentation (user/admin guide)? Other?

- josh

On 09/11/2013 04:17 PM, Pete Carlson wrote:
> All,
>
> I resolved the issue.  Per Eric's suggestion I changed my swappiness 
> setting down to 10, and also disabled all IPV6 stuff on my VM.  Once I 
> did this and changed the IP address of my monitor to 127.0.0.1 my 
> monitor started working.
>
> Thanks guys. :-)
>
> Regards,
>
> Pete
>
> On Wed, Sep 11, 2013 at 2:12 PM, John Vines <vines@apache.org 
> <mailto:vines@apache.org>> wrote:
>
>     Furthermore, tservers being down shouldn't prevent the monitor
>     page from coming up at all, so check for any other relevant
>     messages in of the monitor's logs.
>
>
>     On Wed, Sep 11, 2013 at 2:08 PM, Pete Carlson <pgcarlson@gmail.com
>     <mailto:pgcarlson@gmail.com>> wrote:
>
>         How do I determine if the rest of Accumulo is up and running
>         since I can't use the monitor?
>
>         Inside the monitor logs I see messages that it connected to
>         Hadoop and Zookeeper, but then later on I see this message
>         stating there are no tablet servers:
>
>         2013-09-11 11:39:29,199 [server.Accumulo] WARN :  System
>         swappiness setting is greater than ten (60) which can cause
>         time-sensitive operations to be delayed.  Accumulo is time
>         sensitive because it needs to maintain distributed lock agreement.
>         2013-09-11 11:39:32,506 [impl.ServerClient] WARN :
>         tracer:localhost.localdomain There are no tablet servers:
>         check that zookeeper and accumulo are running.
>
>         And when I log in to the accumulo shell I get the warning:
>         There are no tablet servers: check that zookeeper and accumulo
>         are running.
>
>         Could the system swappiness setting of 60 be causing my issue?
>
>         Regards,
>
>         Pete
>
>
>         On Wed, Sep 11, 2013 at 1:13 PM, John Vines <vines@apache.org
>         <mailto:vines@apache.org>> wrote:
>
>             Is the rest of Accumulo up and running? And is that the
>             only type of error in the Monitor logs?
>
>
>             On Wed, Sep 11, 2013 at 1:04 PM, Pete Carlson
>             <pgcarlson@gmail.com <mailto:pgcarlson@gmail.com>> wrote:
>
>                 Ok, I deleted /accumulo out of hdfs, initialized
>                 accumulo, and re-started Accumulo.  I still can't see
>                 the monitor IN my VM browser.  I still get the same
>                 Monitor info message in the logs.
>
>                 Pete
>
>                 On Wed, Sep 11, 2013 at 11:55 AM, John Vines
>                 <vines@apache.org <mailto:vines@apache.org>> wrote:
>
>                     You need to delete the /accumulo directory out of hdfs
>
>
>                     On Wed, Sep 11, 2013 at 11:54 AM, Pete Carlson
>                     <pgcarlson@gmail.com <mailto:pgcarlson@gmail.com>>
>                     wrote:
>
>                         I removed the monitor certificates as
>                         suggested, and I no longer get any messages in
>                         the monitor_localhost.err file.
>
>                         When I look in the
>                         monitor_localhost.localdomain.log file and saw
>                         the following INFO/Exception message:
>
>                         2013-09-11 11:42:50,012 [monitor.Monitor] INFO
>                         :  Failed to obtain problem reports
>                         java.lang.RuntimeException:
>                         org.apache.accumulo.core.client.impl.ThriftScanner$ScanTimedOutException
>                                 at
>                         org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java:174)
>                                 at
>                         org.apache.accumulo.server.problems.ProblemReports$3.hasNext(ProblemReports.java:241)
>                                 at
>                         org.apache.accumulo.server.problems.ProblemReports.summarize(ProblemReports.java:299)
>                                 at
>                         org.apache.accumulo.server.monitor.Monitor.fetchData(Monitor.java:392)
>                                 at
>                         org.apache.accumulo.server.monitor.Monitor$2.run(Monitor.java:504)
>                                 at
>                         org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
>                                 at java.lang.Thread.run(Thread.java:724)
>                         Caused by:
>                         org.apache.accumulo.core.client.impl.ThriftScanner$ScanTimedOutException
>                                 at
>                         org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:212)
>                                 at
>                         org.apache.accumulo.core.client.impl.ScannerIterator$Reader.run(ScannerIterator.java:82)
>                                 at
>                         org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java:164)
>                                 ... 6 more
>
>                         I was thinking this was related to the fact I
>                         didn't yet have Thrift 0.9.0 installed (due to
>                         dependency issues).  I now have Thrift 0.9.0
>                         installed.  I saved off my config files, blew
>                         away the accumulo directory and extracted it
>                         again from the tar.gz and copied over my conf
>                         files.
>
>                         However, when I run the
>                         accumulo-1.5.0/bin/accumulo init command again
>                         it gives me the following error:
>
>                         [util.Initialize] FATAL: It appears this
>                         location was previously initialized, exiting ...
>
>                         I can still start Accumulo like before with
>                         the bin/start-all.sh command but the Accumulo
>                         monitor still doesn't work IN my VM.
>
>                         Any other ideas to try?
>
>                         Pete
>
>                         On Wed, Sep 11, 2013 at 10:57 AM, John Vines
>                         <vines@apache.org <mailto:vines@apache.org>>
>                         wrote:
>
>                             Either try rerunning the monitor
>                             certificates or remove the portion from
>                             your accumulo-site that defines use of the
>                             certificate. I suggest the latter simply
>                             to remove variables at hand.
>
>
>                             On Wed, Sep 11, 2013 at 10:54 AM, Pete
>                             Carlson <pgcarlson@gmail.com
>                             <mailto:pgcarlson@gmail.com>> wrote:
>
>                                 John,
>
>                                 What is the monitor process called so
>                                 I can verify its running?  I did look
>                                 at the thread monitor logs, and under
>                                 monitor_localhost.err it gives a
>                                 FileNotFoundException because it can't
>                                 find the keystore.jks under
>                                 accumulo-1.5.0/conf/
>
>                                 I would have thought when I ran
>                                 ./generate_monitor_certificate.sh it
>                                 would have created this.
>
>                                 I was trying to run the monitor in a
>                                 browser IN the VM.  Thats how I found
>                                 this problem.
>
>                                 Regards,
>
>                                 Pete
>
>
>                                 On Wed, Sep 11, 2013 at 10:41 AM, John
>                                 Vines <vines@apache.org
>                                 <mailto:vines@apache.org>> wrote:
>
>                                     Again, please check to make sure
>                                     the monitor process is running and
>                                     it's not reporting any errors in
>                                     it's logs.
>
>                                     If you can verify that (and please
>                                     verify that first), try accessing
>                                     the monitor in a browser IN the VM.
>
>
>                                     On Wed, Sep 11, 2013 at 10:36 AM,
>                                     Pete Carlson <pgcarlson@gmail.com
>                                     <mailto:pgcarlson@gmail.com>> wrote:
>
>                                         All,
>
>                                         Just to clarify I have a
>                                         Centos VM running on my
>                                         Windows 7 box, and I have
>                                         Accumulo, Hadoop, and
>                                         Zookeeper running on it
>                                         standalone.  I was trying to
>                                         open the Accumulo monitor
>                                         within that VM and it failed.
>                                          However I can open the Hadoop
>                                         Namenode and Jobtracker web
>                                         interfaces in my browser just
>                                         fine.
>
>                                         Brendan, I was able to connect
>                                         to my VM using netcat on port
>                                         50095.
>
>                                         However, when I try to telnet
>                                         to that port on my Win7 with
>                                         telnet using my VM's ip
>                                         address and port i.e.,
>
>                                         telnet <IP address of vm> 50095
>
>                                         I get a message: ....Could not
>                                         open connection to the host,
>                                         on port 50095: Connect failed.
>
>                                         Pete
>
>
>                                         On Wed, Sep 11, 2013 at 9:15
>                                         AM, Brendan Heussler
>                                         <bheussler@gmail.com
>                                         <mailto:bheussler@gmail.com>>
>                                         wrote:
>
>                                             Can you try to connect to
>                                             your VM using netcat on
>                                             port 50095?
>
>                                             On VM:
>                                             nc -l 50095
>
>                                             On Client:
>                                             nc <ip-address of VM> 50095
>
>                                              Then type "test" or
>                                             something on your Client
>                                             machine.  The message
>                                             should be sent to your VM.
>
>                                             This will help to test the
>                                             connectivity of the two
>                                             machines over port 50095
>
>
>                                             Brendan
>
>
>                                             On Tue, Sep 10, 2013 at
>                                             5:22 PM, John Vines
>                                             <vines@apache.org
>                                             <mailto:vines@apache.org>>
>                                             wrote:
>
>                                                 Are you certain the
>                                                 monitor is running and
>                                                 was able to bind to
>                                                 the port? Please check
>                                                 the monitor logs for
>                                                 any errors.
>
>
>                                                 On Tue, Sep 10, 2013
>                                                 at 5:18 PM, Pete
>                                                 Carlson
>                                                 <pgcarlson@gmail.com
>                                                 <mailto:pgcarlson@gmail.com>>
>                                                 wrote:
>
>                                                     John,
>
>                                                     I don't see port
>                                                     50095 in use in
>                                                     the VM.  When I
>                                                     become root and
>                                                     type lsof -i -P |
>                                                     grep :50095 I
>                                                     don't get anything
>                                                     back.
>
>                                                     Any suggestions on
>                                                     how to get the
>                                                     port running?  I
>                                                     tried editing the
>                                                     iptables with the
>                                                     following command:
>
>                                                     iptables -I INPUT
>                                                     -p tcp --dport
>                                                     50095 -j ACCEPT
>
>                                                     and then I typed
>
>                                                     service iptables save
>
>                                                     to save my iptable
>                                                     changes, but when
>                                                     I restarted my VM
>                                                     I didn't see any
>                                                     difference.
>
>                                                     Regards,
>
>                                                     Pete
>
>                                                     Looking for a
>                                                     unique woodcraft?
>                                                      Check out my
>                                                     Facebook page:
>                                                     www.facebook.com/oqwcrafts
>                                                     <http://www.facebook.com/oqwcrafts>
>
>                                                     Also on Etsy:
>                                                     https://www.etsy.com/shop/OlneyQualityWoodCrft
>
>
>                                                     On Tue, Sep 10,
>                                                     2013 at 4:56 PM,
>                                                     John Vines
>                                                     <vines@apache.org
>                                                     <mailto:vines@apache.org>>
>                                                     wrote:
>
>                                                         You don't see
>                                                         50095 in use
>                                                         locally on the
>                                                         VM or in your
>                                                         box? You
>                                                         should only
>                                                         see it in use
>                                                         in the VM. If
>                                                         not, make sure
>                                                         it's running.
>                                                         If it is,
>                                                         check the
>                                                         monitor logs
>                                                         and see what's
>                                                         up.
>
>
>                                                         On Tue, Sep
>                                                         10, 2013 at
>                                                         4:51 PM, Pete
>                                                         Carlson
>                                                         <pgcarlson@gmail.com
>                                                         <mailto:pgcarlson@gmail.com>>
>                                                         wrote:
>
>                                                             Hi John,
>
>                                                             No, it
>                                                             looks like
>                                                             port 50095
>                                                             wasn't
>                                                             open on my
>                                                             VM.  I
>                                                             tried
>                                                             modifying
>                                                             the IP
>                                                             tables to
>                                                             make it
>                                                             opened,
>                                                             and
>                                                             restarted
>                                                             my VM but
>                                                             when I run
>                                                             the lsof
>                                                             -i command
>                                                             I don't
>                                                             see port
>                                                             50095 listed.
>
>                                                             Pete
>
>                                                             Regards,
>
>                                                             Pete
>
>                                                             Looking
>                                                             for a
>                                                             unique
>                                                             woodcraft?
>                                                              Check out
>                                                             my
>                                                             Facebook
>                                                             page:
>                                                             www.facebook.com/oqwcrafts
>                                                             <http://www.facebook.com/oqwcrafts>
>
>                                                             Also on
>                                                             Etsy:
>                                                             https://www.etsy.com/shop/OlneyQualityWoodCrft
>
>
>                                                             On Tue,
>                                                             Sep 10,
>                                                             2013 at
>                                                             3:56 PM,
>                                                             John Vines
>                                                             <vines@apache.org
>                                                             <mailto:vines@apache.org>>
>                                                             wrote:
>
>                                                                 Do you
>                                                                 have
>                                                                 the
>                                                                 port
>                                                                 open
>                                                                 to
>                                                                 your VM?
>
>
>                                                                 On
>                                                                 Tue,
>                                                                 Sep
>                                                                 10,
>                                                                 2013
>                                                                 at
>                                                                 3:54
>                                                                 PM,
>                                                                 Pete
>                                                                 Carlson <pgcarlson@gmail.com
>                                                                 <mailto:pgcarlson@gmail.com>>
>                                                                 wrote:
>
>                                                                     Hi,
>
>                                                                     I
>                                                                     just
>                                                                     set up
>                                                                     a
>                                                                     free
>                                                                     standing
>                                                                     Accumulo
>                                                                     1.5 instance
>                                                                     on
>                                                                     my
>                                                                     linux
>                                                                     VM, and
>                                                                     when
>                                                                     I
>                                                                     run bin/start-all.sh
>                                                                     it
>                                                                     starts
>                                                                     up
>                                                                     accumulo,
>                                                                     and also
>                                                                     connects
>                                                                     to
>                                                                     my
>                                                                     free
>                                                                     standing
>                                                                     Zookeeper
>                                                                     and HDFS
>                                                                     instances
>                                                                     .
>
>                                                                     However,
>                                                                     when
>                                                                     I
>                                                                     try to
>                                                                     pull
>                                                                     up
>                                                                     the Accumulo
>                                                                     master
>                                                                     in
>                                                                     my
>                                                                     FireFox
>                                                                     browser
>                                                                     (Firefox
>                                                                     ESR 10.0.12)
>                                                                     i.e.,
>                                                                     http://accumulomaster:50095/status
for
>                                                                     health
>                                                                     and monitoring,
>                                                                     I
>                                                                     get an
>                                                                     unable
>                                                                     to
>                                                                     connect
>                                                                     error.
>
>                                                                     I
>                                                                     also
>                                                                     tried
>                                                                     running
>                                                                     ./generate_monitor_certificate.sh
>                                                                     and copied
>                                                                     those
>                                                                     properties
>                                                                     in
>                                                                     my
>                                                                     accumulo-site.xml
>                                                                     but I
>                                                                     still
>                                                                     get the
>                                                                     same
>                                                                     result.
>
>                                                                     Is
>                                                                     there
>                                                                     something
>                                                                     else
>                                                                     I
>                                                                     need
>                                                                     to
>                                                                     configure?
>
>                                                                     Regards,
>
>
>                                                                     Pete
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>


Mime
View raw message