accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Vines <vi...@apache.org>
Subject Re: Can't get Accumulo 1.5 monitor interface to start in my browser
Date Wed, 11 Sep 2013 18:12:13 GMT
Furthermore, tservers being down shouldn't prevent the monitor page from
coming up at all, so check for any other relevant messages in of the
monitor's logs.


On Wed, Sep 11, 2013 at 2:08 PM, Pete Carlson <pgcarlson@gmail.com> wrote:

> How do I determine if the rest of Accumulo is up and running since I can't
> use the monitor?
>
> Inside the monitor logs I see messages that it connected to Hadoop and
> Zookeeper, but then later on I see this message stating there are no tablet
> servers:
>
> 2013-09-11 11:39:29,199 [server.Accumulo] WARN :  System swappiness
> setting is greater than ten (60) which can cause time-sensitive operations
> to be delayed.  Accumulo is time sensitive because it needs to maintain
> distributed lock agreement.
> 2013-09-11 11:39:32,506 [impl.ServerClient] WARN :
> tracer:localhost.localdomain There are no tablet servers: check that
> zookeeper and accumulo are running.
>
> And when I log in to the accumulo shell I get the warning: There are no
> tablet servers: check that zookeeper and accumulo are running.
>
> Could the system swappiness setting of 60 be causing my issue?
>
> Regards,
>
> Pete
>
>
> On Wed, Sep 11, 2013 at 1:13 PM, John Vines <vines@apache.org> wrote:
>
>> Is the rest of Accumulo up and running? And is that the only type of
>> error in the Monitor logs?
>>
>>
>> On Wed, Sep 11, 2013 at 1:04 PM, Pete Carlson <pgcarlson@gmail.com>wrote:
>>
>>> Ok, I deleted /accumulo out of hdfs, initialized accumulo, and
>>> re-started Accumulo.  I still can't see the monitor IN my VM browser.  I
>>> still get the same Monitor info message in the logs.
>>>
>>> Pete
>>>
>>> On Wed, Sep 11, 2013 at 11:55 AM, John Vines <vines@apache.org> wrote:
>>>
>>>> You need to delete the /accumulo directory out of hdfs
>>>>
>>>>
>>>> On Wed, Sep 11, 2013 at 11:54 AM, Pete Carlson <pgcarlson@gmail.com>wrote:
>>>>
>>>>> I removed the monitor certificates as suggested, and I no longer get
>>>>> any messages in the monitor_localhost.err file.
>>>>>
>>>>> When I look in the monitor_localhost.localdomain.log file and saw the
>>>>> following INFO/Exception message:
>>>>>
>>>>> 2013-09-11 11:42:50,012 [monitor.Monitor] INFO :  Failed to obtain
>>>>> problem reports
>>>>> java.lang.RuntimeException:
>>>>> org.apache.accumulo.core.client.impl.ThriftScanner$ScanTimedOutException
>>>>>         at
>>>>> org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java:174)
>>>>>         at
>>>>> org.apache.accumulo.server.problems.ProblemReports$3.hasNext(ProblemReports.java:241)
>>>>>         at
>>>>> org.apache.accumulo.server.problems.ProblemReports.summarize(ProblemReports.java:299)
>>>>>         at
>>>>> org.apache.accumulo.server.monitor.Monitor.fetchData(Monitor.java:392)
>>>>>         at
>>>>> org.apache.accumulo.server.monitor.Monitor$2.run(Monitor.java:504)
>>>>>         at
>>>>> org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
>>>>>         at java.lang.Thread.run(Thread.java:724)
>>>>> Caused by:
>>>>> org.apache.accumulo.core.client.impl.ThriftScanner$ScanTimedOutException
>>>>>         at
>>>>> org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:212)
>>>>>         at
>>>>> org.apache.accumulo.core.client.impl.ScannerIterator$Reader.run(ScannerIterator.java:82)
>>>>>         at
>>>>> org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java:164)
>>>>>         ... 6 more
>>>>>
>>>>> I was thinking this was related to the fact I didn't yet have Thrift
>>>>> 0.9.0 installed (due to dependency issues).  I now have Thrift 0.9.0
>>>>> installed.  I saved off my config files, blew away the accumulo directory
>>>>> and extracted it again from the tar.gz and copied over my conf files.
>>>>>
>>>>> However, when I run the accumulo-1.5.0/bin/accumulo init command again
>>>>> it gives me the following error:
>>>>>
>>>>> [util.Initialize] FATAL: It appears this location was previously
>>>>> initialized, exiting ...
>>>>>
>>>>> I can still start Accumulo like before with the bin/start-all.sh
>>>>> command but the Accumulo monitor still doesn't work IN my VM.
>>>>>
>>>>> Any other ideas to try?
>>>>>
>>>>> Pete
>>>>>
>>>>> On Wed, Sep 11, 2013 at 10:57 AM, John Vines <vines@apache.org>
wrote:
>>>>>
>>>>>> Either try rerunning the monitor certificates or remove the portion
>>>>>> from your accumulo-site that defines use of the certificate. I suggest
the
>>>>>> latter simply to remove variables at hand.
>>>>>>
>>>>>>
>>>>>> On Wed, Sep 11, 2013 at 10:54 AM, Pete Carlson <pgcarlson@gmail.com>wrote:
>>>>>>
>>>>>>> John,
>>>>>>>
>>>>>>> What is the monitor process called so I can verify its running?
 I
>>>>>>> did look at the thread monitor logs, and under monitor_localhost.err
it
>>>>>>> gives a FileNotFoundException because it can't find the keystore.jks
under
>>>>>>> accumulo-1.5.0/conf/
>>>>>>>
>>>>>>> I would have thought when I ran ./generate_monitor_certificate.sh
it
>>>>>>> would have created this.
>>>>>>>
>>>>>>> I was trying to run the monitor in a browser IN the VM.  Thats
how I
>>>>>>> found this problem.
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Pete
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Sep 11, 2013 at 10:41 AM, John Vines <vines@apache.org>wrote:
>>>>>>>
>>>>>>>> Again, please check to make sure the monitor process is running
and
>>>>>>>> it's not reporting any errors in it's logs.
>>>>>>>>
>>>>>>>> If you can verify that (and please verify that first), try
>>>>>>>> accessing the monitor in a browser IN the VM.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Sep 11, 2013 at 10:36 AM, Pete Carlson <pgcarlson@gmail.com
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> All,
>>>>>>>>>
>>>>>>>>> Just to clarify I have a Centos VM running on my Windows
7 box,
>>>>>>>>> and I have Accumulo, Hadoop, and Zookeeper running on
it standalone.  I was
>>>>>>>>> trying to open the Accumulo monitor within that VM and
it failed.  However
>>>>>>>>> I can open the Hadoop Namenode and Jobtracker web interfaces
in my browser
>>>>>>>>> just fine.
>>>>>>>>>
>>>>>>>>> Brendan, I was able to connect to my VM using netcat
on port 50095.
>>>>>>>>>
>>>>>>>>> However, when I try to telnet to that port on my Win7
with telnet
>>>>>>>>> using my VM's ip address and port i.e.,
>>>>>>>>>
>>>>>>>>> telnet <IP address of vm> 50095
>>>>>>>>>
>>>>>>>>> I get a message:   ....Could not open connection to the
host, on
>>>>>>>>> port 50095: Connect failed.
>>>>>>>>>
>>>>>>>>> Pete
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Sep 11, 2013 at 9:15 AM, Brendan Heussler <
>>>>>>>>> bheussler@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Can you try to connect to your VM using netcat on
port 50095?
>>>>>>>>>>
>>>>>>>>>> On VM:
>>>>>>>>>> nc -l 50095
>>>>>>>>>>
>>>>>>>>>> On Client:
>>>>>>>>>> nc <ip-address of VM> 50095
>>>>>>>>>>
>>>>>>>>>>  Then type "test" or something on your Client machine.
 The
>>>>>>>>>> message should be sent to your VM.
>>>>>>>>>>
>>>>>>>>>> This will help to test the connectivity of the two
machines over
>>>>>>>>>> port 50095
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Brendan
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Sep 10, 2013 at 5:22 PM, John Vines <vines@apache.org>wrote:
>>>>>>>>>>
>>>>>>>>>>> Are you certain the monitor is running and was
able to bind to
>>>>>>>>>>> the port? Please check the monitor logs for any
errors.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Sep 10, 2013 at 5:18 PM, Pete Carlson
<
>>>>>>>>>>> pgcarlson@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> John,
>>>>>>>>>>>>
>>>>>>>>>>>> I don't see port 50095 in use in the VM.
 When I become root
>>>>>>>>>>>> and type lsof -i -P | grep :50095 I don't
get anything back.
>>>>>>>>>>>>
>>>>>>>>>>>> Any suggestions on how to get the port running?
 I tried
>>>>>>>>>>>> editing the iptables with the following command:
>>>>>>>>>>>>
>>>>>>>>>>>> iptables -I INPUT -p tcp --dport 50095 -j
ACCEPT
>>>>>>>>>>>>
>>>>>>>>>>>> and then I typed
>>>>>>>>>>>>
>>>>>>>>>>>> service iptables save
>>>>>>>>>>>>
>>>>>>>>>>>> to save my iptable changes, but when I restarted
my VM I didn't
>>>>>>>>>>>> see any difference.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>>
>>>>>>>>>>>> Pete
>>>>>>>>>>>>
>>>>>>>>>>>> Looking for a unique woodcraft?  Check out
my Facebook page:
>>>>>>>>>>>> www.facebook.com/oqwcrafts
>>>>>>>>>>>>
>>>>>>>>>>>> Also on Etsy: https://www.etsy.com/shop/OlneyQualityWoodCrft
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Sep 10, 2013 at 4:56 PM, John Vines
<vines@apache.org>wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> You don't see 50095 in use locally on
the VM or in your box?
>>>>>>>>>>>>> You should only see it in use in the
VM. If not, make sure it's running. If
>>>>>>>>>>>>> it is, check the monitor logs and see
what's up.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Sep 10, 2013 at 4:51 PM, Pete
Carlson <
>>>>>>>>>>>>> pgcarlson@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> No, it looks like port 50095 wasn't
open on my VM.  I tried
>>>>>>>>>>>>>> modifying the IP tables to make it
opened, and restarted my VM but when I
>>>>>>>>>>>>>> run the lsof -i command I don't see
port 50095 listed.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Pete
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Pete
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Looking for a unique woodcraft? 
Check out my Facebook page:
>>>>>>>>>>>>>> www.facebook.com/oqwcrafts
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Also on Etsy: https://www.etsy.com/shop/OlneyQualityWoodCrft
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Sep 10, 2013 at 3:56 PM,
John Vines <vines@apache.org
>>>>>>>>>>>>>> > wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Do you have the port open to
your VM?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Sep 10, 2013 at 3:54
PM, Pete Carlson <
>>>>>>>>>>>>>>> pgcarlson@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I just set up a free standing
Accumulo 1.5 instance on my
>>>>>>>>>>>>>>>> linux VM, and when I run
bin/start-all.sh it starts up accumulo, and also
>>>>>>>>>>>>>>>> connects to my free standing
Zookeeper and HDFS instances .
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> However, when I try to pull
up the Accumulo master in my
>>>>>>>>>>>>>>>> FireFox browser (Firefox
ESR 10.0.12) i.e.,
>>>>>>>>>>>>>>>> http://accumulomaster:50095/status
for health and
>>>>>>>>>>>>>>>> monitoring, I get an unable
to connect error.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I also tried running ./generate_monitor_certificate.sh
and
>>>>>>>>>>>>>>>> copied those properties in
my accumulo-site.xml but I still get the same
>>>>>>>>>>>>>>>> result.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Is there something else I
need to configure?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Pete
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message