accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pete Carlson <pgcarl...@gmail.com>
Subject Re: Can't get Accumulo 1.5 monitor interface to start in my browser
Date Wed, 11 Sep 2013 20:17:45 GMT
All,

I resolved the issue.  Per Eric's suggestion I changed my swappiness
setting down to 10, and also disabled all IPV6 stuff on my VM.  Once I did
this and changed the IP address of my monitor to 127.0.0.1 my monitor
started working.

Thanks guys. :-)

Regards,

Pete

On Wed, Sep 11, 2013 at 2:12 PM, John Vines <vines@apache.org> wrote:

> Furthermore, tservers being down shouldn't prevent the monitor page from
> coming up at all, so check for any other relevant messages in of the
> monitor's logs.
>
>
> On Wed, Sep 11, 2013 at 2:08 PM, Pete Carlson <pgcarlson@gmail.com> wrote:
>
>> How do I determine if the rest of Accumulo is up and running since I
>> can't use the monitor?
>>
>> Inside the monitor logs I see messages that it connected to Hadoop and
>> Zookeeper, but then later on I see this message stating there are no tablet
>> servers:
>>
>> 2013-09-11 11:39:29,199 [server.Accumulo] WARN :  System swappiness
>> setting is greater than ten (60) which can cause time-sensitive operations
>> to be delayed.  Accumulo is time sensitive because it needs to maintain
>> distributed lock agreement.
>> 2013-09-11 11:39:32,506 [impl.ServerClient] WARN :
>> tracer:localhost.localdomain There are no tablet servers: check that
>> zookeeper and accumulo are running.
>>
>> And when I log in to the accumulo shell I get the warning: There are no
>> tablet servers: check that zookeeper and accumulo are running.
>>
>> Could the system swappiness setting of 60 be causing my issue?
>>
>> Regards,
>>
>> Pete
>>
>>
>> On Wed, Sep 11, 2013 at 1:13 PM, John Vines <vines@apache.org> wrote:
>>
>>> Is the rest of Accumulo up and running? And is that the only type of
>>> error in the Monitor logs?
>>>
>>>
>>> On Wed, Sep 11, 2013 at 1:04 PM, Pete Carlson <pgcarlson@gmail.com>wrote:
>>>
>>>> Ok, I deleted /accumulo out of hdfs, initialized accumulo, and
>>>> re-started Accumulo.  I still can't see the monitor IN my VM browser.  I
>>>> still get the same Monitor info message in the logs.
>>>>
>>>> Pete
>>>>
>>>> On Wed, Sep 11, 2013 at 11:55 AM, John Vines <vines@apache.org> wrote:
>>>>
>>>>> You need to delete the /accumulo directory out of hdfs
>>>>>
>>>>>
>>>>> On Wed, Sep 11, 2013 at 11:54 AM, Pete Carlson <pgcarlson@gmail.com>wrote:
>>>>>
>>>>>> I removed the monitor certificates as suggested, and I no longer
get
>>>>>> any messages in the monitor_localhost.err file.
>>>>>>
>>>>>> When I look in the monitor_localhost.localdomain.log file and saw
the
>>>>>> following INFO/Exception message:
>>>>>>
>>>>>> 2013-09-11 11:42:50,012 [monitor.Monitor] INFO :  Failed to obtain
>>>>>> problem reports
>>>>>> java.lang.RuntimeException:
>>>>>> org.apache.accumulo.core.client.impl.ThriftScanner$ScanTimedOutException
>>>>>>         at
>>>>>> org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java:174)
>>>>>>         at
>>>>>> org.apache.accumulo.server.problems.ProblemReports$3.hasNext(ProblemReports.java:241)
>>>>>>         at
>>>>>> org.apache.accumulo.server.problems.ProblemReports.summarize(ProblemReports.java:299)
>>>>>>         at
>>>>>> org.apache.accumulo.server.monitor.Monitor.fetchData(Monitor.java:392)
>>>>>>         at
>>>>>> org.apache.accumulo.server.monitor.Monitor$2.run(Monitor.java:504)
>>>>>>         at
>>>>>> org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
>>>>>>         at java.lang.Thread.run(Thread.java:724)
>>>>>> Caused by:
>>>>>> org.apache.accumulo.core.client.impl.ThriftScanner$ScanTimedOutException
>>>>>>         at
>>>>>> org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:212)
>>>>>>         at
>>>>>> org.apache.accumulo.core.client.impl.ScannerIterator$Reader.run(ScannerIterator.java:82)
>>>>>>         at
>>>>>> org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java:164)
>>>>>>         ... 6 more
>>>>>>
>>>>>> I was thinking this was related to the fact I didn't yet have Thrift
>>>>>> 0.9.0 installed (due to dependency issues).  I now have Thrift 0.9.0
>>>>>> installed.  I saved off my config files, blew away the accumulo directory
>>>>>> and extracted it again from the tar.gz and copied over my conf files.
>>>>>>
>>>>>> However, when I run the accumulo-1.5.0/bin/accumulo init command
>>>>>> again it gives me the following error:
>>>>>>
>>>>>> [util.Initialize] FATAL: It appears this location was previously
>>>>>> initialized, exiting ...
>>>>>>
>>>>>> I can still start Accumulo like before with the bin/start-all.sh
>>>>>> command but the Accumulo monitor still doesn't work IN my VM.
>>>>>>
>>>>>> Any other ideas to try?
>>>>>>
>>>>>> Pete
>>>>>>
>>>>>> On Wed, Sep 11, 2013 at 10:57 AM, John Vines <vines@apache.org>wrote:
>>>>>>
>>>>>>> Either try rerunning the monitor certificates or remove the portion
>>>>>>> from your accumulo-site that defines use of the certificate.
I suggest the
>>>>>>> latter simply to remove variables at hand.
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Sep 11, 2013 at 10:54 AM, Pete Carlson <pgcarlson@gmail.com>wrote:
>>>>>>>
>>>>>>>> John,
>>>>>>>>
>>>>>>>> What is the monitor process called so I can verify its running?
 I
>>>>>>>> did look at the thread monitor logs, and under monitor_localhost.err
it
>>>>>>>> gives a FileNotFoundException because it can't find the keystore.jks
under
>>>>>>>> accumulo-1.5.0/conf/
>>>>>>>>
>>>>>>>> I would have thought when I ran ./generate_monitor_certificate.sh
>>>>>>>> it would have created this.
>>>>>>>>
>>>>>>>> I was trying to run the monitor in a browser IN the VM. 
Thats how
>>>>>>>> I found this problem.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Pete
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Sep 11, 2013 at 10:41 AM, John Vines <vines@apache.org>wrote:
>>>>>>>>
>>>>>>>>> Again, please check to make sure the monitor process
is running
>>>>>>>>> and it's not reporting any errors in it's logs.
>>>>>>>>>
>>>>>>>>> If you can verify that (and please verify that first),
try
>>>>>>>>> accessing the monitor in a browser IN the VM.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Sep 11, 2013 at 10:36 AM, Pete Carlson <
>>>>>>>>> pgcarlson@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> All,
>>>>>>>>>>
>>>>>>>>>> Just to clarify I have a Centos VM running on my
Windows 7 box,
>>>>>>>>>> and I have Accumulo, Hadoop, and Zookeeper running
on it standalone.  I was
>>>>>>>>>> trying to open the Accumulo monitor within that VM
and it failed.  However
>>>>>>>>>> I can open the Hadoop Namenode and Jobtracker web
interfaces in my browser
>>>>>>>>>> just fine.
>>>>>>>>>>
>>>>>>>>>> Brendan, I was able to connect to my VM using netcat
on port
>>>>>>>>>> 50095.
>>>>>>>>>>
>>>>>>>>>> However, when I try to telnet to that port on my
Win7 with telnet
>>>>>>>>>> using my VM's ip address and port i.e.,
>>>>>>>>>>
>>>>>>>>>> telnet <IP address of vm> 50095
>>>>>>>>>>
>>>>>>>>>> I get a message:   ....Could not open connection
to the host, on
>>>>>>>>>> port 50095: Connect failed.
>>>>>>>>>>
>>>>>>>>>> Pete
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Sep 11, 2013 at 9:15 AM, Brendan Heussler
<
>>>>>>>>>> bheussler@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Can you try to connect to your VM using netcat
on port 50095?
>>>>>>>>>>>
>>>>>>>>>>> On VM:
>>>>>>>>>>> nc -l 50095
>>>>>>>>>>>
>>>>>>>>>>> On Client:
>>>>>>>>>>> nc <ip-address of VM> 50095
>>>>>>>>>>>
>>>>>>>>>>>  Then type "test" or something on your Client
machine.  The
>>>>>>>>>>> message should be sent to your VM.
>>>>>>>>>>>
>>>>>>>>>>> This will help to test the connectivity of the
two machines over
>>>>>>>>>>> port 50095
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Brendan
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Sep 10, 2013 at 5:22 PM, John Vines <vines@apache.org>wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Are you certain the monitor is running and
was able to bind to
>>>>>>>>>>>> the port? Please check the monitor logs for
any errors.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Sep 10, 2013 at 5:18 PM, Pete Carlson
<
>>>>>>>>>>>> pgcarlson@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> John,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I don't see port 50095 in use in the
VM.  When I become root
>>>>>>>>>>>>> and type lsof -i -P | grep :50095 I don't
get anything back.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Any suggestions on how to get the port
running?  I tried
>>>>>>>>>>>>> editing the iptables with the following
command:
>>>>>>>>>>>>>
>>>>>>>>>>>>> iptables -I INPUT -p tcp --dport 50095
-j ACCEPT
>>>>>>>>>>>>>
>>>>>>>>>>>>> and then I typed
>>>>>>>>>>>>>
>>>>>>>>>>>>> service iptables save
>>>>>>>>>>>>>
>>>>>>>>>>>>> to save my iptable changes, but when
I restarted my VM I
>>>>>>>>>>>>> didn't see any difference.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Pete
>>>>>>>>>>>>>
>>>>>>>>>>>>> Looking for a unique woodcraft?  Check
out my Facebook page:
>>>>>>>>>>>>> www.facebook.com/oqwcrafts
>>>>>>>>>>>>>
>>>>>>>>>>>>> Also on Etsy: https://www.etsy.com/shop/OlneyQualityWoodCrft
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Sep 10, 2013 at 4:56 PM, John
Vines <vines@apache.org>wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> You don't see 50095 in use locally
on the VM or in your box?
>>>>>>>>>>>>>> You should only see it in use in
the VM. If not, make sure it's running. If
>>>>>>>>>>>>>> it is, check the monitor logs and
see what's up.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Sep 10, 2013 at 4:51 PM,
Pete Carlson <
>>>>>>>>>>>>>> pgcarlson@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> No, it looks like port 50095
wasn't open on my VM.  I tried
>>>>>>>>>>>>>>> modifying the IP tables to make
it opened, and restarted my VM but when I
>>>>>>>>>>>>>>> run the lsof -i command I don't
see port 50095 listed.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Pete
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Pete
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Looking for a unique woodcraft?
 Check out my Facebook page:
>>>>>>>>>>>>>>> www.facebook.com/oqwcrafts
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Also on Etsy: https://www.etsy.com/shop/OlneyQualityWoodCrft
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Sep 10, 2013 at 3:56
PM, John Vines <
>>>>>>>>>>>>>>> vines@apache.org> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Do you have the port open
to your VM?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Sep 10, 2013 at 3:54
PM, Pete Carlson <
>>>>>>>>>>>>>>>> pgcarlson@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I just set up a free
standing Accumulo 1.5 instance on my
>>>>>>>>>>>>>>>>> linux VM, and when I
run bin/start-all.sh it starts up accumulo, and also
>>>>>>>>>>>>>>>>> connects to my free standing
Zookeeper and HDFS instances .
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> However, when I try to
pull up the Accumulo master in my
>>>>>>>>>>>>>>>>> FireFox browser (Firefox
ESR 10.0.12) i.e.,
>>>>>>>>>>>>>>>>> http://accumulomaster:50095/status
for health and
>>>>>>>>>>>>>>>>> monitoring, I get an
unable to connect error.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I also tried running
./generate_monitor_certificate.sh and
>>>>>>>>>>>>>>>>> copied those properties
in my accumulo-site.xml but I still get the same
>>>>>>>>>>>>>>>>> result.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Is there something else
I need to configure?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Pete
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message