qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robbie Gemmell" <robbie.gemm...@gmail.com>
Subject RE: can't list in java cli
Date Sat, 21 Nov 2009 01:30:32 GMT
Ok, so ruling out it taking 15 seconds to finalise a connection that is
actually being established, that's what id expect to happen just now if the
connection attempt times out (which is returning a null instead of throwing
an IOException like it should, leading to the NPE you are seeing) because
the attempt is getting blackholed somehow.

With both ports open, the only other issue I know of that could lead to this
relates to the JMX Server's Connector Stub embedding incorrect host data for
the remote machine, and thus throwing the connecting client to the wrong
destination when it inspects the stub and tries to connect to the
destination contained within it (see
http://docs.sun.com/source/820-6740/images/adm_jmxConnect_wRegistry.gif for
an picture of the process). 

This can happen when running the broker on a Linux box which doesn't
properly resolve its hostname to its host address (though generally this
makes it put in something like 127.0.0.1 or 127.0.1.1 and causes an instant
connection-refused related failure as the JMX connection attempt gets
directed to the local machine instead of the remote machine after picking up
the RMI stub from the remote RMI registry to determine the socket type and
address of the JMX server) or even when it does correctly establish the
hostname but this name/ip isnt reachable from the client machine. 

If you could, try the below info from the management console FAQ to force
the hostname/ip that the JMX server embeds within its RMI stub for the
client to retrieve. 

And failing that try connecting with JConsole, which if it fails will do so
in any circumstance with a 'connection failed' error dialog but can have
some very fine grained logging enabled to see exactly why. Use it to do a
remote connection to <ip/hostname>:port of the remote server after starting
JConsole with the command: jconsole
-J-Djava.util.logging.config.file=logging.properties (where
logging.properties is a file of the form shown at
http://blogs.sun.com/jmxetc/resource/logging.properties - and id probably
turn javax.management.remote.level up to FINEST). Within the huge amount of
stuff it prints during a connection attempt should be the connection details
it is trying to connect to retrieve the RMI stub, and the details contained
in the RMI stub it retrieves from the RMIRegistry before attempting to
establish the JMX connection itself using those details. If you get this
far, feel free to send me (privately, the list will strip anything except
the smallest attachments) the log so I can take a look, as im intrigued by
the difficulties your experiencing here.

Robbie



>From the management console FAQ:

The RMI based JMX ConnectorServer used by the broker requires two ports to
operate. The console connects to an RMI Registry running on the primary
(default 8999) port and retrieves the information actually needed to connect
to the JMX Server. This information embeds the hostname of the remote
machine, and if this is incorrect or unreachable by the connecting client
the connection will fail.

This situation arises due to the hostname configuration on Linux and is
generally encountered when the remote machine does not have a DNS hostname
entry on the local network, causing the hostname command to return a
loopback IP instead of a fully qualified domain name or IP address
accessible by remote client machines. It is described in further detail at:
http://java.sun.com/javase/6/docs/technotes/guides/management/faq.html#linux
1

To remedy this issue you can set the java.rmi.server.hostname system
property to control the hostname/ip reported to the RMI runtime when
advertising the JMX ConnectorServer. This can also be used to dictate the
address returned on a computer with multiple network interfaces to control
reachability. To do so, add the value -Djava.rmi.server.hostname=<desired
hostname/ip> to the QPID_OPTS environment variable before starting the
qpid-server script.


-----Original Message-----
From: mARK bLOORE [mailto:mbloore@gmail.com] 
Sent: 21 November 2009 00:30
To: users@qpid.apache.org
Subject: Re: can't list in java cli

I get the same error, but after a much longer delay.  I tried it twice.


On Fri, Nov 20, 2009 at 3:15 PM, Robbie Gemmell
<robbie.gemmell@gmail.com> wrote:
> I think you might be coming up against the bug I noticed when looking at
> this: http://issues.apache.org/jira/browse/QPID-2152
>
> The person who raised the JIRA hasnt indicated whether the workaround I
> suggested solved their problem, so id be interested to find out if it
solves
> yours. From the above link:
>
>
> However, examining the code I have identified one issue that might cause
the
> problem you are seeing, whereby if a connection is actually in progress
but
> taking an unexpected amount of time to complete then a
NullPointerException
> can occur which would lead to the connection appearing to fail and
generate
> the output you are seeing.
>
> If this is indeed the cause of the problem you are seeing, as a workaround
> you can alter the console configuration to allow an increased time for the
> connection to succeed or fail:
> 1. Open the qpidmc.ini file at the root of the console release [on the
linux
> and windows builds, its buried within the package for Mac OS X]
> 2. Add the value "-Dtimeout=15000" (without the quotes) on the empty line
> below "-Declipse.consoleLog=true"
>
> Let me know if that helps,
> Robbie
>
> -----Original Message-----
> From: mARK bLOORE [mailto:mbloore@gmail.com]
> Sent: 20 November 2009 19:30
> To: users@qpid.apache.org
> Subject: Re: can't list in java cli
>
> Thanks, Robbie, but I'm still getting
> java.lang.NullPointerException
>        at
>
org.apache.qpid.management.ui.jmx.JMXServerRegistry.<init>(JMXServerRegistry
> .java:93)
>        at
>
org.apache.qpid.management.ui.views.NavigationView.createJMXServerConnection
> (NavigationView.java:240)
>        at
>
org.apache.qpid.management.ui.views.NavigationView.addNewServer(NavigationVi
> ew.java:301)
> ...
> trying to connect to the remote broker.
>
> I have tested both ports, and they are open.  My config is pretty
> default, I just added my queues and upped their size, and of course
> made my user an admin.  Is there anything else?
>
>
> On Fri, Nov 20, 2009 at 12:49 PM, Robbie Gemmell
> <robbie.gemmell@gmail.com> wrote:
>> Yes, there is a second port you need to open, which is 100 higher than
>> the jmxport listed in the configuration (8999 in your case, so 9099 is
>> the other one you want).
>>
>> RMI based JMX actually requires 2 ports when used the way basically
>> everyone uses it (an RMIConnectorServer JMX server advertised in an
>> RMIRegistry), so the reason you are not getting connected is that the
>> 2nd port (where the actual JMX Connector Server is listening) is still
>> blocked and when the console tries connecting to it after retrieving
>> its address from the registry it cant reach it. Prior to 0.5 this
>> second port was randomly assigned in the ephemeral range by the JVM,
>> but I modified it to an offset of the configured port (which is where
>> the RMIRegistry is located) for 0.5 for exactly the situation you are
>> in where the ports need to be known to be opened.
>>
>> Robbie
>>
>> 2009/11/20 mARK bLOORE <mbloore@gmail.com>:
>>> Thanks Robbie.  That got me going on a local test broker, but I have
>>> made the same access changes to the remote production server and I
>>> still can't connect.  I opened port 8999/TCP in its firewall; do I
>>> need to do anything else?
>>>
>>>
>>> On Wed, Nov 11, 2009 at 5:09 PM, Robbie Gemmell
>>> <robbie.gemmell@gmail.com> wrote:
>>>> Ok, I have narrowed down the cause of the exception you are seeing.
>>>>
>>>> The problem in this case seems to be that you have used a user account
> which
>>>> is valid, but does not have any management access rights. As a result,
> the
>>>> CLI managed to establish a JMXConnector to the server, but as soon as
> you
>>>> actually try to do something with it you get the SecurityException as
> you
>>>> don't have permissions to invoke any operations.
>>>>
>>>> You need to edit the etc/jmxremote.access file in the broker
> configuration
>>>> to give the user rights. It is of the format username=rights where
> rights
>>>> can be either readonly, readwrite, or admin. Readonly allows all
>>>> non-modifying actions, readwrite also allows modifying actions, and
> admin is
>>>> the same but with access to additional management mbeans (though as far
> as I
>>>> know the CLI cant acually do anything that would require admin - in 0.5
> this
>>>> controls access rights to the ConfigurationManagement, UserManagement
> and
>>>> LoggingManagement MBeans)
>>>>
>>>> Once that is done the CLI should be able to do stuff, though as ive
> noted it
>>>> has its issues (eg https://issues.apache.org/jira/browse/QPID-1878)
>>>>
>>>> Robbie
>>>>
>>>>> -----Original Message-----
>>>>> From: mARK bLOORE [mailto:mbloore@gmail.com]
>>>>> Sent: 11 November 2009 16:57
>>>>> To: users@qpid.apache.org
>>>>> Subject: can't list in java cli
>>>>>
>>>>> I am running the Java 0.5 broker and management tools.  In qpid-cli,
>>>>> if I try to list any type of object, I get
>>>>>
>>>>> qpid-admin-$ queue list
>>>>> java.lang.SecurityException: Access denied
>>>>>       at
>>>>>
org.apache.qpid.server.management.MBeanInvocationHandlerImpl.invoke(MBe
>>>>> anInvocationHandlerImpl.java:159)
>>>>>       at $Proxy7.queryNames(Unknown Source)
>>>>>       at
>>>>>
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnection
>>>>> Impl.java:1468)
>>>>>       at
>>>>>
javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionI
>>>>> mpl.java:90)
>>>>>       at
>>>>>
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(R
>>>>> MIConnectionImpl.java:1284)
>>>>>       at java.security.AccessController.doPrivileged(Native Method)
>>>>>       at
>>>>>
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMI
>>>>> ConnectionImpl.java:1389)
>>>>>       at
>>>>>
javax.management.remote.rmi.RMIConnectionImpl.queryNames(RMIConnectionI
>>>>> mpl.java:552)
>>>>>       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>       at
>>>>>
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja
>>>>> va:57)
>>>>>       at
>>>>>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso
>>>>> rImpl.java:43)
>>>>>       at java.lang.reflect.Method.invoke(Method.java:616)
>>>>>       at
>>>>> sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
>>>>>       at sun.rmi.transport.Transport$1.run(Transport.java:177)
>>>>>       at java.security.AccessController.doPrivileged(Native Method)
>>>>>       at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
>>>>>       at
>>>>>
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:553
>>>>> )
>>>>>       at
>>>>>
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.
>>>>> java:808)
>>>>>       at
>>>>>
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.j
>>>>> ava:667)
>>>>>       at
>>>>>
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ja
>>>>> va:1110)
>>>>>       at
>>>>>
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j
>>>>> ava:603)
>>>>>       at java.lang.Thread.run(Thread.java:636)
>>>>>       at
>>>>>
sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRe
>>>>> moteCall.java:273)
>>>>>       at
>>>>>
sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:25
>>>>> 1)
>>>>>       at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:160)
>>>>>       at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source)
>>>>>       at
>>>>> javax.management.remote.rmi.RMIConnectionImpl_Stub.queryNames(Unknown
>>>>> Source)
>>>>>       at
>>>>>
javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.qu
>>>>> eryNames(RMIConnector.java:829)
>>>>>       at
>>>>>
org.apache.qpid.commands.objects.ObjectNames.returnObjects(ObjectNames.
>>>>> java:52)
>>>>>       at
>>>>> org.apache.qpid.commands.Commandlist.listobjects(Commandlist.java:106)
>>>>>       at
>>>>> org.apache.qpid.commands.Commandlist.execute(Commandlist.java:191)
>>>>>       at
>>>>>
org.apache.qpid.CommandExecutionEngine.runcommand(CommandExecutionEngin
>>>>> e.java:69)
>>>>>       at
>>>>>
org.apache.qpid.CommandLineInterpreter.main(CommandLineInterpreter.java
>>>>> :170)
>>>>> java.lang.NullPointerException
>>>>>       at
>>>>> org.apache.qpid.commands.Commandlist.listobjects(Commandlist.java:107)
>>>>>       at
>>>>> org.apache.qpid.commands.Commandlist.execute(Commandlist.java:191)
>>>>>       at
>>>>>
org.apache.qpid.CommandExecutionEngine.runcommand(CommandExecutionEngin
>>>>> e.java:69)
>>>>>       at
>>>>>
org.apache.qpid.CommandLineInterpreter.main(CommandLineInterpreter.java
>>>>> :170)
>>>>> qpid-admin-$
>>>>>
>>>>>
>>>>> What is wrong?
>>>>>
>>>>> --
>>>>> mARK bLOORE <mbloore@gmail.com>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> Apache Qpid - AMQP Messaging Implementation
>>>>> Project:      http://qpid.apache.org
>>>>> Use/Interact: mailto:users-subscribe@qpid.apache.org
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> Apache Qpid - AMQP Messaging Implementation
>>>> Project:      http://qpid.apache.org
>>>> Use/Interact: mailto:users-subscribe@qpid.apache.org
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> mARK bLOORE <mbloore@gmail.com>
>>>
>>> ---------------------------------------------------------------------
>>> Apache Qpid - AMQP Messaging Implementation
>>> Project:      http://qpid.apache.org
>>> Use/Interact: mailto:users-subscribe@qpid.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> Apache Qpid - AMQP Messaging Implementation
>> Project:      http://qpid.apache.org
>> Use/Interact: mailto:users-subscribe@qpid.apache.org
>>
>>
>
>
>
> --
> mARK bLOORE <mbloore@gmail.com>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project:      http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project:      http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>



-- 
mARK bLOORE <mbloore@gmail.com>

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org



---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org


Mime
View raw message