falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Baptiste Onofré ...@nanthrax.net>
Subject Re: [HEADS UP] Support of ActiveMQ HA/Scalability
Date Thu, 12 Feb 2015 10:49:26 GMT
OK, it's what I thought: stomp is mostly used for python or perl, 
openwire from JMS/NMS/CMS.

Thanks,
Regards
JB

On 02/12/2015 10:58 AM, Idris Ali wrote:
> Platform team here uses stomp from perl to check the ActiveMQ service and
> other monitoring.
>
> Thanks,
> -Idris
>
> On Thu, Feb 12, 2015 at 1:37 AM, Jean-Baptiste Onofré <jb@nanthrax.net>
> wrote:
>
>> By the way, who are the clients which use stomp (maybe python as it's the
>> only protocol supported by python) ? Openwire should be used as default
>> transport connector, at least for JMS and CMS.
>>
>> Regards
>> JB
>>
>> On 02/11/2015 08:59 PM, Idris Ali wrote:
>>
>>> Yeah sure, no hurry.
>>>
>>> I think I figured it out, seems to be the memory config issue in the new
>>> installation.
>>>
>>> 2015-02-11 17:03:44,843 | ERROR | Could not accept connection :
>>> java.lang.Exception: java.lang.OutOfMemoryError: Java heap space |
>>> org.apache.activemq.broker.TransportConnector | Ac
>>>
>>> tiveMQ Transport Server Thread Handler: stomp://localhost:61613
>>>
>>> 2015-02-11 17:02:32,440 | ERROR | Could not accept connection :
>>> java.lang.Exception: java.lang.OutOfMemoryError: Java heap space |
>>> org.apache.activemq.broker.TransportConnector | Ac
>>>
>>> tiveMQ Transport Server Thread Handler: tcp://0.0.0.0:61616
>>>
>>>
>>>
>>> 2015-02-11 17:02:26,140 | DEBUG | Unregistering MBean
>>> org.apache.activemq:type=Broker,brokerName=localhost,
>>> connector=clientConnectors,connectorName=openwire,
>>> connectionViewType=clien
>>>
>>> tId,connectionName=ID_tzns4008.grid.uj1.inmobi.com-
>>> 48815-1422442061155-2_1
>>> | org.apache.activemq.broker.jmx.ManagementContext | ActiveMQ Transport:
>>> tcp:///IP:40234@61616
>>>
>>> 2015-02-11 17:01:45,533 | WARN  | Failed to remove producer:
>>> ID:machine-48831-1423672843718-0:1:1:1 |
>>> org.apache.activemq.broker.TransportConnection | ActiveMQ
>>>
>>> Transport: tcp:///IP:55499@61616
>>>
>>> *java.lang.OutOfMemoryError: Java heap space   *
>>>
>>>
>>>
>>>
>>>           at
>>> java.util.concurrent.CopyOnWriteArrayList.remove(
>>> CopyOnWriteArrayList.java:515)
>>>
>>>
>>>           at
>>> java.util.concurrent.CopyOnWriteArraySet.remove(
>>> CopyOnWriteArraySet.java:232)
>>>
>>> Followed by lot of this messages:
>>>
>>> java.lang.OutOfMemoryError: Java heap space
>>>
>>>
>>> Thanks again,
>>>
>>> -Idris
>>>
>>> On Thu, Feb 12, 2015 at 1:04 AM, Jean-Baptiste Onofré <jb@nanthrax.net>
>>> wrote:
>>>
>>>   Hey Idris,
>>>>
>>>> if you don't mind, I would like to work on this tomorrow (my time).
>>>>
>>>> Regards
>>>> JB
>>>>
>>>> On 02/11/2015 07:36 PM, Idris Ali wrote:
>>>>
>>>>   Hi JB,
>>>>>
>>>>> Slightly off from this discussion, we recently upgraded ActiveMQ version
>>>>> to
>>>>> 5.9.1 from 5.4.3 to work with Falcon.
>>>>>
>>>>> With the default configs, however we see wire format negotiation
>>>>> timedout
>>>>> issue pretty often and the job fails.
>>>>>
>>>>> Failing Oozie Launcher, Main class
>>>>> [org.apache.oozie.action.hadoop.JavaMain], main() threw exception,
>>>>> javax.jms.JMSException: Wire format negotiation timeout: peer did not
>>>>> send his wire format.
>>>>> org.apache.oozie.action.hadoop.JavaMainException:
>>>>> javax.jms.JMSException: Wire format negotiation timeout: peer did not
>>>>> send his wire format.
>>>>>           at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.
>>>>> java:60)
>>>>>           at org.apache.oozie.action.hadoop.LauncherMain.run(
>>>>> LauncherMain.java:42)
>>>>>           at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.
>>>>> java:37)
>>>>>           at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>           at sun.reflect.NativeMethodAccessorImpl.invoke(
>>>>> NativeMethodAccessorImpl.java:57)
>>>>>           at sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>>>> DelegatingMethodAccessorImpl.java:43)
>>>>>           at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>           at org.apache.oozie.action.hadoop.LauncherMapper.map(
>>>>> LauncherMapper.java:293)
>>>>>           at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
>>>>>
>>>>>
>>>>> Initial investigation suggests something to do with Log4j deadlock.
>>>>> Please let me know if this is a known issue in 5.9.1 ?
>>>>>
>>>>> Thanks,
>>>>> -Idris
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Feb 11, 2015 at 9:20 AM, Pallavi Rao <pallavi.rao@inmobi.com>
>>>>> wrote:
>>>>>
>>>>>    JB,
>>>>>
>>>>>> We are working on Falcon talking to ActiveMQ HA setup too.  Our initial
>>>>>> hunch is that as long as the right URL (with fail over scheme) is
>>>>>> specified
>>>>>> in the configuration, startup properties or cluster definition, it
>>>>>> should
>>>>>> work without any code changes. Plan to run a quick test soon. Will
keep
>>>>>> you
>>>>>> posted.
>>>>>>
>>>>>> Regards,
>>>>>> Pallavi
>>>>>> On Feb 10, 2015 7:51 PM, "Jean-Baptiste Onofré" <jb@nanthrax.net>
>>>>>> wrote:
>>>>>>
>>>>>>    Hi all,
>>>>>>
>>>>>>>
>>>>>>> I'm working on the ActiveMQ update (to 5.10.1) and improvements
on the
>>>>>>> broker (configuration flexibility, tuning, etc).
>>>>>>>
>>>>>>> I plan also to improve the support of some ActiveMQ features,
>>>>>>> especially:
>>>>>>> 1/ master/slave support with kahadb and JDBC locking backend
>>>>>>> 2/ network of brokers support to be able to have Falcon instances
>>>>>>> producing on one broker which can dispatch the messages (including
>>>>>>>
>>>>>>>   conduit
>>>>>>
>>>>>>   subscriptions support) to other brokers where Falcon instances
can get
>>>>>>>
>>>>>>>   the
>>>>>>
>>>>>>   messages.
>>>>>>>
>>>>>>> I will send more details and patches very soon.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Regards
>>>>>>> JB
>>>>>>> --
>>>>>>> Jean-Baptiste Onofré
>>>>>>> jbonofre@apache.org
>>>>>>> http://blog.nanthrax.net
>>>>>>> Talend - http://www.talend.com
>>>>>>>
>>>>>>>
>>>>>>>   --
>>>>>> _____________________________________________________________
>>>>>> The information contained in this communication is intended solely
for
>>>>>> the
>>>>>> use of the individual or entity to whom it is addressed and others
>>>>>> authorized to receive it. It may contain confidential or legally
>>>>>> privileged
>>>>>> information. If you are not the intended recipient you are hereby
>>>>>> notified
>>>>>> that any disclosure, copying, distribution or taking any action in
>>>>>> reliance
>>>>>> on the contents of this information is strictly prohibited and may
be
>>>>>> unlawful. If you have received this communication in error, please
>>>>>> notify
>>>>>> us immediately by responding to this email and then delete it from
your
>>>>>> system. The firm is neither liable for the proper and complete
>>>>>> transmission
>>>>>> of the information contained in this communication nor for any delay
in
>>>>>> its
>>>>>> receipt.
>>>>>>
>>>>>>
>>>>>>
>>>>>   --
>>>> Jean-Baptiste Onofré
>>>> jbonofre@apache.org
>>>> http://blog.nanthrax.net
>>>> Talend - http://www.talend.com
>>>>
>>>>
>>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>

-- 
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Mime
View raw message