Return-Path: X-Original-To: apmail-stratos-dev-archive@minotaur.apache.org Delivered-To: apmail-stratos-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9CAE0115F1 for ; Tue, 2 Sep 2014 08:20:32 +0000 (UTC) Received: (qmail 79790 invoked by uid 500); 2 Sep 2014 08:20:32 -0000 Delivered-To: apmail-stratos-dev-archive@stratos.apache.org Received: (qmail 79739 invoked by uid 500); 2 Sep 2014 08:20:32 -0000 Mailing-List: contact dev-help@stratos.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@stratos.apache.org Delivered-To: mailing list dev@stratos.apache.org Received: (qmail 79728 invoked by uid 99); 2 Sep 2014 08:20:31 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Sep 2014 08:20:31 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of paul@wso2.com designates 209.85.219.53 as permitted sender) Received: from [209.85.219.53] (HELO mail-oa0-f53.google.com) (209.85.219.53) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Sep 2014 08:20:28 +0000 Received: by mail-oa0-f53.google.com with SMTP id eb12so4473998oac.40 for ; Tue, 02 Sep 2014 01:20:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wso2.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=84lto92mdo5+3f2d8QQkRLQVTm2KuJ37apzwb8Y8+vE=; b=LFYJioJOrLMLVXJVLu+CC1FV7ezj8eFxFbbOB0/MhI6EAG3eC5zjHhNdWbS0+0ayGf cOxKddwqcrdEzisg8MIAMGE7rSYjeWSQOPZ8KsJWYoMpVorZE6MkLLtPRHYaGSnv3JfO bF9TMIny5jhvH256CYLz8mVGRxfLSp6ouncJQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=84lto92mdo5+3f2d8QQkRLQVTm2KuJ37apzwb8Y8+vE=; b=JxMYDPBS8wdyerKnU7EuZYTP+BeuUE2+Gb95dLqMSvwIt2UnpwKjn16/fRnD1mooB5 +mOQMUZ9aAs0ndlMEAEAvRaXBqOmG1Eud6fcdAjaqNYW6gJUtaR7//RmeK688tfHtPXO FTfNcDqiZMLGl93zWLg1DDsDDLYHBeNvdO0NlHu3qD9Uk+J9GeTpSXXv0tttoYGhrYr9 Y4meZFgvzpL4+8ip+Auk5DihX0fjYYVxk6E1oQXUFNHq5JyFrjHcAojZp9Z85CNN4FEO oC7JzGPAb3Ur8FM8G8o8BPNK408z6gTmcPzKyVX3hSmsLfj/q0QNX29djlzyRQb0LGBx wOEg== X-Gm-Message-State: ALoCoQnnAcHcKdnCcbJdzQkbs1jN/EZKqgCu4BhWlXkQkX+yPW0YvuqsYwYdRBGb29f3sbT0/YtZ X-Received: by 10.182.118.193 with SMTP id ko1mr30190569obb.45.1409646007273; Tue, 02 Sep 2014 01:20:07 -0700 (PDT) MIME-Version: 1.0 Received: by 10.202.64.66 with HTTP; Tue, 2 Sep 2014 01:19:27 -0700 (PDT) In-Reply-To: References: From: Paul Fremantle Date: Tue, 2 Sep 2014 09:19:27 +0100 Message-ID: Subject: Re: Messaging protocol with Stratos python cartridge agent To: Gayan Gunarathne Cc: dev@stratos.apache.org Content-Type: multipart/alternative; boundary=089e0115ea3085df94050210c958 X-Virus-Checked: Checked by ClamAV on apache.org --089e0115ea3085df94050210c958 Content-Type: text/plain; charset=UTF-8 Can you post an example of the headers and body? Paul On 2 September 2014 09:17, Gayan Gunarathne wrote: > Hi Paul, > > Currently we have existing java based message publisher with the headers. > Here we try to use those published messages with the python client. If we > use JSON structured message with headers enclosed we have to change all > other current implementations as well. AFAIK We will distinguished the > messages with their header value.So my point is if we used this sort of > JSON structure for messages, we need to change all the current messaging > model. > > OTOH with the Stomp protocol with python client we were able to get those > header values and used those messages as it is. > > Thanks, > Gayan > > > On Tue, Sep 2, 2014 at 1:04 PM, Paul Fremantle wrote: > >> What headers do you need to pass? >> >> The way of doing headers with MQTT is to use a structured message (e.g. >> JSON) and put some headers into the JSON. >> >> Paul >> >> >> On 2 September 2014 07:58, Gayan Gunarathne wrote: >> >>> >>> We did some analysis regarding the MQTT protocol as well.Following are >>> our findings regarding the MQTT. >>> >>> MQTT is low bandwidth consumption messaging protocol which >>> supports pub/sub protocol with the following limitations. >>> >>> - It has fixed header (2 bytes) . Specifications mentioning that it >>> can be extended.But AFAIU clients[1] didn't support for this header >>> extension. We have checked the MQTT with the messages published with >>> headers and we didn't receive the header on those.Only receive the message >>> body. >>> - The payload size for a MQTT message is 256MB.I guess design point >>> for MQTT usage is targeted at more constrained devices.[3]. MQTT is better >>> at a high volume of low size messages. >>> - It is supports only topic.This will not be a issue currently in >>> our scenario as we used only topic currently AFAIK.[4] >>> >>> >>> I guess it build with above limitations and it is very good choice for >>> the mobile and small application as it is very lightweight and low >>> power.[5] WDYT? >>> >>> Please share your thoughts on this. >>> >>> [1] https://pypi.python.org/pypi/paho-mqtt >>> [2[ https://github.com/iwanbk/nyamuk >>> [3] http://stephendnicholas.com/archives/1217 >>> [4] http://www.mqtechconference.com/sessions_v2013/MQTC_v2013_MQTT.pdf >>> [5] >>> http://blogs.vmware.com/vfabric/2013/02/choosing-your-messaging-protocol-amqp-mqtt-or-stomp.html >>> >>> >>> Thanks, >>> Gayan >>> >>> >>> On Mon, Sep 1, 2014 at 4:32 PM, Gayan Gunarathne >>> wrote: >>> >>>> Thanks Paul. >>>> >>>> We will check on this paho-mgtt python library as well. >>>> >>>> Thanks, >>>> Gayan >>>> >>>> >>>> >>>> >>>> On Mon, Sep 1, 2014 at 4:17 PM, Paul Fremantle wrote: >>>> >>>>> Python has excellent pure python MQTT support. >>>>> >>>>> > pip install paho-mqtt >>>>> >>>>> https://pypi.python.org/pypi/paho-mqtt >>>>> >>>>> Paul >>>>> >>>>> >>>>> On 1 September 2014 11:40, Gayan Gunarathne wrote: >>>>> >>>>>> Hi Paul, >>>>>> >>>>>> Here we try to connect the our existing Apache ActiveMQ with the pure >>>>>> python.(We try to replace our java cartridge agent with the python >>>>>> as after that we can remove the java dependency from the cartridge itself). >>>>>> >>>>>> In the initial research we didn't notice such a pure python libraries >>>>>> with the MQTT which can connect with the ActiveMQ. Also with the initial >>>>>> reading I guess they are recommend it for the mobile applications as its >>>>>> low bandwidth consumption.Also I am noticed lot of web socket based >>>>>> applications are developed through MQTT. >>>>>> >>>>>> Thanks, >>>>>> Gayan >>>>>> >>>>>> >>>>>> On Mon, Sep 1, 2014 at 3:42 PM, Paul Fremantle wrote: >>>>>> >>>>>>> Have you looked at mqtt? >>>>>>> >>>>>>> Paul >>>>>>> On 1 Sep 2014 12:10, "Gayan Gunarathne" wrote: >>>>>>> >>>>>>>> >>>>>>>> Hi All, >>>>>>>> >>>>>>>> As per our initial research on the python based messaging >>>>>>>> implementation of python cartridge agent and discussion[1] we can summarize >>>>>>>> our findings as following. >>>>>>>> >>>>>>>> *1.AMQP* >>>>>>>> >>>>>>>> Apache ActiveMQ used the AMQP 1.0 protocol. We found C based client >>>>>>>> library (proton) [2] which can be used with python. >>>>>>>> But it is a platform dependent library and every time we need to >>>>>>>> install the libraries based on the architecture and OS version.IMO this >>>>>>>> will be difficult us to manage as we need to install the libraries based on >>>>>>>> the platform. >>>>>>>> >>>>>>>> Other python libraries(txAMQP ,pika etc) supports AMQP 0.9 which >>>>>>>> fails to connect with the ActiveMQ. >>>>>>>> >>>>>>>> >>>>>>>> *2.STOMP* >>>>>>>> >>>>>>>> We can connect to the ActiveMQ with the STOMP protocol by using the >>>>>>>> STOMP python client library[3].IMO we can used STOMP with ActiveMQ without >>>>>>>> worry about the platform based library installation.By considering our >>>>>>>> scope of the cartridge agent I think it is better to have a platform >>>>>>>> independence cartridge as far as we can. Then, IMO STOMP based messaging >>>>>>>> queue implementation will be more manageable when we increase our number >>>>>>>> of platform supported. >>>>>>>> WDYT? >>>>>>>> >>>>>>>> When comparing the performance of theses two protocols it looks >>>>>>>> like there are no such huge difference[4].I guess the performance also >>>>>>>> tightly coupled with the message broker you are using as well.OTOH Apache >>>>>>>> ActiveMQ recommended way also to used the STOMP with the python[5]. >>>>>>>> >>>>>>>> >>>>>>>> Please share your thoughts on this. >>>>>>>> >>>>>>>> [1] Stratos Python Cartridge Agent >>>>>>>> [2] https://github.com/apache/qpid-proton >>>>>>>> [3] https://github.com/jasonrbriggs/stomp.py >>>>>>>> [4] >>>>>>>> http://blogs.vmware.com/vfabric/2013/02/choosing-your-messaging-protocol-amqp-mqtt-or-stomp.html >>>>>>>> [5] http://activemq.apache.org/python.html >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Gayan >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> >>>>>>>> Gayan Gunarathne >>>>>>>> Technical Lead >>>>>>>> WSO2 Inc. (http://wso2.com) >>>>>>>> email : gayang@wso2.com | mobile : +94 766819985 >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Best Regards, >>>>>> >>>>>> Gayan Gunarathne >>>>>> Technical Lead >>>>>> WSO2 Inc. (http://wso2.com) >>>>>> email : gayang@wso2.com | mobile : +94 766819985 >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Paul Fremantle >>>>> CTO and Co-Founder, WSO2 >>>>> OASIS WS-RX TC Co-chair, Apache Member >>>>> >>>>> UK: +44 207 096 0336 >>>>> >>>>> blog: http://pzf.fremantle.org >>>>> twitter.com/pzfreo >>>>> paul@wso2.com >>>>> >>>>> wso2.com Lean Enterprise Middleware >>>>> >>>>> Disclaimer: This communication may contain privileged or other >>>>> confidential information and is intended exclusively for the addressee/s. >>>>> If you are not the intended recipient/s, or believe that you may have >>>>> received this communication in error, please reply to the sender indicating >>>>> that fact and delete the copy you received and in addition, you should not >>>>> print, copy, retransmit, disseminate, or otherwise use the information >>>>> contained in this communication. Internet communications cannot be >>>>> guaranteed to be timely, secure, error or virus-free. The sender does not >>>>> accept liability for any errors or omissions. >>>>> >>>> >>>> >>>> >>>> -- >>>> Best Regards, >>>> >>>> Gayan Gunarathne >>>> Technical Lead >>>> WSO2 Inc. (http://wso2.com) >>>> email : gayang@wso2.com | mobile : +94 766819985 >>>> >>>> >>> >>> >>> >>> -- >>> Best Regards, >>> >>> Gayan Gunarathne >>> Technical Lead >>> WSO2 Inc. (http://wso2.com) >>> email : gayang@wso2.com | mobile : +94 766819985 >>> >>> >> >> >> >> -- >> Paul Fremantle >> CTO and Co-Founder, WSO2 >> OASIS WS-RX TC Co-chair, Apache Member >> >> UK: +44 207 096 0336 >> >> blog: http://pzf.fremantle.org >> twitter.com/pzfreo >> paul@wso2.com >> >> wso2.com Lean Enterprise Middleware >> >> Disclaimer: This communication may contain privileged or other >> confidential information and is intended exclusively for the addressee/s. >> If you are not the intended recipient/s, or believe that you may have >> received this communication in error, please reply to the sender indicating >> that fact and delete the copy you received and in addition, you should not >> print, copy, retransmit, disseminate, or otherwise use the information >> contained in this communication. Internet communications cannot be >> guaranteed to be timely, secure, error or virus-free. The sender does not >> accept liability for any errors or omissions. >> > > > > -- > Best Regards, > > Gayan Gunarathne > Technical Lead > WSO2 Inc. (http://wso2.com) > email : gayang@wso2.com | mobile : +94 766819985 > > -- Paul Fremantle CTO and Co-Founder, WSO2 OASIS WS-RX TC Co-chair, Apache Member UK: +44 207 096 0336 blog: http://pzf.fremantle.org twitter.com/pzfreo paul@wso2.com wso2.com Lean Enterprise Middleware Disclaimer: This communication may contain privileged or other confidential information and is intended exclusively for the addressee/s. If you are not the intended recipient/s, or believe that you may have received this communication in error, please reply to the sender indicating that fact and delete the copy you received and in addition, you should not print, copy, retransmit, disseminate, or otherwise use the information contained in this communication. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions. --089e0115ea3085df94050210c958 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Can you post an example of the headers and body?

<= /div>
Paul


On 2 September 2014 09:17, Gayan Gunarathne = <gayang@wso2.com> wrote:
Hi Paul,
Currently we have existing java based message publisher with th= e headers. Here we try to use those published messages with the python clie= nt. If we use JSON structured message with =C2=A0headers enclosed we have t= o change all other current implementations as well. AFAIK We will distingui= shed the messages with their header value.So my point is if we used this so= rt of JSON structure for messages, we need to change all the current messag= ing model.

OTOH with the Stomp protocol with python client we were able= to get those header values and used those messages as it is.=C2=A0

Thanks,
Gayan


On Tue, Sep 2, 2014 at 1:04 PM, Paul Fre= mantle <paul@wso2.com> wrote:
What headers do you need to pass?

The w= ay of doing headers with MQTT is to use a structured message (e.g. JSON) an= d put some headers into the JSON.

Paul


On 2 September 2014 07:58, Gayan Gunarat= hne <gayang@wso2.com> wrote:

We did some analysis regarding the MQTT protocol = as well.Following are our findings regarding the MQTT.

=
MQTT is low bandwidth consumption messaging protocol which supports=C2= =A0pub/sub protocol=C2=A0with the following limitations.=C2=A0
  • It has fixed header (2 bytes) . Specifications mentioning that= it can be extended.But AFAIU clients[1] didn't support for this header= extension. We have checked the MQTT with the messages published with heade= rs and we didn't receive the header on those.Only receive the message b= ody.
  • The payload size for a MQTT message is 256MB.I guess design point for M= QTT usage is targeted at more constrained devices.[3]. MQTT is better at a = high volume of low size messages.
  • It is supports only topic.Thi= s will not be a issue currently in our scenario as we used only topic curre= ntly AFAIK.[4]

I guess it build with above limitations= and it is very good choice for the mobile and small application as it=C2=A0is ver= y lightweight and low power.[5] WDYT?
Please share your thoughts on this.
[3]=C2=A0http://stephendnich= olas.com/archives/1217


On Mon, Sep 1, 2014 at 4:32 PM, Gayan Gunarathne <gayang@wso2.com> wrote:
Thanks Paul.

=
We will check on this paho-mgtt python library as well.

=
Thanks,
Gayan


=


On Mon, Sep 1, 2014 at 4:17 PM, Paul Fremant= le <paul@wso2.com> wrote:
Python has excellent pure python MQTT support.=C2=A0
<= br>
>=C2=A0pip install paho-mqtt


Paul


On = 1 September 2014 11:40, Gayan Gunarathne <gayang@wso2.com> wro= te:
Hi Paul,

Here we try to conn= ect the our existing Apache ActiveMQ with the pure python.(We try to replace our java cart= ridge agent with the python as after that we can remove the java dependency= from the cartridge itself).=C2=A0
In the initial research we didn't notice such a pure python librar= ies with the MQTT=C2=A0=C2=A0which can connect with the ActiveMQ. Also with= the initial reading I guess they are recommend it for the mobile applicati= ons as its low bandwidth consumption.Also I am noticed lot of web socket ba= sed applications are developed through MQTT.

Thanks,
Gayan


On Mon, Sep 1, 2014 at = 3:42 PM, Paul Fremantle <paul@wso2.com> wrote:

Have you looked at mqtt?

<= span>

Paul

On 1 Sep 2014 12:10, "Gayan Gunarathne"= ; <gayang@wso2.com<= /a>> wrote:

Hi All,

As pe= r our initial research on the python based messaging implementation of pyth= on cartridge agent and discussion[1] we can summarize our findings as follo= wing.

1.AMQP

Apache ActiveMQ = used the AMQP 1.0 protocol. We found C based client library (proton) [2] wh= ich can be used with python.
But it is a platform dependent libra= ry and every time we need to install the libraries based on the architectur= e and OS version.IMO this will be difficult us to manage as we need to inst= all the libraries based on the platform.

Other python libraries(txAMQP ,pika etc)=C2=A0= supports AMQP 0.9 which fails to connect with the ActiveMQ.

<= /div>

2.STOMP

We can connect = to the ActiveMQ with the STOMP protocol by using the STOMP python client li= brary[3].IMO we can used STOMP with ActiveMQ without worry about the platfo= rm based library installation.By considering our scope of the cartridge age= nt I think it is better to have a platform independence cartridge as far as= we can. Then, IMO STOMP based messaging queue implementation =C2=A0will be= more manageable when we increase our number of platform supported.
WDYT?

When comparing the performance of these= s two protocols it looks like there are no such huge difference[4].I guess = the performance also tightly coupled with the message broker you are using = as well.OTOH Apache ActiveMQ recommended way also to used the STOMP with th= e python[5].


Please share your thoughts on this.

[1]=C2=A0Stratos Python Cartridge Agent
[4]=C2=A0http://blogs.vmware.com/v= fabric/2013/02/choosing-your-messaging-protocol-amqp-mqtt-or-stomp.html=

=
Thanks,
Gayan



--


Gayan Gunarathne
=
Technical Lead
WSO2 Inc. (http://wso2.com)
=
= email=C2=A0 : gayang@wso2.com=C2=A0 | mobile = : +94 766819985
=C2=A0



<= /div>--
Best Regards,

G= ayan Gunarathne
Technical Lead<= br>
WSO2 Inc. (http://wso2.com)
=
=
email=C2=A0 : gayang@wso2.com=C2=A0 | mo= bile : +94 766819985<= /span>
=C2=A0



--
=
Paul Fremantle
CTO and Co-Founder, WSO2
OASIS WS-RX = TC Co-chair, Apache Member

UK: +44 207 096 0336=

blog: http://pzf= .fremantle.org
twitter.com/pzfreo<= /a>
paul@wso2.com=




--



--
=
Best Re= gards,

Gayan Gunarathne
Technical Lead
= email=C2=A0 : gayang@wso2.com=C2=A0 | mobile = : +94 766819985
=C2=A0



--
=
Paul Fremantle
CTO and Co-Founder, WSO2
OASIS WS-RX = TC Co-chair, Apache Member

UK: +44 207 096 0336=

blog: http://pzf.fre= mantle.org
t= witter.com/pzfreo
paul@wso2.com

wso2.com L= ean Enterprise Middleware

Disclaimer: This communication may contain= privileged or other confidential information and is intended exclusively f= or the addressee/s. If you are not the intended recipient/s, or believe tha= t you may have received this communication in error, please reply to the se= nder indicating that fact and delete the copy you received and in addition,= you should not print, copy, retransmit, disseminate, or otherwise use the = information contained in this communication. Internet communications cannot= be guaranteed to be timely, secure, error or virus-free. The sender does n= ot accept liability for any errors or omissions.



--
=
Best Re= gards,

Gayan Gunarathne
Technical Lead
= email=C2=A0 : gayang@wso2.com=C2=A0 | mobile = : +94 766819985
=C2=A0



--
=
Paul Fremantle
CTO and Co-Founder, WSO2
OASIS WS-RX = TC Co-chair, Apache Member

UK: +44 207 096 0336

blog: http://pzf.fre= mantle.org
t= witter.com/pzfreo
paul@wso2.com

wso2.com L= ean Enterprise Middleware

Disclaimer: This communication may contain= privileged or other confidential information and is intended exclusively f= or the addressee/s. If you are not the intended recipient/s, or believe tha= t you may have received this communication in error, please reply to the se= nder indicating that fact and delete the copy you received and in addition,= you should not print, copy, retransmit, disseminate, or otherwise use the = information contained in this communication. Internet communications cannot= be guaranteed to be timely, secure, error or virus-free. The sender does n= ot accept liability for any errors or omissions.
--089e0115ea3085df94050210c958--