activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Shannon <christopher.l.shan...@gmail.com>
Subject Re: [DISCUSS] ARTEMIS-1009 Pure Messaging Encoding Refactoring Artemis 2.0
Date Mon, 06 Mar 2017 14:34:31 GMT
Protocol free sounds good to me.  Binary can still be Base64 wrapped.  I
guess if we did that then the importer could be configured to write the
message to the broker in different formats, either core or openwire.

Also, one thing I don't currently like with the the XmlImporter
implementation is that it relies on new line characters for parsing and XML
shouldn't need spaces or white space characters to be parsed.  (See
getMessageBodyBytes() in XmlDataImporter).  This doesn't apply in the
export case from the 5.x broker (because we need to load the entire message
into memory when writing it to XML anyways) but for the large message
export case for Artemis instead of having multiple CDATA elements it makes
more sense to have multiple tags. Ie.

<large-message-body>
  <segment>...</segment>
   <segment>...</segment>
</large-message-body>

Also, the use of CDATA is unnecessary if using Base64 (and makes it harder
to use because JaxB doesn't have very good CDATA support)

On Mon, Mar 6, 2017 at 8:56 AM, Clebert Suconic <clebert.suconic@gmail.com>
wrote:

> I would change it to be protocol free.
>
>
>
> <textbody>text goes here </textbody>
>
> Maps binary etc.
>
>
>
> On Mon, Mar 6, 2017 at 8:45 AM Christopher Shannon <
> christopher.l.shannon@gmail.com> wrote:
>
> > Sounds good we can talk monday (i guess today).   I think the current
> > strategy of converting to core and using the existing importer is fine
> for
> > now as i don't want to hold up a 2.0.0 release.  But we can target
> > enhancements for future versions.
> >
> > On Sun, Mar 5, 2017 at 11:01 PM, Clebert Suconic <
> > clebert.suconic@gmail.com>
> > wrote:
> >
> > > PR just arrived :)
> > >
> > > https://github.com/apache/activemq-artemis/pull/1064
> > >
> > > On Sat, Mar 4, 2017 at 1:10 AM, Clebert Suconic
> > > <clebert.suconic@gmail.com> wrote:
> > > > @CShanon: I was actually dealing with the exporter/importer today.
> > > >
> > > > You're right.. the body of the message is for core-only.
> > > >
> > > >
> > > > We could do the following:
> > > >
> > > >
> > > > I - expand the format to be agnostic:
> > > >
> > > >
> > > > <Text></text>
> > > > </bytes></bytes>
> > > > <map></map>
> > > >
> > > >
> > > >
> > > > or we could encode at the protocols....
> > > >
> > > >
> > > > it seems easier to just use the protocol now. we could talk mon
> monday
> > > on IRC?
> > > >
> > > > On Fri, Mar 3, 2017 at 6:56 PM, Clebert Suconic
> > > > <clebert.suconic@gmail.com> wrote:
> > > >> ... almost there... my testsuite is almost cleaned up.. there will
> be
> > > >> a few tests failing that will fix after bringing into master.
> > > >>
> > > >> I am planning to bring this into master between now and monday.
> > > >>
> > > >> this is just a heads up.
> > > >>
> > > >> Meanwhile (weekend anyways)... please don't make any too heavy
> changes
> > > >> :) especially under ./artemis-protocols... or anything that will
> touch
> > > >> Message... (that's a reason I actually want to merge soon, there's
a
> > > >> PR Martyn sent for instance that would clash conceptually (not just
> > > >> lines of code)).
> > > >>
> > > >>
> > > >>
> > > >> On Thu, Mar 2, 2017 at 8:42 AM, Clebert Suconic
> > > >> <clebert.suconic@gmail.com> wrote:
> > > >>> :O. Wow!!!!
> > > >>>
> > > >>>
> > > >>> All this perspiration is paying of at least.  1% inspiration (the
> > idea
> > > to
> > > >>> use messages directly is quite simple). But changing the amount
of
> > > >>> dependencies to it. Required some work.
> > > >>>
> > > >>> I ran the test suite yesterday.  Some failures but I know what
are
> > > they.
> > > >>> That means I am almost there to send a PR.
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Thu, Mar 2, 2017 at 8:34 AM nigro_franz <nigro.fra@gmail.com>
> > > wrote:
> > > >>>>
> > > >>>> the hw/os configuration is:
> > > >>>> - Intel® Xeon® Processor E3-1505M v5
> > > >>>> - Linux 4.9.6 x86_64
> > > >>>> - 32 GB RAM
> > > >>>> - openjdk version "1.8.0_121"
> > > >>>>
> > > >>>> test configuration (1 producer vs 1 consumer):
> > > >>>> -  protocol = amqp
> > > >>>>  - delivery = NonPersistent
> > > >>>>  - destination type = Queue
> > > >>>>  - message bytes = 100
> > > >>>>  - runs = 5
> > > >>>>  - warmup iterations = 100000
> > > >>>>  - iterations = 100000
> > > >>>>
> > > >>>> Clebert encoding patch
> > > >>>>
> > > >>>>
> > > >>>> Producer Throughput: 42713 ops/sec
> > > >>>> Consumer Throughput: 65444 ops/sec
> > > >>>> EndToEnd Throughput: *39095 ops/sec*
> > > >>>>
> > > >>>> Old encoding
> > > >>>>
> > > >>>>
> > > >>>> Producer Throughput: 36617 ops/sec
> > > >>>> Consumer Throughput: 8203 ops/sec
> > > >>>> EndToEnd Throughput: *6451 ops/sec*
> > > >>>>
> > > >>>> The throughput is improved of about 1 order of magnitude,
really
> an
> > > >>>> awesome
> > > >>>> job Clebert!!!!
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> --
> > > >>>> View this message in context:
> > > >>>> http://activemq.2283324.n4.nabble.com/DISCUSS-ARTEMIS-
> > > 1009-Pure-Messaging-Encoding-Refactoring-Artemis-2-0-
> > > tp4722638p4722689.html
> > > >>>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
> > > >>>
> > > >>> --
> > > >>> Clebert Suconic
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Clebert Suconic
> > > >
> > > >
> > > >
> > > > --
> > > > Clebert Suconic
> > >
> > >
> > >
> > > --
> > > Clebert Suconic
> > >
> >
> --
> Clebert Suconic
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message