activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Shannon <christopher.l.shan...@gmail.com>
Subject Re: [DISCUSS] ActiveMQ utility project name
Date Mon, 06 Feb 2017 20:33:17 GMT
Yep, that is a good point.  JAXB supports marshaling fragments by using the
stax XMLStreamWriter as a writer so the entire document doesn't get written
to memory at once.  (The reverse is also true for reading a document as
XMLStreamReader can be used.)  So we should be ok in terms of not writing
the whole thing in memory.

On Mon, Feb 6, 2017 at 3:10 PM, Clebert Suconic <clebert.suconic@gmail.com>
wrote:

> @CShannon: whatever you use, make sure it streams instead of caching /
> putting stuff in memory.
>
> I had been bitten by things like that before and I have been
> preferring using direct streaming because of that. (it was simpler at
> the time).
>
> Not saying you should use System.out.println like I did ;) (well
> something smiilar :) )
>
> ... just be careful to not hold the whole thing in memory before
> sending down the stream, otherwise you won't be able to export large
> journals.
>
> On Mon, Feb 6, 2017 at 2:25 PM, Christopher Shannon
> <christopher.l.shannon@gmail.com> wrote:
> > Cool, thanks.  I created a new Jira to create the JAXB classes from the
> > schema once that is done.
> >
> > On Mon, Feb 6, 2017 at 11:33 AM, Justin Bertram <jbertram@apache.com>
> wrote:
> >
> >> I'm actually working on a schema now as the existing importer/exporter
> >> needs updates to deal with the new addressing scheme.  I'll let you know
> >> when I'm done.
> >>
> >>
> >> Justin
> >>
> >> ----- Original Message -----
> >> From: "Clebert Suconic" <clebert.suconic@gmail.com>
> >> To: dev@activemq.apache.org
> >> Sent: Monday, February 6, 2017 10:27:08 AM
> >> Subject: Re: [DISCUSS] ActiveMQ utility project name
> >>
> >> I don't think so.  Although we could add it.
> >>
> >>
> >> On Mon, Feb 6, 2017 at 11:03 AM Christopher Shannon <
> >> christopher.l.shannon@gmail.com> wrote:
> >>
> >> > Is there a schema for the Artemis import format?  I was thinking it
> would
> >> > be nice to generate some JAXB classes from a schema that we could use
> >> when
> >> > exporting from KahaDB to XML.
> >> >
> >> > On Fri, Feb 3, 2017 at 5:08 PM, Timothy Bish <tabish121@gmail.com>
> >> wrote:
> >> >
> >> > > On 02/03/2017 07:09 AM, Christopher Shannon wrote:
> >> > >
> >> > >> A Jira project has been created.
> >> > >> https://issues.apache.org/jira/browse/AMQCLI/  The permissions
> should
> >> > >> hopefully be set up correctly for PMC members and committers.
> >> > >>
> >> > >> I won't get a chance today but early next week I can start setting
> up
> >> > the
> >> > >> project skeleton (pom file, etc) so we can start working on moving
> >> stuff
> >> > >> into it.  If someone else wants to work on that earlier, feel
free.
> >> > >>
> >> > >
> >> > > I created some initial project bits to get things started.  Looks
> like
> >> we
> >> > > still need to get the gitsvn2jira feature turned on.
> >> > >
> >> > >
> >> > > On Thu, Feb 2, 2017 at 11:59 AM, Jean-Baptiste Onofré <
> jb@nanthrax.net
> >> >
> >> > >> wrote:
> >> > >>
> >> > >> Yes, it just takes some time for the first sync.
> >> > >>>
> >> > >>> Regards
> >> > >>> JB⁣
> >> > >>>
> >> > >>> On Feb 2, 2017, 17:58, at 17:58, Christopher Shannon <
> >> > >>> christopher.l.shannon@gmail.com> wrote:
> >> > >>>
> >> > >>>> I believe the mirror should happen automatically within
24 hours.
> >> > >>>>
> >> > >>>> On Thu, Feb 2, 2017 at 10:57 AM, Andy Taylor <
> >> andy.tayls67@gmail.com>
> >> > >>>> wrote:
> >> > >>>>
> >> > >>>> can we get it mirrored in Github?
> >> > >>>>>
> >> > >>>>> On 2 February 2017 at 15:34, Christopher Shannon <
> >> > >>>>> christopher.l.shannon@gmail.com> wrote:
> >> > >>>>>
> >> > >>>>> The repository is created.
> >> > >>>>>> https://git-wip-us.apache.org/repos/asf?p=activemq-cli-
> >> > >>>>>>
> >> > >>>>> tools.git;a=summary
> >> > >>>>>
> >> > >>>>>> I guess the next step is to get a Jira project
added for it.  I
> >> can
> >> > >>>>>>
> >> > >>>>> put a
> >> > >>>>
> >> > >>>>> ticket in for that.
> >> > >>>>>>
> >> > >>>>>> On Thu, Feb 2, 2017 at 10:11 AM, Clebert Suconic
<
> >> > >>>>>> clebert.suconic@gmail.com>
> >> > >>>>>> wrote:
> >> > >>>>>>
> >> > >>>>>> Artemis broker would be
> >> > >>>>>>>> slow with large XML.
> >> > >>>>>>>>
> >> > >>>>>>> The stream reader is not parsing the whole
file..it is
> streaming
> >> > >>>>>>>
> >> > >>>>>> the
> >> > >>>>
> >> > >>>>> XML instead of loading the whole thing.
> >> > >>>>>>> It will be the same as if it was reading a
lot of data from
> any
> >> > >>>>>>>
> >> > >>>>>> other
> >> > >>>>
> >> > >>>>> storage, the large XML won't make much a difference
AFAIK.
> >> > >>>>>>>
> >> > >>>>>>> Gzip certainly works.  I wasn't thinking of
doing anything
> >> > >>>>>>>>
> >> > >>>>>>> crazy with
> >> > >>>>
> >> > >>>>> compression.  What I was thinking of was just doing
something
> >> > >>>>>>>>
> >> > >>>>>>> simple
> >> > >>>>
> >> > >>>>> like
> >> > >>>>>>
> >> > >>>>>>> have an optional flag the user could set for
the command that
> >> > >>>>>>>>
> >> > >>>>>>> would
> >> > >>>>
> >> > >>>>> just
> >> > >>>>>>
> >> > >>>>>>> end up creating a GZIPOutputStream and write
the XML directly
> >> > >>>>>>>>
> >> > >>>>>>> to a
> >> > >>>>
> >> > >>>>> gzip
> >> > >>>>>
> >> > >>>>>> file which would save time (and disk space) from
having to
> >> > >>>>>>>>
> >> > >>>>>>> create the
> >> > >>>>
> >> > >>>>> XML
> >> > >>>>>>
> >> > >>>>>>> first and then compress it after the fact.
> >> > >>>>>>>>
> >> > >>>>>>>
> >> > >>>>>>> Yep, I thought the same.. it can definitely
be done.
> >> > >>>>>>>
> >> > >>>>>>>
> >> > >
> >> > > --
> >> > > Tim Bish
> >> > > twitter: @tabish121
> >> > > blog: http://timbish.blogspot.com/
> >> > >
> >> > >
> >> >
> >> --
> >> Clebert Suconic
> >>
>
>
>
> --
> Clebert Suconic
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message