activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Shannon <christopher.l.shan...@gmail.com>
Subject Re: [DISCUSS] ActiveMQ utility project name
Date Thu, 02 Feb 2017 15:03:14 GMT
For the XML side of things I was just thinking that if someone wanted to
export a large store (such as 1GB+, etc)  then it would produce very large
XML files which would not be efficient to store or transfer.  A use case I
had in mind is exporting a broker into XML on one machine and then
transferring the data to another machine with an Artemis broker would be
slow with large XML.

Gzip certainly works.  I wasn't thinking of doing anything crazy with
compression.  What I was thinking of was just doing something simple like
have an optional flag the user could set for the command that would just
end up creating a GZIPOutputStream and write the XML directly to a gzip
file which would save time (and disk space) from having to create the XML
first and then compress it after the fact.


On Thu, Feb 2, 2017 at 9:38 AM, Clebert Suconic <clebert.suconic@gmail.com>
wrote:

> On Thu, Feb 2, 2017 at 9:28 AM, Justin Bertram <jbertram@apache.com>
> wrote:
> >> For large stores I think we will need to do something here in terms of
> compression otherwise we could end up with huge XML files.
> >
> > What specifically are your concerns and ideas here?  Storage appears
> cheap and plentiful and files can be compressed after they are created with
> gzip or similar.  Also, huge message stores are (IMO) an anti-pattern.  A
> message broker is not a database after all.  What use-cases do you have in
> mind?
>
>
> I am not sure what CShanon had in mind,
>
> but we could (I"m not saying yet we should.. just listing the possibility):
>
> -  import from KahaDB and do directly what the importer is doing.
> Instead of dumping an XML, it could add directly to the broker running
> from the kahaDB storage.
>
>
> I don't mind a huge XML (or compressed gz in such way the importer
> would read compressed). But it could be an option eventually.
>
> @CShanon: I wouldn't do the directly unless you are aware of user
> demand, meaning the XML is not good for certain cases.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message