activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Strachan" <james.strac...@gmail.com>
Subject Re: BlobMessage support for sending out-of-band BLOBs
Date Wed, 14 Mar 2007 15:55:35 GMT
On 3/14/07, Aleksi Kallio <aleksi.kallio@csc.fi> wrote:
> >> I had a bit of a look at the Jetty code and it doesn't seem to have a
> >> simple 'file servlet' for supporting GET/PUT/DELETE of files in a
> >> directory; so I was thinking we could contribute one to the Jetty6
> >> tree and reuse that. e.g. maybe use the DefaultServlet in Jetty (which
> >> does GET fine) then add PUT/DELETE support
>
> I have this working (upload, download, remove) and have tested it with
> files in the order of a couple of gigabytes.

Awesome!


> I'd have two questions, all input is appreciated...
>
>
> 1) I'm currently using standard Java URL / HttpURLConnection stuff to
> connect to the web server. Unfortunately there is a bug or design issue
> that makes HttpURLConnection to load everything into memory:
>
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5026745
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4212479
>
> That is definitely not appropriate for our purposes. Currently you can
> also use chunked transfer mode
> (HttpUrlConnection.setChunkedStreamingMode()) and it works well with
> Jetty. However not all HTTP servers support this mode, I don't know if
> it is an issue. The bigger issue is that the method only exists in Java
> 1.5. In other words, it is not usable in the ActiveMQ context?
>
> I think the only solution would be to use some other HTTP package,
> probably commons-http. Or do you see some other workarounds? Personally
> I would have preferred if the thing would have not required anything but
> standard JRE classes. Unfortunately it seems impossible, unless one
> writes the HTTP code from scratch, which does not sound like a clever
> thing to do.

Good point. Trunk of ActiveMQ is currently Java 5 specific anyway, so
using chunked streaming mode sounds a fine workaround for now.

We could always switch to using either the Jetty client stuff or
commons-httpclient if its on the classpath. It'd be nice to keep the
dependency list as small as possible; but work better if
commons-httpclient or jetty-client is on the classpath etc.


> 2) While storing files to an intermediary file server they are named
> after their message ID's. Is that wise?

Well the Message IDs are guarrenteed to be system wide unique strings,
so they're quite good for the names of the files - but...

> Message ID's look a bit unfit
> for filenames. Do you think in some environments/FS's these kind of
> names might cause problems?

Agreed. We should definitely make the message IDs both URI and file
name friendly. So how about we search & replace the message ID string
and switch some of the dodgy characters and replace them with
something.

e.g.

switch : to a / maybe?

Then each producer will be in a directory which will be inside a
directory named after the connection ID etc.

Thinking about it - it might make more sense to change ActiveMQ itself
to create MessageId strings which use / rather than : maybe...

-- 

James
-------
http://radio.weblogs.com/0112098/

Mime
View raw message