activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefano Bagnara <apa...@bago.org>
Subject Re: SMTP Server (Apache James) spooling hints
Date Fri, 09 May 2008 08:20:57 GMT
James Strachan ha scritto:
> 2008/5/8 Stefano Bagnara <apache@bago.org>:
>>  I'm starting analysis on how to replace our default spool with ActiveMQ and
>>  [...] in James we currently have a single "message store" and we can
>> "lock on a message" (so no other thread will take it) "retrieve it", "update
>> and unlock it" (alter its state or state+content) or "remove it". How would
>> you manage this with ActiveMQ?
> 
> With ActiveMQ you'd use a queue per state/maillet, remove it from the
> queue, do something with it then put it on some other queue(s) (either
> changed or the same message). The simple JMS/MOM model of sending to a
> queue or consuming from a queue turns out to be very fast; allowing a
> highly SEDA based asynchronous model to go really fast since there's
> no locking or leasing required - and messages can flow very
> asynchronously to boost throughput.

What does it happen under the hood when I use so many queues? Is the 
message fully written to disk each time I move it from a queue to 
another or does it simply update a reference when it belongs to the same 
store?

> If you do find you wanna grab - edit - put back type thing alot you
> could look at using JavaSpaces (or Entity Bean :). But I think for
> JAMES then messaging could work well as it sounds to me (as a newbie
> JAMES person) like what you're doing processing mail is kinda a pipes
> and filters type model...
> http://activemq.apache.org/camel/pipes-and-filters.html
> 
> which maps very well to messaging and queues.

Cool! This is very interesting and I never read about it. So I'm going 
to study it and to play with it a bit.

> For more background see :
> http://activemq.apache.org/camel/enterprise-integration-patterns.html
> 
>>  3) Outgoing spool:
>>  The outgoing spool in JAMES is a spool like the main spool, with the
>> difference that a message delivery could fail and there is a retry schedule.
>> [...]
>>  Any suggestions on how to do this with ActiveMQ?
> 
> It sounds like you could use the delayer pattern...
> http://activemq.apache.org/camel/delayer.html
> 
> Then have separate queues for '30 mins later', '1 hour later', '2 hours later'.
> [...]

I looked at the website and found an error in this page:
http://activemq.apache.org/camel/spring-xml-extensions.html
"An error occurred: Connection refused. The system administrator has 
been notified."
I looked at the CWIKI sources 
(http://cwiki.removeme_apache.org/confluence/display/CAMEL/Spring+XML+Extensions) 
and I see this:
{snippet:id=e3|lang=xml|url=activemq/camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/builder/spring_route_builder_test.xml}
Not sure but maybe you have to add svn.apache.org/repos/asf/ in front of it?

>>  I understand this is a lot of questions, but I would really appreciate any
>> hint, even partial. I'm collecting ideas :-)
> 
> :)

Thank you! Your answers are even more than what I expected! You're 
suggestion seems to be very very useful and I think you saved me weeks 
of thoughts!
I'll start with your hints and I'll come back with more questions as 
soon as I'll have rode the camel! ;-)

>>  PS: we are also evaluating using JCR for inboxes if you was wondering, but
>> this is another story, for another list ;-)
> 
> You could store the mail in JCR and use messaging for the process
> flow. e.g. the JMS messages could just contain a reference (URL?) to
> the message payload.
> 
> How often is the payload of the message mutated as it goes through
> maillets? If it remains kinda static and its more the headers, states
> & mailets that change mostly, it could be worth putting the payload in
> some file system / REST resource / JCR and just referring to the
> payload for large messages (say over 1-10MB)?

This really depends on custom configurations. We provides many mailets 
that will alter the payload and many that simply run checks and route 
the message. I guess an estimation of a generic use case could be this:
- 100% of messages we spool will have some of their header changed.
-  30% of messages will have their body changed a couple of times.

Very much appreciated, thank you again,
Stefano


Mime
View raw message