james-server-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothy Prepscius <timprepsc...@gmail.com>
Subject Re: inmemory mailbox wrong configuration
Date Thu, 07 Jun 2012 21:32:13 GMT
Awesome, thanks!

On Jun 7, 2012, at 10:35 AM, Eric Charles wrote:

> Hi Tim,
> 
> 1. Yes,
> 2. Yes
> 
> 3. StoreMessageManager in mailbox-store has the boiler-plate for you. If your implementation
blocks (lets say in a YourImplMessageMapper method), the thread will block. There are 20 threads
defined by default, so if the first thread blocks, the other one are still there to consume
the mails from the AMQ queue.  Of course, if all the 20 threads block, spooling is stopped.
Btw, there is no time-out on a thread blocking (But you should not implement your mailbox
storage with a blocking mechanism - I suppose you are asking to anticipate a database, network
connection not being released and being blocked?)
> 
> The <threads>20</threads> is for the AMQ consuming. For the remote delivery,
it works completely differently, based on the RemoteDeliveryMailet where you can define there
the number of concurrent threads running for remote delivery.
> 
> Thx for your interest :)
> 
> Eric
> 
> On 06/07/2012 03:16 PM, Timothy Prepscius wrote:
>> Thanks for this!
>> 
>> I am translating this in my head this way:
>> 
>> 1. Mail Receiver receives a mail and queues it in ActiveMQ.
>> 2. Mail Processor has multiple consumer threads watching the ActiveM queue.
>> 3. If one mailbox  write operation blocks for 5 minutes, the other consumer threads
of the Mail Processor will continue.
>> 
>> Is this roughly correct?
>> 
>> Is this relevant or is the spool outgoing: (below xml of mailetcontainer-template.conf
)
>> 
>> <!-- This block is responsible for processing messages on the spool. -->
>> 
>> <mailetcontainer enableJmx="true">
>> 
>>     <!-- MailAddress used for PostMaster -->
>> 	<context>
>> 	<postmaster>postmaster@localhost</postmaster>
>> 	</context>
>> 
>>     <!-- Number of spool threads -->
>>     <spooler>
>> 	<threads>20</threads>
>> 	</spooler>
>> 
>> 
>> 
>> Thanks again for your time!
>> 
>> -tim
>> 
>> On Jun 6, 2012, at 8:43 AM, Eric Charles wrote:
>> 
>>> Hi Tim,
>>> 
>>> 1. James doesn't block on connection (but don't forget the intermediate spooling
via ActiveMQ).
>>> 
>>> 2. To answer this question, you need to realize James reads the mails via a Camel
context, getting them from ActiveMQ.
>>> 
>>> Sorry if these response are too short, I was just wanting to take more time to
provide you with a complete picture, but I understand you need to go forward.
>>> 
>>> What you can do it put a break point in the append method, and send mails to
James (running in your IDE). You will see the various threads and also that it is non-blocking....
exception made of the various lock with do via the JVMMailboxPathLocker, see for example this
snippet from StoreMessageManager
>>> 
>>>            final Message<Id>  message = createMessage(internalDate, size,
bodyStartOctet, contentIn, flags, propertyBuilder);
>>>            return locker.executeWithLock(mailboxSession, new StoreMailboxPath<Id>(getMailboxEntity()),
new MailboxPathLocker.LockAwareExecution<Long>() {
>>> 
>>>                @Override
>>>                public Long execute() throws MailboxException {
>>>                    MessageMetaData data = appendMessageToStore(message, mailboxSession);
>>> 
>>>                    SortedMap<Long, MessageMetaData>  uids = new TreeMap<Long,
MessageMetaData>();
>>>                    uids.put(data.getUid(), data);
>>>                    dispatcher.added(mailboxSession, uids, getMailboxEntity());
>>>                    return data.getUid();
>>>                }
>>>            }, true);
>>> 
>>> 
>>> This being said, you will also need to  take care of the various locking/concurrent
access depending on the technology you use to store your mails (database, file, jcr,...).
>>> 
>>> I don't have a Yes/No answer, it must be looked and analysed in context.
>>> 
>>> Happy to further discuss,
>>> 
>>> Eric
>>> 
>>> 
>>> 
>>> On 06/06/2012 02:24 PM, Timothy Prepscius wrote:
>>>> Ok, I'm sorry to spam about this question, but I would really like a response.
>>>> I am making assumptions, which may be incorrect.
>>>> Because of maven it is difficult for me to get a full source tree to read.
>>>> 
>>>> 
>>>> 1.  Does james block on a single connection?
>>>> (I assume no, because this would be just be crazy)
>>>> 
>>>> 2.  Does james block on a transaction within a connection, meaning server
X talks to you on a single connection, transmits M1, M2, M3.
>>>> Does james receive M1, queue a transaction in a different thread, and continue
on to M2, -OR- does it receive M1, block on transaction processor, and then continue onto
M2?
>>>> 
>>>> (I hope doesn't block, if so I should implement a thread pool for the mailbox-write.)
>>>> 
>>>> 
>>>> Thanks,
>>>> 
>>>> -tim
>>>> 
>>>> On Jun 4, 2012, at 9:28 AM, Timothy Prepscius wrote:
>>>> 
>>>>> 
>>>>> I got rid of the NPE on java7 instead of switching to java6.  So no problems
there anymore.
>>>>> I will investigate how to do the jira in a minute.
>>>>> 
>>>>> 
>>>>> I have (perhaps) only one more question:
>>>>> 
>>>>> Should I implement the Mailbox as a multi-threaded non blocking pool?
>>>>> (Does James block the entire server on a Mailbox write?)
>>>>> 
>>>>> Or should I implement the Mailbox as a single-threaded blocking write
mechanism?
>>>>> (Is there somewhere in James a thread pool handling requests from the
smtp socket?)
>>>>> 
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> -tim
>>>>> 
>>>>> On Jun 4, 2012, at 4:49 AM, Eric Charles wrote:
>>>>> 
>>>>>> Hi Timothy,
>>>>>> 
>>>>>> 1.- I have committed your patch [1], thx for this.
>>>>>> 2.- Do you still have NPE with JDK6. If yes, can you post here the
full stacktrace?
>>>>>> 3.- Please open a JIRA [2] for your proposal for getFullContent()
and explain there in a few words why it would be more optimal.
>>>>>> 
>>>>>> Thx, Eric
>>>>>> 
>>>>>> [1] http://svn.apache.org/viewvc?rev=1345862&view=rev
>>>>>> [2] https://issues.apache.org/jira/browse/MAILBOX
>>>>>> 
>>>>>> 
>>>>>> On 06/04/2012 05:00 AM, Timothy Prepscius wrote:
>>>>>>> 
>>>>>>> On Jun 3, 2012, at 10:59 PM, Timothy Prepscius wrote:
>>>>>>> 
>>>>>>>> Sorry, hit the wrong button I guess.
>>>>>>>> 
>>>>>>>> On Jun 3, 2012, at 10:40 PM, Eric Charles wrote:
>>>>>>>> 
>>>>>>>>> Hi Timothy,
>>>>>>>>> Can you post to the mailing list so everyone can comment
on this?
>>>>>>>>> Thx,  Eric
>>>>>>>>> 
>>>>>>>>> On 06/03/2012 07:23 PM, Timothy Prepscius wrote:
>>>>>>>>>> hey, could I also offer one code change, which will
not conflict with anything anywhere (I think) ?
>>>>>>>>>> 
>>>>>>>>>> It optimizes the getFullContent for the SimpleMessage
>>>>>>>>>> 
>>>>>>>>>> ---
>>>>>>>>>> 
>>>>>>>>>> tprepscius$ svn diff
>>>>>>>>>> Index: src/main/java/org/apache/james/mailbox/store/mail/model/impl/SimpleMessage.java
>>>>>>>>>> ===================================================================
>>>>>>>>>> --- src/main/java/org/apache/james/mailbox/store/mail/model/impl/SimpleMessage.java
(revision 1345622)
>>>>>>>>>> +++ src/main/java/org/apache/james/mailbox/store/mail/model/impl/SimpleMessage.java
(working copy)
>>>>>>>>>> @@ -96,6 +96,10 @@
>>>>>>>>>>        }
>>>>>>>>>>    }
>>>>>>>>>> 
>>>>>>>>>> +    @Override
>>>>>>>>>> +    public InputStream getFullContent() throws IOException
{
>>>>>>>>>> +    	return content.newStream(0, -1);
>>>>>>>>>> +    }
>>>>>>>>>> 
>>>>>>>>>>    public Date getInternalDate() {
>>>>>>>>>>        return internalDate;
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Jun 3, 2012, at 1:14 PM, Timothy Prepscius wrote:
>>>>>>>>>> 
>>>>>>>>>>> hmm, I just added httpclient-4.0.2.jar to the
class path and got the same error.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> a second ago I unzipped the httpclient jar
>>>>>>>>>>> removed the entire META-INF/* dir tree
>>>>>>>>>>> and zipped it up again manually and now it works.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> hmmm
>>>>>>>>>>> 
>>>>>>>>>>> not sure what that means :-)
>>>>>>>>>>> 
>>>>>>>>>>> -tim
>>>>>>>>>>> 
>>>>>>>>>>> On Jun 3, 2012, at 1:03 PM, Timothy Prepscius
wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> vanilla is fine, no prob
>>>>>>>>>>>> 
>>>>>>>>>>>> it is indeed 1.7
>>>>>>>>>>>> 
>>>>>>>>>>>> tprepscius@blue:~/Projects/James/apache-james/bin$
java -version
>>>>>>>>>>>> java version "1.7.0_03"
>>>>>>>>>>>> OpenJDK Runtime Environment (IcedTea7 2.1.1pre)
(7~u3-2.1.1~pre1-1ubuntu3)
>>>>>>>>>>>> OpenJDK Client VM (build 22.0-b10, mixed
mode, sharing)
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On Jun 3, 2012, at 12:47 PM, Eric Charles
wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Thx Timothy, I will commit your patch
in trunk tomorrow.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> For NPE, do you run fine vanilla james?
We had such NPE when running with jdk7, although it was running fine with JDK6.
>>>>>>>>>>>>> Thx, Eric
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On 06/03/2012 03:28 PM, Timothy Prepscius
wrote:
>>>>>>>>>>>>>> yah:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>     <constructor-arg index="0"
ref="memory-sessionMapperFactory"/>
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Hey- I have a question:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> So I used another mail package to
test my ideas, called, JES-2, and now I'm moving to James for production.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I want to create a new james-mailbox.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I struggled for many many hours doing
this and hit the same road block over and over.
>>>>>>>>>>>>>> Unfortunately it is a configuration
problem, not a coding problem, so it becomes an indeterminate time problem- which isn't fun
;-)
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Here is what I did:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 1.  tar/untar clone the james-mailbox/memory-mailbox
to james-mailbox/custom-mailbox
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 2.  rename the java packages and
class names to xxx.customer.CustomerBlahBlahBlah.
>>>>>>>>>>>>>> (things compile and have different
names/packages than the memory)
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 3.  cloned the spring-mailbox-memory.xml
to spring-mailbox-custom.xml, changed names inside to reflect names of #2
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 4.  modified the spring-mailbox.xml
to include this new spring-mailbox-custom.xml
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> ---------
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 5.  mvn'd the code, shut down the
mail server, transferred the jars for mailbox-custom and the modified mailbox-spring.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> (at other points I was transferring
everything I could find, but it made no difference)
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 6.  modified the run.sh to include
the jar in the class path.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 7. ran sudo ./run.sh
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> ----------
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Error is something like:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> a NullPointerException when parsing
XML for spring-server.xml.
>>>>>>>>>>>>>> with a huge stack trace which doesn't
yield too much interesting (to my eyes) except that I know it is init-ing.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I can't remember where that XML file
is.  I found it yesterday, but now I've forgotten where it is.
>>>>>>>>>>>>>> Anyhow, when I found it I noticed
that it had an import to the spring-mailbox.xml…
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> But I've unzipped those jars I transferred,
and the spring-mailbox.xml seems to be there.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> ----
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Any thoughts?
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Is the guy/girl who wrote the hbase
on this list?
>>>>>>>>>>>>>> Do you have any notes on the process
of integrating that mailbox into james?
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> (just noticed my writing is incredibly
bad this morning, couldn't sleep, apologize if I'm writing like a 2 year old.)
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Jun 3, 2012, at 3:17 AM, Eric
Charles wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Hi Tim,
>>>>>>>>>>>>>>> Thx for the feedback.
>>>>>>>>>>>>>>> Any patch to correct this?
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Thx, Eric
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On 06/03/2012 08:06 AM, Timothy
Prepscius wrote:
>>>>>>>>>>>>>>>> http://svn.apache.org/repos/asf/james/mailbox/trunk/spring/src/main/resources/META-INF/org/apache/james/spring-mailbox-memory.xml
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>     <constructor-arg index="0"
ref="maildir-sessionMapperFactory"/>
>>>>>>>>>>>>>>>> might be wrong.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> -tim
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> eric | http://about.echarles.net
| @echarles
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>>>>> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
>>>>>>>>>>>>>>> For additional commands, e-mail:
server-user-help@james.apache.org
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>>>> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
>>>>>>>>>>>>>> For additional commands, e-mail:
server-user-help@james.apache.org
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> --
>>>>>>>>>>>>> eric | http://about.echarles.net | @echarles
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> eric | http://about.echarles.net | @echarles
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
>>>>>>> For additional commands, e-mail: server-user-help@james.apache.org
>>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> eric | http://about.echarles.net | @echarles
>>>>>> 
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
>>>>>> For additional commands, e-mail: server-user-help@james.apache.org
>>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
>>>> For additional commands, e-mail: server-user-help@james.apache.org
>>>> 
>>> 
>>> --
>>> eric | http://about.echarles.net | @echarles
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
>>> For additional commands, e-mail: server-user-help@james.apache.org
>>> 
>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
>> For additional commands, e-mail: server-user-help@james.apache.org
>> 
> 
> -- 
> eric | http://about.echarles.net | @echarles
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
> For additional commands, e-mail: server-user-help@james.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org


Mime
View raw message