james-server-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothy Prepscius <timprepsc...@gmail.com>
Subject Re: inmemory mailbox wrong configuration
Date Thu, 07 Jun 2012 13:16:37 GMT
Thanks for this!

I am translating this in my head this way:

1. Mail Receiver receives a mail and queues it in ActiveMQ.
2. Mail Processor has multiple consumer threads watching the ActiveM queue.
3. If one mailbox  write operation blocks for 5 minutes, the other consumer threads of the
Mail Processor will continue.

Is this roughly correct?

Is this relevant or is the spool outgoing: (below xml of mailetcontainer-template.conf )

<!-- This block is responsible for processing messages on the spool. -->

<mailetcontainer enableJmx="true">
 
    <!-- MailAddress used for PostMaster -->
	<context>
	    <postmaster>postmaster@localhost</postmaster>
	</context>

    <!-- Number of spool threads -->
    <spooler>
	  <threads>20</threads>
	</spooler>
      


Thanks again for your time!

-tim

On Jun 6, 2012, at 8:43 AM, Eric Charles wrote:

> Hi Tim,
> 
> 1. James doesn't block on connection (but don't forget the intermediate spooling via
ActiveMQ).
> 
> 2. To answer this question, you need to realize James reads the mails via a Camel context,
getting them from ActiveMQ.
> 
> Sorry if these response are too short, I was just wanting to take more time to provide
you with a complete picture, but I understand you need to go forward.
> 
> What you can do it put a break point in the append method, and send mails to James (running
in your IDE). You will see the various threads and also that it is non-blocking.... exception
made of the various lock with do via the JVMMailboxPathLocker, see for example this snippet
from StoreMessageManager
> 
>            final Message<Id> message = createMessage(internalDate, size, bodyStartOctet,
contentIn, flags, propertyBuilder);
>            return locker.executeWithLock(mailboxSession, new StoreMailboxPath<Id>(getMailboxEntity()),
new MailboxPathLocker.LockAwareExecution<Long>() {
> 
>                @Override
>                public Long execute() throws MailboxException {
>                    MessageMetaData data = appendMessageToStore(message, mailboxSession);
> 
>                    SortedMap<Long, MessageMetaData> uids = new TreeMap<Long,
MessageMetaData>();
>                    uids.put(data.getUid(), data);
>                    dispatcher.added(mailboxSession, uids, getMailboxEntity());
>                    return data.getUid();
>                }
>            }, true);
> 
> 
> This being said, you will also need to  take care of the various locking/concurrent access
depending on the technology you use to store your mails (database, file, jcr,...).
> 
> I don't have a Yes/No answer, it must be looked and analysed in context.
> 
> Happy to further discuss,
> 
> Eric
> 
> 
> 
> On 06/06/2012 02:24 PM, Timothy Prepscius wrote:
>> Ok, I'm sorry to spam about this question, but I would really like a response.
>> I am making assumptions, which may be incorrect.
>> Because of maven it is difficult for me to get a full source tree to read.
>> 
>> 
>> 1.  Does james block on a single connection?
>> (I assume no, because this would be just be crazy)
>> 
>> 2.  Does james block on a transaction within a connection, meaning server X talks
to you on a single connection, transmits M1, M2, M3.
>> Does james receive M1, queue a transaction in a different thread, and continue on
to M2, -OR- does it receive M1, block on transaction processor, and then continue onto M2?
>> 
>> (I hope doesn't block, if so I should implement a thread pool for the mailbox-write.)
>> 
>> 
>> Thanks,
>> 
>> -tim
>> 
>> On Jun 4, 2012, at 9:28 AM, Timothy Prepscius wrote:
>> 
>>> 
>>> I got rid of the NPE on java7 instead of switching to java6.  So no problems
there anymore.
>>> I will investigate how to do the jira in a minute.
>>> 
>>> 
>>> I have (perhaps) only one more question:
>>> 
>>> Should I implement the Mailbox as a multi-threaded non blocking pool?
>>> (Does James block the entire server on a Mailbox write?)
>>> 
>>> Or should I implement the Mailbox as a single-threaded blocking write mechanism?
>>> (Is there somewhere in James a thread pool handling requests from the smtp socket?)
>>> 
>>> 
>>> Thanks,
>>> 
>>> -tim
>>> 
>>> On Jun 4, 2012, at 4:49 AM, Eric Charles wrote:
>>> 
>>>> Hi Timothy,
>>>> 
>>>> 1.- I have committed your patch [1], thx for this.
>>>> 2.- Do you still have NPE with JDK6. If yes, can you post here the full stacktrace?
>>>> 3.- Please open a JIRA [2] for your proposal for getFullContent() and explain
there in a few words why it would be more optimal.
>>>> 
>>>> Thx, Eric
>>>> 
>>>> [1] http://svn.apache.org/viewvc?rev=1345862&view=rev
>>>> [2] https://issues.apache.org/jira/browse/MAILBOX
>>>> 
>>>> 
>>>> On 06/04/2012 05:00 AM, Timothy Prepscius wrote:
>>>>> 
>>>>> On Jun 3, 2012, at 10:59 PM, Timothy Prepscius wrote:
>>>>> 
>>>>>> Sorry, hit the wrong button I guess.
>>>>>> 
>>>>>> On Jun 3, 2012, at 10:40 PM, Eric Charles wrote:
>>>>>> 
>>>>>>> Hi Timothy,
>>>>>>> Can you post to the mailing list so everyone can comment on this?
>>>>>>> Thx,  Eric
>>>>>>> 
>>>>>>> On 06/03/2012 07:23 PM, Timothy Prepscius wrote:
>>>>>>>> hey, could I also offer one code change, which will not conflict
with anything anywhere (I think) ?
>>>>>>>> 
>>>>>>>> It optimizes the getFullContent for the SimpleMessage
>>>>>>>> 
>>>>>>>> ---
>>>>>>>> 
>>>>>>>> tprepscius$ svn diff
>>>>>>>> Index: src/main/java/org/apache/james/mailbox/store/mail/model/impl/SimpleMessage.java
>>>>>>>> ===================================================================
>>>>>>>> --- src/main/java/org/apache/james/mailbox/store/mail/model/impl/SimpleMessage.java
(revision 1345622)
>>>>>>>> +++ src/main/java/org/apache/james/mailbox/store/mail/model/impl/SimpleMessage.java
(working copy)
>>>>>>>> @@ -96,6 +96,10 @@
>>>>>>>>        }
>>>>>>>>    }
>>>>>>>> 
>>>>>>>> +    @Override
>>>>>>>> +    public InputStream getFullContent() throws IOException
{
>>>>>>>> +    	return content.newStream(0, -1);
>>>>>>>> +    }
>>>>>>>> 
>>>>>>>>    public Date getInternalDate() {
>>>>>>>>        return internalDate;
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Jun 3, 2012, at 1:14 PM, Timothy Prepscius wrote:
>>>>>>>> 
>>>>>>>>> hmm, I just added httpclient-4.0.2.jar to the class path
and got the same error.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> a second ago I unzipped the httpclient jar
>>>>>>>>> removed the entire META-INF/* dir tree
>>>>>>>>> and zipped it up again manually and now it works.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> hmmm
>>>>>>>>> 
>>>>>>>>> not sure what that means :-)
>>>>>>>>> 
>>>>>>>>> -tim
>>>>>>>>> 
>>>>>>>>> On Jun 3, 2012, at 1:03 PM, Timothy Prepscius wrote:
>>>>>>>>> 
>>>>>>>>>> vanilla is fine, no prob
>>>>>>>>>> 
>>>>>>>>>> it is indeed 1.7
>>>>>>>>>> 
>>>>>>>>>> tprepscius@blue:~/Projects/James/apache-james/bin$
java -version
>>>>>>>>>> java version "1.7.0_03"
>>>>>>>>>> OpenJDK Runtime Environment (IcedTea7 2.1.1pre) (7~u3-2.1.1~pre1-1ubuntu3)
>>>>>>>>>> OpenJDK Client VM (build 22.0-b10, mixed mode, sharing)
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Jun 3, 2012, at 12:47 PM, Eric Charles wrote:
>>>>>>>>>> 
>>>>>>>>>>> Thx Timothy, I will commit your patch in trunk
tomorrow.
>>>>>>>>>>> 
>>>>>>>>>>> For NPE, do you run fine vanilla james? We had
such NPE when running with jdk7, although it was running fine with JDK6.
>>>>>>>>>>> Thx, Eric
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On 06/03/2012 03:28 PM, Timothy Prepscius wrote:
>>>>>>>>>>>> yah:
>>>>>>>>>>>> 
>>>>>>>>>>>>     <constructor-arg index="0" ref="memory-sessionMapperFactory"/>
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> Hey- I have a question:
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> So I used another mail package to test my
ideas, called, JES-2, and now I'm moving to James for production.
>>>>>>>>>>>> 
>>>>>>>>>>>> I want to create a new james-mailbox.
>>>>>>>>>>>> 
>>>>>>>>>>>> I struggled for many many hours doing this
and hit the same road block over and over.
>>>>>>>>>>>> Unfortunately it is a configuration problem,
not a coding problem, so it becomes an indeterminate time problem- which isn't fun ;-)
>>>>>>>>>>>> 
>>>>>>>>>>>> Here is what I did:
>>>>>>>>>>>> 
>>>>>>>>>>>> 1.  tar/untar clone the james-mailbox/memory-mailbox
to james-mailbox/custom-mailbox
>>>>>>>>>>>> 
>>>>>>>>>>>> 2.  rename the java packages and class names
to xxx.customer.CustomerBlahBlahBlah.
>>>>>>>>>>>> (things compile and have different names/packages
than the memory)
>>>>>>>>>>>> 
>>>>>>>>>>>> 3.  cloned the spring-mailbox-memory.xml
to spring-mailbox-custom.xml, changed names inside to reflect names of #2
>>>>>>>>>>>> 
>>>>>>>>>>>> 4.  modified the spring-mailbox.xml to include
this new spring-mailbox-custom.xml
>>>>>>>>>>>> 
>>>>>>>>>>>> ---------
>>>>>>>>>>>> 
>>>>>>>>>>>> 5.  mvn'd the code, shut down the mail server,
transferred the jars for mailbox-custom and the modified mailbox-spring.
>>>>>>>>>>>> 
>>>>>>>>>>>> (at other points I was transferring everything
I could find, but it made no difference)
>>>>>>>>>>>> 
>>>>>>>>>>>> 6.  modified the run.sh to include the jar
in the class path.
>>>>>>>>>>>> 
>>>>>>>>>>>> 7. ran sudo ./run.sh
>>>>>>>>>>>> 
>>>>>>>>>>>> ----------
>>>>>>>>>>>> 
>>>>>>>>>>>> Error is something like:
>>>>>>>>>>>> 
>>>>>>>>>>>> a NullPointerException when parsing XML for
spring-server.xml.
>>>>>>>>>>>> with a huge stack trace which doesn't yield
too much interesting (to my eyes) except that I know it is init-ing.
>>>>>>>>>>>> 
>>>>>>>>>>>> I can't remember where that XML file is.
 I found it yesterday, but now I've forgotten where it is.
>>>>>>>>>>>> Anyhow, when I found it I noticed that it
had an import to the spring-mailbox.xml…
>>>>>>>>>>>> 
>>>>>>>>>>>> But I've unzipped those jars I transferred,
and the spring-mailbox.xml seems to be there.
>>>>>>>>>>>> 
>>>>>>>>>>>> ----
>>>>>>>>>>>> 
>>>>>>>>>>>> Any thoughts?
>>>>>>>>>>>> 
>>>>>>>>>>>> Is the guy/girl who wrote the hbase on this
list?
>>>>>>>>>>>> Do you have any notes on the process of integrating
that mailbox into james?
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> (just noticed my writing is incredibly bad
this morning, couldn't sleep, apologize if I'm writing like a 2 year old.)
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On Jun 3, 2012, at 3:17 AM, Eric Charles
wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Hi Tim,
>>>>>>>>>>>>> Thx for the feedback.
>>>>>>>>>>>>> Any patch to correct this?
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thx, Eric
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On 06/03/2012 08:06 AM, Timothy Prepscius
wrote:
>>>>>>>>>>>>>> http://svn.apache.org/repos/asf/james/mailbox/trunk/spring/src/main/resources/META-INF/org/apache/james/spring-mailbox-memory.xml
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>     <constructor-arg index="0"
ref="maildir-sessionMapperFactory"/>
>>>>>>>>>>>>>> might be wrong.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> -tim
>>>>>>>>>>>>> 
>>>>>>>>>>>>> --
>>>>>>>>>>>>> eric | http://about.echarles.net | @echarles
>>>>>>>>>>>>> 
>>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>>> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
>>>>>>>>>>>>> For additional commands, e-mail: server-user-help@james.apache.org
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
>>>>>>>>>>>> For additional commands, e-mail: server-user-help@james.apache.org
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> --
>>>>>>>>>>> eric | http://about.echarles.net | @echarles
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> eric | http://about.echarles.net | @echarles
>>>>>> 
>>>>> 
>>>>> 
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
>>>>> For additional commands, e-mail: server-user-help@james.apache.org
>>>>> 
>>>> 
>>>> --
>>>> eric | http://about.echarles.net | @echarles
>>>> 
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
>>>> For additional commands, e-mail: server-user-help@james.apache.org
>>>> 
>>> 
>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
>> For additional commands, e-mail: server-user-help@james.apache.org
>> 
> 
> -- 
> eric | http://about.echarles.net | @echarles
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
> For additional commands, e-mail: server-user-help@james.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org


Mime
View raw message