Return-Path: X-Original-To: apmail-james-server-user-archive@www.apache.org Delivered-To: apmail-james-server-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 138369CAE for ; Thu, 7 Jun 2012 21:32:45 +0000 (UTC) Received: (qmail 30218 invoked by uid 500); 7 Jun 2012 21:32:43 -0000 Delivered-To: apmail-james-server-user-archive@james.apache.org Received: (qmail 30170 invoked by uid 500); 7 Jun 2012 21:32:43 -0000 Mailing-List: contact server-user-help@james.apache.org; run by ezmlm Precedence: bulk List-Unsubscribe: List-Help: List-Post: List-Id: "James Users List" Reply-To: "James Users List" Delivered-To: mailing list server-user@james.apache.org Received: (qmail 30086 invoked by uid 99); 7 Jun 2012 21:32:43 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Jun 2012 21:32:43 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of timprepscius@gmail.com designates 209.85.212.46 as permitted sender) Received: from [209.85.212.46] (HELO mail-vb0-f46.google.com) (209.85.212.46) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Jun 2012 21:32:37 +0000 Received: by vbbff1 with SMTP id ff1so742100vbb.5 for ; Thu, 07 Jun 2012 14:32:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to:x-mailer; bh=yDwa7mcxikzQNZu0Ka0k3UatuqAfH6DnP/2rEqYZM64=; b=eK9FgFBpZ9SCmnifh9wcHCYfqKNzM5OYjIJwmaSrcJlKjGaJ+eqCbqMtS5fKWN8iQY yyiqHHzQGONpXMIJW9Ab5hMaK6KrqBiIEawi+qB2Rf3l/93r/Xpblo29DfqOYEkRiSjs vPiMXbQLdphSJeYdbW3njBtzuslAHaJQcDxSzwi9rRxDJExwbkyu9RMvI7lMAKDDjxHa ge6WHAx1lblHTZtEHJEc6XOMcUUTKJ6csl4ds7Y3yuGTtdolzWExZirP+YxMV2yarRlv 8WrIqyg04S4U22NOXiVOsONGTV9lGvHYwM3CogcZD/s2xO3dzd2D2/Dq1hhh6V/t8wvj LhwA== Received: by 10.52.36.116 with SMTP id p20mr3154276vdj.129.1339104736674; Thu, 07 Jun 2012 14:32:16 -0700 (PDT) Received: from [192.168.1.109] (cpe-98-14-81-235.nyc.res.rr.com. [98.14.81.235]) by mx.google.com with ESMTPS id cr4sm6296758vdb.7.2012.06.07.14.32.15 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 07 Jun 2012 14:32:15 -0700 (PDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Apple Message framework v1278) Subject: Re: inmemory mailbox wrong configuration From: Timothy Prepscius In-Reply-To: <4FD0BC1C.7060203@apache.org> Date: Thu, 7 Jun 2012 17:32:13 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: References: <8B2614FA-88F4-44FE-A71C-666AEEACF5C2@gmail.com> <4FCB0F70.8070305@apache.org> <0E46E8A4-62C7-4448-AE0F-2F71E599B13A@gmail.com> <4FCB952C.9040306@apache.org> <16E446EA-1407-4DC0-9681-E496E0E2FB64@gmail.com> <28FA977D-14DA-4DD4-98A7-78C094172ADE@gmail.com> <4FCC2023.90200@u-mangate.com> <27AA17ED-8ACE-4912-9FE7-939B875BEB09@gmail.com> <4FCC76A3.2030707@apache.org> <4FCF506A.1010303@apache.org> <7743D2C4-3310-450C-A9C7-002C1743ED4A@gmail.com> <4FD0BC1C.7060203@apache.org> To: "James Users List" X-Mailer: Apple Mail (2.1278) X-Virus-Checked: Checked by ClamAV on apache.org Awesome, thanks! On Jun 7, 2012, at 10:35 AM, Eric Charles wrote: > Hi Tim, >=20 > 1. Yes, > 2. Yes >=20 > 3. StoreMessageManager in mailbox-store has the boiler-plate for you. = If your implementation blocks (lets say in a YourImplMessageMapper = method), the thread will block. There are 20 threads defined by default, = so if the first thread blocks, the other one are still there to consume = the mails from the AMQ queue. Of course, if all the 20 threads block, = spooling is stopped. Btw, there is no time-out on a thread blocking (But = you should not implement your mailbox storage with a blocking mechanism = - I suppose you are asking to anticipate a database, network connection = not being released and being blocked?) >=20 > The 20 is for the AMQ consuming. For the remote = delivery, it works completely differently, based on the = RemoteDeliveryMailet where you can define there the number of concurrent = threads running for remote delivery. >=20 > Thx for your interest :) >=20 > Eric >=20 > On 06/07/2012 03:16 PM, Timothy Prepscius wrote: >> Thanks for this! >>=20 >> I am translating this in my head this way: >>=20 >> 1. Mail Receiver receives a mail and queues it in ActiveMQ. >> 2. Mail Processor has multiple consumer threads watching the ActiveM = queue. >> 3. If one mailbox write operation blocks for 5 minutes, the other = consumer threads of the Mail Processor will continue. >>=20 >> Is this roughly correct? >>=20 >> Is this relevant or is the spool outgoing: (below xml of = mailetcontainer-template.conf ) >>=20 >> >>=20 >> >>=20 >> >> >> postmaster@localhost >> >>=20 >> >> >> 20 >> >>=20 >>=20 >>=20 >> Thanks again for your time! >>=20 >> -tim >>=20 >> On Jun 6, 2012, at 8:43 AM, Eric Charles wrote: >>=20 >>> Hi Tim, >>>=20 >>> 1. James doesn't block on connection (but don't forget the = intermediate spooling via ActiveMQ). >>>=20 >>> 2. To answer this question, you need to realize James reads the = mails via a Camel context, getting them from ActiveMQ. >>>=20 >>> Sorry if these response are too short, I was just wanting to take = more time to provide you with a complete picture, but I understand you = need to go forward. >>>=20 >>> What you can do it put a break point in the append method, and send = mails to James (running in your IDE). You will see the various threads = and also that it is non-blocking.... exception made of the various lock = with do via the JVMMailboxPathLocker, see for example this snippet from = StoreMessageManager >>>=20 >>> final Message message =3D = createMessage(internalDate, size, bodyStartOctet, contentIn, flags, = propertyBuilder); >>> return locker.executeWithLock(mailboxSession, new = StoreMailboxPath(getMailboxEntity()), new = MailboxPathLocker.LockAwareExecution() { >>>=20 >>> @Override >>> public Long execute() throws MailboxException { >>> MessageMetaData data =3D = appendMessageToStore(message, mailboxSession); >>>=20 >>> SortedMap uids =3D new = TreeMap(); >>> uids.put(data.getUid(), data); >>> dispatcher.added(mailboxSession, uids, = getMailboxEntity()); >>> return data.getUid(); >>> } >>> }, true); >>>=20 >>>=20 >>> This being said, you will also need to take care of the various = locking/concurrent access depending on the technology you use to store = your mails (database, file, jcr,...). >>>=20 >>> I don't have a Yes/No answer, it must be looked and analysed in = context. >>>=20 >>> Happy to further discuss, >>>=20 >>> Eric >>>=20 >>>=20 >>>=20 >>> On 06/06/2012 02:24 PM, Timothy Prepscius wrote: >>>> Ok, I'm sorry to spam about this question, but I would really like = a response. >>>> I am making assumptions, which may be incorrect. >>>> Because of maven it is difficult for me to get a full source tree = to read. >>>>=20 >>>>=20 >>>> 1. Does james block on a single connection? >>>> (I assume no, because this would be just be crazy) >>>>=20 >>>> 2. Does james block on a transaction within a connection, meaning = server X talks to you on a single connection, transmits M1, M2, M3. >>>> Does james receive M1, queue a transaction in a different thread, = and continue on to M2, -OR- does it receive M1, block on transaction = processor, and then continue onto M2? >>>>=20 >>>> (I hope doesn't block, if so I should implement a thread pool for = the mailbox-write.) >>>>=20 >>>>=20 >>>> Thanks, >>>>=20 >>>> -tim >>>>=20 >>>> On Jun 4, 2012, at 9:28 AM, Timothy Prepscius wrote: >>>>=20 >>>>>=20 >>>>> I got rid of the NPE on java7 instead of switching to java6. So = no problems there anymore. >>>>> I will investigate how to do the jira in a minute. >>>>>=20 >>>>>=20 >>>>> I have (perhaps) only one more question: >>>>>=20 >>>>> Should I implement the Mailbox as a multi-threaded non blocking = pool? >>>>> (Does James block the entire server on a Mailbox write?) >>>>>=20 >>>>> Or should I implement the Mailbox as a single-threaded blocking = write mechanism? >>>>> (Is there somewhere in James a thread pool handling requests from = the smtp socket?) >>>>>=20 >>>>>=20 >>>>> Thanks, >>>>>=20 >>>>> -tim >>>>>=20 >>>>> On Jun 4, 2012, at 4:49 AM, Eric Charles wrote: >>>>>=20 >>>>>> Hi Timothy, >>>>>>=20 >>>>>> 1.- I have committed your patch [1], thx for this. >>>>>> 2.- Do you still have NPE with JDK6. If yes, can you post here = the full stacktrace? >>>>>> 3.- Please open a JIRA [2] for your proposal for getFullContent() = and explain there in a few words why it would be more optimal. >>>>>>=20 >>>>>> Thx, Eric >>>>>>=20 >>>>>> [1] http://svn.apache.org/viewvc?rev=3D1345862&view=3Drev >>>>>> [2] https://issues.apache.org/jira/browse/MAILBOX >>>>>>=20 >>>>>>=20 >>>>>> On 06/04/2012 05:00 AM, Timothy Prepscius wrote: >>>>>>>=20 >>>>>>> On Jun 3, 2012, at 10:59 PM, Timothy Prepscius wrote: >>>>>>>=20 >>>>>>>> Sorry, hit the wrong button I guess. >>>>>>>>=20 >>>>>>>> On Jun 3, 2012, at 10:40 PM, Eric Charles wrote: >>>>>>>>=20 >>>>>>>>> Hi Timothy, >>>>>>>>> Can you post to the mailing list so everyone can comment on = this? >>>>>>>>> Thx, Eric >>>>>>>>>=20 >>>>>>>>> On 06/03/2012 07:23 PM, Timothy Prepscius wrote: >>>>>>>>>> hey, could I also offer one code change, which will not = conflict with anything anywhere (I think) ? >>>>>>>>>>=20 >>>>>>>>>> It optimizes the getFullContent for the SimpleMessage >>>>>>>>>>=20 >>>>>>>>>> --- >>>>>>>>>>=20 >>>>>>>>>> tprepscius$ svn diff >>>>>>>>>> Index: = src/main/java/org/apache/james/mailbox/store/mail/model/impl/SimpleMessage= .java >>>>>>>>>> = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>>>>>>>>> --- = src/main/java/org/apache/james/mailbox/store/mail/model/impl/SimpleMessage= .java (revision 1345622) >>>>>>>>>> +++ = src/main/java/org/apache/james/mailbox/store/mail/model/impl/SimpleMessage= .java (working copy) >>>>>>>>>> @@ -96,6 +96,10 @@ >>>>>>>>>> } >>>>>>>>>> } >>>>>>>>>>=20 >>>>>>>>>> + @Override >>>>>>>>>> + public InputStream getFullContent() throws IOException { >>>>>>>>>> + return content.newStream(0, -1); >>>>>>>>>> + } >>>>>>>>>>=20 >>>>>>>>>> public Date getInternalDate() { >>>>>>>>>> return internalDate; >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> On Jun 3, 2012, at 1:14 PM, Timothy Prepscius wrote: >>>>>>>>>>=20 >>>>>>>>>>> hmm, I just added httpclient-4.0.2.jar to the class path and = got the same error. >>>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>>>>>> a second ago I unzipped the httpclient jar >>>>>>>>>>> removed the entire META-INF/* dir tree >>>>>>>>>>> and zipped it up again manually and now it works. >>>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>>>>>> hmmm >>>>>>>>>>>=20 >>>>>>>>>>> not sure what that means :-) >>>>>>>>>>>=20 >>>>>>>>>>> -tim >>>>>>>>>>>=20 >>>>>>>>>>> On Jun 3, 2012, at 1:03 PM, Timothy Prepscius wrote: >>>>>>>>>>>=20 >>>>>>>>>>>> vanilla is fine, no prob >>>>>>>>>>>>=20 >>>>>>>>>>>> it is indeed 1.7 >>>>>>>>>>>>=20 >>>>>>>>>>>> tprepscius@blue:~/Projects/James/apache-james/bin$ java = -version >>>>>>>>>>>> java version "1.7.0_03" >>>>>>>>>>>> OpenJDK Runtime Environment (IcedTea7 2.1.1pre) = (7~u3-2.1.1~pre1-1ubuntu3) >>>>>>>>>>>> OpenJDK Client VM (build 22.0-b10, mixed mode, sharing) >>>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>> On Jun 3, 2012, at 12:47 PM, Eric Charles wrote: >>>>>>>>>>>>=20 >>>>>>>>>>>>> Thx Timothy, I will commit your patch in trunk tomorrow. >>>>>>>>>>>>>=20 >>>>>>>>>>>>> For NPE, do you run fine vanilla james? We had such NPE = when running with jdk7, although it was running fine with JDK6. >>>>>>>>>>>>> Thx, Eric >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> On 06/03/2012 03:28 PM, Timothy Prepscius wrote: >>>>>>>>>>>>>> yah: >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Hey- I have a question: >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> So I used another mail package to test my ideas, called, = JES-2, and now I'm moving to James for production. >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> I want to create a new james-mailbox. >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> I struggled for many many hours doing this and hit the = same road block over and over. >>>>>>>>>>>>>> Unfortunately it is a configuration problem, not a coding = problem, so it becomes an indeterminate time problem- which isn't fun = ;-) >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Here is what I did: >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> 1. tar/untar clone the james-mailbox/memory-mailbox to = james-mailbox/custom-mailbox >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> 2. rename the java packages and class names to = xxx.customer.CustomerBlahBlahBlah. >>>>>>>>>>>>>> (things compile and have different names/packages than = the memory) >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> 3. cloned the spring-mailbox-memory.xml to = spring-mailbox-custom.xml, changed names inside to reflect names of #2 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> 4. modified the spring-mailbox.xml to include this new = spring-mailbox-custom.xml >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> --------- >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> 5. mvn'd the code, shut down the mail server, = transferred the jars for mailbox-custom and the modified mailbox-spring. >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> (at other points I was transferring everything I could = find, but it made no difference) >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> 6. modified the run.sh to include the jar in the class = path. >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> 7. ran sudo ./run.sh >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> ---------- >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Error is something like: >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> a NullPointerException when parsing XML for = spring-server.xml. >>>>>>>>>>>>>> with a huge stack trace which doesn't yield too much = interesting (to my eyes) except that I know it is init-ing. >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> I can't remember where that XML file is. I found it = yesterday, but now I've forgotten where it is. >>>>>>>>>>>>>> Anyhow, when I found it I noticed that it had an import = to the spring-mailbox.xml=85 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> But I've unzipped those jars I transferred, and the = spring-mailbox.xml seems to be there. >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> ---- >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Any thoughts? >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Is the guy/girl who wrote the hbase on this list? >>>>>>>>>>>>>> Do you have any notes on the process of integrating that = mailbox into james? >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> (just noticed my writing is incredibly bad this morning, = couldn't sleep, apologize if I'm writing like a 2 year old.) >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> On Jun 3, 2012, at 3:17 AM, Eric Charles wrote: >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>> Hi Tim, >>>>>>>>>>>>>>> Thx for the feedback. >>>>>>>>>>>>>>> Any patch to correct this? >>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>> Thx, Eric >>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>> On 06/03/2012 08:06 AM, Timothy Prepscius wrote: >>>>>>>>>>>>>>>> = http://svn.apache.org/repos/asf/james/mailbox/trunk/spring/src/main/resour= ces/META-INF/org/apache/james/spring-mailbox-memory.xml >>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> might be wrong. >>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>> -tim >>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> eric | http://about.echarles.net | @echarles >>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>> = --------------------------------------------------------------------- >>>>>>>>>>>>>>> To unsubscribe, e-mail: = server-user-unsubscribe@james.apache.org >>>>>>>>>>>>>>> For additional commands, e-mail: = server-user-help@james.apache.org >>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> = --------------------------------------------------------------------- >>>>>>>>>>>>>> To unsubscribe, e-mail: = server-user-unsubscribe@james.apache.org >>>>>>>>>>>>>> For additional commands, e-mail: = server-user-help@james.apache.org >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> -- >>>>>>>>>>>>> eric | http://about.echarles.net | @echarles >>>>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> -- >>>>>>>>> eric | http://about.echarles.net | @echarles >>>>>>>>=20 >>>>>>>=20 >>>>>>>=20 >>>>>>> = --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org >>>>>>> For additional commands, e-mail: = server-user-help@james.apache.org >>>>>>>=20 >>>>>>=20 >>>>>> -- >>>>>> eric | http://about.echarles.net | @echarles >>>>>>=20 >>>>>> = --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org >>>>>> For additional commands, e-mail: = server-user-help@james.apache.org >>>>>>=20 >>>>>=20 >>>>=20 >>>>=20 >>>> = --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org >>>> For additional commands, e-mail: server-user-help@james.apache.org >>>>=20 >>>=20 >>> -- >>> eric | http://about.echarles.net | @echarles >>>=20 >>> = --------------------------------------------------------------------- >>> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org >>> For additional commands, e-mail: server-user-help@james.apache.org >>>=20 >>=20 >>=20 >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org >> For additional commands, e-mail: server-user-help@james.apache.org >>=20 >=20 > --=20 > eric | http://about.echarles.net | @echarles >=20 > --------------------------------------------------------------------- > To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org > For additional commands, e-mail: server-user-help@james.apache.org >=20 --------------------------------------------------------------------- To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org For additional commands, e-mail: server-user-help@james.apache.org