From api-return-198-apmail-directory-api-archive=directory.apache.org@directory.apache.org Tue Aug 10 08:27:30 2010 Return-Path: Delivered-To: apmail-directory-api-archive@minotaur.apache.org Received: (qmail 74682 invoked from network); 10 Aug 2010 08:27:30 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 10 Aug 2010 08:27:30 -0000 Received: (qmail 72832 invoked by uid 500); 10 Aug 2010 08:27:30 -0000 Delivered-To: apmail-directory-api-archive@directory.apache.org Received: (qmail 72792 invoked by uid 500); 10 Aug 2010 08:27:29 -0000 Mailing-List: contact api-help@directory.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: api@directory.apache.org Delivered-To: mailing list api@directory.apache.org Received: (qmail 72784 invoked by uid 99); 10 Aug 2010 08:27:28 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 Aug 2010 08:27:28 +0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests=FREEMAIL_FROM,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of elecharny@gmail.com designates 74.125.82.178 as permitted sender) Received: from [74.125.82.178] (HELO mail-wy0-f178.google.com) (74.125.82.178) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 Aug 2010 08:27:22 +0000 Received: by wyf22 with SMTP id 22so608400wyf.37 for ; Tue, 10 Aug 2010 01:27:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:reply-to :user-agent:mime-version:to:subject:references:in-reply-to :content-type:content-transfer-encoding; bh=svVsxmMgy9fbstSWxNMBe94DKGXzwZKIa3MJ51bmv0s=; b=Hq6iYRxeIJs3gQPV4SThOUlakBUj+xU/j8itM7YAJW2QNS0xjn+SlIyuRcnOJ4xUKO iowqNlljeP8n6iR2VDdp0YbuIg7fgQzObV1XyYztCH7efe6u5l8th5SfrUydG0kyINUI 01BT7G7yA+TFOLHzb9H0SBb6zYN4CvyvGpj+M= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:reply-to:user-agent:mime-version:to:subject :references:in-reply-to:content-type:content-transfer-encoding; b=XaARsWMbTbiuBx7tdeyGcaQ6KwJk+n9a0rnHihd3aFSP2RpiFfe0BWu8agypFhu3tN ZtvNmthuJndK/B2dBkVCyZw+mN2vHnbXc9mOv1jwvBzrVB6ThczhFbHQBoUpNv4bXvHx 4E//g9LPcQg35MBpb/BMRKZDCDohQF/cjtLLU= Received: by 10.216.175.83 with SMTP id y61mr14917332wel.30.1281428820947; Tue, 10 Aug 2010 01:27:00 -0700 (PDT) Received: from emmanuel-lecharnys-MacBook-Pro.local ([78.192.106.184]) by mx.google.com with ESMTPS id h37sm3131244wej.23.2010.08.10.01.26.58 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 10 Aug 2010 01:26:59 -0700 (PDT) Message-ID: <4C610DFB.80403@gmail.com> Date: Tue, 10 Aug 2010 10:29:47 +0200 From: Emmanuel Lecharny Reply-To: elecharny@apache.org User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.7) Gecko/20100713 Thunderbird/3.1.1 MIME-Version: 1.0 To: api@directory.apache.org Subject: Re: DN and valueOf( "" ) method References: <4C4FF0D4.2040108@gmail.com> <4C500F87.10806@gmail.com> <4C600DAF.1030102@oracle.com> <4C601673.9030201@gmail.com> <4C6106BE.8030702@oracle.com> In-Reply-To: <4C6106BE.8030702@oracle.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 8/10/10 9:58 AM, Matthew Swift wrote: > [...] >> But are you going to have 16K threads anyway ? > > > Perhaps not today, but perhaps tomorrow. Here, we are entering into the plain IO vs NIO debate. There is no limit in the number of threads you can start on a JVM except the available memory, the JVM configuration (especially the -Xss size. It defaults to 1024, which is way to big if you are going to have 100K threads...) and obviously the OS you are running on. > > There are already very large CMT (core multi-threading) machines in > the pipeline from various vendors and I have heard and read > projections of 10K+ thread CMT machines in the next few years. Hence > the fork/join work going on in JDK7 at the moment and the push for > closures. > > Since many server apps size their thread pools based on the number of > available processors (core threads on CMT machines) then it is not > that unlikely in the next few years. > > I think that even today some app server environments run with tens of > thousands of threads. Most certainly... > > Basically, we have no control over how other people use our SDK so, > while I might not use 16K threads, someone else might :-( Well, I see no benefit in having 16K threads if the server is NIO based. But it might well be a good idea to switch to plain IOs, if we can scale to tens of thousands of connected sessions. This is the main issue here, compared to a Http server, for instance (where session are short lived). But even for a LDAP server, I'm not sure that we really need to deal with hundred of thousands opened sessions... Generally speaking, I'm not sure it's secure to 'open' the LDAP server to end users, so it's very likely that the server will be accessed by an application, which will probably not open a session per user, but instead use a pool of sessions and reuse it. This is an interesting debate... We are using MINA here, which is a NIO based framework, but I foresee a version which would be a pure IO framework, using either plain IO or NIO. That could help to conduct tests to see what will be the best solution (some like Paul Tyma think - and he even demonstrated - that IO is 30% fatser than NIO, even with thousands of threads : http://mailinator.blogspot.com/2008/02/kill-myth-please-nio-is-not-faster-than.html) > > >>> >>> A major problem with this approach if we choose to use it in the >>> OpenDS SDK is that common ancestor DNs (e.g. "dc=com") are going to >>> end up in a single LHM so, using our current design (RDN + parent >>> DN), all decoding attempts will usually end up contending on the >>> same lock anyway :-( So we may need to change our DN implementation >>> to better cope with this caching strategy. >>> >>> We are not alone though: a concurrent Map implementation which can >>> be used for caching in a similar manner to LHM is one of the most >>> frequently demanded enhancements to the java.util.concurrent library. >> There might be some other data structure available, we may need to do >> some research in this area... >> >> > > I did have a poke around some time ago and found a couple. I didn't > evaluate them though as I seem to remember that they introduced a lot > of extra "baggage" and I'd like to keep the size of the SDK to a > minimum (to be honest, I think our OpenDS SDK is already too big). It would be a pain to carry 1 more Mb of useless libs, true... -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com