Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of cgsmcmlxxv@gmail.com
 designates 74.125.82.54 as permitted sender)
Message-ID: <4EDFD65A.2000202@gmail.com>
Date: Wed, 07 Dec 2011 22:10:50 +0100
From: CGS <cgsmcmlxxv@gmail.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
 rv:1.9.2.23) Gecko/20110922 Thunderbird/3.1.15
MIME-Version: 1.0
To: user@couchdb.apache.org
Subject: Re: User authentication failures
References: 
 <CACjU+5OCBXDtaduBi0Ri7XgO_+9ZggbQeB9TOhmQm9cTdYdRiA@mail.gmail.com>
 <CACjU+5OqiHXKSPcuyhmOLyNuDn2wDunoAV0=Y+56JfhXsDYqZA@mail.gmail.com>
 <CAL3q7H4YP8VOco7U_f+Yd+_ncy+6j-rO1cxwx_GRXMW-Y57Q=g@mail.gmail.com>
 <CAOnDDh56Qq2jnz9=4AOPFd5i=0ANMCvsLJbRag=JRjyyVQ5e9g@mail.gmail.com>
 <4EDF2612.6040303@gmail.com>
 <CAOnDDh7Scm2JMT+R6wRV_2n+Lp4=+UemJP1AXym2h5pSG+AAAg@mail.gmail.com>
In-Reply-To: 
 <CAOnDDh7Scm2JMT+R6wRV_2n+Lp4=+UemJP1AXym2h5pSG+AAAg@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

I tested on 2 GB RAM increasing every time by one order of magnitude at 
the time. At 10M it no longer started, complaining about not being able 
to allocate resources. I don't know exactly if it is the RAM or my linux 
file descriptors (CentOS 6.0). So, I remained at 1M. As far as I know, 
Windows 7 supports at maximum 16M file descriptors, so, you may try to 
raise it up to that limit for the test.

Another test which I've done was if I can really have 1M spawned threads 
which can interact with the OS (Erlang process in which I started 1M 
processes in parallel which each should have brought something with 
cURL). The answer was no (the OS stopped increasing the number of cURL 
instances, invoking lack of resources). The full memory was occupied and 
so I couldn't determine if I reached the OS limit or I was starving my 
processes.

In case the RAM is the problem, I suppose your 8 GB can bring up a lot 
of threads, but if the OS is imposing the limit, then you may want to 
leave some file descriptors for other processes as well.

I would be interested at what numbers you reach. :)

CGS


On 12/07/2011 08:39 PM, Pete Vander Giessen wrote:
> On Wed, Dec 7, 2011 at 12:38 AM, CGS<cgsmcmlxxv@gmail.com>  wrote:
>> It looks like Erlang reached the 1024 processes default limit. That should
>> be solved by increasing ERL_MAX_PORTS ... To be noted that
>> Erlang checks for the working memory availability for each thread, so, the
>> maximum ports cannot be insanely high.
> Thank you for that piece of info. How high would you say would be
> "insanely" high for, say, a server w/ 8GB of RAM?
>
> Broader question: if I start a continuous replication, will that hold
> a thread open for the (theoretically unlimited) duration of the
> replication?
>
> They key thing is that I'm trying to figure out how we hit that limit
> without doing anything _too_ mean to the server. Increasing the thread
> limit sounds like it might be part of the solution, but I'd like to
> understand the problem a bit more, and I'm doing a bit of blind
> flailing at the moment ...
>
> ~ PeteVG