Return-Path: Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: (qmail 97843 invoked from network); 29 Sep 2010 19:36:55 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 29 Sep 2010 19:36:55 -0000 Received: (qmail 67195 invoked by uid 500); 29 Sep 2010 19:36:53 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 66982 invoked by uid 500); 29 Sep 2010 19:36:53 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 66974 invoked by uid 99); 29 Sep 2010 19:36:53 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Sep 2010 19:36:52 +0000 X-ASF-Spam-Status: No, hits=0.7 required=10.0 tests=RCVD_IN_DNSWL_NONE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [209.85.214.180] (HELO mail-iw0-f180.google.com) (209.85.214.180) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Sep 2010 19:36:45 +0000 Received: by iwn8 with SMTP id 8so1921478iwn.11 for ; Wed, 29 Sep 2010 12:36:22 -0700 (PDT) MIME-Version: 1.0 Received: by 10.231.33.71 with SMTP id g7mr2298144ibd.54.1285788981537; Wed, 29 Sep 2010 12:36:21 -0700 (PDT) Received: by 10.231.143.213 with HTTP; Wed, 29 Sep 2010 12:36:21 -0700 (PDT) In-Reply-To: <4CA3520C.7050606@aol.com> References:

<4C9DDCE2.3050307@aol.com> <4C9DE9D0.5090002@aol.com> <4CA3520C.7050606@aol.com> Date: Thu, 30 Sep 2010 08:36:21 +1300 Message-ID: Subject: Re: Couchdb crashes on Windows From: Dave Cottlehuber To: user@couchdb.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org awesome cliff - can you let us know the flags please? also doing testing. On 30 September 2010 03:49, Cliff Williams wrote: > =A0Dave, > > I hope you are well > > I too began to look at this a couple of nights ago. > > The process that I was going to follow; > =A0 =A0Establish a base case > =A0 =A0Write some file IO tests > =A0 =A0Apply these tests to the Couchdb environment and see what differen= ces > popped out. > > At the very least I thought that I could produce some tests to ascertain = why > 1.01 is misbehaving on windows, if Erlang is problematic =A0and to use as > future tests for new releases of Couchdb on windows. > > I downloaded the latest full Erlang windows binary as my base case > (currently R14B) and wrote the following very simple bit of code as my fi= rst > test > >


> -module(filetest).
> -export([main/0]).
> main() ->
> {ok, Binary}=3Dfile:read_file("blanks"),
> {ok, WriteDescr} =3D file:open(filetest.txt, [raw, append]),
>
> loop(1000, WriteDescr,Binary),
> file:close(WriteDescr).
>
> loop(0,_NotNeeded,_NotNeeded) -> ok;
>
> loop(N,WriteDescr,Binary) ->
> file:write(WriteDescr,Binary),
> loop(N-1,WriteDescr,Binary).
>

> > Note blanks is a zero filled file of about 2.2MB that i originally used t= o > confirm that i had the same issues as Peter first reported. > > Run with a loop of 10 and you get a twenty MB file ........... nice > Run again with a loop of 10 and you get a Forty MB file ......... nice > delete filetest.txt > Run again with a loop of 2000 and you get a file in excess of 4GB .......= ... > nice .......... but slow > delete filetest.txt > Run again with a loop of 4000 and you get a file in excess of 8GB > ...........nice............but very slow > > Now everything seems fine and dandy but ....... > > Delete filetest.txt > Run with a loop of 10 and you get a twenty MB file ........... nice > Run again with a loop of 10 and you get a Forty MB file ......... nice > Run again with a loop of 2000 and you get a file in excess of 4GB .......= ... > nice .......... but slow > Run again with a loop of 2000 and the program begins to "append" from the > beginning of the file so the file does not grow but instead gets > overwritten. This behaviour seems to occur at the 4GB file size on change= of > file descriptor ............. not nice ....... not good. > > I've had a look at the way Erlang is compiled for windows and there are a > couple of flags pertaining to large file support. Over the next few days = I > will attempt my own compilation of Erlang with these flags set and report > back. (Need to get hold of VC++ first) > > regards > > Cliff > > > On 29/09/10 01:02, Dave Cottlehuber wrote: >> >> On 26 September 2010 10:11, Peter Somers =A0wrote: >>> >>> Hi Dave, >>> >>> No luck. Couchdb crashed again :-( >>> >>> I really hope someone can help me on this issue... >>> >>> Best regards, >>> >>> Peter >> >> Hi Peter >> >> There may be 2 things going on here. >> >> - your current crash appears to be due to file corruption; >> =A0- {{badmatch,{error,file_corruption}}, >> this is after couch reads an md5 hash in read_raw_iolist_int - maybe >> due to going over length, I can't tell. >> >> - your first crash is the one I can replicate - the whole VM dies with >> a memory allocation issue. I can replicate this on my win7 enterprise >> couch running 1.01, but not 0.11.0. >> >> I can load the DB up to this size; 0xffd9a0f6d, couchdb dies: >> >> =A0 =A0 {{badmatch,{error,enomem}}, [{couch_file,read_raw_iolist_int,3}, >> {couch_file,handle_call,3}, >> >> Restarting windows/couchdb& =A0then curl http://localhost:5984/fatso yie= lds >> {"error":"kill","reason":"{gen_server,call, >> =A0 =A0 [couch_server, >> =A0 =A0 =A0{open,<<\"fatso\">>, >> =A0 =A0 =A0 =A0 =A0[{user_ctx, >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 {user_ctx,null, >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 [<<\"_admin\">>], >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 <<\"{couch_httpd_auth, >> default_authentication_handler}\">>}}]}, >> =A0 =A0 =A0infinity]}"} >> >> Anyway some more work ahead to identify exactly where this comes from >> - thanks for raising the jira. >> >> cheers >> Dave >> >