incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Cottlehuber <d...@muse.net.nz>
Subject Re: Couchdb crashes on Windows
Date Wed, 29 Sep 2010 19:36:21 GMT
awesome cliff - can you let us know the flags please? also doing testing.

On 30 September 2010 03:49, Cliff Williams <cliffywills@aol.com> wrote:
>  Dave,
>
> I hope you are well
>
> I too began to look at this a couple of nights ago.
>
> The process that I was going to follow;
>    Establish a base case
>    Write some file IO tests
>    Apply these tests to the Couchdb environment and see what differences
> popped out.
>
> At the very least I thought that I could produce some tests to ascertain why
> 1.01 is misbehaving on windows, if Erlang is problematic  and to use as
> future tests for new releases of Couchdb on windows.
>
> I downloaded the latest full Erlang windows binary as my base case
> (currently R14B) and wrote the following very simple bit of code as my first
> test
>
> <code>
> -module(filetest).
> -export([main/0]).
> main() ->
> {ok, Binary}=file:read_file("blanks"),
> {ok, WriteDescr} = file:open(filetest.txt, [raw, append]),
>
> loop(1000, WriteDescr,Binary),
> file:close(WriteDescr).
>
> loop(0,_NotNeeded,_NotNeeded) -> ok;
>
> loop(N,WriteDescr,Binary) ->
> file:write(WriteDescr,Binary),
> loop(N-1,WriteDescr,Binary).
> </code>
>
> Note blanks is a zero filled file of about 2.2MB that i originally used to
> confirm that i had the same issues as Peter first reported.
>
> Run with a loop of 10 and you get a twenty MB file ........... nice
> Run again with a loop of 10 and you get a Forty MB file ......... nice
> delete filetest.txt
> Run again with a loop of 2000 and you get a file in excess of 4GB ..........
> nice .......... but slow
> delete filetest.txt
> Run again with a loop of 4000 and you get a file in excess of 8GB
> ...........nice............but very slow
>
> Now everything seems fine and dandy but .......
>
> Delete filetest.txt
> Run with a loop of 10 and you get a twenty MB file ........... nice
> Run again with a loop of 10 and you get a Forty MB file ......... nice
> Run again with a loop of 2000 and you get a file in excess of 4GB ..........
> nice .......... but slow
> Run again with a loop of 2000 and the program begins to "append" from the
> beginning of the file so the file does not grow but instead gets
> overwritten. This behaviour seems to occur at the 4GB file size on change of
> file descriptor ............. not nice ....... not good.
>
> I've had a look at the way Erlang is compiled for windows and there are a
> couple of flags pertaining to large file support. Over the next few days I
> will attempt my own compilation of Erlang with these flags set and report
> back. (Need to get hold of VC++ first)
>
> regards
>
> Cliff
>
>
> On 29/09/10 01:02, Dave Cottlehuber wrote:
>>
>> On 26 September 2010 10:11, Peter Somers<petsomers@gmail.com>  wrote:
>>>
>>> Hi Dave,
>>>
>>> No luck. Couchdb crashed again :-(
>>>
>>> I really hope someone can help me on this issue...
>>>
>>> Best regards,
>>>
>>> Peter
>>
>> Hi Peter
>>
>> There may be 2 things going on here.
>>
>> - your current crash appears to be due to file corruption;
>>  - {{badmatch,{error,file_corruption}},
>> this is after couch reads an md5 hash in read_raw_iolist_int - maybe
>> due to going over length, I can't tell.
>>
>> - your first crash is the one I can replicate - the whole VM dies with
>> a memory allocation issue. I can replicate this on my win7 enterprise
>> couch running 1.01, but not 0.11.0.
>>
>> I can load the DB up to this size; 0xffd9a0f6d, couchdb dies:
>>
>>     {{badmatch,{error,enomem}}, [{couch_file,read_raw_iolist_int,3},
>> {couch_file,handle_call,3},
>>
>> Restarting windows/couchdb&  then curl http://localhost:5984/fatso yields
>> {"error":"kill","reason":"{gen_server,call,
>>     [couch_server,
>>      {open,<<\"fatso\">>,
>>          [{user_ctx,
>>               {user_ctx,null,
>>                   [<<\"_admin\">>],
>>                   <<\"{couch_httpd_auth,
>> default_authentication_handler}\">>}}]},
>>      infinity]}"}
>>
>> Anyway some more work ahead to identify exactly where this comes from
>> - thanks for raising the jira.
>>
>> cheers
>> Dave
>>
>

Mime
View raw message