Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of adam.kocoloski@gmail.com
 designates 209.85.221.179 as permitted sender)
Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes
Mime-Version: 1.0 (Apple Message framework v1076)
Subject: Re: Timeout Error when trying to access views + Indexing problems
From: Adam Kocoloski <kocolosk@apache.org>
In-Reply-To: <6cfccb3d0910032307i1df64f79u437d5c803bff2c50@mail.gmail.com>
Date: Sun, 4 Oct 2009 13:19:16 -0400
Content-Transfer-Encoding: 7bit
Message-Id: <A60F21CC-9C57-4202-84C1-04BEF84CD141@apache.org>
References: <6cfccb3d0910030910m678c9157o811a57e02ffbeff0@mail.gmail.com>
  <6cfccb3d0910031824q5d9362c2k7719b32a779e956f@mail.gmail.com>
  <1521F562-7878-423B-BEC3-FF2A1916DA0C@gmail.com>
 <6cfccb3d0910032307i1df64f79u437d5c803bff2c50@mail.gmail.com>
To: user@couchdb.apache.org

Hi Glenn, I saw something like this once, but unfortunately I wasn't  
able to resolve it.  Can you try the following?

1) Start couchdb with the -i option to get an interactive shell

2) At the prompt, try to open the view file and read the header

{ok, Fd} = couch_file:open("/vol/couchdb/var/lib/ 
couchdb/.searchlight_production_design/ 
5f190ffb2ed589746e8796d2423869ac.view").
couch_file:read_header(Fd).

In my case, that call timed out, and the stacktrace during normal  
operation was exactly the one you showed earlier in this thread.  Best,

Adam

On Oct 4, 2009, at 2:07 AM, Glenn Rempe wrote:

> Thanks for the reply Paul.  Some comments below.
>
> Also, just for full disclosure, the CouchDB I am working on was  
> moved out of
> another couchdb and it was originally created using CDB 0.9.1.  I  
> show a dir
> listing below that indicates exactly what was moved.
>
> On Sat, Oct 3, 2009 at 6:46 PM, Paul Joseph Davis <
> paul.joseph.davis@gmail.com> wrote:
>
>> Glenn,
>>
>> This sounds like your map function is timing out which causes the  
>> error.
>> You could try upping the os process timeout setting in the config.
>>
>>
> When I go into futon and select one of my views in my design doc it  
> *always*
> consistently pops up a javascript alert with the error text at ~5  
> seconds
> after selecting the view.  It doesn't seem to matter what else I  
> do.  It
> also didn't vary when I changed the os_process_timeout value in  
> futon as you
> suggested from 5000 to 25000.  Can you explain exactly what this  
> particular
> param is doing?  I assume the value is milliseconds?
>
>
>> To see what's going on you can increase to debug logging or use the  
>> log
>> function in your maps. There's also the status page in futon which  
>> I think
>> you said you were looking at.
>>
>>
> Yes, I was previously looking at the status page.  But now since I've
> upgraded to trunk I never see any indexing activity happening in the  
> status
> page no matter what I do.
>
>
>> If indexing crashes it should just pick up where it left off when you
>> retrigger. Use the status page to verify. If it's not then let us  
>> know.
>>
>>
> Can you clarify, is this also the case when no index has ever  
> successfully
> run?  I was wondering if I first need to get through at least one  
> index
> session (maybe with a smaller amount of records) prior to incremental
> indexing working as expected.
>
> Is there any way to determine what percentage of the total records  
> have been
> added to the index?
>
> For your info, here are the contents of the DB dir.  You can see the  
> main DB
> is 42GB now (~17 million records).
>
> root@ip-10-250-55-239:/home/rails/underscore-sync-mysql-to-couchdb/ 
> shared/log#
> ls -la /vol/couchdb/var/lib/couchdb
> total 41674956
> drwxr-xr-x 3 couchdb root             78 2009-10-04 00:46 .
> drwxr-xr-x 3 couchdb root             20 2009-10-03 05:02 ..
> -rw-r--r-- 1 couchdb couchdb 42675073133 2009-10-04 02:13
> searchlight_production.couch
> drwxr-xr-x 2 couchdb couchdb         120 2009-10-03 06:02
> .searchlight_production_design
>
> root@ip-10-250-55-239:/home/rails/underscore-sync-mysql-to-couchdb/ 
> shared/log#
> ls -la /vol/couchdb/var/lib/couchdb/.searchlight_production_design/
> total 33700196
> drwxr-xr-x 2 couchdb couchdb         120 2009-10-03 06:02 .
> drwxr-xr-x 3 couchdb root             78 2009-10-04 00:46 ..
> -rw-r--r-- 1 couchdb couchdb  9819347287 2009-10-03 08:04
> 5f190ffb2ed589746e8796d2423869ac.view
> -rw-r--r-- 1 couchdb couchdb    91402872 2009-10-03 06:03
> b127a58306fb8e7858cd1a92f8398511.view
> -rw-r--r-- 1 couchdb couchdb 24598236884 2009-10-02 13:00
> SearchDocument.view
>
>
>
>> If you can't find anything in the debug logs then ping the lust and  
>> we'll
>> get into trying to duplicate.
>>
>>
> I have turned on the 'debug' level in the logs and that provided me  
> with the
> info I previously provided.  I'll try to use the log function in the  
> map and
> see if that shows anything.
>
> Thanks for helping.  If it comes to it, I may be able to make a  
> snapshot of
> this EBS volume and start a host that you could login to and get  
> your hands
> directly on it if that would be helpful.
>
> Glenn