couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Samuel Newson <rnew...@apache.org>
Subject Re: _cleanup_view is deleting all views
Date Fri, 31 Jan 2014 16:52:41 GMT

Ownership is interesting. Would the bigcouch user have the right to delete the file but not
open it for reading?

There’s definitely an issue in bigcouch (fixed long since in couchdb) where any failure
to open a view file makes us delete it.

OS/fs all check out fine, You see the filename that should be retained in that log output?
you’re 100% sure? You do have a live design doc pointing to it?

B.

On 31 Jan 2014, at 16:39, Vladimir Ralev <vladimir.ralev@gmail.com> wrote:

> Thanks a lot. The database was moved from older machines so some other file
> system metadata might be scrambled. But I don't see what can cause a
> problem like this.
> 
> Yes the debug output is seen "deleting unused view index files:" and it
> deletes every view in every database, little doubt about it. It doesn't
> delete fresh views though that are fully regenerated afterwards. I think
> the original views somehow got corrupted, but I need to figure out why and
> may be fix it manually with a script
> 
> OS is Debian 64, file system is ext4, there is a little scramble of the
> file ownership, some directories are owned by old bigcouch user, others by
> root, so that's one thing I am investigating. I reset the ownership, but
> will have to repeat it for my next tests.
> 
> 
> 
> 
> On Fri, Jan 31, 2014 at 6:21 PM, Robert Samuel Newson <rnewson@apache.org>wrote:
> 
>> and details of OS, filesystem, anything you think might be relevant.
>> 
>> B.
>> 
>> On 31 Jan 2014, at 16:20, Robert Samuel Newson <rnewson@apache.org> wrote:
>> 
>>> First thing to note is that bigcouch development is over, but we can at
>> least confirm this;
>>> 
>>> This function fetches all the design docs of the database, grabs all the
>> signatures from each (you'll have noticed view filenames look uuid/randomy,
>> that's a 'sig'), and then sweeps the dir where all views for the given
>> database should be and deletes those not in the 'keep' list.
>>> 
>>> Can you enable debug level logging (curl
>> localhost:5984/_config/log/level -X PUT -d '"debug"' to *all* bigcouch
>> nodes) and tell us if ;
>>> 
>>> ?LOG_DEBUG("deleting unused view index files: ~p",[DeleteFiles]),
>>> 
>>> actually gets printed?
>>> 
>>> B.
>>> 
>>> On 31 Jan 2014, at 16:09, Vladimir Ralev <vladimir.ralev@gmail.com>
>> wrote:
>>> 
>>>> Hi guys,
>>>> 
>>>> bigcouch 0.4.2 has the following code that handles view cleanup:
>>>> 
>>>> cleanup_index_files(Db) ->
>>>> 
>>>>  % load all ddocs
>>>> 
>>>>  {ok, DesignDocs} = couch_db:get_design_docs(Db),
>>>> 
>>>> 
>>>>  % make unique list of group sigs
>>>> 
>>>>  Sigs = lists:map(fun(#doc{id = GroupId}) ->
>>>> 
>>>>      {ok, Info} = get_group_info(Db, GroupId),
>>>> 
>>>>      ?b2l(couch_util:get_value(signature, Info))
>>>> 
>>>>  end, [DD||DD <- DesignDocs, DD#doc.deleted == false]),
>>>> 
>>>> 
>>>>  FileList = list_index_files(Db),
>>>> 
>>>> 
>>>>  DeleteFiles =
>>>> 
>>>>  if length(Sigs) =:= 0 ->
>>>> 
>>>>      FileList;
>>>> 
>>>>  true ->
>>>> 
>>>>      % regex that matches all ddocs
>>>> 
>>>>      RegExp = "("++ string:join(Sigs, "|") ++")",
>>>> 
>>>> 
>>>>  % filter out the ones in use
>>>> 
>>>>      [FilePath || FilePath <- FileList,
>>>> 
>>>>          re:run(FilePath, RegExp, [{capture, none}]) =:= nomatch]
>>>> 
>>>>  end,
>>>> 
>>>> 
>>>>  % delete unused files
>>>> 
>>>>  ?LOG_DEBUG("deleting unused view index files: ~p",[DeleteFiles]),
>>>> 
>>>>  RootDir = couch_config:get("couchdb", "view_index_dir"),
>>>> 
>>>>  [couch_file:delete(RootDir,File,false)||File <- DeleteFiles],
>>>> 
>>>>  ok.
>>>> 
>>>> 
>>>> From here
>>>> 
>> https://github.com/cloudant/bigcouch/blob/master/apps/couch/src/couch_view.erl#L84
>>>> 
>>>> It's supposed to delete only unused views, but in my case it deletes
>>>> everything and then starts building from scratch. Can you help me
>>>> understand the condition used here to filter the files that are
>> currently
>>>> in use? How is the regex supposed to work.
>>> 
>> 
>> 


Mime
View raw message