incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From J Chris Anderson <jch...@apache.org>
Subject Re: Crash when using builtin reduce _stats
Date Sat, 07 Aug 2010 01:00:51 GMT

On Aug 6, 2010, at 5:57 PM, Yue Chuan Lim wrote:

> Also, I can verify that a temp view with any number of trailing linefeeds
> (and whitespace) on the reduce function works fine (looks like for a
> temp-view the text gets tossed into a javascript eval?), but for an actual
> view, the reduce function needs to be just "_stats".
> 
> Should I still open a JIRA ticket?
> 

Modifying the test case to test that "_stats-and-then-some-junk" works would be the most helpful
thing you could do.

http://github.com/apache/couchdb/blob/trunk/share/www/script/test/reduce_builtin.js

It may be that Futon is trimming the whitespace for you. I bet a tempview with more whitespace
(when sent to couch) would also break.

Chris

> Thanks for your help in tracking this down, much appreciated!
> Yue Chuan
> 
> On Sat, Aug 7, 2010 at 8:50 AM, J Chris Anderson <jchris@apache.org> wrote:
> 
>> 
>> On Aug 6, 2010, at 5:41 PM, Yue Chuan Lim wrote:
>> 
>>> It appears that a trailing linefeed is the cause of my problems.
>>> 
>>> On the working function, the reduce function is simply "_stats" on the
>>> failing function it is "_stats\u000a"
>>> 
>> 
>> ahhh, ye old linefeed bug.
>> 
>> nuff said. if someone can provide a test case for this, the actual bug fix
>> should be trivial.
>> 
>> I think the behavior should be a prefix test, so that even "_statsorama" or
>> "_stats\nare\awesome" should work just as "_stats" does.
>> 
>> Chris
>> 
>>> --- message below retained for prosperity, in the event I am wrong
>>> 
>>> After playing around with the function (as above, ensuring that there are
>> no
>>> non-numeric values, as well as removing and eventually readding the > 0
>>> test), it suddenly starts working.
>>> 
>>> // revision 18 - works, renamed to stats_pass
>>> {
>>>  "_id": "_design/stats",
>>>  "_rev": "18-776464ea3e4e548f44c55aa8b889f8a1",
>>>  "langauge": "javascript",
>>>  "views": {
>>>      "active_power": {
>>>          "map": "function(doc) {\u000a  if(doc.data[12] > 0) {\u000a
>>> key = doc._id.split(\"|\")[0];\u000a    emit(key, doc.data[12]);\u000a
>>> }\u000a}\u000a\u000a",
>>>          "reduce": "_stats"
>>>      }
>>>  },
>>>  "language": "javascript"
>>> }
>>> 
>>> // revision 14 (15, 16, 17 are variants of this), when I removed the
>> reduce
>>> function to run the tests as per above
>>> {
>>>  "_id": "_design/stats",
>>>  "_rev": "14-416ed97296c70d0c53215c210ebcf464",
>>>  "langauge": "javascript",
>>>  "views": {
>>>      "active_power": {
>>>          "map": "function(doc) {\u000a  if(doc.data[12] > 0) {\u000a
>>> key = doc._id.split(\"|\")[0];\u000a    emit(key, doc.data[12]);\u000a
>>> }\u000a}\u000a\u000a"
>>>      }
>>>  },
>>>  "language": "javascript"
>>> }
>>> 
>>> // revision 13, the failing revision
>>> {
>>>  "_id": "_design/stats",
>>>  "_rev": "13-9549c38bb346d590749adc81f3f6c7e2",
>>>  "langauge": "javascript",
>>>  "views": {
>>>      "active_power": {
>>>          "map": "function(doc) {\u000a  if(doc.data[12] > 0) {\u000a
>>> key = doc._id.split(\"|\")[0];\u000a    emit(key, doc.data[12]);\u000a
>>> }\u000a}\u000a\u000a",
>>>          "reduce": "_stats\u000a"
>>>      }
>>>  },
>>>  "language": "javascript"
>>> }
>>> 
>>> These appear to me as essentially identical views (13 and 18). but 13
>> fails
>>> and 18 passes. Copy/pasting them to a new view (i.e. stats_pass (rev 18)
>> and
>>> stats_fail (rev 13)) on a couch.io instance, and they continue to pass
>> and
>>> fail respectively.
>>> 
>>> http://shasderias.couchone_remove_me_
>>> .com/gsc_lt2/_design/stats_pass/_view/active_power
>>> http://shasderias.couchone_remove_me_
>>> .com/gsc_lt2/_design/stats_fails/_view/active_power
>>> 
>>> On Sat, Aug 7, 2010 at 8:24 AM, J Chris Anderson <jchris@gmail.com>
>> wrote:
>>> 
>>>> 
>>>> On Aug 6, 2010, at 5:21 PM, Yue Chuan Lim wrote:
>>>> 
>>>>> I highly doubt it, modifying the function to:
>>>>> 
>>>>> function(doc) {
>>>>> if(doc.data[12] > 0) {
>>>>>  key = doc._id.split("|")[0];
>>>>>  emit(doc.data[12], doc.data[12]);
>>>>> }
>>>>> }
>>>>> 
>>>>> and sorting by key, it appears all the numbers are indeed numbers.
>>>>> 
>>>>> function(doc) {
>>>>> if(doc.data[12] > 0) {
>>>>>  key = doc._id.split("|")[0];
>>>>>  if(doc.data[12] != parseInt(doc.data[12])) {
>>>>>    emit(doc.data[12], doc.data[12]);
>>>>>  }
>>>>> }
>>>>> }
>>>>> 
>>>>> Returns a empty set.
>>>>> 
>>>>> On a hunch, I removed the doc.data[12] > 0 line, and the query seems
to
>>>> run
>>>>> fine
>>>>> 
>>>>> (with group=true)
>>>>> 
>>>>> {"rows":[
>>>>> 
>>>> 
>> {"key":"gsc_test_01","value":{"sum":335168,"count":1000,"min":0,"max":389,"sumsqr":128263850}},
>>>>> 
>>>> 
>> {"key":"gsc_test_02","value":{"sum":381805,"count":1000,"min":0,"max":765,"sumsqr":176920147}},
>>>>> 
>>>> 
>> {"key":"gsc_test_03","value":{"sum":411391,"count":1000,"min":187,"max":761,"sumsqr":178307983}}
>>>>> ]}
>>>>> 
>>>>> So it appears that _stats doesn't like value sets without 0s :P (or
>> maybe
>>>>> reduce + group=true has something against unequal count values)
>>>>> 
>>>>> Thanks for looking into this :)
>>>> 
>>>> Since there isn't an easy answer in sight, and I'm up to my neck in
>> other
>>>> patches right now, maybe the best thing to do is start a Jira ticket to
>>>> record your observations while they are still fresh.
>>>> 
>>>> http://issues.apache.org/jira/browse/COUCHDB
>>>> 
>>>> Thanks,
>>>> Chris
>>>> 
>>>>> Yue Chuan
>>>>> 
>>>>> 
>>>>> On Sat, Aug 7, 2010 at 8:13 AM, J Chris Anderson <jchris@apache.org>
>>>> wrote:
>>>>> 
>>>>>> 
>>>>>> On Aug 6, 2010, at 5:09 PM, Yue Chuan Lim wrote:
>>>>>> 
>>>>>>> Further tested on the couch.io windows installer as well as a
>>>>>>> couch.ioinstance with the same results, works on a temp view,
crashes
>>>>>>> on a actual
>>>>>>> view.
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>> 
>> shasderias.couchone_remove_me_.com/_utils/database.html?gsc_lt2/_design/stats/_view/active_power
>>>>>>> 
>>>>>>> is the couchone instance.
>>>>>> 
>>>>>> Is there any chance you have a few docs where the value is not a
>> number?
>>>> If
>>>>>> you remove the reduce function completely and query the map is every
>>>> single
>>>>>> row's value a number?
>>>>>> 
>>>>>> Thanks for helping us get to the bottom of this.
>>>>>> 
>>>>>> Chris
>>>>>> 
>>>>>>> 
>>>>>>> On Fri, Aug 6, 2010 at 9:01 PM, Yue Chuan Lim <shasderias@gmail.com>
>>>>>> wrote:
>>>>>>> 
>>>>>>>> And curiously enough, running it as a temp view works just
fine.
>>>>>>>> 
>>>>>>>> [info] [<0.1859.0>] 127.0.0.1 - - 'POST'
>>>>>>>> /gsc_lt2/_temp_view?limit=11&descending=true&group=true
200
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Fri, Aug 6, 2010 at 8:38 PM, Yue Chuan Lim <shasderias@gmail.com
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Fairly standard map/reduce function, map looks something
like
>>>>>>>>> 
>>>>>>>>> function(doc) {
>>>>>>>>> if(doc.data[12] > 0) {
>>>>>>>>> key = doc._id.split("|")[0];
>>>>>>>>> emit(key, doc.data[12]);
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>> 
>>>>>>>>> Data returned by map is along the lines of: ("test1",
123),
>> ("test1",
>>>>>>>>> 245).
>>>>>>>>> 
>>>>>>>>> Reduce is just: _stats
>>>>>>>>> 
>>>>>>>>> Crash log: http://pastebin.com/aX7jeeLm
>>>>>>>>> Platform: Windows 7 (32-bit)
>>>>>>>>> 
>>>>>>>>> Using the installer by Mark Hammond. Incidentally, is
the installer
>>>> on
>>>>>>>>> couch.io the preferred one for windows deployment?
>>>>>>>>> 
>>>>>>>>> Thanks
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>> 
>> 


Mime
View raw message