incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lennart Melzer <l.mel...@tu-bs.de>
Subject Re: POST/PUTing binary Attachments to external Handler
Date Wed, 11 Nov 2009 07:29:37 GMT
Hi Paul,

I double checked the response of my external ruby script.

I reduced the code to the bare minimum (nearly representing what the  
python example from the wiki looks like)

#!/usr/bin/env ruby
require 'rubygems'
require 'json'

def request
   begin
     while line = STDIN.readline
       yield JSON.parse(line)
     end
   rescue EOFError
     return
   end
end

def respond code = 200, data = {}, headers = {}
   output = "#{{:code => code,:body => data, :headers =>  
headers}.to_json}\n"
   STDOUT.write output
   STDOUT.flush
end
request do |line|
   respond 200, "random"
end

When I do a GET request to the external handler I get "random" back,  
when I do a POST with a simple form I get "random" back, but when  
POSTing an image (by curl) as multipart/form-data it breaks with the  
aforementioned error returning nothing. So since the return value  
seems to work with other types of request, it can't be my external  
handler.

Sorry for bothering, but I really want to get this working. Been  
reading through the code for some time now, but I don't understand why  
couchdb is even calling send_json in the first place.

Greetings,

Lennart
On Nov 11, 2009, at 12:11 AM, Paul Davis wrote:

> On Tue, Nov 10, 2009 at 5:56 PM, Lennart Melzer <l.melzer@tu-bs.de>  
> wrote:
>> Hi Paul,
>>
>> I'm not trying to return arbitrary binary data,I really just try to  
>> post an
>> attached file to an external handler. Right now the handler returns  
>> a static
>> response which works when posting form data (without binary data in  
>> it) and
>> with get request. It doesn't work when posting a binary file. I  
>> don't think
>> it's a problem with the handler itself.
>>
>> I don't know why the couchdb is trying to encode the binary post  
>> request as
>> json.
>>
>> Greetings,
>>
>> Lennart
>>
>> On Nov 10, 2009, at 11:26 PM, Paul Davis wrote:
>>
>>> On Tue, Nov 10, 2009 at 5:22 PM, Lennart Melzer <l.melzer@tu- 
>>> bs.de> wrote:
>>>>
>>>> I just looked through the error logs and the code a bit more. I  
>>>> think
>>>> this
>>>> passage describes best what seems to happen (though I don't know  
>>>> why)
>>>>
>>>> So the stracktrace shows that the mochiweb passes the request to
>>>> couch_httpd:handle_request where it seems to try to call send_json
>>>> (whysoever). It all fails when it tries to encode the binary data  
>>>> as a
>>>> json
>>>> string. This all seems to happen before couch_httpd_external.erl  
>>>> stuff
>>>> gets
>>>> called, which might be able to handle binary data correctly.
>>>>
>>>> Here's the log snippet:
>>>>  crasher:
>>>>   initial call: mochiweb_socket_server:acceptor_loop/1
>>>>   pid: <0.1059.0>
>>>>   registered_name: []
>>>>   exception exit: {ucs,{bad_utf8_character_code}}
>>>>     in function  xmerl_ucs:from_utf8/1
>>>>     in call from mochijson2:json_encode_string/2
>>>>     in call from mochijson2:'-json_encode_proplist/2-fun-0-'/3
>>>>     in call from lists:foldl/3
>>>>     in call from mochijson2:json_encode_proplist/2
>>>>     in call from couch_httpd:send_json/4
>>>>     in call from couch_httpd:handle_request/5
>>>>     in call from mochiweb_http:headers/5
>>>>
>>>> Anyone out that knows more about the internals of couchdb than I  
>>>> do? I
>>>> would
>>>> be grateful for any advice (gotta learn some erlang programming  
>>>> soonish).
>>>>
>>>> Greetings,
>>>>
>>>> Lennart
>>>>
>>>>
>>>> On Nov 10, 2009, at 12:13 AM, Paul Davis wrote:
>>>>
>>>>> On Mon, Nov 9, 2009 at 12:15 PM, Lennart Melzer <l.melzer@tu- 
>>>>> bs.de>
>>>>> wrote:
>>>>>>
>>>>>> I am running Revision 832477 of the couchdb trunk
>>>>>> On Nov 9, 2009, at 6:09 PM, Paul Davis wrote:
>>>>>>
>>>>>>> On Mon, Nov 9, 2009 at 11:54 AM, Lennart Melzer <l.melzer@tu-bs.de

>>>>>>> >
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Right now, posting or putting binary data to an external
 
>>>>>>>> handler
>>>>>>>> fails,
>>>>>>>> since the data is parsed as a UTF-8 String and not treated
as  
>>>>>>>> binary.
>>>>>>>> The
>>>>>>>> external handler never gets to do its work, so this might
be  
>>>>>>>> an issue
>>>>>>>> with
>>>>>>>> the handling of data sent to an external handler. Am I  
>>>>>>>> misusing the
>>>>>>>> external-handler interface?
>>>>>>>>
>>>>>>>> Error snippet:
>>>>>>>> [error] [<0.95.0>] {error_report,<0.30.0>,
>>>>>>>>  {<0.95.0>,std_error,
>>>>>>>>  {mochiweb_socket_server,235,
>>>>>>>>      {child_error,{ucs,{bad_utf8_character_code}}}}}}
>>>>>>>>
>>>>>>>> Greetings,
>>>>>>>>
>>>>>>>> Lennart
>>>>>>>>
>>>>>>>
>>>>>>> Lennart,
>>>>>>>
>>>>>>> What version of CouchDB are you on? I seem to remember a patch
 
>>>>>>> for
>>>>>>> this going into trunk at some point. Not sure if that was pre/

>>>>>>> post
>>>>>>> 0.10 though.
>>>>>>>
>>>>>>> Paul Davis
>>>>>>
>>>>>>
>>>>>
>>>>> Lennart,
>>>>>
>>>>> Hmm, looking through the svn log I'm not seeing a commit that  
>>>>> mentions
>>>>> body or posts. I swear I remember something like that coming  
>>>>> through
>>>>> at one point. Benoit, didn't you have something to do with that?
>>>>>
>>>>> Paul Davis
>>>>
>>>>
>>>
>>> Lennart,
>>>
>>> That stack trace suggests that you're trying to return a JSON  
>>> response
>>> that is not valid JSON. If your external is trying to return  
>>> arbitrary
>>> binary data in the _external response object you'll need to make  
>>> sure
>>> and use the body member.
>>>
>>> Paul Davis
>>
>>
>
> Lennart,
>
> What are you returning from your _external handler? You should be
> returning something like:
>
> {
>    "code": 200,
>    "body": "arbitrary string of data and some stuff",
>    "headers": {"Content-Type": "application/my-stuff"}
> }
>
> I'd have to remind myself on the exact rules, but returning {"code":
> 200, "json": "stuff"} will send "stuff" through the JSON serializer,
> which means it has to conform to unicode escaping rules that
> mochijson2 exposes via xmerl:ucs16_to_char or whatever.
>
> Perhaps you can log the response you're sending and paste a copy of  
> that?
>
> HTH,
> Paul Davis


Mime
View raw message