Return-Path: Delivered-To: apmail-axis-c-user-archive@www.apache.org Received: (qmail 25798 invoked from network); 4 Feb 2011 20:35:57 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 4 Feb 2011 20:35:57 -0000 Received: (qmail 38946 invoked by uid 500); 4 Feb 2011 20:35:57 -0000 Delivered-To: apmail-axis-c-user-archive@axis.apache.org Received: (qmail 38895 invoked by uid 500); 4 Feb 2011 20:35:56 -0000 Mailing-List: contact c-user-help@axis.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "Apache AXIS C User List" Delivered-To: mailing list c-user@axis.apache.org Received: (qmail 38887 invoked by uid 99); 4 Feb 2011 20:35:55 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Feb 2011 20:35:55 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=RCVD_IN_DNSWL_NONE,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [213.174.32.97] (HELO mailout02.ims-firmen.de) (213.174.32.97) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Feb 2011 20:35:48 +0000 Received: from [192.168.1.110] (helo=mailin01.ims-firmen.de) by mailout02.ims-firmen.de with esmtp (Exim 4.72) (envelope-from ) id 1PlSNK-0006sP-50 for c-user@axis.apache.org; Fri, 04 Feb 2011 21:35:26 +0100 Received: from [144.41.253.148] (helo=[10.0.0.53]) by mailin01.ims-firmen.de with esmtpa (Exim 4.72) (envelope-from ) id 1PlSNJ-0006nO-U7 for c-user@axis.apache.org; Fri, 04 Feb 2011 21:35:26 +0100 Subject: Re: XML reader/writer buffer size problem? From: Thomas Gentsch To: Apache AXIS C User List In-Reply-To: <1296822109.2274.176.camel@parsley> References: <1276465246.2264.5.camel@parsley> <1276465624.2264.8.camel@parsley> <4C15A7EA.40706@wso2.com> <1276500772.2268.8.camel@parsley> <4C15FAE2.9000306@wso2.com> <1276850673.2221.62.camel@parsley> <1276854782.2221.74.camel@parsley> <1278673353.2367.24.camel@parsley> <1296129803.2333.124.camel@parsley> <1296130561.2333.126.camel@parsley> <1296146857.6873.28.camel@parsley> <4D41A3AB.1090702@wso2.com> <1296150199.6873.35.camel@parsley> <1296390072.2333.37.camel@parsley> <1296816794.2274.134.camel@parsley> <1296822109.2274.176.camel@parsley> Content-Type: text/plain; charset="UTF-8" Date: Fri, 04 Feb 2011 21:35:23 +0100 Message-ID: <1296851723.2274.622.camel@parsley> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit OK, I give up. I tried with the libxml2 parser and with that it works. Here are my latest findings, just in case somebody wants to look into this further ... =================== Digging further, now I'm back at the XML writer again. Have found this: axis2_http_transport_sender_invoke() { ... buffer = (axis2_char_t *) axiom_xml_writer_get_xml(xml_writer, env); buffer_size = axiom_xml_writer_get_xml_size(xml_writer, env); /* TGE */ printf("TGE: http sender: %d bytes\n", buffer_size); printf("TGE: http sender: '%s'\n", buffer); fflush(stdout); ... int rc = axutil_stream_write(out_stream, env, buffer, buffer_size); printf("TGE: http sender stream_write returned %d.\n", rc); } and I get again the truncated XML. In other words, here happens the same as I had before in my dump() function - the axiom node gets printed into a buffer, but not completely. The guththila XML writer prints everything correctly - but not, as I initially thought, into the socket, but again into another buffer (as above). The size of the buffer is correct (975000), however the actual buffer is not: ... TGE: http sender: 975921 bytes TGE: http sender: '2411339386 Must be something like that. > Essentially: > > 1. In the SOAP server I can see all data correctly, and I can see that > the guththila XML writer prints all XML tags correctly. > 2. In the SOAP client (I have added prinouts to the guththila XML > parser) I see the truncated XML - at some point it just stops. > > Now I have added a few more trace statements into the network handler > and now I can see this (PID 9419 is the server, 9428 the client): > ======== > [Fri Feb 4 13:07:44 2011] 9419 [...TRACE...] msg_recv.c(459) > [axis2]Exit:axis2_msg_recv_receive_impl > [Fri Feb 4 13:07:44 2011] 9419 [...TRACE...] engine.c(323) > Exit:axis2_engine_receive > [Fri Feb 4 13:07:44 2011] 9428 [...TRACE...] http_sender.c(1593) > Entry:axis2_http_sender_process_response > [Fri Feb 4 13:07:44 2011] 9428 [...TRACE...] http_sender.c(1614) > Exit:axis2_http_sender_process_response > [Fri Feb 4 13:07:44 2011] 9428 [debug] http_transport_sender.c(822) OP > name axutil_qname_get_localpart = http://www.w3.org/2004/08/wsdl/out-in > [Fri Feb 4 13:07:44 2011] 9428 [debug] soap_builder.c(945) identified > soap version is soap12 > [Fri Feb 4 13:07:44 2011] 9419 [error] network_handler.c(208) TGE: > Shutting down socket > [Fri Feb 4 13:07:44 2011] 9419 [error] network_handler.c(218) TGE: > Waiting for response ... > [Fri Feb 4 13:07:44 2011] 9419 [error] network_handler.c(224) TGE: > Returned. Closing socket. > [Fri Feb 4 13:07:44 2011] [info] Request served in 22.036 seconds > [Fri Feb 4 13:07:45 2011] 9428 [...TRACE...] > http_transport_sender.c(596) Exit:axis2_http_transport_sender_invoke > [Fri Feb 4 13:07:45 2011] 9428 [...TRACE...] engine.c(192) > axis2_engine_send end successfully > [Fri Feb 4 13:07:45 2011] 9428 [...TRACE...] engine.c(211) > Start:axis2_engine_receive > [Fri Feb 4 13:07:45 2011] 9428 [...TRACE...] engine.c(680) > Start:axis2_engine_invoke_phases > ======== > > Means that the server closes the connection already while the client has > not even started to receive the response data ... That in turn would > mean that we now rely on the OS buffering data written into the socket. > > Hmmm, is that intended? Is that more a developers topic? > > Rgds, > tge > > On Fri, 2011-02-04 at 11:53 +0100, Thomas Gentsch wrote: > > Hi all, > > > > sorry, I have to re-open this, the corrupted XML was my fault (while > > editing the test data). > > Basically it still does not work. In my real prog I have added some more > > printouts (also in Axis code): > > > > axiom_node_t * SOAPServer::invokeFunc(const axutil_env_t * env, > > axiom_node_t * n) > > { > > DataObjectPtr req = sdoFromAxiom(n); > > > > DataObjectPtr resp = doTheWork(req); > > > > printf("TGE: SDO Resp:\n"); > > dumpSDO(resp); > > > > axiom_node_t * ax = sdoToAxiom(resp); > > printf("TGE: Axiom Resp:\n"); > > dumpAxiom(ax); > > > > return ax; > > } > > > > Now what I see is that the incoming request gets transformed from Axiom > > to SDO nicely, will be processed, and I also see the response SDO "resp" > > as complete printed out (dumpSDO). > > Then, in sdoToAxiom() I convert the response SDO back to an Axiom node > > (as stated earlier, using the SDO AxiomHelper) and print it to stdout > > (see below). Here I have added a printout into the guththila XML writer > > and can see that *all* tags are properly written. > > Now something funny happens: > > In dumpAxiom() I originally had this: > > > > axiom_xml_writer_t * w = axiom_xml_writer_create_for_memory(env, > > NULL, AXIS2_TRUE, 0, AXIS2_XML_PARSER_TYPE_BUFFER); > > axiom_output_t * om_output = axiom_output_create(env, w); > > axiom_node_serialize(n, env, om_output); > > axis2_char_t * buf = (axis2_char_t *) axiom_xml_writer_get_xml(w, env); > > fprintf(f, "\nData: '%s'\n", buf); > > > > which yields a truncated output string: > > Data: '24 > However, note the closing single quote which is part of the fprintf > > above. > > Now, (before noticing that) I have added: > > fflush(f); > > after the fprintf and voila, the printout was complete. Does not make > > any sense to me ... :-| > > > > However, now, still my Axis2c client prog receives a truncated response > > only, even (now the next interesting piece), again the guththila XML > > writer runs again in the server (serializing the Axiom into the socket, > > I guess ...) after the fprintf above and I can see all XML tags written > > correctly. > > > > Is it possible that this is not a question of the XML reader/writer but > > rather that the SOAP client is not fast enough reading the response > > while the server already closes the connection or something like this? > > I'm still digging further, but any hints are welcome! :-) > > > > Rgds, > > tge > > > > On Sun, 2011-01-30 at 13:21 +0100, Thomas Gentsch wrote: > > > Hi all, > > > > > > sorry, not Axis' fault ... the input.xml is wrong. Hard to spot, but > > > there was an opening tag missing. > > > However > > > - still unclear why the two writer types behave that differently ?? > > > - the error message in the log might be a bit better (my input.xml > > > consists of *one* line of a length of 460000 chars, hence line 1 :-) > > > maybe a char offset might have helped) > > > I'll update Jira ... > > > > > > Rgds, > > > tge > > > > > > On Thu, 2011-01-27 at 18:43 +0100, Thomas Gentsch wrote: > > > > Hi Damitha, > > > > > > > > if you refer to AXIS2C-448 - I have that in. > > > > > > > > Besides of that I haven't found anything :-| (searching for > > > > axiom_xml_reader or axiom_xml_writer) > > > > I'll dig more but into it but of course I could save that effort of > > > > somebody else had an idea :-) > > > > > > > > Also, might be interesting to run that against 1.7 ... > > > > > > > > Rgds + thx, > > > > tge > > > > > > > > On Thu, 2011-01-27 at 22:26 +0530, Damitha Kumarage wrote: > > > > > Hi Thomas, > > > > > I remember similar problem appeared in a previous thread. Could you > > > > > please search in mail archives? > > > > > > > > > > Thanks, > > > > > Damitha > > > > > Thomas Gentsch wrote: > > > > > > Just opened AXIS2C-1534 and attached the test progs + data. > > > > > > Any help is appreciated!! > > > > > > Rgds, > > > > > > tge > > > > > > > > > > > > On Thu, 2011-01-27 at 13:16 +0100, Thomas Gentsch wrote: > > > > > > > > > > > >> Minor correction in the code below (I merged two functions) ... > > > > > >> > > > > > >> On Thu, 2011-01-27 at 13:03 +0100, Thomas Gentsch wrote: > > > > > >> > > > > > >>> Hi all, > > > > > >>> > > > > > >>> I'm still with Axis2c 1.6 and have a problem (de)serializing an XML > > > > > >>> string. Is there anything know about such a problem (haven't found > > > > > >>> anything in Jira)? > > > > > >>> > > > > > >>> The essential piece is a string buffer containing an XML structure, > > > > > >>> which is created from a Tuscany SDO. Then that string buffer gets > > > > > >>> converted to an Axiom Node (using the Tuscany AxiomHelper) to be > > > > > >>> returned to the calling Axis2c client. Normally this works nicely, but > > > > > >>> if the XML is very long (in my example 484460 bytes), it gets truncated. > > > > > >>> > > > > > >>> I can see the string buffer "str" (below) as still intact. Then, > > > > > >>> essentially the following happens: > > > > > >>> =========== > > > > > >>> AxiomHelper::toAxiomNode() > > > > > >>> { > > > > > >>> ... > > > > > >>> axiom_xml_reader_t * reader = > > > > > >>> axiom_xml_reader_create_for_memory( > > > > > >>> the_env, (void*)str, strlen(str), > > > > > >>> (const axis2_char_t *)"UTF-8", AXIS2_XML_PARSER_TYPE_BUFFER); > > > > > >>> axiom_stax_builder_t* builder = > > > > > >>> axiom_stax_builder_create(the_env, reader); > > > > > >>> axiom_document_t* document = > > > > > >>> axiom_stax_builder_get_document(builder, the_env); > > > > > >>> axiom_node_t* root_node = > > > > > >>> axiom_document_get_root_element(document, the_env); > > > > > >>> axiom_document_build_all(document, the_env); > > > > > >>> > > > > > >> The code above is function1 and returns "document". > > > > > >> Below is function2 dealing with that document. > > > > > >> > > > > > >> > > > > > >>> axiom_node_t* root_node = > > > > > >>> axiom_document_get_root_element(doc, the_env); > > > > > >>> return root_node; > > > > > >>> } > > > > > >>> > > > > > >>> Now I have the Axiom node "n" and dump it (just for testing): > > > > > >>> > > > > > >>> axiom_xml_writer_t * w = axiom_xml_writer_create_for_memory(env, > > > > > >>> NULL, AXIS2_TRUE, 0, AXIS2_XML_PARSER_TYPE_BUFFER); > > > > > >>> axiom_output_t * om_output = axiom_output_create(env, w); > > > > > >>> axiom_node_serialize(n, env, om_output); > > > > > >>> axis2_char_t * buf = (axis2_char_t *) axiom_xml_writer_get_xml(w, env); > > > > > >>> > > > > > >>> fprintf(f, "\nData: '%s'\n", buf); > > > > > >>> ============= > > > > > >>> > > > > > >>> and the output is truncated to 17253 bytes ending with some control > > > > > >>> characters. > > > > > >>> As said, that code has been working for months/years now w/o any > > > > > >>> problems, so I believe that it is OK in general and I rather suspect the > > > > > >>> Axis2c XML handling in conjunction with large buffers ... > > > > > >>> > > > > > >>> I'll extract the pieces into a standalone testprog for easier > > > > > >>> troubleshooting - but still: Any ideas? > > > > > >>> > > > > > >>> Many thx!!! > > > > > >>> tge > > > > > >>> > > > > > >>> > > > > > >> > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: c-user-unsubscribe@axis.apache.org > > For additional commands, e-mail: c-user-help@axis.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: c-user-unsubscribe@axis.apache.org For additional commands, e-mail: c-user-help@axis.apache.org