axis-c-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Gentsch ...@e-tge.de>
Subject Re: XML reader/writer buffer size problem?
Date Fri, 04 Feb 2011 20:35:23 GMT
OK, I give up. I tried with the libxml2 parser and with that it works.
Here are my latest findings, just in case somebody wants to look into
this further ...

===================

Digging further, now I'm back at the XML writer again. Have found this:
axis2_http_transport_sender_invoke()
{
 ...
 buffer = (axis2_char_t *) axiom_xml_writer_get_xml(xml_writer, env);
 buffer_size = axiom_xml_writer_get_xml_size(xml_writer, env);
/* TGE */
printf("TGE: http sender: %d bytes\n", buffer_size);
printf("TGE: http sender: '%s'\n", buffer);
fflush(stdout);

...
  int rc = axutil_stream_write(out_stream, env, buffer, buffer_size);
printf("TGE: http sender stream_write returned %d.\n", rc);
}

and I get again the truncated XML. In other words, here happens the same
as I had before in my dump() function - the axiom node gets printed into
a buffer, but not completely.

The guththila XML writer prints everything correctly - but not, as I
initially thought, into the socket, but again into another buffer (as
above).
The size of the buffer is correct (975000), however the actual buffer is
not:
...
TGE: http sender: 975921 bytes
TGE: http sender: '<soapenv:Envelope ... <tns3:num300000000>24<//^M'
TGE: http sender stream_write returned 975921.^M
TGE: axutil_stream_write_socket 17 bytes^M
TGE: axutil_stream_write_socket wrote 17 bytes^M
TGE: axutil_stream_write_socket 36 bytes^M
TGE: axutil_stream_write_socket wrote 36 bytes^M
TGE: axutil_stream_write_socket 49 bytes^M
TGE: axutil_stream_write_socket wrote 49 bytes^M
TGE: axutil_stream_write_socket 50 bytes^M
TGE: axutil_stream_write_socket wrote 50 bytes^M
TGE: axutil_stream_write_socket 24 bytes^M
TGE: axutil_stream_write_socket wrote 24 bytes^M
TGE: axutil_stream_write_socket 2 bytes^M
TGE: axutil_stream_write_socket wrote 2 bytes^M
TGE: axutil_stream_write_socket 245761 bytes^M
TGE: axutil_stream_write_socket wrote 245761 bytes^M
TGE: axutil_stream_write_socket 2 bytes^M
TGE: axutil_stream_write_socket wrote 2 bytes^M

The 245761 is about the length of data actually sent over the wire and
the same as printed as out truncated.

Now, further ... it turned out that while constructing the buffer, the
guththila XML writer finds an empty/illegal prefix:

TGE Guth chars: '1'^M
TGE Guth elem end pfx: ''^M
TGE Guth elem end: 'num300000000'^M

and writes:
TGE: Copying buffer[4](262137): 'num300000000>1</'^M
or:
... 31 3c 2f 00 0d 00 04 3a ...
equals:
... ></ rubbish :

Now I tried to understand what's going on in the guththila parser but
give up for now.

The crazy thing is that it works ok for 25000 tags within that stupid
structure and fails 2 times (found another occurence of the same) ...
actually I just notice:
Both cases are the first closing tags in buffers 4 and 5:
TGE: Copying buffer[4](262137): 'num300000000>1</'
TGE: Copying buffer[5](468040): 'num8000000>339386</'

and the previous buffer ends like this:
  ...><tns3:
in both cases

===================

On Fri, 2011-02-04 at 13:21 +0100, Thomas Gentsch wrote:
> Must be something like that.
> Essentially:
> 
> 1. In the SOAP server I can see all data correctly, and I can see that
> the guththila XML writer prints all XML tags correctly.
> 2. In the SOAP client (I have added prinouts to the guththila XML
> parser) I see the truncated XML - at some point it just stops.
> 
> Now I have added a few more trace statements into the network handler
> and now I can see this (PID 9419 is the server, 9428 the client):
> ========
> [Fri Feb  4 13:07:44 2011] 9419 [...TRACE...] msg_recv.c(459)
> [axis2]Exit:axis2_msg_recv_receive_impl
> [Fri Feb  4 13:07:44 2011] 9419 [...TRACE...] engine.c(323)
> Exit:axis2_engine_receive
> [Fri Feb  4 13:07:44 2011] 9428 [...TRACE...] http_sender.c(1593)
> Entry:axis2_http_sender_process_response
> [Fri Feb  4 13:07:44 2011] 9428 [...TRACE...] http_sender.c(1614)
> Exit:axis2_http_sender_process_response
> [Fri Feb  4 13:07:44 2011] 9428 [debug] http_transport_sender.c(822) OP
> name axutil_qname_get_localpart = http://www.w3.org/2004/08/wsdl/out-in
> [Fri Feb  4 13:07:44 2011] 9428 [debug] soap_builder.c(945) identified
> soap version is soap12
> [Fri Feb  4 13:07:44 2011] 9419 [error] network_handler.c(208) TGE:
> Shutting down socket
> [Fri Feb  4 13:07:44 2011] 9419 [error] network_handler.c(218) TGE:
> Waiting for response ...
> [Fri Feb  4 13:07:44 2011] 9419 [error] network_handler.c(224) TGE:
> Returned. Closing socket.
> [Fri Feb  4 13:07:44 2011] [info]  Request served in 22.036 seconds
> [Fri Feb  4 13:07:45 2011] 9428 [...TRACE...]
> http_transport_sender.c(596) Exit:axis2_http_transport_sender_invoke
> [Fri Feb  4 13:07:45 2011] 9428 [...TRACE...] engine.c(192)
> axis2_engine_send end successfully
> [Fri Feb  4 13:07:45 2011] 9428 [...TRACE...] engine.c(211)
> Start:axis2_engine_receive
> [Fri Feb  4 13:07:45 2011] 9428 [...TRACE...] engine.c(680)
> Start:axis2_engine_invoke_phases
> ========
> 
> Means that the server closes the connection already while the client has
> not even started to receive the response data ... That in turn would
> mean that we now rely on the OS buffering data written into the socket.
> 
> Hmmm, is that intended? Is that more a developers topic?
> 
> Rgds,
>   tge
> 
> On Fri, 2011-02-04 at 11:53 +0100, Thomas Gentsch wrote:
> > Hi all,
> > 
> > sorry, I have to re-open this, the corrupted XML was my fault (while
> > editing the test data).
> > Basically it still does not work. In my real prog I have added some more
> > printouts (also in Axis code):
> > 
> > axiom_node_t * SOAPServer::invokeFunc(const axutil_env_t * env,
> >                                       axiom_node_t * n)
> > {
> >   DataObjectPtr req = sdoFromAxiom(n);
> > 
> >   DataObjectPtr resp = doTheWork(req);
> > 
> >   printf("TGE: SDO Resp:\n");
> >   dumpSDO(resp);
> > 
> >   axiom_node_t * ax = sdoToAxiom(resp);
> >   printf("TGE: Axiom Resp:\n");
> >   dumpAxiom(ax);
> > 
> >   return ax;
> > }
> > 
> > Now what I see is that the incoming request gets transformed from Axiom
> > to SDO nicely, will be processed, and I also see the response SDO "resp"
> > as complete printed out (dumpSDO).
> > Then, in sdoToAxiom() I convert the response SDO back to an Axiom node
> > (as stated earlier, using the SDO AxiomHelper) and print it to stdout
> > (see below). Here I have added a printout into the guththila XML writer
> > and can see that *all* tags are properly written.
> > Now something funny happens:
> > In dumpAxiom() I originally had this:
> > 
> > axiom_xml_writer_t * w = axiom_xml_writer_create_for_memory(env,
> >   NULL, AXIS2_TRUE, 0, AXIS2_XML_PARSER_TYPE_BUFFER);
> > axiom_output_t     * om_output = axiom_output_create(env, w);
> > axiom_node_serialize(n, env, om_output);
> > axis2_char_t * buf = (axis2_char_t *) axiom_xml_writer_get_xml(w, env);
> > fprintf(f, "\nData: '%s'\n", buf);
> > 
> > which yields a truncated output string:
> >  Data: '<listresponse ... <tns3:num300000000>24<//^M'
> > However, note the closing single quote which is part of the fprintf
> > above.
> > Now, (before noticing that) I have added:
> >  fflush(f);
> > after the fprintf and voila, the printout was complete. Does not make
> > any sense to me ... :-|
> > 
> > However, now, still my Axis2c client prog receives a truncated response
> > only, even (now the next interesting piece), again the guththila XML
> > writer runs again in the server (serializing the Axiom into the socket,
> > I guess ...) after the fprintf above and I can see all XML tags written
> > correctly.
> > 
> > Is it possible that this is not a question of the XML reader/writer but
> > rather that the SOAP client is not fast enough reading the response
> > while the server already closes the connection or something like this?
> > I'm still digging further, but any hints are welcome! :-)
> > 
> > Rgds,
> >   tge
> > 
> > On Sun, 2011-01-30 at 13:21 +0100, Thomas Gentsch wrote:
> > > Hi all,
> > > 
> > > sorry, not Axis' fault ... the input.xml is wrong. Hard to spot, but
> > > there was an opening tag <num50000000> missing.
> > > However
> > >  - still unclear why the two writer types behave that differently ??
> > >  - the error message in the log might be a bit better (my input.xml 
> > >    consists of *one* line of a length of 460000 chars, hence line 1 :-) 
> > >    maybe a char offset might have helped)
> > > I'll update Jira ...
> > > 
> > > Rgds,
> > >   tge
> > > 
> > > On Thu, 2011-01-27 at 18:43 +0100, Thomas Gentsch wrote:
> > > > Hi Damitha,
> > > > 
> > > > if you refer to AXIS2C-448 - I have that in.
> > > > 
> > > > Besides of that I haven't found anything :-| (searching for
> > > > axiom_xml_reader or axiom_xml_writer)
> > > > I'll dig more but into it but of course I could save that effort of
> > > > somebody else had an idea :-)
> > > > 
> > > > Also, might be interesting to run that against 1.7 ...
> > > > 
> > > > Rgds + thx,
> > > >   tge
> > > > 
> > > > On Thu, 2011-01-27 at 22:26 +0530, Damitha Kumarage wrote:
> > > > > Hi Thomas,
> > > > > I remember similar problem appeared in a previous thread. Could you

> > > > > please search in mail archives?
> > > > > 
> > > > > Thanks,
> > > > > Damitha
> > > > > Thomas Gentsch wrote:
> > > > > > Just opened AXIS2C-1534 and attached the test progs + data.
> > > > > > Any help is appreciated!!
> > > > > > Rgds,
> > > > > >   tge
> > > > > >
> > > > > > On Thu, 2011-01-27 at 13:16 +0100, Thomas Gentsch wrote:
> > > > > >   
> > > > > >> Minor correction in the code below (I merged two functions)
...
> > > > > >>
> > > > > >> On Thu, 2011-01-27 at 13:03 +0100, Thomas Gentsch wrote:
> > > > > >>     
> > > > > >>> Hi all,
> > > > > >>>
> > > > > >>> I'm still with Axis2c 1.6 and have a problem (de)serializing
an XML
> > > > > >>> string. Is there anything know about such a problem
(haven't found
> > > > > >>> anything in Jira)?
> > > > > >>>
> > > > > >>> The essential piece is a string buffer containing an
XML structure,
> > > > > >>> which is created from a Tuscany SDO. Then that string
buffer gets
> > > > > >>> converted to an Axiom Node (using the Tuscany AxiomHelper)
to be
> > > > > >>> returned to the calling Axis2c client. Normally this
works nicely, but
> > > > > >>> if the XML is very long (in my example 484460 bytes),
it gets truncated.
> > > > > >>>
> > > > > >>> I can see the string buffer "str" (below) as still intact.
Then,
> > > > > >>> essentially the following happens:
> > > > > >>> ===========
> > > > > >>> AxiomHelper::toAxiomNode()
> > > > > >>> {
> > > > > >>> ...
> > > > > >>> axiom_xml_reader_t * reader =
> > > > > >>> axiom_xml_reader_create_for_memory(
> > > > > >>>   the_env, (void*)str, strlen(str),
> > > > > >>>   (const axis2_char_t *)"UTF-8", AXIS2_XML_PARSER_TYPE_BUFFER);
> > > > > >>> axiom_stax_builder_t* builder =
> > > > > >>>   axiom_stax_builder_create(the_env, reader);
> > > > > >>> axiom_document_t* document =
> > > > > >>>   axiom_stax_builder_get_document(builder, the_env);
> > > > > >>> axiom_node_t* root_node =
> > > > > >>>   axiom_document_get_root_element(document, the_env);
> > > > > >>> axiom_document_build_all(document, the_env);
> > > > > >>>       
> > > > > >> The code above is function1 and returns "document".
> > > > > >> Below is function2 dealing with that document.
> > > > > >>
> > > > > >>     
> > > > > >>> axiom_node_t* root_node =
> > > > > >>>   axiom_document_get_root_element(doc, the_env);
> > > > > >>> return  root_node;
> > > > > >>> }
> > > > > >>>
> > > > > >>> Now I have the Axiom node "n" and dump it (just for
testing):
> > > > > >>>
> > > > > >>> axiom_xml_writer_t * w = axiom_xml_writer_create_for_memory(env,
> > > > > >>>   NULL, AXIS2_TRUE, 0, AXIS2_XML_PARSER_TYPE_BUFFER);
> > > > > >>> axiom_output_t * om_output = axiom_output_create(env,
w);
> > > > > >>> axiom_node_serialize(n, env, om_output);
> > > > > >>> axis2_char_t * buf = (axis2_char_t *) axiom_xml_writer_get_xml(w,
env);
> > > > > >>>
> > > > > >>> fprintf(f, "\nData: '%s'\n", buf);
> > > > > >>> =============
> > > > > >>>
> > > > > >>> and the output is truncated to 17253 bytes ending with
some control
> > > > > >>> characters.
> > > > > >>> As said, that code has been working for months/years
now w/o any
> > > > > >>> problems, so I believe that it is OK in general and
I rather suspect the
> > > > > >>> Axis2c XML handling in conjunction with large buffers
...
> > > > > >>>
> > > > > >>> I'll extract the pieces into a standalone testprog for
easier
> > > > > >>> troubleshooting - but still: Any ideas?
> > > > > >>>
> > > > > >>> Many thx!!!
> > > > > >>>   tge
> > > > > >>>
> > > > > >>>       
> > > > > >>
> > > 
> > 
> > 
> > 
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: c-user-unsubscribe@axis.apache.org
> > For additional commands, e-mail: c-user-help@axis.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: c-user-unsubscribe@axis.apache.org
For additional commands, e-mail: c-user-help@axis.apache.org


Mime
View raw message