axis-c-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stadelmann Josef" <josef.stadelm...@axa-winterthur.ch>
Subject AW: AW: transfer limitations
Date Mon, 05 Jul 2010 12:50:04 GMT
Hope this helps you?`!

 

Yes, how did I strike it out? I am using WSIT from HP to integrate legacy systems. In short,
big (huge) OMElement is arriving at the JAVA side, (could well contain Bas64 encoded data)
it is then serialized to a very large (huge) xml string; which is then pass to an out-of-process
server via some sort of WsiIpcContext (IPC Interprocess Communication Channel); JNI is involved
too. 

 

….

 

When my large data arrives at the C-Legacy Server passed by WSIT ToolKit, I get a OpenVMS
Descriptor which has a Length Field and a Pointer to the data. 

 

….

 

And now comes what I did not understand so far but since I try to help you, I was able to
fix my own problem today. 

So what I did not understand is as follow.

 

My data arrives at WSI$$fktmap() in *pbIndata with a inLen of Bystes set to 91344

The code you see below is HP-WSIT-generated-code, and it works fine. 

 

The second green code line shows how this data is passed to my own routine fktmap() which
is declared as

‘__int32 fktmap (char * p1, int inLen, dsc_d * p2);’

 

Given that pbInData can be passed further on via a P1 of type char* then please explain me
the problem I am faced with.

 

1.  In one case when I do

Unsigned __int16 scnt = strlen(P1); 

I get scnt back as 25808 bytes 

BUT 

inLen shows me 91344

Why?

 

2.  When I save P1 in a file, all bytes are there
BUT
When I pass it to 

xml_reader = axiom_xml_reader_create_for_memory(env, payload_inp, scnt, encoding, AXIS2_XML_PARSER_TYPE_BUFFER);

Then I get the following errors in my log file for one transaction, at different call levels.

 

[Mon Jul  5 14:10:28 2010] [error] DKB3:[SW-PROJEKTE.webservices.axis2.trunk.c.axiom.src.parser]libxml2_reader_wrapper.c;36(886)


  expected '>'

***************

 -- SEVERITY_ERROR

***************

[Mon Jul  5 14:10:28 2010] [error] DKB3:[SW-PROJEKTE.webservices.axis2.trunk.c.axiom.src.parser]libxml2_reader_wrapper.c;36(886)


  Opening and ending tag mismatch: Fldnam line 0 and F

***************

 -- SEVERITY_ERROR

***************

[Mon Jul  5 14:10:28 2010] [error] DKB3:[SW-PROJEKTE.webservices.axis2.trunk.c.axiom.src.parser]libxml2_reader_wrapper.c;36(886)


  Extra content at the end of the document

***************

 -- SEVERITY_ERROR

***************

[Mon Jul  5 14:10:28 2010] [error] DKB3:[SW-PROJEKTE.webservices.axis2.trunk.c.axiom.src.parser]libxml2_reader_wrapper.c;36(427)
 

  error occured in reading xml stream

[Mon Jul  5 14:10:28 2010] [debug] DKB3:[SW-PROJEKTE.SPEZSRVW.axawl.spezpla.servers.SPgServer]SPg-legacy.c;100(1593)

***************

 

 

The source of the problem is now fixed; 

 

scnt receiving from strlen(P1) the byte count was of unsigned __int16, too short to keep a
value bigger then 65535!!!!!

 

scnt must be of type unsigned __int32 and only then it is able to keep the number 91344 !

 

One can take the calculator and enter 91344 as decimal number, then convert to hex and subtract
10000 Hex, then convert back to decimal.

A too short type of an integer maskes out what strlen() returns correctly.

 

 

BUT when the proper length of bytes is passed to axiom_xml_reader_create_for_memory() 

the routine works fine and does not bail out with a -- SEVERITY ERROR

 

Conclusion of the day! If you work with very long strings (longer then FFFF === 65535) it
is essential that you use 

“unsigned __int32 OR unsigned __int64 as type for then “scnt” = strlen(P1)” with P1
pointing to a string longer than 65535 characters.

 

 

Now back to you: How do you detect that your XML is broken? Which routine informs you about
that?

 

Sepp

 

 

///////////////////////////////////////////////////////////

//

// function: WSI$$fktmap

//

//

//

///////////////////////////////////////////////////////////

static unsigned int WSI$$fktmap (unsigned int  sessionID,

                                  MSGB          *pbInData,

                                  int           inLen,

                                  MSGB          **ppbOutData,

                                  int           *pOutLen)

{

#ifdef __DEBUG__

    lib$signal(SS$_DEBUG);

#endif

    char    wsi$ret [4];

    dsc_sd  *wsi$p_dx;

    dsc_a   *wsi$arydsc;

    int     wsi$pnum = 2;

    int     wsi$size = 0;

    encoder wsi$enc;

    dsc_sd  wsi$ret_dx = {4, DSC$K_DTYPE_L, DSC$K_CLASS_S, wsi$ret, 0, 0, 0};

    internal_context_t *wsi$context = (internal_context_t *) sessionID;

 

    wsi$p_dx = (dsc_sd *) pbInData;

 

    // All pointers are converted from relative to absolute addresses here.

    wsi$p_dx[0].dsc$a_pointer += (int) wsi$p_dx;

    wsi$p_dx[1].dsc$a_pointer += (int) wsi$p_dx;

 

    // Special parameters, such as Floats & Dynamic Strings, are fixup here.

    wsi$dstr_fix( &wsi$p_dx[1] );

 

    // Structure parameters are fixed up here.

 

#pragma names restore   // we escape to Spg-Legacy and Axis2 code here which is compiled AS_IS
& SHORTENED

    // user-side function call

    *(__int32 *)wsi$ret_dx.dsc$a_pointer =  fktmap((char *) wsi$p_dx[0].dsc$a_pointer, inLen,
(dsc_d *) &wsi$p_dx[1]);

#pragma names save 

#pragma names uppercase, shortened        // and back in HP WSIt world go for UPPERCASE &
SHORTENED

 

    // OUT & IN/OUT Floats get converted back to IEEE here.

    // OUT & IN/OUT structures get fixup up here.

    // OUT & IN/OUT strings get fixed up here 

 

    // Determine size of output buffer, size of static + dynamic out params

    wsi$size = 28 + wsi$p_dx[1].dsc$w_length;

    wsi$context->mem_disp->malloc( wsi$context->mem_disp->cnxid, wsi$size, (void
**) ppbOutData );

    wsi$init_encoder( &wsi$enc, *ppbOutData, wsi$pnum );

 

    // Package up the parameters for returning here.

    wsi$add_param( &wsi$ret_dx,  &wsi$enc );

    wsi$add_param( &wsi$p_dx[1], &wsi$enc );

 

    // Deallocate any memory used by Dynamic Strings here.

    wsi$dstr_free( &wsi$p_dx[1] );

 

    // Return here

    *pOutLen = wsi$size;

    return error_status_ok;

}

 

 

-----Ursprüngliche Nachricht-----
Von: Akos Marton [mailto:makos999@gmail.com] 
Gesendet: Montag, 5. Juli 2010 11:13
An: Apache AXIS C User List
Betreff: Re: AW: transfer limitations

 

Hi Sepp,

 

On Mon, Jul 5, 2010 at 9:24 AM, Stadelmann Josef

<josef.stadelmann@axa-winterthur.ch> wrote:

> Given you send ALL your XML data into a file and parse it. Does the parser bail out somewhere?

I also thought it, but varying the sent amount of data, thus XML is

not broken at the same point in every case.

 

> 

> I just have similar situation. There is (in my mind) too much data delivered; AND - Are
you working with some kind of Strings through char*?

Yes, axis2_char_t*.

 

> 

> How long can a char* String be? How far does this pointer reach? Writing all my XML data
into a file and checking the XML Structure shows a very well formed XML file; But any attempt
to parse it through the axis2_libxml2_reader_wrapper is a no go and it bails out with various
errors toward the end of the files-content.- no I  memory.

I may not got what you mean above about the long of char*. You can

allocate arbitrary (depends on your OS) amount of space, using a char*

if you use dynamic memory management. How did you strike it out?

 

> 

> Pointer size? 16, 32, 64?

The size of pointer depends on your hardware and operating system, so

that on 32bit operating system pointer has size of 4bytes.

 

Now, I'm experimenting and learning how to configure the .Net side

using WCF-technologies and set it to send data in chunks using MTOM.

It seems for me, .Net-.Net can communicate with each other very well

(of course), but needs more portability configuration to make it to be

able to handle MTOM.

In spite this we don't understand the reason of broken XMLs.

 

Akos

 

> 

> Sepp

> 

> -----Ursprüngliche Nachricht-----

> Von: Akos Marton [mailto:makos999@gmail.com]

> Gesendet: Freitag, 18. Juni 2010 10:20

> An: Apache AXIS C User List

> Betreff: Re: AW: transfer limitations

> 

> The problem is still not solved, so that any idea and answer is appreciated.

> 

> You can read a mail on c-user list about the same problem by Jan-fon,

> except that he found the same situation on client-side.

> subject:        'Client to receive large data problem'

> date:           Thu, 25 Feb 2010 11:07:53 GMT

> http://mail-archives.apache.org/mod_mbox/axis-c-user/201002.mbox/%3CCC8FF067F7F4E54796AFDF21339BA0FC134A94BA@orsmsx505.amr.corp.intel.com%3E

> 

> Hope it helps,

> Akos Marton

> 

> 

> Stadelmann Josef wrote:

>> Have you setup to send in chunks from CL to SV or do you expect that more than 4
MB will go at once, within one request object?

>> Sepp

>> 

>> -----Ursprüngliche Nachricht-----

>> Von: Akos Marton [mailto:makos999@gmail.com]

>> Gesendet: Freitag, 11. Juni 2010 15:31

>> An: Apache AXIS C User List

>> Betreff: Re: transfer limitations

>> 

>> The XML is broken every time at different part. I tried to configure httpd server
with switching out many known size limitation and run axis2c under this, but it also doesn't
send the whole XML.

>> 

>> The sent data is just broken somewhere. I watched it with tcpdump.

>> Smaller XMLs can be transferred.

>> 

>> Well, written a short tester code, which reads the necessary XML from file and parses,
decodes it perfectly, so that the algorithm and code must do the right.

>> 

>> Any idea why the sending process is being broken?

>> 

>> Thanks,

>> mAkos

>> 

>> 

>> Akos Marton wrote:

>>> Hi List,

>>> 

>>> Is there any limitation about the amount of data in a certain XML-tag?

>>> In case of true, could I increase it somehow?

>>> For instance <image> ... </image> by the size of 4Megabytes of base64

>>> code. The whole XML data does not reach the server (have cut at

>>> certain part), but a smaller amount of data reaches it fully.

>>> 

>>> Could be httpd.conf a solution? I do not know how to locate it for

>>> axis2c binary version.

>>> 

>>> Thanks,

>>> mAkos

>>> 

>>> 

>> 

>> 

>> 

>> ---------------------------------------------------------------------

>> To unsubscribe, e-mail: c-user-unsubscribe@axis.apache.org

>> For additional commands, e-mail: c-user-help@axis.apache.org

>> 

>> 

> 

> 

> ---------------------------------------------------------------------

> To unsubscribe, e-mail: c-user-unsubscribe@axis.apache.org

> For additional commands, e-mail: c-user-help@axis.apache.org

> 

> 

> ---------------------------------------------------------------------

> To unsubscribe, e-mail: c-user-unsubscribe@axis.apache.org

> For additional commands, e-mail: c-user-help@axis.apache.org

> 

> 

 

 

 

-- 

People seldom notice clothes, if you wear a big smile.

My grandmother uses Ubuntu, don't you think so?!

OLVASD: http://www.xkcd.org/

 

---------------------------------------------------------------------

To unsubscribe, e-mail: c-user-unsubscribe@axis.apache.org

For additional commands, e-mail: c-user-help@axis.apache.org

 

Mime
View raw message