axis-c-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Samisa Abeysinghe <sam...@wso2.com>
Subject Re: AW: transfer limitations
Date Mon, 05 Jul 2010 14:13:10 GMT
You can mix them up. That is the whole point of MTOM.

Samisa...

On Mon, Jul 5, 2010 at 7:38 PM, Stadelmann Josef <
josef.stadelmann@axa-winterthur.ch> wrote:

>  Samisa,
>
>
>
> you right;
>
>
>
> But that is why I like to know if I can mix the two forms in one web
> service, in one web service server;
>
>
>
> My current SpezplaService knows
>
> OMElement login(OMElement e),
>
> OMElement fktmap(OMElement e),
>
> OMElement logout(OMElement),
>
>
>
> Now, can I add 2 more methods to do
>
> putFile( . . . FileData, String FilePath ); and
>
> . . . getFile(String FilePath);
>
>
>
> IN THE SAME SERVICE CLASS?
>
>
>
>
>
> And as nearly everybody sooner then later has to transive larger files, why
> not make it a AXIS2 standard somehow?
>
>
>
> Maybe someone has good examples for me, best in either direction, and with
> promise and hope that WCF 3.5 .NET can cooperate too! J
>
>
>
> Josef
>
>
>
>
>
> *Von:* Samisa Abeysinghe [mailto:samisa@wso2.com]
> *Gesendet:* Montag, 5. Juli 2010 15:58
>
> *An:* Apache AXIS C User List
> *Betreff:* Re: AW: transfer limitations
>
>
>
> If you want to dig into the content and pick stuff, then you got to build
> AXIOM and get the job done. So MTOM would not help in that case.
>
>
>
> However, if you are mainly interested in getting the file to the other
> side, then MTOM is the choice.
>
>
>
> It looks to me that, you want something int he middle, as you have to pick
> key/value from what you transfer.
>
>
>
> I got to admit, that building the whole thing as AXIOM for bulky stuff is
> going to be very costly, and we have seen Axis2/C degrade with larger
> messages in terms of performance. So if you try and get the content as MTOM
> and use some other logic to do the processing rather than AXIOM build, that
> would be more efficient.
>
>
>
> Samisa...
>
> On Mon, Jul 5, 2010 at 6:57 PM, Stadelmann Josef <
> josef.stadelmann@axa-winterthur.ch> wrote:
>
> Samisa,
>
>
>
> Thank you, and yes, it is not a too good idea to build a mega AXIOM. But it
> is as it is; We mainly use AXIOM to reflect buckets full of name/value pairs
> from transaction mask fields. Transactions in long lasting sessions; So we
> send two buckets (call it workspaces) to pass the past and current workspace
> to the server and receive the output-workspace from the server. On arrival
> we de-serialize the xml into an Axiom model and from the Axiom we fill
> hash-tables, we can merge hash-table easy. Then our legacy apps gets/puts by
> key-name the value; once done, code returns to create the output axiom
> model, which gets serialized into xml.
>
>
>
> It was designed for transaction masks (VB Forms with many fields); it was
> then overused by VB programmers to transmit files! In the future I am going
> to enhance my server/service to use MTOM for file-transfers.
>
> The problem I am faced with is
>
>
>
>             Scope=”soapsession”
>
>
>
> Can you tell me how I can mix-in into my existing service 2 methods which
> runs in scope=”soapsession” to send/receive MTOM coded data.
>
>
>
> Why did I not do it until today;
>
>
>
> Because it is somewhere said, either at Microsoft .NET WCF 3.5 or AXIS2/J
> or /C that one can’t mix SOAP/XML and MTOM on the same service?
>
>
>
> If I am wrong I am more than happy to sniff into my existing larger service
> object the two MTOM methods getFile() and putFile();
>
>
>
> Josef
>
>
>
>
>
>
>
>
>
>
>
> *Von:* Samisa Abeysinghe [mailto:samisa@wso2.com]
> *Gesendet:* Montag, 5. Juli 2010 15:09
> *An:* Apache AXIS C User List
>
>
> *Betreff:* Re: AW: transfer limitations
>
>
>
> If you want to handle large chunks of data,  why not use MTOM?
>
>
>
> Also, there is a data source mode in Axis2/C to stream the chunk through.
>
>
>
> We use used Axis2/C to send mega bytes nearing giga range and it works for
> us for MTOM.
>
>
>
> For service case, I am not sure if it is a good idea to use AXIOM to build
> a mega range doc - that means the service design is busted.
>
>
>
> Samisa...
>
> On Mon, Jul 5, 2010 at 6:20 PM, Stadelmann Josef <
> josef.stadelmann@axa-winterthur.ch> wrote:
>
> Hope this helps you?`!
>
>
>
> Yes, how did I strike it out? I am using WSIT from HP to integrate legacy
> systems. In short, big (huge) OMElement is arriving at the JAVA side, (could
> well contain Bas64 encoded data) it is then serialized to a very large
> (huge) xml string; which is then pass to an out-of-process server via some
> sort of WsiIpcContext (IPC Interprocess Communication Channel); JNI is
> involved too.
>
>
>
> ….
>
>
>
> When my large data arrives at the C-Legacy Server passed by WSIT ToolKit, I
> get a OpenVMS Descriptor which has a Length Field and a Pointer to the data.
>
>
>
>
> ….
>
>
>
> And now comes what I did not understand so far but since I try to help you,
> I was able to fix my own problem today.
>
> So what I did not understand is as follow.
>
>
>
> My data arrives at WSI$$fktmap() in *pbIndata with a inLen of Bystes set to
> 91344
>
> The code you see below is HP-WSIT-generated-code, and it works fine.
>
>
>
> The second green code line shows how this data is passed to my own routine
> fktmap() which is declared as
>
> ‘__int32 fktmap (char * p1, int inLen, dsc_d * p2);’
>
>
>
> Given that pbInData can be passed further on via a P1 of type char* then
> please explain me the problem I am faced with.
>
>
>
> 1.  In one case when I do
>
> Unsigned __int16 scnt = strlen(P1);
>
> I get scnt back as 25808 bytes
>
> BUT
>
> inLen shows me 91344
>
> Why?
>
>
>
> 2.  When I save P1 in a file, all bytes are there
> BUT
> When I pass it to
>
> xml_reader = axiom_xml_reader_create_for_memory(env, payload_inp, scnt,
> encoding, AXIS2_XML_PARSER_TYPE_BUFFER);
>
> Then I get the following errors in my log file for one transaction, at
> different call levels.
>
>
>
> [Mon Jul  5 14:10:28 2010] [error]
> DKB3:[SW-PROJEKTE.webservices.axis2.trunk.c.axiom.src.parser]libxml2_reader_wrapper.c;36(886)
>
>
>   expected '>'
>
> ***************
>
>  -- SEVERITY_ERROR
>
> ***************
>
> [Mon Jul  5 14:10:28 2010] [error]
> DKB3:[SW-PROJEKTE.webservices.axis2.trunk.c.axiom.src.parser]libxml2_reader_wrapper.c;36(886)
>
>
>   Opening and ending tag mismatch: Fldnam line 0 and F
>
> ***************
>
>  -- SEVERITY_ERROR
>
> ***************
>
> [Mon Jul  5 14:10:28 2010] [error]
> DKB3:[SW-PROJEKTE.webservices.axis2.trunk.c.axiom.src.parser]libxml2_reader_wrapper.c;36(886)
>
>
>   Extra content at the end of the document
>
> ***************
>
>  -- SEVERITY_ERROR
>
> ***************
>
> [Mon Jul  5 14:10:28 2010] [error]
> DKB3:[SW-PROJEKTE.webservices.axis2.trunk.c.axiom.src.parser]libxml2_reader_wrapper.c;36(427)
>
>
>   error occured in reading xml stream
>
> [Mon Jul  5 14:10:28 2010] [debug]
> DKB3:[SW-PROJEKTE.SPEZSRVW.axawl.spezpla.servers.SPgServer]SPg-legacy.c;100(1593)
>
> ***************
>
>
>
>
>
> The source of the problem is now fixed;
>
>
>
> scnt receiving from strlen(P1) the byte count was of unsigned __int16, too
> short to keep a value bigger then 65535!!!!!
>
>
>
> scnt must be of type unsigned __int32 and only then it is able to keep the
> number 91344 !
>
>
>
> One can take the calculator and enter 91344 as decimal number, then convert
> to hex and subtract 10000 Hex, then convert back to decimal.
>
> A too short type of an integer maskes out what strlen() returns correctly.
>
>
>
>
>
> BUT when the proper length of bytes is passed to
> axiom_xml_reader_create_for_memory()
>
> the routine works fine and does not bail out with a -- SEVERITY ERROR
>
>
>
> Conclusion of the day! If you work with very long strings (longer then FFFF
> === 65535) it is essential that you use
>
> “unsigned __int32 OR unsigned __int64 as type for then “scnt” = strlen(P1)”
> with P1 pointing to a string longer than 65535 characters.
>
>
>
>
>
> Now back to you: How do you detect that your XML is broken? Which routine
> informs you about that?
>
>
>
> Sepp
>
>
>
>
>
> ///////////////////////////////////////////////////////////
>
> //
>
> // function: WSI$$fktmap
>
> //
>
> //
>
> //
>
> ///////////////////////////////////////////////////////////
>
> static unsigned int WSI$$fktmap (unsigned int  sessionID,
>
>                                   *MSGB          *pbInData,*
>
>                                   *int           inLen,*
>
>                                   MSGB          **ppbOutData,
>
>                                   int           *pOutLen)
>
> {
>
> #ifdef __DEBUG__
>
>     lib$signal(SS$_DEBUG);
>
> #endif
>
>     char    wsi$ret [4];
>
>     dsc_sd  *wsi$p_dx;
>
>     dsc_a   *wsi$arydsc;
>
>     int     wsi$pnum = 2;
>
>     int     wsi$size = 0;
>
>     encoder wsi$enc;
>
>     dsc_sd  wsi$ret_dx = {4, DSC$K_DTYPE_L, DSC$K_CLASS_S, wsi$ret, 0, 0,
> 0};
>
>     internal_context_t *wsi$context = (internal_context_t *) sessionID;
>
>
>
>     wsi$p_dx = (dsc_sd *) pbInData;
>
>
>
>     // All pointers are converted from relative to absolute addresses
> here.
>
>     wsi$p_dx[0].dsc$a_pointer += (int) wsi$p_dx;
>
>     wsi$p_dx[1].dsc$a_pointer += (int) wsi$p_dx;
>
>
>
>     // Special parameters, such as Floats & Dynamic Strings, are fixup
> here.
>
>     wsi$dstr_fix( &wsi$p_dx[1] );
>
>
>
>     // Structure parameters are fixed up here.
>
>
>
> #pragma names restore   // we escape to Spg-Legacy and Axis2 code here
> which is compiled AS_IS & SHORTENED
>
>     // user-side function call
>
>     *(__int32 *)wsi$ret_dx.dsc$a_pointer =  fktmap((char *)
> wsi$p_dx[0].dsc$a_pointer, inLen, (dsc_d *) &wsi$p_dx[1]);
>
> #pragma names save
>
> #pragma names uppercase, shortened        // and back in HP WSIt world go
> for UPPERCASE & SHORTENED
>
>
>
>     // OUT & IN/OUT Floats get converted back to IEEE here.
>
>     // OUT & IN/OUT structures get fixup up here.
>
>     // OUT & IN/OUT strings get fixed up here
>
>
>
>     // Determine size of output buffer, size of static + dynamic out params
>
>     wsi$size = 28 + wsi$p_dx[1].dsc$w_length;
>
>     wsi$context->mem_disp->malloc( wsi$context->mem_disp->cnxid, wsi$size,
> (void **) ppbOutData );
>
>     wsi$init_encoder( &wsi$enc, *ppbOutData, wsi$pnum );
>
>
>
>     // Package up the parameters for returning here.
>
>     wsi$add_param( &wsi$ret_dx,  &wsi$enc );
>
>     wsi$add_param( &wsi$p_dx[1], &wsi$enc );
>
>
>
>     // Deallocate any memory used by Dynamic Strings here.
>
>     wsi$dstr_free( &wsi$p_dx[1] );
>
>
>
>     // Return here
>
>     *pOutLen = wsi$size;
>
>     return error_status_ok;
>
> }
>
>
>
>
>
> -----Ursprüngliche Nachricht-----
> Von: Akos Marton [mailto:makos999@gmail.com]
>
> Gesendet: Montag, 5. Juli 2010 11:13
>
>
> An: Apache AXIS C User List
> Betreff: Re: AW: transfer limitations
>
>
>
> Hi Sepp,
>
>
>
> On Mon, Jul 5, 2010 at 9:24 AM, Stadelmann Josef
>
> <josef.stadelmann@axa-winterthur.ch> wrote:
>
> > Given you send ALL your XML data into a file and parse it. Does the
> parser bail out somewhere?
>
> I also thought it, but varying the sent amount of data, thus XML is
>
> not broken at the same point in every case.
>
>
>
> >
>
> > I just have similar situation. There is (in my mind) too much data
> delivered; AND - Are you working with some kind of Strings through char*?
>
> Yes, axis2_char_t*.
>
>
>
> >
>
> > How long can a char* String be? How far does this pointer reach? Writing
> all my XML data into a file and checking the XML Structure shows a very well
> formed XML file; But any attempt to parse it through the
> axis2_libxml2_reader_wrapper is a no go and it bails out with various errors
> toward the end of the files-content.- no I  memory.
>
> I may not got what you mean above about the long of char*. You can
>
> allocate arbitrary (depends on your OS) amount of space, using a char*
>
> if you use dynamic memory management. How did you strike it out?
>
>
>
> >
>
> > Pointer size? 16, 32, 64?
>
> The size of pointer depends on your hardware and operating system, so
>
> that on 32bit operating system pointer has size of 4bytes.
>
>
>
> Now, I'm experimenting and learning how to configure the .Net side
>
> using WCF-technologies and set it to send data in chunks using MTOM.
>
> It seems for me, .Net-.Net can communicate with each other very well
>
> (of course), but needs more portability configuration to make it to be
>
> able to handle MTOM.
>
> In spite this we don't understand the reason of broken XMLs.
>
>
>
> Akos
>
>
>
> >
>
> > Sepp
>
> >
>
> > -----Ursprüngliche Nachricht-----
>
> > Von: Akos Marton [mailto:makos999@gmail.com]
>
> > Gesendet: Freitag, 18. Juni 2010 10:20
>
> > An: Apache AXIS C User List
>
> > Betreff: Re: AW: transfer limitations
>
> >
>
> > The problem is still not solved, so that any idea and answer is
> appreciated.
>
> >
>
> > You can read a mail on c-user list about the same problem by Jan-fon,
>
> > except that he found the same situation on client-side.
>
> > subject:        'Client to receive large data problem'
>
> > date:           Thu, 25 Feb 2010 11:07:53 GMT
>
> >
> http://mail-archives.apache.org/mod_mbox/axis-c-user/201002.mbox/%3CCC8FF067F7F4E54796AFDF21339BA0FC134A94BA@orsmsx505.amr.corp.intel.com%3E
>
> >
>
> > Hope it helps,
>
> > Akos Marton
>
> >
>
> >
>
> > Stadelmann Josef wrote:
>
> >> Have you setup to send in chunks from CL to SV or do you expect that
> more than 4 MB will go at once, within one request object?
>
> >> Sepp
>
> >>
>
> >> -----Ursprüngliche Nachricht-----
>
> >> Von: Akos Marton [mailto:makos999@gmail.com]
>
> >> Gesendet: Freitag, 11. Juni 2010 15:31
>
> >> An: Apache AXIS C User List
>
> >> Betreff: Re: transfer limitations
>
> >>
>
> >> The XML is broken every time at different part. I tried to configure
> httpd server with switching out many known size limitation and run axis2c
> under this, but it also doesn't send the whole XML.
>
> >>
>
> >> The sent data is just broken somewhere. I watched it with tcpdump.
>
> >> Smaller XMLs can be transferred.
>
> >>
>
> >> Well, written a short tester code, which reads the necessary XML from
> file and parses, decodes it perfectly, so that the algorithm and code must
> do the right.
>
> >>
>
> >> Any idea why the sending process is being broken?
>
> >>
>
> >> Thanks,
>
> >> mAkos
>
> >>
>
> >>
>
> >> Akos Marton wrote:
>
> >>> Hi List,
>
> >>>
>
> >>> Is there any limitation about the amount of data in a certain XML-tag?
>
> >>> In case of true, could I increase it somehow?
>
> >>> For instance <image> ... </image> by the size of 4Megabytes
of base64
>
> >>> code. The whole XML data does not reach the server (have cut at
>
> >>> certain part), but a smaller amount of data reaches it fully.
>
> >>>
>
> >>> Could be httpd.conf a solution? I do not know how to locate it for
>
> >>> axis2c binary version.
>
> >>>
>
> >>> Thanks,
>
> >>> mAkos
>
> >>>
>
> >>>
>
> >>
>
> >>
>
> >>
>
> >> ---------------------------------------------------------------------
>
> >> To unsubscribe, e-mail: c-user-unsubscribe@axis.apache.org
>
> >> For additional commands, e-mail: c-user-help@axis.apache.org
>
> >>
>
> >>
>
> >
>
> >
>
> > ---------------------------------------------------------------------
>
> > To unsubscribe, e-mail: c-user-unsubscribe@axis.apache.org
>
> > For additional commands, e-mail: c-user-help@axis.apache.org
>
> >
>
> >
>
> > ---------------------------------------------------------------------
>
> > To unsubscribe, e-mail: c-user-unsubscribe@axis.apache.org
>
> > For additional commands, e-mail: c-user-help@axis.apache.org
>
> >
>
> >
>
>
>
>
>
>
>
> --
>
> People seldom notice clothes, if you wear a big smile.
>
> My grandmother uses Ubuntu, don't you think so?!
>
> OLVASD: http://www.xkcd.org/
>
>
>
> ---------------------------------------------------------------------
>
> To unsubscribe, e-mail: c-user-unsubscribe@axis.apache.org
>
> For additional commands, e-mail: c-user-help@axis.apache.org
>
>
>
> Samisa Abeysinghe
> VP Engineering
> WSO2 Inc.
> http://wso2.com
> http://wso2.org
>
>
>
> Samisa Abeysinghe
> VP Engineering
> WSO2 Inc.
> http://wso2.com
> http://wso2.org
>
>
>
Samisa Abeysinghe
VP Engineering
WSO2 Inc.
http://wso2.com
http://wso2.org

Mime
View raw message