Return-Path: 
 <c-users-return-989-apmail-xerces-c-users-archive=xerces.apache.org@xerces.apache.org>
Delivered-To: apmail-xerces-c-users-archive@www.apache.org
Received: (qmail 45200 invoked from network); 14 Jan 2007 12:07:38 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2)
  by minotaur.apache.org with SMTP; 14 Jan 2007 12:07:38 -0000
Received: (qmail 85036 invoked by uid 500); 14 Jan 2007 12:07:44 -0000
Delivered-To: apmail-xerces-c-users-archive@xerces.apache.org
Received: (qmail 85017 invoked by uid 500); 14 Jan 2007 12:07:44 -0000
Mailing-List: contact c-users-help@xerces.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:c-users-help@xerces.apache.org>
List-Unsubscribe: <mailto:c-users-unsubscribe@xerces.apache.org>
List-Post: <mailto:c-users@xerces.apache.org>
List-Id: <c-users.xerces.apache.org>
Reply-To: c-users@xerces.apache.org
Delivered-To: mailing list c-users@xerces.apache.org
Received: (qmail 85006 invoked by uid 99); 14 Jan 2007 12:07:44 -0000
Received: from herse.apache.org (HELO herse.apache.org) (140.211.11.133)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 14 Jan 2007 04:07:44 -0800
X-ASF-Spam-Status: No, hits=2.0 required=10.0
	tests=HTML_MESSAGE,SPF_HELO_PASS,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (herse.apache.org: domain of Martin.Harm@cimt-ag.de
 designates 85.182.132.161 as permitted sender)
Received: from [85.182.132.161] (HELO apollon.cimt-ag.de) (85.182.132.161)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 14 Jan 2007 04:07:34 -0800
Received: from hermes.cimt-ag.de ([85.182.132.163])
	by apollon.cimt-ag.de (8.13.8/8.13.6) with ESMTP id l0EC3onN037197;
	Sun, 14 Jan 2007 13:07:05 +0100 (CET)
	(envelope-from Martin.Harm@cimt-ag.de)
X-MimeOLE: Produced By Microsoft Exchange V6.5
Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C737D4.774B4A10"
Subject: AW: Limitation of file size with DOM
Date: Sun, 14 Jan 2007 13:06:47 +0100
Message-ID: <CFF0C8E79AF583438440F1AC76CED2F70145FC38@dehbgcimtex1>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: Limitation of file size with DOM
Thread-Index: Acc2CY1pUjXfSrXQTt2YJKDCWI2u1wAYZ2QgAFn1ZCs=
References: <19D68C9110AB5646A4A28153FCB1FC3A017FA619@de01exm68.ds.mot.com>
From: "Martin Harm" <Martin.Harm@cimt-ag.de>
To: "Kosaraju Karuna-C9013Z" <C9013Z@motorola.com>,
        <c-users@xerces.apache.org>
X-Spam-Level: *
X-Spam-Checker-Version: SpamAssassin 3.1.6-thbe (2006-10-03) on
	apollon.cimt-ag.de
X-Virus-Checked: Checked by ClamAV on apache.org
X-Old-Spam-Status: No, score=1.2 required=5.0 tests=AWL,BAYES_40,HTML_30_40,
	HTML_MESSAGE,NO_DNS_FOR_FROM,SPF_HELO_PASS,SPF_PASS autolearn=no
	version=3.1.6-thbe

------_=_NextPart_001_01C737D4.774B4A10
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Hi,
well it didnt work just rigth now:
 - xml-file: 661M 14. Jan 12:29 accountStatement.400000.xml
 - call: xerces-c-src_2_7_0/bin/DOMCount accountStatement.400000.xml
 - on:
 uname: Linux tolero 2.6.18-1.2257.fc5 #1 Fri Dec 15 16:06:24 EST 2006 =
i686 athlon i386 GNU/Linux
 gcc:  gcc (GCC) 4.1.1 20060525 (Red Hat 4.1.1-1)
 free
             total       used       free     shared    buffers     =
cached
Mem:       1555628     389748    1165880          0      23496     =
260140
-/+ buffers/cache:     106112    1449516
Swap:      1060208          0    1060208
=20
because of abolute no memory left .. the system freezes, and I have to =
reset it.

The symtoms of having strange errors occured on older versions of xerces =
and linux
and are not reproducable on my current system.

You should test it on your own.. mayby this litte perl script helps on =
generating
large XML-Files.

use strict;
my @cnt =3D ( 1,3, 100, 200, 5000, 10000, 20000, 40000, 50000, =
60000,80000, 100000,200000,400000,800000);

my $baseName=3Dshift;

open FF,"$baseName.header"; my @headerLines=3D<FF>; close FF;
open FF,"$baseName.entry"; my @contentLines=3D<FF>;close FF;
open FF,"$baseName.footer";my @footerLines=3D<FF>;close FF;

foreach my $entryCnt (@cnt) {
    my $fname =3D sprintf("$baseName.%d.xml",$entryCnt);
    print "Building: $fname\n";
    my $fout;
    open($fout, ">$fname");
    print $fout  @headerLines;
    for(my $i=3D0; $i<$entryCnt;$i++) {
        print $fout @contentLines;
    }
    print $fout  @footerLines;
    close($fout);

}
1;

ciao
martin



-----Urspr=FCngliche Nachricht-----
Von: Kosaraju Karuna-C9013Z [mailto:C9013Z@motorola.com]
Gesendet: Fr 2007-01-12 18:04
An: c-users@xerces.apache.org
Cc: jason.e.stewart@gmail.com; Martin Harm
Betreff: RE: Limitation of file size with DOM
=20
Thanks Jason and Martin for the replies,
I am now confused with the contradictory experiences of both of you.
Is it possible to let me know the xerces version, OS, kernel version
etc., on which you have seen this.
I just wanted to understand the scenario where it worked (for Jason) and
it dint work (for Martin)

Thanks again,
Karuna

-----Original Message-----
From: Jason Stewart [mailto:jason.e.stewart@gmail.com]=20
Sent: Thursday, January 11, 2007 11:21 PM
To: c-users@xerces.apache.org
Subject: Re: Limitation of file size with DOM

Hi,

On 1/11/07, Kosaraju Karuna-C9013Z <C9013Z@motorola.com> wrote:
> Hi,
>
> We are using Xserces-c DOM APIS to parse our applications XML file.=20
> Our XML file size may go to MB.
> Wanted to know whether there is any restriction on xerces-c DOM on the

> size of the file it can handle.
> The version we are using is xercesc-c.2.6.0. Any replies would be=20
> appreciated
>

I've used Xerces to parse file sizes in the 500Mb range...

The only limitation I had was on the memory of the machine...

Cheers, jas.


Hi,
=20
I tested parsing large files. The memory consumption was aprox. 7 to 10
times of the XML-File-Size (pretty printed). But there was at least a
limit on the size of the XML-File (aprox. 250Mb) where the xerces (2.6)
bounces with strange errors.. and there *was* still some RAM left :-(
(tested on linux). So I was forced to switch to SAX.
=20
cu
martin


------_=_NextPart_001_01C737D4.774B4A10--