cocoon-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jos Snellings <Jos.Snelli...@pandora.be>
Subject RE: Very large Generator file
Date Mon, 10 May 2010 15:04:12 GMT
Yes, Jeff, my guess is that the tfactory stuff will accomplish the job
for you without complaining about memory shortage! Your tomcat instance
may have other things around increasing the memory footprint all the
time. 


On Mon, 2010-05-10 at 09:43 -0500, Schmitz, Jeffrey A wrote:
> Yes, the resulting XHTML file is not for online use.
> 
> And yes, I've given max memory to the tomcat instance.  So, will having my specialized
transformer do the xsl transform directly using a SAX transformer not work either?  e.g. something
like:
> 
>    TransformerFactory tfactory = TransformerFactory.newInstance();
>    Transformer transformer = tfactory.newTransformer(new StreamSource(
> 				xslID));
>    transformer.transform(new StreamSource(sourceFile), new StreamResult(
> 				new File(modelFile)));
> 
> 
> I guess I was under the impression that SAX based xsl transformers didn't use much memory
regardless file size, but perhaps that's not true?
> 
> If that's not going to work, would moving to a 64 bit machine so I can give more memory
to Tomcat be an option (is there a 64 bit Tomcat)?  
> 
> > If this is the case a short java program can make that xhtml file for
> > you.
> I'm not exactly sure what you're suggesting here.  Do you mean for the tomcat/cocoon
process to exec a separate java program that will do the xsl transform on the large file?
 Won't any java program come up against the 2 Gig max memory limitation?
> 
> Thanks,
> Jeff
> 
> > -----Original Message-----
> > From: Jos Snellings [mailto:Jos.Snellings@pandora.be]
> > Sent: Monday, May 10, 2010 9:30 AM
> > To: users@cocoon.apache.org
> > Subject: RE: Very large Generator file
> > 
> > Hi Jeff,
> > 
> > If you have been trying giving all the available memory to your tomcat
> > instance (JVM options) you are at the end of you tether for the current
> > solutions, unless somebody has a different idea.
> > 
> > I understood from your previous mail that the resulting xhtml is not
> > for
> > on-line use, it is a sort of report/summary.
> > If this is the case a short java program can make that xhtml file for
> > you.
> > 
> > Cheers,
> > Jos
> > 
> > 
> > On Mon, 2010-05-10 at 09:17 -0500, Schmitz, Jeffrey A wrote:
> > > Hi Jos,
> > >    I've already written my own Transformer (extends
> > AbstractSAXTransformer) that currently takes the output of the default
> > cocoon xsl transformer (via the sitemap) and performs subsequent, non-
> > xsl transforms, writes the results to file and generates the returned
> > web-page
> > > e.g.
> > >       <map:match pattern="*/*/*In/*/*In.flow/*" >
> > >          <map:generate src="inputs/{4}" />
> > >          <map:transform src="xsl/{3}InAdapter.xsl">
> > >             <map:parameter name="instanceName" value="{2}"/>
> > >          </map:transform>
> > >          <map:transform type="SPARQLinTrans">
> > >             <map:parameter name="inputFile" value="{4}"/>
> > >             <map:parameter name="SPARQLFlow" value="{5}"/>
> > >          </map:transform>
> > >          <map:serialize type="xhtml"/>
> > >       </map:match>
> > >
> > > I'm currently heading in the direction of just adding the initial xsl
> > transformation as part of my own Transformer's duties (I think your
> > option 3) so that it can hopefully handle the larger files, but I was
> > wondering if there might be a way to easily keep things as they are and
> > continue using the default cocoon xsl tranform as the first step.
> > Sounds like there's not?
> > >
> > >
> > > > -----Original Message-----
> > > > From: Jos Snellings [mailto:Jos.Snellings@pandora.be]
> > > > Sent: Monday, May 10, 2010 8:39 AM
> > > > To: users@cocoon.apache.org
> > > > Subject: RE: Very large Generator file
> > > >
> > > > A SAX pipeline is part of cocoon.
> > > > Can you explain briefly what you want to do?
> > > > Consider using a cocoon-3 pipeline.
> > > > Consider using a SAX transformation:
> > > > - read xml
> > > > - do something with it (transforming, for instance)
> > > > - write output to a file
> > > >
> > > > Jos
> > > >
> > > > On Mon, 2010-05-10 at 08:23 -0500, Schmitz, Jeffrey A wrote:
> > > > > Correct, it's not being turned into a web page.  Is a SAX
> > pipeline
> > > > part of Cocoon?  I'm still on Cocoon 2.1, but I've never heard that
> > > > term specifically, I thought the default generator did use SAX.
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Jos Snellings [mailto:Jos.Snellings@pandora.be]
> > > > > > Sent: Saturday, May 08, 2010 12:06 AM
> > > > > > To: users@cocoon.apache.org
> > > > > > Subject: Re: Very large Generator file
> > > > > >
> > > > > > Hi Jeff,
> > > > > >
> > > > > > That depends a great deal on what you want to do exactly with
> > the
> > > > large
> > > > > > file. I guess you do not want to transform it into one huge
web
> > > > page?
> > > > > > In general, if you have to deal with lengthy inputs, a SAX
> > pipeline
> > > > is
> > > > > > the way to go.
> > > > > >
> > > > > > Jos
> > > > > >
> > > > > > On Fri, 2010-05-07 at 18:14 -0500, Schmitz, Jeffrey A wrote:
> > > > > > > Hello,
> > > > > > >
> > > > > > >    I was wondering if there’s a way to handle very large
> > files as
> > > > > > > inputs to the default cocoon generator?  As it is, I am
> > getting a
> > > > > > > memory heap error when trying to process an input file
> > because
> > > > it’s
> > > > > > so
> > > > > > > large.  Am I going to have to process it in some other
> > manner, or
> > > > is
> > > > > > > there a way to make Cocoon perform the transform for me?
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Jeff
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > ---------------------------------------------------------------
> > ----
> > > > --
> > > > > > To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
> > > > > > For additional commands, e-mail: users-help@cocoon.apache.org
> > > > >
> > > >
> > > >
> > > >
> > > > -------------------------------------------------------------------
> > --
> > > > To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
> > > > For additional commands, e-mail: users-help@cocoon.apache.org
> > >
> > 
> > 
> > 
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
> > For additional commands, e-mail: users-help@cocoon.apache.org
> 



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org


Mime
View raw message