xmlgraphics-fop-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeremias Maerki <...@jeremias-maerki.ch>
Subject Re: Major issue with image loading in FOP 0.95beta
Date Tue, 06 May 2008 07:40:44 GMT
Hmm, I guess it was a wise decision to make a beta release before the
final. Please note that the image loading framework is completely new.
The old one (while working fine for a specific set of image types) was a
paing to maintain and extend and it often ate much more memory than
necessary. Some hickups were to be expected and I was hoping I have been
able to fix most of them by now. But maybe not...

http://xmlgraphics.apache.org/fop/0.95/releaseNotes_0.95beta.html

I've done some experiments and put about 20 PNGs (some of which pretty
large) in my private SVN repository (SVN 1.4.5 over HTTPD 2.2). I ran a
multi-threaded test (Sun JVM 1.5.0_15, 4 threads, FOP 0.95beta with
XG Commons 1.3 and FOP Trunk with XG Commons Trunk) against that setup.

My machine: WinXP SP2, Intel Core2Duo E6600

But I cannot see any errors or strange behaviour. Once all the images
have been loaded once they are not requested from the SVN server again 
(at least as long there's enough memory in the JVM). The only thing I've
seen is that if I start a number of threads simultaneously and let them
render the same FO document, each document causes the same files to be
loaded as the framework currently doesn't synchronize concurrent initial
requests for the same URL. Once the PNG is in the cache there are no
more requests against the SVN server.

I dialed the available memory down so the image cache cannot hold all
the images in cache to force the image framework to reload the images
every time. Nothing bad happened. But I don't have a large hardware park
that would allow me to throw more at my SVN repository.

Another thing I've done is put an TCP proxy in between to find out
what's going on on that level. There was no obvious misbehaviour to be
seen there.

I fear you'll have to dive in yourself and find out what exactly is
going wrong. Just to give you an idea what's going on:

FOP always "preloads" an image before it fully loads it. That means that
it detects the image format and extracts some basic metadata so FOP
knows about the image's intrinsic size. That is used for layout and
allows us not to fully load the image into memory at layout time. Only
when rendering the output file the image is fully loaded. Whenever
possible the original connection that the preloading has been done with
is reused (because some people generate dynamic images using servlets).
The basic change related to your problem with the new image loading
framework: Instead of working with normal InputStreams, the new
framework works with ImageIO's ImageInputStream which provides
convenient methods for working with image files. For non-File-URLs, the
ImageInputStream is created using ImageIO.createImageInputStream(Object).
By default, ImageIO uses a "cached stream" where the stream data is
copied to a local file in a temp directory from which the actual data is
accessed. One thing you can try is to disable this caching to see if
anything changes: ImageIO.setUseCache(false) should do that (call it at
application startup or before you create the FopFactory). If that
changes the behaviour it may give us an idea where to look next.

You can also enable some logging in the image loading framework to get
some information what's going on inside. That would be the
"org.apache.xmlgraphics.image.loader" category.

Without being able to reproduce the behaviour it is difficult to help.
Some further questions from my side:
- How many different PNGs are being accessed?
- Are they smaller or bigger files?
- Are you running FOP in a multi-threaded fashion? Or from many
different machines against the SVN repository?

On 05.05.2008 18:09:55 Jean-François El Fouly wrote:
> Andreas Delmelle a écrit :
> > On May 5, 2008, at 17:22, Jean-François El Fouly wrote:
> >
> > Hi
> >
> >> Sorry, I guess in the strictest sense this is xmlgraphics-common 
> >> related now, but it was discovered and investigated using FOP 0.95beta.
> >>
> >> We upgraded from 0.94 to use the new bookmarks features and our 
> >> software became globally unstable, hanging randomly during PDF 
> >> generation -- to be precise, getting all kinds of random problems 
> >> during image loading.
> >> The images we use are stored in SVN and retrieved via their HTTP URLs 
> >> from an SVN server. Reading the SVN logs, we see a very different 
> >> behaviour between 0.94 (which worked fine and has been retested 
> >> since) and 0.95beta: the FOP requests litteraly "suffocate" the SVN 
> >> server, sending quite a lot of requests for images and taking way too 
> >> long to consume the responses. So, after a (random) while, the SVN 
> >> server gives up (timeout) and the FOP application on the other side 
> >> crashes immediately after. At least that is how we understand the 
> >> problem after testing and reading all sorts of server logs (with a 
> >> sysadmin) for a whole day.
> >> This looks like a major regression and makes 0.95beta completely 
> >> unusable for us -- though we badly need the new features :-(
> >>
> >> If anybody had an idea, I must say I'd be extremely grateful...
> >
> > Two questions, for the moment:
> > Which image format(s) are you using?
> PNG only (but lots of them).
>
> > Does the JAI/ImageIO implementation on the box where FOP runs, have a 
> > native codec to read that format?
> >
> Will ask / investigate (it's a production server to which I don't have 
> direct access). Debian 4.0 on JDK 1.5.0_11 and JBoss 4.2.1.GA.
> Thanks !



Jeremias Maerki


---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org


Mime
View raw message