Mailing-List: contact users-help@pdfbox.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@pdfbox.apache.org
Date: Thu, 23 Feb 2017 18:54:19 +0000 (UTC)
From: Viraf Bankwalla <viraf_bankwalla@yahoo.com.INVALID>
Reply-To: Viraf Bankwalla <bankwalla@pobox.com>
To: "users@pdfbox.apache.org" <users@pdfbox.apache.org>
Message-ID: <151524595.3471006.1487876059752@mail.yahoo.com>
In-Reply-To: <3140a206-576d-4088-bd78-485f28c6c3fd@t-online.de>
References: <168441298.3196491.1487858877764.ref@mail.yahoo.com> <168441298.3196491.1487858877764@mail.yahoo.com> <3140a206-576d-4088-bd78-485f28c6c3fd@t-online.de>
Subject: Re: OutOfMemoryException converting PDF to TIFF Images
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_3471005_1481140317.1487876059748"
archived-at: Thu, 23 Feb 2017 19:05:04 -0000

------=_Part_3471005_1481140317.1487876059748
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Yes, I saw that=C2=A0DefaultResourceCache uses SoftReference, however for s=
ome reason when I look at the heap dump XObjects are prevalent. =C2=A0
The TIFF images are saved to file, using ImageIOUtil.write, and I simply ke=
ep the File reference for downstream processing.=C2=A0
The change made to PDResources.isAllowedCache is:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 COSBase image =3D xobject.getCOSO=
bject().getDictionaryObject(COSName.SUBTYPE);=C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 if (image instanceof COSName && ((COSName) image).equals(COSN=
ame.IMAGE))=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 {=C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0  return false;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 }
After sending my earlier e-mail, I realized that I may not be able to add u=
ser defined filter to PDResource (still figuring out the code), and that ma=
ybe it would be added to=C2=A0DefaultResourceCache.
I would like to be able to get a fix for the issue in the next release - th=
ere may be a better fix to the problem than I have provided above, as I am =
unfamiliar with the code base.
Thanks
- viraf


      From: Tilman Hausherr <THausherr@t-online.de>
 To: users@pdfbox.apache.org=20
 Sent: Thursday, February 23, 2017 1:06 PM
 Subject: Re: OutOfMemoryException converting PDF to TIFF Images
  =20
Am 23.02.2017 um 15:07 schrieb viraf.bankwalla@yahoo.com.INVALID:
> I am using PDFBox to convert PDF documents to a series of TIFF images (on=
e for each page).=C2=A0 The implementation uses PDFRenderer to render each =
page.=C2=A0 Things work fine when I am processing a single document in a si=
ngle thread, however when I try to process multiple documents (each in its =
own thread) I get an OutOfMemoryException.
> In analyzing the heap dump, I see that this is caused by the images cache=
d in DefaultResourceCache.=C2=A0 Objects are added added to the cache in PD=
Resources, which includes a method private boolean isAllowedCache(PDXObject=
 xobject) that is used to determine whether an PDXObject can be cached.=C2=
=A0 I have extended this to filter out COSName.IMAGE, and am now able to pr=
ocess multiple documents in parallel.
> I'd like to contribute this change back to the community.=C2=A0 However p=
rior to adding this, I though some feedback on the filtering mechanism may =
be appropriate.=C2=A0 Some options include:
>=C2=A0 =C2=A0=20
>=C2=A0 =C2=A0 - Always exclude images
>=C2=A0 =C2=A0 - Allow user to specify whether images should be cached or n=
ot (add a method to PDResource to toggle filtering of images).=C2=A0 Defaul=
t would including caching of images to be backwards compatible.
>=C2=A0 =C2=A0 - Defer image caching decision to user through callback.=C2=
=A0 Default callback would cache all images to provide backwards compatibil=
ity.
>
> I also wanted to know how best to submit my patch for inclusion.
> Thanks

In theory, the cache should dump its contents when memory becomes low=20
thanks to the SoftReference. Thus I'm wondering if it doesn't work at=20
all, that's the real question.

About your own code - are you "losing" any reference to the TIFF you=20
produce after each page? Or are they in an array of images?


re submitting stuff, see=20
https://pdfbox.apache.org/codingconventions.html , open an issue in JIRA=20
and submit a .diff / .patch file. Be aware that we don't accept every=20
submission. I'm skeptical about callbacks, we don't do such a thing=20
anywhere.

Tilman

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org


  =20
------=_Part_3471005_1481140317.1487876059748--