Mailing-List: contact user-help@uima.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@uima.apache.org
From: <Armin.Wegner@bka.bund.de>
To: <user@uima.apache.org>
Subject: AW: CPE memory usage
Thread-Topic: CPE memory usage
Thread-Index: AdHxOp4dSWla09CvTcyuufWE+lndZv//62CA///cijCAAIK3AP//0+Ug
Date: Mon, 8 Aug 2016 13:31:54 +0000
Message-ID: <A734E21E566ADB40A70F2EBC5D3C2EB051FCF56E@SWMMBX12.bk.bka.bund.de>
References: <A734E21E566ADB40A70F2EBC5D3C2EB051FCF2CF@SWMMBX12.bk.bka.bund.de>
 <FDF97D68-3E82-4AE8-95C5-3D499834CE2F@apache.org>
 <A734E21E566ADB40A70F2EBC5D3C2EB051FCF53E@SWMMBX12.bk.bka.bund.de>
 <A0CDC356-755A-46BF-8B1C-28708A660F1F@apache.org>
In-Reply-To: <A0CDC356-755A-46BF-8B1C-28708A660F1F@apache.org>
Accept-Language: de-DE, en-US
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="barc315ef1f2711f95b00b12a058e9a87bc"; micalg=pgp-sha1; protocol="application/pgp-signature"
archived-at: Mon, 08 Aug 2016 13:32:17 -0000

--barc315ef1f2711f95b00b12a058e9a87bc
Content-Language: de-DE
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable

Hi Richard!

I've changed the document reader to a kind of no-op-reader, that always set=
s the document text to an empty string: same behavior, but much slower incr=
ease in memory usage.

Cheers,
Armin

-----Urspr=FCngliche Nachricht-----
Von: Richard Eckart de Castilho [mailto:rec@apache.org]=20
Gesendet: Montag, 8. August 2016 14:31
An: user@uima.apache.org
Betreff: Re: CPE memory usage

I am not aware of any resource leaks in uimaFIT or UIMA-J either.
Maybe Solr doesn't handle resources in the way you expect? E.g.
queries or documents may have to be closed/returned when they
are no longer needed. You could to generate a heap dump using
JVisualVM to figure our what kind of objects are accumulating.

Cheers,

-- Richard

> On 08.08.2016, at 09:22, Armin.Wegner@bka.bund.de wrote:
>=20
> Hello Richard!
>=20
> No, I can't change the reader. It's reading from Solr. The response docum=
ents are put in a queue. The querying logic is done in hasNext(). hasNext()=
 returns true if the queue is not empty. If the queue is empty, hasNext() s=
ends a request to Solr and puts the response documents in the empty queue. =
If there are no more response documents from Solr, the queue remains empty =
and the reader is done. getNext() pulls a document from the queue and sets =
the CAS's document text. The number of documents returned from Solr can be =
given as a parameter. Currently it's set to 1, that is one document per req=
uest. The only fields of the reader class are the parameters and the docume=
nt queue. All other variables are local to their methods. It's pretty simpl=
e. There shouldn't be any resource leaks.
>=20
> Best,
> Armin
>=20
> -----Urspr=FCngliche Nachricht-----
> Von: Richard Eckart de Castilho [mailto:rec@apache.org]=20
> Gesendet: Montag, 8. August 2016 08:50
> An: user@uima.apache.org
> Betreff: Re: CPE memory usage
>=20
> Did you try using a different reader?
>=20
> Cheers,
>=20
> -- Richard
>=20
>> On 08.08.2016, at 08:10, <Armin.Wegner@bka.bund.de> <Armin.Wegner@bka.bu=
nd.de> wrote:
>>=20
>> Hi!
>>=20
>> I'm using uimaFIT 2.2.0 and uimaj 2.8.1. The collectection processing en=
gine is slowy eating up all memory until it gets killed by the system. This=
 happens even when I'm just runnging a collection reader and no other compo=
ments (no analysis at all). Does anyone has experiented a similar behavior =
or has any ideas?
>>=20
>> Best,
>> Armin
>=20


--barc315ef1f2711f95b00b12a058e9a87bc
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iQEcBAABAgAGBQJXqInMAAoJEAk50sqYef+/iMoH/j1KFqaYnc9B5yAY2j+QECts
fkISeMq8+bnpwk8Cn/HXH/2FHyt3vAvaXaVjzoTve2XzT0El30KdPjxhjzf+GJLz
3vau9eLm9HL4N57FWggqrcF7MiYGQfvnmxj2MMFtBx7VnGhz7Dt7PM9fNMS7AmXF
JxNUn1LEyYqc+io1kDJLz+k3Qngl+B/0CprNRWtFyKZ29/dI1i2axvdro2lf5XJL
KvXyob+J1Qe94yvwC0a5jZfrVXBMKjHFAGDTFzS5Cyelw1EBjBywG+cjYynKjkyG
AvORjjuE5TVFcM7FyYZ3HKYKaGsh0venJYeef9ZvqdGcI9S/0yMyzmRkPWXTY94=
=9SdD
-----END PGP SIGNATURE-----
--barc315ef1f2711f95b00b12a058e9a87bc--