jackrabbit-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Hobi <d.h...@gmx.ch>
Subject Re: JVM hangs during startup (indexing) (2)
Date Fri, 08 Mar 2013 08:49:31 GMT
Hi Marek

Thanks for your suggestions!

Unfortunately, changing the JVM gc options so that the JVM will no longer hang just worked
ONCE.
Therefore I was not able to get a "nice" stack trace anymore :-(

Connection Pool:
We had set the max connection pool size to 5. As you suggested I set it to 15. Same effect.
Even 50 or 100 did not help.
Interestingly, postgres shows just 3 open connection for jackrabbit. Is this a suspicious
behavior or does it simply not need more connections and is therefore not spawning new connections?

Enabling debug, setting login and socket timeout on the postgres-jdbc-driver did not help
either.
I even changed the pool implementation from dbcp to c3p0 in jackrabbit-core-2.6.0 to be "sure"
it is not a pool bug.
Further we are not able to reproduce the hang with:
- Slower machine, postgres 8.3
- Very fast machine, postgres 8.4 (only hanged 1 of 5 times so far)
- Faster machine, mysql

This issue is slowly driving me nuts ;-)

Thanks!
Daniel
 


Am 07.03.2013 um 16:24 schrieb mslama@email.cz:

> Also randomly we had problem with not released connections in JBoss 
> connection pool and Postgres. Now long time no occurrence.
> In any case you can check number/list of open connections in psql . In our 
> case it was
> 97 open connection (by default 100 open connections in postgres and 3 are 
> reserved for admin). It happenned even if we had much smaller
> max connection pool size. As it happened randomly we did not find any 
> solution. We tried to change connection pool size and it does not
> happen any more. Still no idea what was cause of this. In any case using 
> psql you can find out easily if it is also your problem.
> 
> Query is for 9.x: SELECT * FROM pg_stat_activity;
> Marek
> 
> -- 
> Marek Slama
> mslama@email.cz
> 
> 
> 
> ---------- Původní zpráva ----------
> Od: mslama@email.cz
> Datum: 7. 3. 2013
> Předmět: Re: JVM hangs during startup (indexing) (2)
> 
> "Quick look: It hangs in connection pool - connection pool if it cannot 
> return free connection to caller it waits forever by default.
> How big is your connection pool? (Maximum size? It would be good to patch 
> connection pool to see who/how many connections is
> used and why connections are used not release. In our config we use 
> connection max pool size 15 and it is enough.) So first
> fast fix would be to increase max connection pool size but if there is 
> problem with not released connection it will not help for long.
> 
> Good news is it is not problem with memory or gc (at least IMO).
> 
> Marek
> 
> -- 
> Marek Slama
> mslama@email.cz
> 
> 
> 
> ---------- Původní zpráva ----------
> Od: Daniel Hobi <d.hobi@gmx.ch>
> Datum: 7. 3. 2013
> Předmět: JVM hangs during startup (indexing) (2)
> 
> "Hi there
> 
> 
> 
> Some updates on our issue started here:
> 
> http://mail-archives.apache.org/mod_mbox/jackrabbit-users/201301.mbox/%3C201
> 30115135117.186960%40gmx.net%3E
> 
> 
> 
> We're still facing the JVM freeze (possibly during minor gcs) even if...
> 
> ...we updated Jackrabbit from 2.2.13 to 2.6.0.
> 
> ...we updated the Sun JVM to the latest version (1.6.39) and even tried
> OpenJDK.
> 
> ...we increased the JVM heap size (up to 1.5gb).
> 
> ...we upgraded PostgreSQL to the latest version (8.4.x / 9.2.x) available.
> 
> 
> 
> But there is good news too:
> 
> - I've just played around with the JVM's garbage collection
> options. Even if the index process still hangs the JVM will no longer block.
> I was able to get an interesting thread dump:
> <http://pastebin.com/uSrXpej8> http://pastebin.com/uSrXpej8
> 
> - With the addition JVM Param -XX:MaxNewSize=10m we can make the
> hang occur faster (less than 100 minor gcs)
> 
> 
> 
> I would be very grateful if someone could have a look at this. We're running
> (slowly but surely) out of ideas :-(
> 
> 
> 
> Thanks!
> 
> Daniel""


Mime
View raw message