Mailing-List: contact users-help@tomcat.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Tomcat Users List" <users@tomcat.apache.org>
Received-SPF: pass (athena.apache.org: domain of popanik@gmail.com designates
 209.85.214.45 as permitted sender)
Message-ID: <506D47F6.6030101@gmail.com>
Date: Thu, 04 Oct 2012 11:25:26 +0300
From: Nick Katsipoulakis <popanik@gmail.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:15.0) Gecko/20120912 Thunderbird/15.0.1
MIME-Version: 1.0
To: Tomcat Users List <users@tomcat.apache.org>
Subject: Re: Memory leaks and IllegalStateException caused (probably) by a
 Singleton object
References: <506C4F0B.1020906@gmail.com>
 <CANcS8PmV=+i1B+JR1YBNq9S3jUzuO59gRM-u=3DC984DohP3JQ@mail.gmail.com>
In-Reply-To: 
 <CANcS8PmV=+i1B+JR1YBNq9S3jUzuO59gRM-u=3DC984DohP3JQ@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

On 10/03/2012 06:03 PM, Filippo Machi wrote:
> Ciao Nick,
> according to my own experience, I saw some logs at SEVERE very similar to
> what you posted here when I used hot deploy for a web application I worked
> on and that application used threadlocal variables and launched some
> threads that tomcat didn't control directly.
> These logs are part of some checks tomcat performs when an application is
> unloaded.
> Basically tomcat it's claiming there could be problems since thread locals
> variable are not emptied correctly by the webapp and so tomcat (the java
> process) it's not able to release memory.
> Even for threads there is the same check.
> When something like that occurred, resources used by the old version of
> your webapp are not released correctly and so you may incur in an
> OutOfMemoryException affecting the perm gen space.
> Besides, if you see the log statements, it seems some external packages are
> causing that, am I wrong?
> Except for the naming convention you used, I didn't see anything harming in
> your code :-)
>
>
>
> On Wed, Oct 3, 2012 at 4:43 PM, Nick Katsipoulakis <popanik@gmail.com>wrote:
>
>> Dear list,
>> I am currently developing a Java Web application on a machine with the
>> following
>> attributes:
>>
>> - Ubuntu-Server Edition Linux 12.04 64-bit
>> - Sun java JDK version 7
>> - Apache Tomcat 7.0.30
>> - Netbeans IDE, version 7.1.2
>>
>> My project consists of a SOAP Web service interface, that maintains a
>> shared in-
>> memory object. In order to make the object visible to all threads, I
>> developed a
>> Singleton. I post the code of my application below:
>>
>>      @WebService()
>>      public class ETL_WS {
>>
>>          private Singleton CAP_offer_coupon_map;
>>
>>          public ETL_WS() { }
>>
>>          /**
>>           * This method adds a single coupon record to the memory map.
>>           */
>>          @WebMethod
>>          synchronized public int singleCouponLoad(@WebParam(**name =
>> "CouponID") long coupon_id,
>>          @WebParam(name = "ProductCategoryID") long product_category_id,
>> @WebParam(name = "DateTo") Date dateTo,
>>          @WebParam(name = "LocationID") Location location_id) {
>>          Coupon _c = new Coupon(coupon_id, product_category_id, dateTo);
>>          if(location_id != null)
>>              _c.setLocation(location_id);
>>          CAP_CouponOfferCollection _data = CAP_offer_coupon_map.getModel(**
>> );
>>          Coupon _tmp = _data.getCoupon(coupon_id);
>>          if(_tmp == null)
>>              return -1;
>>          _data.insertCoupon(_c);
>>          return 0;
>>          }
>>
>>          /**
>>           * This method adds a single offer record to the memory map.
>>           */
>>          @WebMethod
>>          synchronized public int singleOfferLoad(@WebParam(name =
>> "OfferID") long offer_id,
>>          @WebParam(name = "ProductCategoryID") long product_category_id,
>> @WebParam(name = "DateTo") Date dateTo,
>>          @WebParam(name = "LocationID") Location location_id) {
>>          Offer _o = new Offer(offer_id, product_category_id, dateTo);
>>          if(location_id != null)
>>              _o.setLocation(location_id);
>>          CAP_CouponOfferCollection _data = CAP_offer_coupon_map.getModel(**
>> );
>>          Offer _tmp = _data.getOffer(offer_id);
>>          if(_tmp == null)
>>              return -1;
>>          _data.insertOffer(_o);
>>          return 0;
>>          }
>>
>>          @WebMethod
>>          synchronized public String getAllCoupons() {
>>          CAP_CouponOfferCollection _data = CAP_offer_coupon_map.getModel(**
>> );
>>          HashMap<Long, Coupon> _c = _data.getCoupons_map();
>>          return _c.toString();
>>          }
>>
>>          @WebMethod
>>          synchronized public String getAllOffers() {
>>          CAP_CouponOfferCollection _data = CAP_offer_coupon_map.getModel(**
>> );
>>          HashMap<Long, Offer> _o = _data.getOffers_map();
>>          return _o.toString();
>>          }
>>
>>          @WebMethod
>>          synchronized public long getProductIdFromCouponId(@**WebParam(name
>> = "CouponID") long coupon_id) {
>>          long _product_id = -1;
>>          CAP_CouponOfferCollection _data = CAP_offer_coupon_map.getModel(**
>> );
>>          Coupon _c = _data.getCoupon(coupon_id);
>>          if(_c != null)
>>              _product_id = _c.getCoupon_id();
>>          return _product_id;
>>          }
>>
>>          @WebMethod
>>          synchronized public long getProductIdFromOfferId(@**WebParam(name
>> = "OfferID") long offer_id) {
>>          long _product_id = -1;
>>          CAP_CouponOfferCollection _data = CAP_offer_coupon_map.getModel(**
>> );
>>          Offer _o = _data.getOffer(offer_id);
>>          if(_o != null)
>>              _product_id = _o.getOffer_id();
>>          return _product_id;
>>          }
>>
>>      }
>>
>> The Singleton wrapper-class is shown below:
>>
>>      public class Singleton {
>>
>>          private static boolean _instanceFlag = false;
>>
>>          private static final Singleton _instance = new Singleton();
>>
>>          private static CAP_CouponOfferCollection _data;
>>
>>          private Singleton() {
>>          _data = new CAP_CouponOfferCollection();
>>          _instanceFlag = true;
>>          }
>>
>>          public static synchronized CAP_CouponOfferCollection getModel() {
>>          if(_instanceFlag == false) {
>>              _data = new CAP_CouponOfferCollection();
>>              _instanceFlag = true;
>>          }
>>          return _data;
>>          }
>>      }
>>
>> and the CAP_CouponOfferCollection class is shown below:
>>
>> public class CAP_CouponOfferCollection {
>>
>>      private HashMap<Long, Coupon> _coupons_map;
>>
>>      private HashMap<Long, ArrayList<Long>> _product_to_coupons_map;
>>
>>      private HashMap<Long, Offer> _offers_map;
>>
>>      private HashMap<Long, ArrayList<Long>> _product_cat_to_offers_map;
>>
>>      private static long _creation_time;
>>
>>      public CAP_CouponOfferCollection() {
>>          _creation_time = System.currentTimeMillis();
>>          System.out.println("Creation of CAP_CouponOffer object: " +
>>                  _creation_time);
>>      }
>>
>>      synchronized public void insertCoupon(Coupon newCoupon) {
>>              if(_coupons_map == null) {
>>                  _coupons_map = new HashMap<Long, Coupon>();
>>                  _product_to_coupons_map =
>>                          new HashMap<Long, ArrayList<Long>>();
>>              }
>>              Long key = newCoupon.getCoupon_id();
>>              if(!_coupons_map.containsKey(**key)) {
>>                  _coupons_map.put(key, newCoupon);
>>                  key = newCoupon.getProductCategory_**id();
>>                  if(_product_to_coupons_map.**containsKey(key)) {
>>                      ArrayList<Long> _c_list = _product_to_coupons_map.get(
>> **key);
>>                      _c_list.add(newCoupon.**getCoupon_id());
>>                      _product_to_coupons_map.**remove(key);
>>                      _product_to_coupons_map.put(**key, _c_list);
>>                  }else {
>>                      ArrayList<Long> _c_list = new ArrayList<Long>();
>>                      _c_list.add(newCoupon.**getCoupon_id());
>>                      _product_to_coupons_map.put(**key, _c_list);
>>                  }
>>              }
>>      }
>>
>>      synchronized public void insertOffer(Offer newOffer) {
>>              if(_offers_map == null) {
>>                  _offers_map = new HashMap<Long, Offer>();
>>                  _product_cat_to_offers_map =
>>                          new HashMap<Long, ArrayList<Long>>();
>>              }
>>              Long key = newOffer.getOffer_id();
>>              if(!_offers_map.containsKey(**key)) {
>>                  _offers_map.put(key, newOffer);
>>                  key = newOffer.getProductCategory_**id();
>>                  if(_product_cat_to_offers_map.**containsKey(key)) {
>>                      ArrayList<Long> _o_list = _product_cat_to_offers_map.*
>> *get(key);
>>                      _o_list.add(newOffer.getOffer_**id());
>>                      _product_cat_to_offers_map.**remove(key);
>>                      _product_cat_to_offers_map.**put(key, _o_list);
>>                  }else {
>>                      ArrayList<Long> _o_list = new ArrayList<Long>();
>>                      _o_list.add(newOffer.getOffer_**id());
>>                      _product_cat_to_offers_map.**put(key, _o_list);
>>                  }
>>              }
>>      }
>>
>>      synchronized public void removeCoupon(long couponId) {
>>              Coupon _c;
>>              Long key = new Long(couponId);
>>              if(_coupons_map != null && _coupons_map.containsKey(key)) {
>>                  _c = (Coupon) _coupons_map.get(key);
>>                  _coupons_map.remove(key);
>>                  Long product = new Long(_c.getCoupon_id());
>>                  ArrayList<Long> _c_list =
>>                          (ArrayList<Long>) _product_to_coupons_map.get(**
>> product);
>>                  _c_list.remove(key);
>>                  _product_to_coupons_map.**remove(product);
>>                  _product_to_coupons_map.put(**product, _c_list);
>>              }
>>      }
>>
>>      synchronized public void removeOffer(long offerId) {
>>              Offer _o;
>>              Long key = new Long(offerId);
>>              if(_offers_map != null && _offers_map.containsKey(key)) {
>>                  _o = (Offer) _offers_map.get(key);
>>                  _offers_map.remove(key);
>>                  Long product = new Long(_o.getOffer_id());
>>                  ArrayList<Long> _o_list =
>>                          (ArrayList<Long>) _product_cat_to_offers_map.**
>> get(product);
>>                  _o_list.remove(key);
>>                  _product_cat_to_offers_map.**remove(product);
>>                  _product_cat_to_offers_map.**put(product, _o_list);
>>              }
>>      }
>>
>>      synchronized public Coupon getCoupon(long CouponID) {
>>          Long key = new Long(CouponID);
>>          if(_coupons_map != null && _coupons_map.containsKey(key)) {
>>              Coupon _c = (Coupon) _coupons_map.get(key);
>>              Date _now = new Date();
>>              if(_now.compareTo(_c.**getDateTo()) > 0) {
>>                  this.removeCoupon(CouponID);
>>                  return null;
>>              }
>>              return (Coupon) _coupons_map.get(key);
>>          }else
>>              return null;
>>      }
>>
>>      synchronized public Offer getOffer(long OfferID) {
>>          Long key = new Long(OfferID);
>>          if(_offers_map != null && _offers_map.containsKey(key)) {
>>              Offer _o = (Offer) _offers_map.get(key);
>>              Date _now = new Date();
>>              if(_now.compareTo(_o.**getDateTo()) > 0) {
>>                  this.removeOffer(OfferID);
>>                  return null;
>>              }
>>              return (Offer) _offers_map.get(key);
>>          }else
>>              return null;
>>      }
>>
>>      synchronized public ArrayList<Long> getCoupons(long ProductID) {
>>          Long key = new Long(ProductID);
>>          if(_product_to_coupons_map != null && _product_to_coupons_map.**containsKey(key))
>> {
>>              ArrayList<Long> _c_list = (ArrayList<Long>)
>> _product_to_coupons_map.get(**key);
>>              Iterator itr = _c_list.iterator();
>>              while(itr.hasNext()) {
>>                  Long l = (Long) itr.next();
>>                  if(this.getCoupon(l.longValue(**)) == null)
>>                      _c_list.remove(l.intValue());
>>              }
>>              _product_to_coupons_map.**remove(key);
>>              _product_to_coupons_map.put(**key, _c_list);
>>              return _c_list;
>>          }else
>>              return null;
>>      }
>>
>>      synchronized public ArrayList<Long> getOffers(long ProductID) {
>>          Long key = new Long(ProductID);
>>          if(_product_cat_to_offers_map != null &&
>> _product_cat_to_offers_map.**containsKey(key)) {
>>              ArrayList<Long> _o_list = _product_cat_to_offers_map.**
>> get(key);
>>              Iterator itr = _o_list.iterator();
>>              while(itr.hasNext()) {
>>                  Long l = (Long) itr.next();
>>                  if(this.getOffer(l.longValue()**) == null)
>>                      _o_list.remove(l.intValue());
>>              }
>>              _product_cat_to_offers_map.**remove(key);
>>              _product_cat_to_offers_map.**put(key, _o_list);
>>              return _o_list;
>>          }else
>>              return null;
>>      }
>>
>>      synchronized public HashMap<Long, Coupon> getCoupons_map() {
>>          return _coupons_map;
>>      }
>>
>>      synchronized public void setCoupons_map(HashMap<Long, Coupon>
>> _coupons_map) {
>>              this._coupons_map = _coupons_map;
>>      }
>>
>>      synchronized public static long getCreation_time() {
>>          return _creation_time;
>>      }
>>
>>      synchronized public void cleanup_offers() {
>>          if(_product_cat_to_offers_map != null) {
>>              Set _offers_key_set = _product_cat_to_offers_map.**keySet();
>>              Iterator itr = _offers_key_set.iterator();
>>              while(itr.hasNext()) {
>>                  Long l = (Long) itr.next();
>>                  this.getOffers(l.longValue());
>>              }
>>          }
>>      }
>>
>>      synchronized public void cleanup_coupons() {
>>          if(_product_to_coupons_map != null) {
>>              Set _coupons_key_set = _product_to_coupons_map.**keySet();
>>              Iterator itr = _coupons_key_set.iterator();
>>              while(itr.hasNext()) {
>>                  Long l = (Long) itr.next();
>>                  this.getCoupons(l.longValue())**;
>>              }
>>          }
>>      }
>> }
>>
>> The problem is that when I deploy the above application to the Apache
>> Tomcat I get
>> the following SEVERE log entries:
>>
>> SEVERE: The web application [/ETL_Procedures] appears to have started a
>> thread named [maintenance-task-executor-**thread-1] but has failed to
>> stop it.
>>   This is very likely to create a memory leak.
>> Oct 03, 2012 5:35:03 PM org.apache.catalina.loader.**WebappClassLoader
>> checkThreadLocalMapForLeaks
>>
>> SEVERE: The web application [/ETL_Procedures] created a ThreadLocal with
>> key of type [org.glassfish.gmbal.generic.**OperationTracer$1] (value
>> [org.glassfish.gmbal.generic.**OperationTracer$1@4c24821]) and a value of
>> type [java.util.ArrayList] (value [[]]) but failed to remove it
>> when the web application was stopped. Threads are going to be renewed over
>> time to try and avoid a probable memory leak.
>> Oct 03, 2012 5:35:03 PM org.apache.catalina.loader.**WebappClassLoader
>> checkThreadLocalMapForLeaks
>>
>> SEVERE: The web application [/ETL_Procedures] created a ThreadLocal with
>> key of type [com.sun.xml.bind.v2.**ClassFactory$1]
>> (value [com.sun.xml.bind.v2.**ClassFactory$1@6f0d70f7]) and a value of
>> type [java.util.WeakHashMap] (value [{class com.sun.xml.ws.runtime.config.
>> **TubeFactoryList=java.lang.ref.**WeakReference@5b73a116,
>> class javax.xml.bind.annotation.**adapters.**CollapsedStringAdapter=java.*
>> *lang.ref.WeakReference@454da42**,
>> class com.sun.xml.ws.runtime.config.**MetroConfig=java.lang.ref.**
>> WeakReference@5ec52546,
>> class com.sun.xml.ws.runtime.config.**TubeFactoryConfig=java.lang.**
>> ref.WeakReference@61124745,
>> class java.util.ArrayList=java.lang.**ref.WeakReference@770534cc, class
>> com.sun.xml.ws.runtime.config.**Tubelines=java.lang.ref.**
>> WeakReference@76cd7a1f,
>> class javax.xml.bind.annotation.**W3CDomHandler=java.lang.ref.**
>> WeakReference@2c0cc628,
>> class com.sun.xml.ws.runtime.config.**TubelineDefinition=java.lang.**
>> ref.WeakReference@7aa582af}]) but failed to remove it when
>> the web application was stopped. Threads are going to be renewed over time
>> to try and avoid a probable memory leak.
>> Oct 03, 2012 5:35:03 PM org.apache.catalina.loader.**WebappClassLoader
>> checkThreadLocalMapForLeaks
>>
>> SEVERE: The web application [/ETL_Procedures] created a ThreadLocal with
>> key of type [com.sun.xml.bind.v2.runtime.**Coordinator$1] (value
>> [com.sun.xml.bind.v2.runtime.**Coordinator$1@826ee11]) and a value of
>> type [java.lang.Object[]] (value [[Ljava.lang.Object;@33d7a245]**)
>> but failed to remove it when the web application was stopped. Threads are
>> going to be renewed over time to try and avoid a probable memory leak.
>> Oct 03, 2012 5:35:03 PM org.apache.catalina.loader.**WebappClassLoader
>> checkThreadLocalMapForLeaks
>>
>> SEVERE: The web application [/ETL_Procedures] created a ThreadLocal with
>> key of type [org.glassfish.gmbal.generic.**OperationTracer$1] (value
>>   [org.glassfish.gmbal.generic.**OperationTracer$1@4c24821]) and a value
>> of type [java.util.ArrayList] (value [[]]) but failed to
>> remove it when the web application was stopped. Threads are going to be
>> renewed over time to try and avoid a probable memory leak.
>>
>> I really do not know what causes those memory leaks. My questions are the
>> following:
>> 1) Can anyone suspect what may be the issue with my web service? If the
>> cause of the memory leaks is the Singleton object,
>> what else can I do to meet my applications requirements and avoid memory
>> leaks.
>> 2) Is there any tool that I can use in order to monitor my threads, and
>> what exactly causes these SEVERE messages?
>> 3) If I let my application deployed for a long time, I get an
>> IllegalStateException with the following message:
>>
>> INFO: Illegal access: this web application instance has been stopped
>> already.  Could not load com.sun.xml.ws.rx.rm.**localization.**
>> LocalizationMessages.
>> The eventual following stack trace is caused by an error thrown for
>> debugging purposes as well as to attempt to terminate the thread
>> which caused the illegal access, and has no functional impact.
>> java.lang.**IllegalStateException
>>      at org.apache.catalina.loader.**WebappClassLoader.loadClass(**
>> WebappClassLoader.java:1600)
>>      at org.apache.catalina.loader.**WebappClassLoader.loadClass(**
>> WebappClassLoader.java:1559)
>>      at com.sun.xml.ws.rx.rm.runtime.**sequence.**
>> SequenceMaintenanceTask.run(**SequenceMaintenanceTask.java:**81)
>>      at com.sun.xml.ws.commons.**DelayedTaskManager$Worker.run(**
>> DelayedTaskManager.java:91)
>>      at java.util.concurrent.**Executors$RunnableAdapter.**
>> call(Executors.java:471)
>>      at java.util.concurrent.**FutureTask$Sync.innerRun(**
>> FutureTask.java:334)
>>      at java.util.concurrent.**FutureTask.run(FutureTask.**java:166)
>>      at java.util.concurrent.**ScheduledThreadPoolExecutor$**
>> ScheduledFutureTask.access$**201(**ScheduledThreadPoolExecutor.**java:178)
>>      at java.util.concurrent.**ScheduledThreadPoolExecutor$**
>> ScheduledFutureTask.run(**ScheduledThreadPoolExecutor.**java:292)
>>      at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
>> ThreadPoolExecutor.java:1110)
>>      at java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
>> ThreadPoolExecutor.java:603)
>>      at java.lang.Thread.run(Thread.**java:722)
>>
>> How can I see the whole call path that causes this Exception?
>>
>> Thank you for your time and I am really sorry for the long message.
>>
>> ------------------------------**------------------------------**---------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.**apache.org<users-unsubscribe@tomcat.apache.org>
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
Bonjorno Filippo and Gracie for your immediate answer.
I also do not see anything wrong with my code, since the Singleton class 
I am using has been tested in other web
services as well. My suspicion is that Apache Tomcat has some cleaning 
up mechanisms, that react negatively when they come across my Singleton 
object.
I am investigating on the matter and I am trying to dig deeper on the 
Memory leaks that may occur on Tomcat.
Do you happen to know (or anyone else) any good tools for monitoring 
memory of each thread in Tomcat?
A tool that can get integrated into Netbeans would be a plus.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org