camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Carman <ja...@carmanconsulting.com>
Subject Re: Does Camel JPA component implement it's own connection pooling?
Date Thu, 31 Jan 2013 19:29:56 GMT
Well, you're managing your entity manager yourself, right?  You could
have a bug in there that's leaving it open?  Can you use Spring or
some other technology to do this for you in a way where you can be
sure everything gets cleaned up?


On Thu, Jan 31, 2013 at 2:28 PM, Chris Wolf <cwolf.algo@gmail.com> wrote:
> Howdy Matthias,
>
> Yes, in an earlier post to this thread, I mentioned that 120
> connections were being opened (the max). So, yes, the max connections
> limit is being hit.
>
> I observe this by tailing the TNS listener log
>
> c:/app/<userId>/diag/tnslsnr/<userId>/listener/trace/listener.log
>
> (Oracle-11g on Windows7)
>
>
> The question is why?  Unless I have a connection leak.
>
>    -Chris
>
>
> On Thu, Jan 31, 2013 at 2:21 PM, Matthias Lüneberg
> <matthias.lueneberg@googlemail.com> wrote:
>> Hi,
>>
>> Is it possible that the oracle database has reached its max connections/processes
limit?
>>
>> I found an article: http://www.dadbm.com/2011/11/oracle-listener-refused-connection-ora-12519-troubleshooting/
>>
>> Maybe its a database failure. Can you have a look at this?
>>
>> HTH, Matthias
>>
>> Am 31.01.2013 um 18:48 schrieb Chris Wolf <cwolf.algo@gmail.com>:
>>
>>> James,
>>>
>>> Here is what it looks like.  At first, I was using the "jpa://"
>>> endpoint, but then I needed to implement a custom processor to handle
>>> additional logic.  Thanks for having a look...
>>>
>>>   -Chris
>>>
>>> from("file:/tmp/local/data?filter=#filter&preMove=/tmp/local/holding/${file:onlyname}.${id}&move=/tmp/local/archive&moveFailed=/tmp/local/errors&readLock=changed")
>>> .unmarshal(cpmdfileFormat)
>>> .processRef("dataStore")
>>> //.to("jpa:?persistenceUnit=marketdata")
>>> .beanRef("oneshotPoller", "resumeJpaConsumer"); // signal consumer
>>> route to begin
>>>
>>>
>>> Bean for "dataStore":
>>>
>>> public class CPMDPersister implements Processor {
>>>       private static final transient Logger log =
>>> LoggerFactory.getLogger(CPMDPersister.class);
>>>       IGenericDAO<MdBaseData> mdBaseDataDAO;
>>>       IGenericDAO<MdCurveData> mdCurveDataDAO;
>>>
>>>       public CPMDPersister(){}
>>>
>>>    public CPMDPersister(IGenericDAO<MdBaseData> mdBaseDataDAO,
>>> IGenericDAO<MdCurveData> mdCurveDataDAO) {
>>>       this.mdBaseDataDAO = mdBaseDataDAO;
>>>       this.mdCurveDataDAO = mdCurveDataDAO;
>>>    }
>>>
>>>       @Override
>>>       public void process(Exchange exchange) throws Exception {
>>>               Message inMsg = exchange.getIn();
>>>
>>>               @SuppressWarnings("unchecked")
>>>               List<MdBaseData> mdBaseData =
>>> (ArrayList<MdBaseData>)inMsg.getBody(ArrayList.class);
>>>
>>>               IntervalElapsedTimer avg = new IntervalElapsedTimer();
>>>               avg.start();
>>>               mdBaseDataDAO.begin();
>>>
>>>               for (MdBaseData md : mdBaseData) {
>>>                       setParentReferences(md);
>>>                       mdBaseDataDAO.save(md);
>>>                   avg.lap();
>>>                   log.debug("JPA Persist MD: {}, {}",
>>>                       new Object[] {md.toString(), avg.toString()});
>>>               }
>>>
>>>               mdBaseDataDAO.commit();
>>>
>>>               log.debug("JPA Persist: {}", avg.toString());
>>>               int a = 0;
>>>       }
>>>
>>>       void setParentReferences(MdBaseData md) {
>>>               for (MdCurveData cd : md.getMdCurveData()) {
>>>                       if (cd.getMdBaseData() == null)
>>>                               cd.setMdBaseData(new ArrayList<MdBaseData>());
>>>                       if (!cd.getMdBaseData().contains(md))
>>>                               cd.getMdBaseData().add(md);
>>>               }
>>>       }
>>> }
>>>
>>> The DAOs, are a thin wrapper around EntityManager, like:
>>>
>>>       @SuppressWarnings("unchecked")
>>>       public List<T> findAll() {
>>>               return entityManager.createQuery("from " + entityType.getName())
>>>                               .getResultList();
>>>       }
>>>       public void begin() {
>>>               this.entityManager.getTransaction().begin();
>>>       }
>>>
>>>       public void commit() {
>>>               this.entityManager.getTransaction().commit();
>>>               this.entityManager.clear();
>>>       }
>>>
>>>       public void rollback() {
>>>               this.entityManager.getTransaction().rollback();
>>>       }
>>>
>>> On Thu, Jan 31, 2013 at 12:34 PM, James Carman
>>> <james@carmanconsulting.com> wrote:
>>>> What does your route look like?
>>>>
>>>>
>>>> On Thu, Jan 31, 2013 at 11:58 AM, Chris Wolf <cwolf.algo@gmail.com>
wrote:
>>>>> I have further determined that it's an OpenJPA issue (or user error,
>>>>> but how can that be?) ;)
>>>>>
>>>>> I will report back when I get to the bottom of it.  Thanks a lot,
>>>>>
>>>>> Chris
>>>>>
>>>>> On Sat, Jan 26, 2013 at 2:47 AM, Willem Jiang <willem.jiang@gmail.com>
wrote:
>>>>>> Can you share us your camel route? It is important for us to find
out the key reason of your issue.
>>>>>>
>>>>>>
>>>>>> 发自我的 iPhone
>>>>>>
>>>>>> 在 2013-1-26,上午5:22,Chris Wolf <cwolf.algo@gmail.com>
写道:
>>>>>>
>>>>>>> On Fri, Jan 25, 2013 at 3:55 PM, Chris Wolf <cwolf.algo@gmail.com>
wrote:
>>>>>>>> I ask because when I use jpa:  in producer mode, I get this
error:
>>>>>>>> Caused by: oracle.net.ns.NetException: Listener refused the
connection
>>>>>>>> with the following error:
>>>>>>>> ORA-12519, TNS:no appropriate service handler found
>>>>>>>>
>>>>>>>> Strangely, using  jpa: in consumer mode, this does not happen.
 In the
>>>>>>>> tnslsnr.log, I observe 120 connections being opened when
>>>>>>>> running the route with jpa: in producer mode.  If I instead
use a
>>>>>>>> custom Processor and do my own JPA persistence code,
>>>>>>>> the issue does not occur.
>>>>>>>>
>>>>>>>> Thanks for any advice,
>>>>>>>>
>>>>>>>> Chris
>>>>>>>
>>>>>>>
>>>>>>> I stepped through with the debugger and the problem went away.
 The
>>>>>>> point at which a lot of connections were opened was
>>>>>>> after I stepped over:
>>>>>>>
>>>>>>>               return jpaTemplate.execute(new JpaCallback<Object>()
{
>>>>>>>                   public Object doInJpa(EntityManager entityManager)
>>>>>>> throws PersistenceException {
>>>>>>>                       return callback.doInJpa(entityManager);
>>>>>>>                   }
>>>>>>>
>>>>>>> ..and when running in real time (not debugging) it seems to be
running
>>>>>>> in an AsyncProcessor - I wonder if it's too many worker
>>>>>>> threads eating up the connections?  BTW, this is mt first project
>>>>>>> using Camel, so forgive the lack of experience...
>>>>>>>
>>>>>>>
>>>>>>> [ad #1 - file:///tmp/local/data] UnmarshallingContext       
   DEBUG
>>>>>>> EOF reached at line 2007
>>>>>>> [ad #1 - file:///tmp/local/data] SendProcessor              
   DEBUG
>>>>>>>>>>> Endpoint[jpa://?persistenceUnit=marketdata] Exchange[null]
>>>>>>> [ad #1 - file:///tmp/local/data] JpaTransactionManager      
   DEBUG
>>>>>>> Creating new transaction with name [null]:
>>>>>>> PROPAGATION_REQUIRED,ISOLATION_DEFAULT
>>>>>>> 8505  marketdata  INFO   [Camel (camel-1) thread #1 -
>>>>>>> file:///tmp/local/data] openjpa.Runtime - Starting OpenJPA 2.2.1
>>>>>>> 8535  marketdata  INFO   [Camel (camel-1) thread #1 -
>>>>>>> file:///tmp/local/data] openjpa.jdbc.JDBC - Using dictionary
class
>>>>>>> "org.apache.openjpa.jdbc.sql.OracleDictionary".
>>>>>>> [ad #1 - file:///tmp/local/data] JpaTransactionManager      
   DEBUG
>>>>>>> Opened new EntityManager
>>>>>>> [org.apache.openjpa.persistence.EntityManagerImpl@798b429b] for
JPA
>>>>>>> transaction
>>>>>>> [ad #1 - file:///tmp/local/data] JpaTransactionManager      
   DEBUG
>>>>>>> Initiating transaction commit
>>>>>>> [ad #1 - file:///tmp/local/data] JpaTransactionManager      
   DEBUG
>>>>>>> Committing JPA transaction on EntityManager
>>>>>>> [org.apache.openjpa.persistence.EntityManagerImpl@798b429b]
>>>>>>> [ad #1 - file:///tmp/local/data] JpaTransactionManager      
   DEBUG
>>>>>>> Closing JPA EntityManager
>>>>>>> [org.apache.openjpa.persistence.EntityManagerImpl@798b429b] after
>>>>>>> transaction
>>>>>>> [ad #1 - file:///tmp/local/data] EntityManagerFactoryUtils  
   DEBUG
>>>>>>> Closing JPA EntityManager
>>

Mime
View raw message