camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Wolf <cwolf.a...@gmail.com>
Subject Re: Does Camel JPA component implement it's own connection pooling?
Date Thu, 31 Jan 2013 17:48:28 GMT
James,

Here is what it looks like.  At first, I was using the "jpa://"
endpoint, but then I needed to implement a custom processor to handle
additional logic.  Thanks for having a look...

   -Chris

from("file:/tmp/local/data?filter=#filter&preMove=/tmp/local/holding/${file:onlyname}.${id}&move=/tmp/local/archive&moveFailed=/tmp/local/errors&readLock=changed")
.unmarshal(cpmdfileFormat)
.processRef("dataStore")
 //.to("jpa:?persistenceUnit=marketdata")
 .beanRef("oneshotPoller", "resumeJpaConsumer"); // signal consumer
route to begin


Bean for "dataStore":

public class CPMDPersister implements Processor {
	private static final transient Logger log =
LoggerFactory.getLogger(CPMDPersister.class);
	IGenericDAO<MdBaseData> mdBaseDataDAO;
	IGenericDAO<MdCurveData> mdCurveDataDAO;

	public CPMDPersister(){}
	
    public CPMDPersister(IGenericDAO<MdBaseData> mdBaseDataDAO,
IGenericDAO<MdCurveData> mdCurveDataDAO) {
    	this.mdBaseDataDAO = mdBaseDataDAO;
    	this.mdCurveDataDAO = mdCurveDataDAO;
    }

	@Override
	public void process(Exchange exchange) throws Exception {
		Message inMsg = exchange.getIn();
		
		@SuppressWarnings("unchecked")
		List<MdBaseData> mdBaseData =
(ArrayList<MdBaseData>)inMsg.getBody(ArrayList.class);

		IntervalElapsedTimer avg = new IntervalElapsedTimer();
		avg.start();
		mdBaseDataDAO.begin();

		for (MdBaseData md : mdBaseData) {
			setParentReferences(md);
			mdBaseDataDAO.save(md);
		    avg.lap();
		    log.debug("JPA Persist MD: {}, {}",
		        new Object[] {md.toString(), avg.toString()});
		}
		
		mdBaseDataDAO.commit();
		
		log.debug("JPA Persist: {}", avg.toString());
		int a = 0;
	}
	
	void setParentReferences(MdBaseData md) {		
		for (MdCurveData cd : md.getMdCurveData()) {
			if (cd.getMdBaseData() == null)
				cd.setMdBaseData(new ArrayList<MdBaseData>());
			if (!cd.getMdBaseData().contains(md))
				cd.getMdBaseData().add(md);
		}
	}
}

The DAOs, are a thin wrapper around EntityManager, like:

	@SuppressWarnings("unchecked")
	public List<T> findAll() {
		return entityManager.createQuery("from " + entityType.getName())
				.getResultList();
	}
	public void begin() {
		this.entityManager.getTransaction().begin();
	}

	public void commit() {
		this.entityManager.getTransaction().commit();
		this.entityManager.clear();
	}

	public void rollback() {
		this.entityManager.getTransaction().rollback();
	}

On Thu, Jan 31, 2013 at 12:34 PM, James Carman
<james@carmanconsulting.com> wrote:
> What does your route look like?
>
>
> On Thu, Jan 31, 2013 at 11:58 AM, Chris Wolf <cwolf.algo@gmail.com> wrote:
>> I have further determined that it's an OpenJPA issue (or user error,
>> but how can that be?) ;)
>>
>> I will report back when I get to the bottom of it.  Thanks a lot,
>>
>> Chris
>>
>> On Sat, Jan 26, 2013 at 2:47 AM, Willem Jiang <willem.jiang@gmail.com> wrote:
>>> Can you share us your camel route? It is important for us to find out the key
reason of your issue.
>>>
>>>
>>> 发自我的 iPhone
>>>
>>> 在 2013-1-26,上午5:22,Chris Wolf <cwolf.algo@gmail.com> 写道:
>>>
>>>> On Fri, Jan 25, 2013 at 3:55 PM, Chris Wolf <cwolf.algo@gmail.com>
wrote:
>>>>> I ask because when I use jpa:  in producer mode, I get this error:
>>>>> Caused by: oracle.net.ns.NetException: Listener refused the connection
>>>>> with the following error:
>>>>> ORA-12519, TNS:no appropriate service handler found
>>>>>
>>>>> Strangely, using  jpa: in consumer mode, this does not happen.  In the
>>>>> tnslsnr.log, I observe 120 connections being opened when
>>>>> running the route with jpa: in producer mode.  If I instead use a
>>>>> custom Processor and do my own JPA persistence code,
>>>>> the issue does not occur.
>>>>>
>>>>> Thanks for any advice,
>>>>>
>>>>> Chris
>>>>
>>>>
>>>> I stepped through with the debugger and the problem went away.  The
>>>> point at which a lot of connections were opened was
>>>> after I stepped over:
>>>>
>>>>                return jpaTemplate.execute(new JpaCallback<Object>()
{
>>>>                    public Object doInJpa(EntityManager entityManager)
>>>> throws PersistenceException {
>>>>                        return callback.doInJpa(entityManager);
>>>>                    }
>>>>
>>>> ..and when running in real time (not debugging) it seems to be running
>>>> in an AsyncProcessor - I wonder if it's too many worker
>>>> threads eating up the connections?  BTW, this is mt first project
>>>> using Camel, so forgive the lack of experience...
>>>>
>>>>
>>>> [ad #1 - file:///tmp/local/data] UnmarshallingContext           DEBUG
>>>> EOF reached at line 2007
>>>> [ad #1 - file:///tmp/local/data] SendProcessor                  DEBUG
>>>>>>>> Endpoint[jpa://?persistenceUnit=marketdata] Exchange[null]
>>>> [ad #1 - file:///tmp/local/data] JpaTransactionManager          DEBUG
>>>> Creating new transaction with name [null]:
>>>> PROPAGATION_REQUIRED,ISOLATION_DEFAULT
>>>> 8505  marketdata  INFO   [Camel (camel-1) thread #1 -
>>>> file:///tmp/local/data] openjpa.Runtime - Starting OpenJPA 2.2.1
>>>> 8535  marketdata  INFO   [Camel (camel-1) thread #1 -
>>>> file:///tmp/local/data] openjpa.jdbc.JDBC - Using dictionary class
>>>> "org.apache.openjpa.jdbc.sql.OracleDictionary".
>>>> [ad #1 - file:///tmp/local/data] JpaTransactionManager          DEBUG
>>>> Opened new EntityManager
>>>> [org.apache.openjpa.persistence.EntityManagerImpl@798b429b] for JPA
>>>> transaction
>>>> [ad #1 - file:///tmp/local/data] JpaTransactionManager          DEBUG
>>>> Initiating transaction commit
>>>> [ad #1 - file:///tmp/local/data] JpaTransactionManager          DEBUG
>>>> Committing JPA transaction on EntityManager
>>>> [org.apache.openjpa.persistence.EntityManagerImpl@798b429b]
>>>> [ad #1 - file:///tmp/local/data] JpaTransactionManager          DEBUG
>>>> Closing JPA EntityManager
>>>> [org.apache.openjpa.persistence.EntityManagerImpl@798b429b] after
>>>> transaction
>>>> [ad #1 - file:///tmp/local/data] EntityManagerFactoryUtils      DEBUG
>>>> Closing JPA EntityManager

Mime
View raw message