ctakes-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vijay garla <vnga...@gmail.com>
Subject Re: cTAKES CPE MySQL Exception
Date Thu, 24 Jul 2014 18:05:39 GMT
My guess is that this exception is coming out of the DictionaryLookup (it
creates a connection and holds on to it for the life of the AE).

If it is coming out of the DBCollectionReader/DBConsumer you're in luck, as
those use a connection pool, and you can configure it to check the
connection upon pulling from the pool

The file is: resources\org\apache\ctakes\ytex\beans-datasource.xml
see
http://commons.apache.org/proper/commons-dbcp/api-1.4/org/apache/commons/dbcp/BasicDataSource.html
- you want to set testOnBorrow to true, and set the validtionQuery to
something like "select 1"

You should also set the errorRateThreshold in the CPE config (you can't do
this via the gui - you have to do this in the xml) - that way the cpe
doesn't bomb on the first error it sees - a few bad apples shouldn't kill
the processing.

HTH,

VJ



On Thu, Jul 24, 2014 at 4:32 PM, Clayton Turner <caturner3@g.cofc.edu>
wrote:

> Hi, everyone.
>
> First off, I'd like to say "awesome and thank you" for the cTAKES 3.2
> release and information. I've been following those pages and it's been
> really helpful for helping me move along in my own progress. Really cool
> stuff.
>
> So I'm using the Collection Processing Engine (with ytex and umls) and I'm
> trying to process ~1 million notes (as opposed to the about 30 in the given
> demo).
>
> I've tried this the past 2 days and when I come back in to check the
> progress I see that I've received an error about 14000 notes into the
> process:
>
> org.apache.uima.analysis_engine.AnalysisEngineProcessException: Annotator
> processing failed.
> CausedBy: org.springframework.transaction.CannotCreateTransactionException:
> Could not open Hibernate Session for transaction; nested exception is
> com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet
> successfully received from the server was 53,888,249 milliseconds ago. The
> last packet sent successfully to the server was 53,888,249 milliseconds
> ago. is longer than the server configured value of 'wait_timeout'. You
> should consider either expiring and/or testing connection validity before
> use in your application, increasing the server configured values for client
> timeouts, or using the Connector/J connection property 'autoReconnect=true'
> to avoid this problem.
>
> So, in my own debugging, I have ensured that autoReconnect true was on (it
> always has been).
>
> I looked at my CPE output in the command prompt and noticed a
> "PacketTooBigException" so I increased the packet max size to 1G (the max
> for sql server).
>
> I increased the time allowed for timeouts.
>
> I'm really unsure of what to do here. Should I find a way to see if there
> is a problematic note that is giving me issues (though I can't understand
> how 1 note would make a packet too large)? Should I try to do some
> horizontal sharding and break the problem into smaller chunks (though I
> would think this program could handle large datasets since it's using a
> query language)? I'm just at a loss with this error, especially since it
> takes so long to actually spit the error out at me.
>
> Thanks in advance everyone,
> Clayton
>
> --
> --
> Clayton Turner
> email: caturner3@g.cofc.edu
> phone: (843)-424-3784
> web: claytonturner.blogspot.com
>
> -------------------------------------------------------------------------------------------------
> “When scientifically investigating the natural world, the only thing worse
> than a blind believer is a seeing denier.”
> - Neil deGrasse Tyson
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message