ctakes-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Finan, Sean" <Sean.Fi...@childrens.harvard.edu>
Subject RE: Best combination of analysis engines to consider negation, family history, uncertainty, etc.
Date Wed, 19 Oct 2016 14:53:47 GMT
Hi Yiming,

Thank you very much for letting the community know what has and has not worked for you.  I
have also had better results with the Assertion annotators than the ClearTk alternatives,
but that could be because of the note types/formats that I am using.

Regarding the "Clear" in names, it is because ClearTk (Clear ToolKit) is used to train machine
learning models for detection of the indicated property.  You can find information on ClearTk
starting here:  http://clear.colorado.edu/compsem/ 

If you prefer to read a paper, you can check out http://www.lrec-conf.org/proceedings/lrec2014/pdf/218_Paper.pdf

Others no the devlist can provide much more information than can I, so you could post a question
if you like.

Cheers,
Sean

-----Original Message-----
From: Zuo Yiming [mailto:yimingzuo@gmail.com] 
Sent: Wednesday, October 19, 2016 10:04 AM
To: user@ctakes.apache.org; dev@ctakes.apache.org
Subject: Best combination of analysis engines to consider negation, family history, uncertainty,
etc.

Hi everyone,

I've spent the last a few months working on a clinical NLP project using cTAKES. It's a very
complex system to me and every time I dig into it some new discoveries will come out. Since
last week, I tried to figure out which analysis engine can help to do a good job to consider
cases like negation, family history, uncertainty, etc. By now, I had some experience and would
like to share with the community.

The best combination for me is to use assertionMiniPipelineAnalysisEngine
for negation, uncertainty, generic and subject detection, and HistoryCleartkAnalysisEngine
for history detection. Both engines are in desc/ctakes-assertion folder. The assertionMiniPipelineAnalysisEngine
also claims to be useful for conditional detection, which I haven't verified using my test
files yet.

I'm using the AggregatePlaintextFastUMLSProcessor on the higher level. The default analysis
engines in AggregatePlaintextFastUMLSProcessor for negation, uncertainty, generic, etc. are
StatusAnnotator + NegationAnnotator + PolarityCleartkAnalysisEngine + SubjectCleartkAnalysisEngine
+ UncertaintyCleartkAnalysisEngine + GenericCleartkAnalysisEngine + HistoryCleartkAnalysisEngine.
It looks like in the node part, StatusAnnotator and NegationAnnotator are commented out, so
only the remaining five analysis engines are actually used and all of them are in the same
desc/ctakes-assertion folder. These five analysis engines were not effective in my test files
and I'm still confused by their relationship to the assertionaAnalysisEngine, conceptConverterAnalysisEngine,
GenericAttributeAnalysisEngine and SubjectAttributeAnalysisEngine used in assertionMiniPipelineAnalysisEngine.
It looks to me the Clear in their names indicate something but I couldn't figure it out without
going through the java code, which I intend not to do at this level.

That's pretty much all of it for now. Anyone familiar with this topic are welcome to jump
in to provide my insights or correction. Hopefully, we can have a nice discussion that can
be useful to other users and developers.

ps. The reason for using AggregatePlaintextFastUMLSProcessor rather than AggregatePlaintextProcessor
is that I find the preferred words property in the former very useful while it can't be detected
using the latter.

Best,
Yiming
--
Yiming Zuo <https://urldefense.proofpoint.com/v2/url?u=https-3A__sites.google.com_site_yimingzuo_&d=DQIBaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m=4at7fOO27JCueBfJFn7Hv2vKWlUAK-nuYYdmMyGRJPQ&s=vSmSOvLXuCa-Pwp8qu05VTzZgGA0P3Y2CL8q3JBhppQ&e=
> Georgetown U. Medical Center:
Dr. Ressom's Omics Lab <https://urldefense.proofpoint.com/v2/url?u=http-3A__omics.georgetown.edu_&d=DQIBaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m=4at7fOO27JCueBfJFn7Hv2vKWlUAK-nuYYdmMyGRJPQ&s=yNsVaS7s20e-125SmdmQqKHvQ0lAQ7si98GefPRDxT0&e=
> ECE Department of Virginia Tech:
Computational Bioinformatics & Bio-imaging Laboratory <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.cbil.ece.vt.edu_&d=DQIBaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m=4at7fOO27JCueBfJFn7Hv2vKWlUAK-nuYYdmMyGRJPQ&s=DpORI1TH9yITkdlRX_RLjxejH2jMJUq8yFaTPjWAar4&e=
>
Mime
View raw message