Mailing-List: contact user-help@ctakes.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@ctakes.apache.org
Received-SPF: error (nike.apache.org: encountered temporary error during SPF
 processing of domain of lance@iodinesoftware.com)
MIME-Version: 1.0
From: Lance Eason <lance@iodinesoftware.com>
Date: Fri, 9 Jan 2015 11:35:29 -0600
Message-ID: 
 <CAPR0Ys8bq1SFpKn_vOsWDx=0r7hXG1n2=N-9aYLUqTSau-FZYg@mail.gmail.com>
Subject: How to train new models for the ClearTK based assertion analyzers?
To: user@ctakes.apache.org
Content-Type: multipart/alternative; boundary=001a11c37c1868d92b050c3b9649

--001a11c37c1868d92b050c3b9649
Content-Type: text/plain; charset=UTF-8

The new ClearTK assertion analyzers in 3.2.1 (GenericCleartkAnalysisEngine,
HistoryCleartkAnalysisEngine, etc.) are a welcome change from the
perspective that they're much, much faster than the previous MedFacts
implementation.  Unfortunately though I'm finding them significantly less
accurate at actually flagging the assertion attributes correctly.

Could someone point me in the direction of how to train new models?  I've
found org.apache.ctakes.assertion.train.TrainAttributeModels which looks
promising but I can't find the current training data anywhere (I'd like to
use it as the starting point) and without samples have no idea what the
format it's expecting is.


Some real world examples from clinical notes:

I would have expected each of the following to be 'generic' (or maybe
'conditional') as they're referring to a hypothetical future problem.  (To
be fair the previous implementation did no better on these):
   - Gel foam cushion is also required in order to prevent pressure ulcers
from forming, as patient will spend many hours of the day in chair.
   - Ordered wound care protocol and applied mepliex to bottom to prevent
skin breakdown.
   - Encouraged patient to shift weight more frequently to prevent any
pressure ulcers.
   - Showed patient how to use foam to avoid pressure ulcers.
   - Gave her pamphlet about pressure ulcers.
   - I educated patient on causes and prevention of pressure ulcers.
   - Instructed on need to adjust position Q2 hours to avoid pressure
ulcers.

This statement is about as direct an instance of negation as possible and
is instead ruled as conditional by the new models instead of negated:
   - Patient does not have pneumonia.

"Ruled out" is no longer understood as negation:
   - Ruled out pneumonia.

"h/o" is no longer understood as historic ("hx" and "history of" are still
picked up):
   - h/o heart failure.

Trailing conditionals are no longer picked up:
   - "Likely tuberculosis" (found) vs. "Tuberculosis likely" (missed)
   - "Possible tuberculosis" (found" vs. "Tuberculosis is possible" (missed)

--001a11c37c1868d92b050c3b9649
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">The new ClearTK assertion analyzers in 3.2.1 (GenericClear=
tkAnalysisEngine, HistoryCleartkAnalysisEngine, etc.) are a welcome change =
from the perspective that they&#39;re much, much faster than the previous M=
edFacts implementation.=C2=A0 Unfortunately though I&#39;m finding them sig=
nificantly less accurate at actually flagging the assertion attributes corr=
ectly.<div><br></div><div>Could someone point me in the direction of how to=
 train new models?=C2=A0 I&#39;ve found org.apache.ctakes.assertion.train.T=
rainAttributeModels which looks promising but I can&#39;t find the current =
training data anywhere (I&#39;d like to use it as the starting point) and w=
ithout samples have no idea what the format it&#39;s expecting is.</div><di=
v><br></div><div><br></div><div>Some real world examples from clinical note=
s:</div><div><br></div><div>I would have expected each of the following to =
be &#39;generic&#39; (or maybe &#39;conditional&#39;) as they&#39;re referr=
ing to a hypothetical future problem. =C2=A0(To be fair the previous implem=
entation did no better on these):<br></div><div><div>=C2=A0 =C2=A0- Gel foa=
m cushion is also required in order to prevent pressure ulcers from forming=
, as patient will spend many hours of the day in chair.<br></div><div>=C2=
=A0 =C2=A0- Ordered wound care protocol and applied mepliex to bottom to pr=
event skin breakdown.</div><div>=C2=A0 =C2=A0- Encouraged patient to shift =
weight more frequently to prevent any pressure ulcers.</div><div>=C2=A0 =C2=
=A0- Showed patient how to use foam to avoid pressure ulcers.</div><div>=C2=
=A0 =C2=A0- Gave her pamphlet about pressure ulcers.</div><div>=C2=A0 =C2=
=A0- I educated patient on causes and prevention of pressure ulcers.</div><=
div>=C2=A0 =C2=A0- Instructed on need to adjust position Q2 hours to avoid =
pressure ulcers.</div></div><div><br></div><div>This statement is about as =
direct an instance of negation as possible and is instead ruled as conditio=
nal by the new models instead of negated:</div><div>=C2=A0 =C2=A0-=C2=A0Pat=
ient does not have pneumonia.</div><div><br></div><div><div class=3D"gmail_=
signature">&quot;Ruled out&quot; is no longer understood as negation:</div>=
<div class=3D"gmail_signature">=C2=A0 =C2=A0-=C2=A0Ruled out pneumonia.</di=
v><div class=3D"gmail_signature"><br></div><div class=3D"gmail_signature">&=
quot;h/o&quot; is no longer understood as historic (&quot;hx&quot; and &quo=
t;history of&quot; are still picked up):</div><div class=3D"gmail_signature=
">=C2=A0 =C2=A0-=C2=A0h/o heart failure.</div><div class=3D"gmail_signature=
"><br></div><div class=3D"gmail_signature">Trailing conditionals are no lon=
ger picked up:</div><div class=3D"gmail_signature">=C2=A0 =C2=A0- &quot;Lik=
ely tuberculosis&quot; (found) vs. &quot;Tuberculosis likely&quot; (missed)=
</div><div class=3D"gmail_signature">=C2=A0 =C2=A0- &quot;Possible tubercul=
osis&quot; (found&quot; vs. &quot;Tuberculosis is possible&quot; (missed)</=
div><div class=3D"gmail_signature"><br></div><div class=3D"gmail_signature"=
><br></div>
</div></div>

--001a11c37c1868d92b050c3b9649--