ctakes-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clayton Turner <caturn...@g.cofc.edu>
Subject Re: v_snomed_fword_lookup view
Date Wed, 13 Aug 2014 16:24:09 GMT
Okay, I believe I have ctakes dictionary fast working now. Something I'm
curious about, though, is how you extract the data in order to conduct
analysis.

I've, in the past, been using the SparseDataExporterImpl from ytex in order
to create a .arff file for use in weka, but the ctakes pipeline I'm using
doesn't seem to be compatible with this ytex exporting as I'm not getting
any cuis in my arff file.

I'm using the aggregate plain text umls processor analysis engine from
ctakes and then using the dbconsumer analysis engine from ytex (for storing
into the database with regard to analysis batch).

Any tips for exporting or some simple issue I'm missing?

Thanks,
Clayton


On Mon, Aug 11, 2014 at 2:09 PM, Harpreet Khanduja <hsk5004@rit.edu> wrote:

> Yes, absolutely and
> no problem at all.
>
> Regards,
> Harpreet
>
>
> On Mon, Aug 11, 2014 at 1:16 PM, Finan, Sean <
> Sean.Finan@childrens.harvard.edu> wrote:
>
> > Thanks Harpreet,
> > That is definitely necessary to build!
> >
> > Those lines should already be in the pom, but commented out.  I think
> that
> > some version/branching issues may have arisen at some point wrt this
> module
> > ...
> >
> > If somebody beats me to it then cheers, otherwise I will try to check out
> > tonight and get all the bits in place.
> >
> > Sean
> >
> > > -----Original Message-----
> > > From: Harpreet Khanduja [mailto:hsk5004@rit.edu]
> > > Sent: Monday, August 11, 2014 1:12 PM
> > > To: dev@ctakes.apache.org
> > > Subject: Re: v_snomed_fword_lookup view
> > >
> > > Hello Clayton,
> > >   I do not know about ytex, but I did switch from dictionary-lookup to
> > dictionary-
> > > lookup-fast.
> > >   I update my ctakes-dictionary-lookup-fast project using maven.
> > >   I think I used Team- Update and switched to the latest revision
> > available and
> > > then
> > >   I downloaded new 3.2 resources from the for umls. and then I added
> > these
> > > resources to my
> > >   ctakes-dictionary-lookup-fast resources folder and also the classpath
> > in ctakes-
> > > clinical-pipeline.
> > >
> > >  Then I changed the pom.xml file which belongs to the whole ctakes
> > project and
> > > added <dependency> <groupId>org.apache.ctakes</groupId>
> > > <artifactId>ctakes-dictionary-lookup-res</artifactId>
> > > <version>${ctakes.version}</version>
> > > </dependency>
> > > <dependency>
> > > <groupId>org.apache.ctakes</groupId>
> > > <artifactId>ctakes-dictionary-lookup-fast</artifactId>
> > > <version>${ctakes.version}</version>
> > > </dependency>
> > >
> > >
> > >  these two dependencies to the file.
> > >
> > >
> > > After this, I also added the dependency
> > >     <dependency>
> > > <groupId>org.apache.ctakes</groupId>
> > > <artifactId>ctakes-dictionary-lookup-fast</artifactId>
> > > </dependency>
> > >
> > > to the pom.xml of ctakes-clinical-pipeline.
> > >
> > > And then add the resources folder in ctakes-clinical-pipeline using
> > build path
> > > configuration under "add class" option.
> > >
> > > After this it should work.
> > >
> > >
> > > Regards,
> > > Harpreet
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Mon, Aug 11, 2014 at 12:44 PM, Clayton Turner <caturner3@g.cofc.edu
> >
> > > wrote:
> > >
> > > > I still get the same error with the ctakes3.2 branch. Any
> suggestions?
> > > >
> > > >
> > > > On Mon, Aug 11, 2014 at 12:06 PM, Clayton Turner
> > > > <caturner3@g.cofc.edu>
> > > > wrote:
> > > >
> > > > > I'm going to do a clean install through the repo rather than the
> > > > > binaries and see if that fixes my issue because I think I just read
> > > > > a past post saying the lookup2 folders exist there.
> > > > >
> > > > >
> > > > > On Mon, Aug 11, 2014 at 11:52 AM, Clayton Turner
> > > > > <caturner3@g.cofc.edu>
> > > > > wrote:
> > > > >
> > > > >> When navigating to
> > > > >> ctakes-dictionary-lookup-fast\desc\analysis_engine
> > > > >> there are 2 files, assumedly analysis engines.
> > > > >>
> > > > >> SnomedLookupAnnotator.xml and SnomedOvLookupAnnotator.xml
> > > > >>
> > > > >> If I pick either, I put in my UMLS information but receive an
> error
> > > > >> when trying to run the CPE:
> > > > >>
> > > > >> Initialization of CAS Processor with name
> "SnomedOvLookupAnnotator"
> > > > >> failed.
> > > > >> CausedBy: org.apache.uima.resource.ResourceConfigurationException:
> > > > >> Initialization of CAS processor with name
> "SnomedOvLookupAnnotator"
> > > > >> failed.
> > > > >> CausedBy:
> org.apache.uima.resource.ResourceInitializationException:
> > > > Error
> > > > >> initializing "org.apache.uima.resource.impl.DataResource_impl"
> from
> > > > >> descriptor file:..............SnomedLookupAnnotator.xml
> > > > >> CausedBy:
> org.apache.uima.resource.ResourceInitializationException:
> > > > Could
> > > > >> not
> > > > >> access the resource data at
> > > > >>
> > > > >>
> > > >
> file:org\apache\ctakes\dictionary\lookup2\Snomed2011ab_ctakesTui\cTake
> > > > sSnomed.xml
> > > > >>
> > > > >> Now, I don't even have a "lookup2" folder and, subsequently the
> Tui
> > > > >> folder and cTakesSnomed.xml file. This seems to be the problem,
> but
> > > > >> I'm
> > > > not
> > > > >> sure where these files are supposed to be grabbed from.
> > > > >>
> > > > >>
> > > > >> On Mon, Aug 11, 2014 at 11:47 AM, Clayton Turner
> > > > >> <caturner3@g.cofc.edu>
> > > > >> wrote:
> > > > >>
> > > > >>> Hi again:
> > > > >>>
> > > > >>> How exactly do you switch to using the cTakes
> > dictionary-lookup-fast.
> > > > Do
> > > > >>> I need to go in and alter xml files or is it as simple as
adding
> a
> > > > certain
> > > > >>> item to the list of analysis engines?
> > > > >>>
> > > > >>>
> > > > >>> On Fri, Aug 8, 2014 at 3:48 PM, Finan, Sean <
> > > > >>> Sean.Finan@childrens.harvard.edu> wrote:
> > > > >>>
> > > > >>>> Hi Clayton,
> > > > >>>>
> > > > >>>> I don't know how the ytex dictionary lookup works, so
I'm afraid
> > > > >>>> that
> > > > I
> > > > >>>> can't help you with an answer.  Maybe Vijay is the best
person
> to
> > > > >>>> do
> > > > this.
> > > > >>>>  If you aren't tied to ytex you could try the new cTakes
> > > > >>>> dictionary-lookup-fast.  I tested "Patient came in with
a malar
> > rash"
> > > > and
> > > > >>>> it found "malar" and "malar rash".
> > > > >>>>
> > > > >>>> Vijay,
> > > > >>>>
> > > > >>>> At some point the lookup-fast module will be the default
for the
> > > > cTakes
> > > > >>>> clinical pipeline.  In order to synchronize the ytex
lookup with
> > > > cTakes,
> > > > >>>> would you like to eventually work together on reusing
the same
> > > > >>>> code
> > > > for
> > > > >>>> ytex?  I have no idea what ytex does, but I know the
ins and
> outs
> > > > >>>> of
> > > > the
> > > > >>>> cdl-fast module.
> > > > >>>>
> > > > >>>> Sean
> > > > >>>>
> > > > >>>> > -----Original Message-----
> > > > >>>> > From: clayclay911@gmail.com [mailto:clayclay911@gmail.com]
On
> > > > Behalf
> > > > >>>> Of
> > > > >>>> > Clayton Turner
> > > > >>>> > Sent: Friday, August 08, 2014 2:08 PM
> > > > >>>> > To: dev@ctakes.apache.org
> > > > >>>> > Subject: v_snomed_fword_lookup view
> > > > >>>> >
> > > > >>>> > Hi Everyone:
> > > > >>>> >
> > > > >>>> > I have a question about how the v_snomed_fword_lookup
view
> > > > >>>> > works
> > > > when
> > > > >>>> > running the CPE.
> > > > >>>> >
> > > > >>>> > So my understanding of the view is that it is a
view comprised
> > > > >>>> > of
> > > > the
> > > > >>>> > ytex.umls_aui_fword table, the umls.mrconso table
and
> > > > >>>> > bits/pieces
> > > > from
> > > > >>>> > other umls tables.
> > > > >>>> >
> > > > >>>> > I feel like this is not completely correct or my
idea of how
> > > > >>>> > the
> > > > join
> > > > >>>> to
> > > > >>>> > create the view works is off. For example, let's
say I want
> the
> > > > >>>> > CPE
> > > > >>>> to find
> > > > >>>> > "malar ____" (e.g. malar rash) as a concept in the
> annotations.
> > > > >>>> > It
> > > > >>>> never
> > > > >>>> > happens after running my CPE descriptor and I cannot
find it
> in
> > > > >>>> > my v_snomed_fword_lookup view.
> > > > >>>> >
> > > > >>>> > select count(*) from umls_aui_fword where fword='malar';
> yields
> > > > >>>> > 34
> > > > >>>> results
> > > > >>>> >
> > > > >>>> > select count(*) from umls.mrconso where str='malar';
yields 3
> > > > results.
> > > > >>>> >
> > > > >>>> > So clearly these two tables know what the cui and
context(s)
> > > > >>>> > are for
> > > > >>>> malar
> > > > >>>> > ____. Yet, whenever I run a gold standard set of
notes through
> > > > >>>> > the
> > > > >>>> CPE,
> > > > >>>> > malar is constantly flagged as just a word token
and the
> > > > >>>> > concept is
> > > > >>>> never
> > > > >>>> > grabbed. This is recurrent for lots of other concepts,
as
> well,
> > > > >>>> > I
> > > > just
> > > > >>>> > wanted to use an example to illustrate my issue.
> > > > >>>> >
> > > > >>>> > Some troubleshooting I already went through:
> > > > >>>> > 1) Reinstalled ytex and umls database objects
> > > > >>>> > 2) Reinstalled a second time after redownloading
umls through
> > > > >>>> > metamorphosys, ensuring that snomed vocabularies
were included
> > > > >>>> > (also checked file sizes and noticed a big difference
so I
> know
> > > > >>>> > those vocabularies ARE included
> > > > >>>> >
> > > > >>>> > Anyone got any ideas as to what the issue could
be?
> > > > >>>> >
> > > > >>>> > Thank you,
> > > > >>>> > Clayton Turner
> > > > >>>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> --
> > > > >>> --
> > > > >>> Clayton Turner
> > > > >>> email: caturner3@g.cofc.edu
> > > > >>> phone: (843)-424-3784
> > > > >>> web: claytonturner.blogspot.com
> > > > >>>
> > > > >>>
> > > >
> ----------------------------------------------------------------------
> > > > ---------------------------
> > > > >>> “When scientifically investigating the natural world, the
only
> > > > >>> thing worse than a blind believer is a seeing denier.”
> > > > >>> - Neil deGrasse Tyson
> > > > >>>
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> --
> > > > >> Clayton Turner
> > > > >> email: caturner3@g.cofc.edu
> > > > >> phone: (843)-424-3784
> > > > >> web: claytonturner.blogspot.com
> > > > >>
> > > > >>
> > > >
> ----------------------------------------------------------------------
> > > > ---------------------------
> > > > >> “When scientifically investigating the natural world, the only
> > > > >> thing worse than a blind believer is a seeing denier.”
> > > > >> - Neil deGrasse Tyson
> > > > >>
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > --
> > > > > Clayton Turner
> > > > > email: caturner3@g.cofc.edu
> > > > > phone: (843)-424-3784
> > > > > web: claytonturner.blogspot.com
> > > > >
> > > > >
> > > >
> ----------------------------------------------------------------------
> > > > ---------------------------
> > > > > “When scientifically investigating the natural world, the only
> thing
> > > > worse
> > > > > than a blind believer is a seeing denier.”
> > > > > - Neil deGrasse Tyson
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > --
> > > > Clayton Turner
> > > > email: caturner3@g.cofc.edu
> > > > phone: (843)-424-3784
> > > > web: claytonturner.blogspot.com
> > > >
> > > >
> ----------------------------------------------------------------------
> > > > --------------------------- “When scientifically investigating the
> > > > natural world, the only thing worse than a blind believer is a seeing
> > > > denier.”
> > > > - Neil deGrasse Tyson
> > > >
> >
>



-- 
--
Clayton Turner
email: caturner3@g.cofc.edu
phone: (843)-424-3784
web: claytonturner.blogspot.com
-------------------------------------------------------------------------------------------------
“When scientifically investigating the natural world, the only thing worse
than a blind believer is a seeing denier.”
- Neil deGrasse Tyson

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message