Mailing-List: contact notifications-help@ctakes.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ctakes.apache.org
Date: Mon, 25 Nov 2013 13:17:35 +0000 (UTC)
From: "Roberto Costumero Moreno (JIRA)" <jira@apache.org>
To: notifications@ctakes.apache.org
Message-ID: <JIRA.12680365.1384979055052.23666.1385385455494@arcas>
In-Reply-To: <JIRA.12680365.1384979055052@arcas>
References: <JIRA.12680365.1384979055052@arcas>
Subject: [jira] [Updated] (CTAKES-268) Fix SentenceDetector training with
 updated OpenNLP API
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/CTAKES-268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Roberto Costumero Moreno updated CTAKES-268:
--------------------------------------------

    Description: 
Fixed the problem where SentenceDetector did not work as expected due to changes in the OpenNLP API.

I have attached a patch file and the sentence file I have used and the model that it has generated.

Seems to be working but need to be checked. I have successfully generated models from the examples and a new one in Spanish in which I am currently working.

Updated: Attached new Patch.

  was:
Fixed the problem where SentenceDetector did not work as expected due to changes in the OpenNLP API.

I have attached a patch file and the sentence file I have used and the model that it has generated.

I have changed code around line 300:

logger.error("----------------------------------------------------------------------------------");
logger.error("Need to update yet for OpenNLP changes "); // TODO
logger.error("Commented out code that no longer compiles due to OpenNLP API incompatible changes"); // TODO
logger.error("----------------------------------------------------------------------------------");
		
		FileReader datafr = new FileReader(inFile);
        EventStream es = new BasicEventStream(new PlainTextByLineDataStream(datafr));
		
		GISModel mod = GIS.trainModel(es, iters, cut);
		SuffixSensitiveGISModelWriter ssgmw = new
		SuffixSensitiveGISModelWriter(
		mod, outFile);
		logger.info("Saving the model as: " + outFile.getAbsolutePath());
		ssgmw.persist();


with this code:


Charset charset = Charset.forName("UTF-8");
		
FileInputStream inStream = new FileInputStream(inFile);
ObjectStream<String> lineStream = new PlainTextByLineStream(inStream, charset);
ObjectStream<SentenceSample> sampleStream = new SentenceSampleStream(lineStream);
		
SentenceModel mod;
		
try {
	mod = SentenceDetectorME.train("en", sampleStream, true, null, ModelUtil.createTrainingParameters(iters, cut));
} finally {
	sampleStream.close();
	inStream.close();
}

SuffixSensitiveGISModelWriter ssgmw = new SuffixSensitiveGISModelWriter(
				 mod.getMaxentModel(), outFile);
logger.info("Saving the model as: " + outFile.getAbsolutePath());
ssgmw.persist();


Seems to be working but need to be checked. I have successfully generated models from the examples and a new one in Spanish in which I am currently working.


> Fix SentenceDetector training with updated OpenNLP API
> ------------------------------------------------------
>
>                 Key: CTAKES-268
>                 URL: https://issues.apache.org/jira/browse/CTAKES-268
>             Project: cTAKES
>          Issue Type: Improvement
>          Components: ctakes-core
>    Affects Versions: 3.1, 3.2, 3.1.1
>         Environment: Mac OS X
>            Reporter: Roberto Costumero Moreno
>              Labels: patch
>             Fix For: 3.1, 3.2, 3.1.1
>
>         Attachments: SentenceDetector.java.patch, sample_sd_en_en.mod, sample_sd_training_sentences.txt
>
>
> Fixed the problem where SentenceDetector did not work as expected due to changes in the OpenNLP API.
> I have attached a patch file and the sentence file I have used and the model that it has generated.
> Seems to be working but need to be checked. I have successfully generated models from the examples and a new one in Spanish in which I am currently working.
> Updated: Attached new Patch.


--
This message was sent by Atlassian JIRA
(v6.1#6144)