opennlp-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Kosin <james.ko...@gmail.com>
Subject Re: svn commit: r1300236 - /opennlp/trunk/opennlp-tools/src/test/java/opennlp/tools/namefind/TokenNameFinderCrossValidatorTest.java
Date Thu, 15 Mar 2012 02:25:04 GMT
William,

I'm testing now... will let you know in a few minutes.

James

On 3/14/2012 10:37 AM, william.colen@gmail.com wrote:
> To fix that I am trying to create the file from the resource URI. I am not
> sure it will work in other systems, here it works. Can you confirm if it
> works for you, James?
>
> Thank you
> William
>
> On Wed, Mar 14, 2012 at 1:55 AM, James Kosin <james.kosin@gmail.com> wrote:
>
>> William,
>>
>> I found this:
>>
>>
>> http://stackoverflow.com/questions/2308188/getresourceasstream-vs-fileinputstream
>>
>> The discussion seems to suggest another issue.
>>
>> James
>>
>> On 3/14/2012 12:52 AM, william.colen@gmail.com wrote:
>>> I had to do it this way otherwise it raises an exception saying that the
>>> stream can not be marked. I think it is required by cross-validation.
>>>
>>> It is working here. Do you think it is something related to the OS? I
>> only
>>> tried it in Linux.
>>>
>>> On Wed, Mar 14, 2012 at 1:42 AM, James Kosin <james.kosin@gmail.com>
>> wrote:
>>>> Hi,
>>>>
>>>> Sadly, yes....
>>>>
>>>> All the other modules get the resource like this:
>>>>>     InputStream in = POSTaggerFactoryTest.class.getClassLoader()
>>>>>
>>>>> .getResourceAsStream("opennlp/tools/postag/AnnotatedSentences.txt");
>>>> I'm not sure that getResource() is going to work.
>>>>
>>>> I end up with errors like this:
>>>>> Tests in error:
>>>>>
>>>>>
>> testWithNameEvaluationErrorListener(opennlp.tools.namefind.TokenNameFinderCrossValidatorTest):
>> C:\Users\James%20Kosin\Documents\NetBeansProjects\thesis\apache\opennlp\opennlp-tools\target\test-classes\opennlp\tools\namefind\AnnotatedSentences.txt
>>>>> (The system cannot find the path specified)
>>>>>
>>>>>
>> testWithNullResources(opennlp.tools.namefind.TokenNameFinderCrossValidatorTest):
>> C:\Users\James%20Kosin\Documents\NetBeansProjects\thesis\apache\opennlp\opennlp-tools\target\test-classes\opennlp\tools\namefind\AnnotatedSentences.txt
>>>>> (The system cannot find the path specified)
>>>> I'll give it another try on the command line with mvn to build.
>>>>
>>>> BRB,
>>>> James
>>>>
>>>> On 3/14/2012 12:11 AM, william.colen@gmail.com wrote:
>>>>> Hi James,
>>>>>
>>>>> The file "AnnotatedSentences.txt" was already in SVN. It is used by
>>>> another
>>>>> unit test.
>>>>>
>>>>> Is the test failing?
>>>>>
>>>>> Thank you,
>>>>> William
>>>>>
>>>>> On Wed, Mar 14, 2012 at 1:03 AM, James Kosin <james.kosin@gmail.com>
>>>> wrote:
>>>>>> -------- Original Message --------
>>>>>> Subject:        Re: svn commit: r1300236 -
>>>>>>
>>>>>>
>> /opennlp/trunk/opennlp-tools/src/test/java/opennlp/tools/namefind/TokenNameFinderCrossValidatorTest.java
>>>>>> Date:   Wed, 14 Mar 2012 00:02:18 -0400
>>>>>> From:   James Kosin <james.kosin@gmail.com>
>>>>>> To:     opennlp-dev@opennlp.apache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>> William,
>>>>>>
>>>>>> On this, don't forget to add the "AnnotatedSentences.txt" file.
>>>>>>
>>>>>> James
>>>>>>
>>>>>> On 3/13/2012 12:48 PM, colen@apache.org wrote:
>>>>>>> Author: colen
>>>>>>> Date: Tue Mar 13 16:48:55 2012
>>>>>>> New Revision: 1300236
>>>>>>>
>>>>>>> URL: http://svn.apache.org/viewvc?rev=1300236&view=rev
>>>>>>> Log:
>>>>>>> OPENNLP-466: Added JUnit that tries to reproduce the issue, but
could
>>>>>> not reproduce the error.
>>>>>>> Modified:
>>>>>>>
>> opennlp/trunk/opennlp-tools/src/test/java/opennlp/tools/namefind/TokenNameFinderCrossValidatorTest.java
>>>>>>> Modified:
>> opennlp/trunk/opennlp-tools/src/test/java/opennlp/tools/namefind/TokenNameFinderCrossValidatorTest.java
>>>>>>> URL:
>> http://svn.apache.org/viewvc/opennlp/trunk/opennlp-tools/src/test/java/opennlp/tools/namefind/TokenNameFinderCrossValidatorTest.java?rev=1300236&r1=1300235&r2=1300236&view=diff
>> ==============================================================================
>>>>>>> ---
>> opennlp/trunk/opennlp-tools/src/test/java/opennlp/tools/namefind/TokenNameFinderCrossValidatorTest.java
>>>>>> (original)
>>>>>>> +++
>> opennlp/trunk/opennlp-tools/src/test/java/opennlp/tools/namefind/TokenNameFinderCrossValidatorTest.java
>>>>>> Tue Mar 13 16:48:55 2012
>>>>>>> @@ -17,10 +17,14 @@
>>>>>>>
>>>>>>>  package opennlp.tools.namefind;
>>>>>>>
>>>>>>> -import static org.junit.Assert.assertNotNull;
>>>>>>> +import static org.junit.Assert.*;
>>>>>>>
>>>>>>> +import java.io.ByteArrayOutputStream;
>>>>>>>  import java.io.FileInputStream;
>>>>>>> +import java.util.Collections;
>>>>>>> +import java.util.Map;
>>>>>>>
>>>>>>> +import opennlp.tools.cmdline.namefind.NameEvaluationErrorListener;
>>>>>>>  import opennlp.tools.util.ObjectStream;
>>>>>>>  import opennlp.tools.util.PlainTextByLineStream;
>>>>>>>  import opennlp.tools.util.TrainingParameters;
>>>>>>> @@ -34,6 +38,9 @@ public class TokenNameFinderCrossValidat
>>>>>>>    private final String TYPE = "default";
>>>>>>>
>>>>>>>    @Test
>>>>>>> +  /**
>>>>>>> +   * Test that reproduces jira OPENNLP-463
>>>>>>> +   */
>>>>>>>    public void testWithNullResources() throws Exception {
>>>>>>>
>>>>>>>      FileInputStream sampleDataIn = new FileInputStream(getClass()
>>>>>>> @@ -49,8 +56,37 @@ public class TokenNameFinderCrossValidat
>>>>>>>      TokenNameFinderCrossValidator cv = new
>>>>>> TokenNameFinderCrossValidator("en",
>>>>>>>          TYPE, mlParams, null, null);
>>>>>>>
>>>>>>> -    cv.evaluate(sampleStream, 1);
>>>>>>> +    cv.evaluate(sampleStream, 2);
>>>>>>>
>>>>>>>      assertNotNull(cv.getFMeasure());
>>>>>>>    }
>>>>>>> +
>>>>>>> +  @Test
>>>>>>> +  /**
>>>>>>> +   * Test that tries to reproduce jira OPENNLP-466
>>>>>>> +   */
>>>>>>> +  public void testWithNameEvaluationErrorListener() throws
>> Exception {
>>>>>>> +
>>>>>>> +    FileInputStream sampleDataIn = new FileInputStream(getClass()
>>>>>>> +        .getClassLoader()
>>>>>>> +
>>  .getResource("opennlp/tools/namefind/AnnotatedSentences.txt").getFile());
>>>>>>> +    ObjectStream<NameSample> sampleStream = new
>> NameSampleDataStream(
>>>>>>> +        new PlainTextByLineStream(sampleDataIn.getChannel(),
>>>>>> "ISO-8859-1"));
>>>>>>> +
>>>>>>> +    TrainingParameters mlParams =
>>>>>> ModelUtil.createTrainingParameters(70, 1);
>>>>>>> +    mlParams.put(TrainingParameters.ALGORITHM_PARAM,
>>>>>>> +        ModelType.MAXENT.toString());
>>>>>>> +
>>>>>>> +    ByteArrayOutputStream out = new ByteArrayOutputStream();
>>>>>>> +    NameEvaluationErrorListener listener = new
>>>>>> NameEvaluationErrorListener(out);
>>>>>>> +
>>>>>>> +    Map<String, Object> resources = Collections.emptyMap();
>>>>>>> +    TokenNameFinderCrossValidator cv = new
>>>>>> TokenNameFinderCrossValidator("en",
>>>>>>> +        TYPE, mlParams, null, resources, listener);
>>>>>>> +
>>>>>>> +    cv.evaluate(sampleStream, 2);
>>>>>>> +
>>>>>>> +    assertTrue(out.size() > 0);
>>>>>>> +    assertNotNull(cv.getFMeasure());
>>>>>>> +  }
>>>>>>>  }
>>>>>>>
>>>>>>>
>>


Mime
View raw message