mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joe Prasanna Kumar (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAHOUT-271) Make WikipediaDatasetCreatorMapper fuzzy category match respect word boundaries
Date Wed, 22 Sep 2010 03:44:36 GMT

    [ https://issues.apache.org/jira/browse/MAHOUT-271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913370#action_12913370
] 

Joe Prasanna Kumar commented on MAHOUT-271:
-------------------------------------------

Hi Olivier,

I was just looking at this issue to see if I can write a patch.
In WikipediaDatasetCreatorMapper, we check to see if the exactMatchOnly option is specified;
if so we check if the Set of Strings (inputCategories) contains the desired category like
below

// inputCategories is a Set<String>
if (exactMatchOnly && inputCategories.contains(category)) {
        return category;
      }
If not (ie if the exactMatchOnly is not specified), only then we are using "category.contains(inputCategory)"

We should not be using a word boundary and restricting the match to an exact word since the
condition itself is that the exactMatchOnly is not set. so whatever code we currently have
looks good.

I am trying to find out what the actual issue is ?!

reg,
Joe.


> Make WikipediaDatasetCreatorMapper fuzzy category match respect word boundaries
> -------------------------------------------------------------------------------
>
>                 Key: MAHOUT-271
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-271
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Classification
>    Affects Versions: 0.2
>            Reporter: Olivier Grisel
>            Assignee: Olivier Grisel
>            Priority: Trivial
>             Fix For: 0.4
>
>
> WikipediaDatasetCreatorDriver is useful to create categorisation corpora out of wikipedia,
however the category match just do a String#contains check which can catch a lot of unrelated
categories.
> Checking the word boundaries with a regexp such as String.format("\\b%s\\b", theCategoryNameIAmLookingFor);
should fix the  issue.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message