mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joe Prasanna Kumar (JIRA)" <>
Subject [jira] Commented: (MAHOUT-271) Make WikipediaDatasetCreatorMapper fuzzy category match respect word boundaries
Date Wed, 22 Sep 2010 03:44:36 GMT


Joe Prasanna Kumar commented on MAHOUT-271:

Hi Olivier,

I was just looking at this issue to see if I can write a patch.
In WikipediaDatasetCreatorMapper, we check to see if the exactMatchOnly option is specified;
if so we check if the Set of Strings (inputCategories) contains the desired category like

// inputCategories is a Set<String>
if (exactMatchOnly && inputCategories.contains(category)) {
        return category;
If not (ie if the exactMatchOnly is not specified), only then we are using "category.contains(inputCategory)"

We should not be using a word boundary and restricting the match to an exact word since the
condition itself is that the exactMatchOnly is not set. so whatever code we currently have
looks good.

I am trying to find out what the actual issue is ?!


> Make WikipediaDatasetCreatorMapper fuzzy category match respect word boundaries
> -------------------------------------------------------------------------------
>                 Key: MAHOUT-271
>                 URL:
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Classification
>    Affects Versions: 0.2
>            Reporter: Olivier Grisel
>            Assignee: Olivier Grisel
>            Priority: Trivial
>             Fix For: 0.4
> WikipediaDatasetCreatorDriver is useful to create categorisation corpora out of wikipedia,
however the category match just do a String#contains check which can catch a lot of unrelated
> Checking the word boundaries with a regexp such as String.format("\\b%s\\b", theCategoryNameIAmLookingFor);
should fix the  issue.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message