lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (Commented) (JIRA)" <>
Subject [jira] [Commented] (SOLR-2839) add alternative language detection impl
Date Sun, 16 Oct 2011 03:16:11 GMT


Robert Muir commented on SOLR-2839:

ok, i'd like to add this basic implementation first.

later, we should add support for some advanced parameters and refactoring:
* whitelisting should not happen in the base class as a post-filter (though this is fine as
a default implementation), but subclasses should override i think. For this detector, it could
improve performance.
* for this detector whitelist should support priors too (e.g. en=0.5, fr=0.1).
* we should add support for configuring smoothing parameter and maxTextLength (and, the base
class's concat should respect that too).
* both this implementation and the tika implementation are copying objects across lists of
language information, i think this is not very efficient to do per-document. So I think we
should change the API from List<DetectedLanguage> detectLanguage() to Iterable<DetectedLanguage>
detectLanguage. It seems in general it just wants the first one anyway.
> add alternative language detection impl
> ---------------------------------------
>                 Key: SOLR-2839
>                 URL:
>             Project: Solr
>          Issue Type: Improvement
>            Reporter: Robert Muir
>         Attachments: SOLR-2839.patch
> based on (apache license), supports 53 languages.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message