lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (JIRA)" <>
Subject [jira] Commented: (SOLR-2202) Money FieldType
Date Wed, 27 Oct 2010 17:35:19 GMT


Robert Muir commented on SOLR-2202:

Hi Greg, these are excellent questions... I'll reply only to the localization ones and let
Uwe or others talk about the Trie stuff.

bq. First, currency parsing in Java appears locale-dependent (which obviously makes sense.)
The concern here is that the locale of the end-user performing queries is likely not the same
as the locale of the search engine. Is there currently a standard mechanism in Solr to acquire
the user's locale? What do we do for other internationalized components?

Well, in general components are internationalized, but this is actually a localization problem.
Usually for Solr, the Solr service is not handlign the end-users request, so its best if Locale
is somehow a parameter, and the default Locale never used at all. In other words, its up to
you to figure out how you want to determine what Locale to use, and Solr would just respect

bq. NumberFormat parsing fails to parse "10.00USD", or "10.00 USD", instead relying upon the
symbol. ("$10.00"). This seems like a limitation since generally using the currency code as
suffix is a locale-independent way of specifying a monetary value, making indexing code easy
to write (simply append the currency code for the document to the value). It may very well
be a good idea to simply standardize on this approach for the purposes of indexing, and avoid
all the locale-specific issues that come up regarding currency symbols.

Its not really a limitation, it depends upon the NumberFormat in use. The one you used for
parsing is just the Locale default format for that Locale from getCurrencyInstance... but
you can supply your own [DecimalFormat|]
pattern too. This is a printf/scanf like pattern that can contain special characters, particularly
ยค (\u00A4): Currency sign, replaced by currency symbol. If doubled, replaced by international
currency symbol. If present in a pattern, the monetary decimal separator is used instead of
the decimal separator.

Ideally here, you could allow this pattern to be a parameter.

bq. The NumberFormat parsing does not yield back the currency, just the value. It seems the
currency itself still needs to be extracted somehow. Is there a built in mechanism to do this?
Currently the patch iterates over all currencies attempting to extract the symbol or code
from the value.

Right, NumberFormat parses the actual number. Really its best if the currency ISO code (e.g.
USD) itself is supplied as a parameter, because these symbols are not unique, for example
$ is used for many currencies.  I think this is what Solr should do, if the end-user application
doesn't know this somehow, it can use more sophisticated mechanisms to "guess" it, particularly
things like ICU's "CurrencyMetaInfo" allow you to supply a "filter" based on things like region,
and timeframes, to get a list of the currencies used in that region at that time.

bq. How important is it that users have control over the currencies table? It was quite useful
to have the ability to define fake currencies for testing (as is done in the example currency.xml
file), it seems that if I changed the implementation to use Java's currency table this might
be a limitation if non-testing oriented use-cases exist.

I don't think its import to have fake currencies, but such things can be done with the Locale
SPI ... I think. I think we could just use real currencies for testing.

> Money FieldType
> ---------------
>                 Key: SOLR-2202
>                 URL:
>             Project: Solr
>          Issue Type: New Feature
>          Components: Schema and Analysis
>    Affects Versions: 1.5
>            Reporter: Greg Fodor
>         Attachments: SOLR-2202-lucene-1.patch, SOLR-2202-solr-1.patch, SOLR-2202-solr-2.patch
> Attached please find patches to add support for monetary values to Solr/Lucene with query-time
currency conversion. The following features are supported:
> - Point queries (ex: "price:4.00USD")
> - Range quries (ex: "price:[$5.00 TO $10.00]")
> - Sorting.
> - Currency parsing by either currency code or symbol.
> - Symmetric & Asymmetric exchange rates. (Asymmetric exchange rates are useful if
there are fees associated with exchanging the currency.)
> At indexing time, money fields can be indexed in a native currency. For example, if a
product on an e-commerce site is listed in Euros, indexing the price field as "10.00EUR" will
index it appropriately. By altering the currency.xml file, the sorting and querying against
Solr can take into account fluctuations in currency exchange rates without having to re-index
the documents.
> The new "money" field type is a polyfield which indexes two fields, one which contains
the amount of the value and another which contains the currency code or symbol. The currency
metadata (names, symbols, codes, and exchange rates) are expected to be in an xml file which
is pointed to by the field type declaration in the schema.xml.
> The current patch is factored such that Money utility functions and configuration metadata
lie in Lucene (see MoneyUtil and CurrencyConfig), while the MoneyType and MoneyValueSource
lie in Solr. This was meant to mirror the work being done on the spacial field types.
> This patch has not yet been deployed to production but will be getting used to power
the international search capabilities of the search engine at Etsy.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message