lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <hossman_luc...@fucit.org>
Subject Re: resin and UTF-8 in URLs
Date Fri, 02 Feb 2007 02:00:56 GMT

: The XML spec says that XML parsers are only required to support
: UTF-8, UTF-16, ISO 8859-1, and US-ASCII. If you use a different
: encoding for XML, there is no guarantee that a conforming parser
: will accept it.

there may not be a garuntee -- but shouldn't we at least try to respect
the clients wishes?

: Ultraseek has been indexing XML for the past nine years, and
: I remember a single customer that had XML in a non-standard
: encoding. Effectively all real-world XML is in one of the
: standard encodings.

That may be, but Solr was only publicly available for 9 months before we
had someone running into confusion because they were tyring to post an XML
file that wasn't UTF-8 :)

    http://www.nabble.com/wana-use-CJKAnalyzer-tf2303256.html#a6498685

: The right spec for XML over HTTP is RFC 3023. For text/xml
: with no charset spec, the XML must be interpreted as US-ASCII.

I can go along with that ... if there is a specification for a file format
that says which charset should be assumed if it can't be determined then i
agree, that's a case where it makes sense to hardcode "UTF-8" or
"US-ASCII" in Solr ... but that's not justification for using something
like request.setCharacterEncoding("UTF-8") in the SolrDispatcher where it
applies to everything -- it's a justification for hardcoding a default of
US-ASCII or UTF-8 in the XmlUpdateRequestHandler.

as a general rule, it seems like trusting the ServletContainer for the
default is hte rightthing to do.





-Hoss


Mime
View raw message