nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Fellows <ccfell...@sbcglobal.net>
Subject Nutch-18 illegal chars in urls: Not sure what the problem is
Date Wed, 26 Apr 2006 20:15:05 GMT
Hello,

Was looking into nutch-18 bug that revolves around
illegal, non-ascii, characters in a url. An example of
a high bit character is 'ç' that maps to a high bit
set.

Before applying any fix, did a brief test with 0.8
trunk. After fetching and indexing on
http://en.wiktionary.org/wiki/ç, I was able to search
on ç and got the following result off the browser
results:

ç - Wiktionary
... Letter [ edit ] Translingual [ edit ] Letter Ç , ç
C with a cedilla ... visit IRC or Wiktionary:AOL . ç
...
http://en.wiktionary.org/wiki/%C3%A7 (cached)
(explain) (anchors) (more from en.wiktionary.org) 

So it looks like it will fetch and parse off of urls
with high bit set characters. Additionally, the
display url has the ç encoded correctly as %C3%A7. 

Is this really a bug?

Doing a similar test off of Google on keywords: "ç"
wiktionary. yields:

ç - Wiktionary
AOL users can access Wiktionary through this link
after accepting the CACERT certificate. ... Ç, ç.
"t&#643;&#601;", the fourth letter of the Albanian
alphabet. ...
en.wiktionary.org/wiki/ç - 15k - Cached - Similar
pages
[ More results from en.wiktionary.org ]

Nearly identical, but the see that the ç is in its
decoded form, not %C3%A7.

If there's any interest in this issue, let me know.

Chris

Mime
View raw message