commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Craig McClanahan <>
Subject Re: Robots.txt parser
Date Thu, 11 Nov 2004 20:41:37 GMT
As to interest, you might consider incorporating this in the
httpclient project but packaging it as a separate JAR for people who
want to use some other HTTP client implementation.  That way the
HttpClient developers could ensure that it meets your needs, without
forcing users of this parser to import all of HttpClient.

As to names, I think Norbert is cute, but there's a Commons guideline
that packages have boring functional names instead of cute marketing


On Thu, 11 Nov 2004 13:01:04 -0500, Henri Yandell <> wrote:
> Following on from a mail on the httpclient-dev list; I'm interested in
> submitting a codebase of mine into Commons that parses robots.txt
> files. It would definitely be of use with HttpClient's future plans
> but I'd like to keep it stand-alone and not hidden away inside
> HttpClient.
> Is there any interest?
> Probably the biggest -ve point for it is that as the robots.txt RFC is
> very well written and unlikely to change in the future, it's not a
> component that is likely to change beyond an option to use HttpClient
> as its GET mechanism.
> If there is interest; is Norbert a bad name? :) I should probably go
> with the simpler NoRobots name as that's the RFC title.
> Hen
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message