jakarta-taglibs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bugzi...@apache.org
Subject DO NOT REPLY [Bug 16779] New: - scrape taglib functionality
Date Tue, 04 Feb 2003 21:02:49 GMT
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG 
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://nagoya.apache.org/bugzilla/show_bug.cgi?id=16779>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND 
INSERTED IN THE BUG DATABASE.

http://nagoya.apache.org/bugzilla/show_bug.cgi?id=16779

scrape taglib functionality 

           Summary: scrape taglib functionality
           Product: Taglibs
           Version: 1.0
          Platform: All
        OS/Version: Other
            Status: NEW
          Severity: Enhancement
          Priority: Other
         Component: Scrape Taglib
        AssignedTo: taglibs-dev@jakarta.apache.org
        ReportedBy: strutsuser@hotmail.com


can someone advise on how i might send user-agent and referer headers on 
scrapes... 

by default, according to logs, the referer and user-agent headers are not sent, 
they show as "-" "-" in http-access.log, some sites don't serve content to 
blank user agents. 

i tried using proxomitron (a freeware proxy utility 
http://www.proxomitron.org/), to specify agent using scrapes provision for 
proxy and proxyport, but still blank referer and user-agent are sent??? this is 
baffling, because when i setup my browser to use proxomitron's proxy, the 
specified referer and agent appear in logs.

 
also, how might i lower the default 10-minute default before a rescrape?

---------------------------------------------------------------------
To unsubscribe, e-mail: taglibs-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: taglibs-dev-help@jakarta.apache.org


Mime
View raw message