incubator-ooo-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bugzi...@apache.org
Subject DO NOT REPLY [Bug 118940] New: Tune robots.txt for openoffice.org website
Date Mon, 20 Feb 2012 23:06:59 GMT
https://issues.apache.org/ooo/show_bug.cgi?id=118940

             Bug #: 118940
        Issue Type: DEFECT
           Summary: Tune robots.txt for openoffice.org website
    Classification: Infrastructure
           Product: www
           Version: version58
          Platform: PC
        OS/Version: Linux
            Status: CONFIRMED
          Severity: normal
          Priority: P3
         Component: openoffice.org website general issues
        AssignedTo: issues@openoffice.org
        ReportedBy: robweir@apache.org
                CC: ooo-issues@incubator.apache.org


We currently have:

==============
User-agent: daumoa
Disallow: /

User-Agent: Sosospider
Disallow: /

User-Agent: Snoopy
Disallow: /

User-Agent: *
Crawl-delay: 5
Disallow: /nonav/
Disallow: /welcome/
Disallow: /white_papers/
==============

Some of this doesn't make sense.  For example, there is nothing at /nonav,
/welcome seems to have a redirect issue that we should just fix.  And
/white-papers/ has reasonable content that we should not be hiding from index
spiders.

-- 
Configure bugmail: https://issues.apache.org/ooo/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

Mime
View raw message