Mailing-List: contact user-help@manifoldcf.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@manifoldcf.apache.org
Received-SPF: pass (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type;
  b=V+BD1Wjp97DoGjnE3EfNaY0X8mAcaV2MNc5D5wIvBAn0Tf9eZ85Cwa2jK2U+prPY0HelgifXRvH/CWU3h+oZSjkrgDBIkah6WXkxPmEBNaSBgTRvx9y704oY0tFlKFwss9bKEYo3VU2yU2vuHnAg0lPBRD1s6RCoTzK9DXTdsOQ=
  ;
Message-ID: <1372686977.45563.YahooMailNeo@web125304.mail.ne1.yahoo.com>
Date: Mon, 1 Jul 2013 06:56:17 -0700 (PDT)
From: Ahmet Arslan <iorixxx@yahoo.com>
Reply-To: Ahmet Arslan <iorixxx@yahoo.com>
Subject: web crawler job settings
To: "user@manifoldcf.apache.org" <user@manifoldcf.apache.org>
MIME-Version: 1.0
Content-Type: multipart/alternative;
 boundary="-1622010646-1600686120-1372686977=:45563"

---1622010646-1600686120-1372686977=:45563
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

Hi,=0A=0AI am crawling main pages of some online newspaper web sites.=A0=0A=
I don't need deletes at all. I am using crawl once model.=0A=0AHere is the =
settings I use :=A0=0A=0ASchedule type:Scan every document once=0AStart Met=
hod : Start at beginning of schedule window=0A=0AScheduled time: Any day of=
 week at 1 am 3 am 5 am 7 am 9 am 11 am 1 pm 3 pm 5 pm 7 pm 9 pm 11 pm plus=
 0 minutes=0AMaximum run time: No limit=0A=0AMaximum hop count for link typ=
e 'link': 1=0AMaximum hop count for link type 'redirect': Unlimited=0AHop c=
ount mode: No deletes, forever=0A=0AInclude only hosts matching seeds? yes=
=0ASeeds: A few URLs in the form of http://main.page.com/{category} where c=
ategory is Sports, Politics etc.=0A=0ABy setting hop count to 1 ( or 2) and=
 'no deletes, forever', I am expecting this crawl to be super fast and most=
 efficient. Minimal DB queries etc. Am I correct?=0A=0AThanks,=0AAhmet=0A
---1622010646-1600686120-1372686977=:45563
Content-Type: text/html; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

<html><body><div style=3D"color:#000; background-color:#fff; font-family:ti=
mes new roman, new york, times, serif;font-size:12pt"><div><div>Hi,</div><d=
iv><br></div><div>I am crawling main pages of some online newspaper web sit=
es.&nbsp;</div><div>I don't need deletes at all. I am using crawl once mode=
l.</div><div><br></div><div style=3D"color: rgb(0, 0, 0); font-size: 16px; =
font-family: 'times new roman', 'new york', times, serif; background-color:=
 transparent; font-style: normal; ">Here is the settings I use :&nbsp;</div=
><div><br></div><div>Schedule type:<span class=3D"Apple-tab-span" style=3D"=
white-space:pre">=09</span>Scan every document once</div><div>Start Method =
: Start at beginning of schedule window</div><div><br></div><div>Scheduled =
time:<span class=3D"Apple-tab-span" style=3D"white-space:pre">=09</span> An=
y day of week at 1 am 3 am 5 am 7 am 9 am 11 am 1 pm 3 pm 5 pm 7 pm 9 pm 11=
 pm plus 0 minutes</div><div>Maximum run time:<span class=3D"Apple-tab-span=
"
 style=3D"white-space:pre">=09</span> No limit</div><div><br></div><div>Max=
imum hop count for link type 'link':<span class=3D"Apple-tab-span" style=3D=
"white-space:pre">=09</span> 1</div><div>Maximum hop count for link type 'r=
edirect':<span class=3D"Apple-tab-span" style=3D"white-space:pre">=09</span=
> Unlimited</div><div>Hop count mode:<span class=3D"Apple-tab-span" style=
=3D"white-space:pre">=09</span> No deletes, forever</div><div><br></div><di=
v>Include only hosts matching seeds?<span class=3D"Apple-tab-span" style=3D=
"white-space:pre">=09</span> yes</div><div>Seeds: A few URLs in the form of=
 http://main.page.com/{category} where category is Sports, Politics etc.</d=
iv><div><br></div><div>By setting hop count to 1 ( or 2) and 'no deletes, f=
orever', I am expecting this crawl to be super fast and most efficient. Min=
imal DB queries etc. Am I correct?</div><div><br></div><div style=3D"color:=
 rgb(0, 0, 0); font-size: 16px; font-family: 'times new roman', 'new york',=
 times, serif;
 background-color: transparent; font-style: normal; ">Thanks,</div><div sty=
le=3D"color: rgb(0, 0, 0); font-size: 16px; font-family: 'times new roman',=
 'new york', times, serif; background-color: transparent; font-style: norma=
l; ">Ahmet</div><div><br></div></div></div></body></html>
---1622010646-1600686120-1372686977=:45563--