hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thanh Do <than...@cs.wisc.edu>
Subject Re: DataBlockScanner scan period
Date Thu, 14 Oct 2010 00:14:21 GMT
Brian,

When you say *attempt* to complete and *entire* node scan,
you mean for example, if a node has 100 block files, it will
try to verify all 100 block every 3 weeks?
That is in average, a block is scanned every (3 weeks / 100 time interval)?

Thanks
Thanh


On Wed, Oct 13, 2010 at 7:07 PM, Brian Bockelman <bbockelm@cse.unl.edu>wrote:

> Hi Thanh,
>
> The scan period is the period that hadoop *attempts* to complete an entire
> node scan.  That is, if it's set to 3 weeks, HDFS will try to scan each
> block once every 3 weeks.
>
> Obviously, depending on the bandwidth you have made available to the
> scanning thread, you can specify impossibly small periods.
>
> Brian
>
> On Oct 13, 2010, at 7:01 PM, Thanh Do wrote:
>
> > Hi again,
> >
> > Could any body explain to me about the scanning period
> > policy of DataBlockScanner? That is who often it wake up
> > and scan a block file.
> > When looking at the code, I found
> >
> > static final long DEFAULT_SCAN_PERIOD_HOURS = 21*24L; // three weeks
> >
> >
> > but definitely it does not wake up and pick a random block
> > to verify every three weeks, right?
> >
> > Thanks a lot,
> > Thanh
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message