Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-dev@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of varun@pinterest.com designates
 209.85.214.177 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAOcnVr3P4AfaTR-munv=QJ8tFMoQKgc=o-Nxx=9XmAmq=Lovow@mail.gmail.com>
References: 
 <CAKxWWm0W1=0qOm91aE40sUL7R+56D=cbEET27aYxCEHVcLaT+g@mail.gmail.com>
	<CAOcnVr3P4AfaTR-munv=QJ8tFMoQKgc=o-Nxx=9XmAmq=Lovow@mail.gmail.com>
Date: Sun, 7 Jul 2013 09:37:46 -0700
Message-ID: 
 <CAKxWWm30-YFiReZJcF0ryxYP+ig9Z4YNQmBrXohT2ZgxWx-Etw@mail.gmail.com>
Subject: Re: Mysterious 7 byte reads from .meta files
From: Varun Sharma <varun@pinterest.com>
To: hdfs-dev@hadoop.apache.org
Content-Type: multipart/alternative; boundary=e89a8fb1f52242ce6904e0ee8c78

--e89a8fb1f52242ce6904e0ee8c78
Content-Type: text/plain; charset=ISO-8859-1

Created https://issues.apache.org/jira/browse/HDFS-4960 and attached a
patch... Thanks !


On Sun, Jul 7, 2013 at 9:06 AM, Harsh J <harsh@cloudera.com> wrote:

> The header of the meta file is read for block metadata version
> validation. The checksum checks are performed after the header is read
> (and skipped if the config's so). The code can be viewed in the
> BlockReaderLocal.java, which is used when local reads are turned on.
>
> I suppose we could skip reading the version as well, though it seems
> "general". Please file a JIRA for this!
>
> On Sat, Jul 6, 2013 at 5:55 AM, Varun Sharma <varun@pinterest.com> wrote:
> > Hi,
> >
> > We are running hbase with application level checksums turned on. We
> > basically have the following configuration:
> >
> > dfs.client.read.shortcircuit.skip.checksum -> true
> > hbase.regionserver.checksum.verify -> true
> >
> > We have shortcircuit reads enabled and have verified that its working -
> > only the HBase region server is accessing the disk. However, an strace on
> > the region server shows some mysterious 7 byte seeks on the .meta files
> > (which are also supposed to contain the checksum). We are running
> > hadoop-2.0.0-alpha.
> >
> > Do 7 byte reads from .meta files ring a bell to someone's ears - I am not
> > totally sure if these are checksums or not and I also don't know enough
> > about hdfs ?
> >
> > Thanks
> > Varun
>
>
>
> --
> Harsh J
>

--e89a8fb1f52242ce6904e0ee8c78--