Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of qwertymaniac@gmail.com
 designates 209.85.161.48 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type:content-transfer-encoding;
        b=GQ+J3XkYwbSEstt5AKg3Rm5q3FOPcroF+vRGYeMvG0aDzD8D444QNrj9NpnG87+Bch
         AeUAMb6iitH38Aw2GBgj/gvqf+53EZ9eEj1oKe1M39nAQOl3JKG/jkbwVz1oYq9z/E7k
         4p3a1dpyE59ilV11YlkafTjOXMenvhsFPL7tI=
MIME-Version: 1.0
In-Reply-To: <A8EA35F5-C930-4358-A2A7-253C27879989@umail.ucsb.edu>
References: <A8EA35F5-C930-4358-A2A7-253C27879989@umail.ucsb.edu>
From: Harsh J <qwertymaniac@gmail.com>
Date: Fri, 1 Apr 2011 12:29:34 +0530
Message-ID: <AANLkTin=z-uf_D=Mxs9ZoDocgwG=TmYSTGHk=g=KFzJH@mail.gmail.com>
Subject: Re: Reading Records from a Sequence File
To: common-user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Fri, Apr 1, 2011 at 9:00 AM, maha <maha@umail.ucsb.edu> wrote:
> Hello Everyone,
>
> =A0 =A0 =A0 =A0As far as I know, when my java program opens a sequence fi=
le for a map calculations, from hdfs. Using SequenceFile.Reader(key,value) =
will actually read the file in dfs.block.size then grabes record-by-record =
from memory.
>
> =A0Is that right?

The dfs.block.size part is partially right when applied in MapReduce
(actually, it would look for sync points for read start and read end).
And no, the reader does not load the entire data in the memory in
one-go. It buffers and reads off the stream just like any other
reader.

Could we have some more information on what your java program does,
and what exactly you are measuring? :)

--=20
Harsh J
http://harshj.com