Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of weishung@gmail.com designates
 74.125.82.176 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :cc:content-type;
        b=Yr2XJTo2xoFCvTMe3V18mVlnL/rtohaXeP4alN18tYPtHNUA/bH/dOni73NfnreXwi
         CJOQ9U9hRr3yuEAL0yh564fyVkwsHn/MgYejk1f7fEPGQI9SfgPfRLvmj1dmmUSy8X+U
         HiDJ7PbERF4s29cvfDVfXoceFG9nZqlqqraCM=
MIME-Version: 1.0
In-Reply-To: <AANLkTi=rO99cD2hZ-bbQNbU99=zxswb6aUr6tjXLO0q7@mail.gmail.com>
References: <AANLkTim2E9mOvScrewudQPM3kYkrbxX=vKXeHhDv9EXp@mail.gmail.com>
	<4D877FD5.5010501@apache.org>
	<AANLkTik7cDZJmyoVpTYj_pUyK=xuSSqtK6Gs7McDYG7X@mail.gmail.com>
	<AANLkTimNKgUWQ_+i0LdVzVh+K=9HCjMYaHPATQhjUTOO@mail.gmail.com>
	<AANLkTi=rO99cD2hZ-bbQNbU99=zxswb6aUr6tjXLO0q7@mail.gmail.com>
Date: Tue, 22 Mar 2011 11:31:40 -0500
Message-ID: <AANLkTik-=5UFkO2LH9zFP3z5ejYk4vjHZd0G87jAdd9x@mail.gmail.com>
Subject: Re: File formats in Hadoop
From: Weishung Chung <weishung@gmail.com>
To: Vivek Krishna <vivekrishna@gmail.com>
Cc: user@hbase.apache.org, common-user@hadoop.apache.org,
	qwertymaniac@gmail.com, Doug Cutting <cutting@apache.org>
Content-Type: multipart/alternative; boundary=0016364ef9946fc205049f14c7d3

--0016364ef9946fc205049f14c7d3
Content-Type: text/plain; charset=ISO-8859-1

I also found this informative article
http://cloudepr.blogspot.com/2009/09/hfile-block-indexed-file-format-to.html


<http://cloudepr.blogspot.com/2009/09/hfile-block-indexed-file-format-to.html>is
the key value pair be
eg column family1 with one qualifier 1 with 2 versions

key1 : rowkey1+column family1:qualifier1+timestamp1
value1: corresponding cell value1
key2 :  rowkey1+column family1:qualifier1+timestamp2
value2: corresponding cell value 2
key3:  rowkey2+column family1:qualifier1+timestamp1
value3: corresponding cell value 3
<http://cloudepr.blogspot.com/2009/09/hfile-block-indexed-file-format-to.html>
On Tue, Mar 22, 2011 at 10:58 AM, Vivek Krishna <vivekrishna@gmail.com>wrote:

> http://nosql.mypopescu.com/post/3220921756/hbase-internals-hfile-explained
> might help.
>
> Viv
>
>
>
>
> On Tue, Mar 22, 2011 at 11:43 AM, Weishung Chung <weishung@gmail.com>wrote:
>
>> My fellow superb hbase experts,
>>
>> Looking at the HFile specs and have some questions.
>> How is a particular table cell in a HBase table being represented in the
>> HFile? Does the key of the key value pair represent the rowkey+column
>> family:qualifier+timestamp and the value represent the corresponding cell
>> value? If so, to read a row, multiple key/value pair reads have to be
>> done?
>>
>> Thank you :)
>>
>>
>> On Tue, Mar 22, 2011 at 9:09 AM, Weishung Chung <weishung@gmail.com>
>> wrote:
>>
>> > Thank you, I will definitely take a look. Also, the TFile spec below
>> helps
>> > me to understand more,
>> > what an exciting work !
>> >
>> >
>> >
>> https://issues.apache.org/jira/secure/attachment/12396286/TFile+Specification+20081217.pdf
>> >
>> > <
>> https://issues.apache.org/jira/secure/attachment/12396286/TFile+Specification+20081217.pdf
>> >
>> > On Mon, Mar 21, 2011 at 11:41 AM, Doug Cutting <cutting@apache.org>
>> wrote:
>> >
>> >> On 03/19/2011 09:01 AM, Weishung Chung wrote:
>> >> > I am browsing through the hadoop.io package and was wondering what
>> >> other
>> >> > file formats are available in hadoop other than SequenceFile and
>> TFile?
>> >> > Is all data written through hadoop including those from hbase saved
>> in
>> >> the
>> >> > above formats? It seems like SequenceFile is in key value pair
>> format.
>> >>
>> >> Avro includes a file format that works with Hadoop.
>> >>
>> >>
>> >>
>> http://avro.apache.org/docs/current/api/java/org/apache/avro/mapred/package-summary.html
>> >>
>> >> Doug
>> >>
>> >
>> >
>>
>
>

--0016364ef9946fc205049f14c7d3--