Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of calvin.lists@gmail.com
 designates 209.85.219.226 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type;
        b=v6OU9PCg76ZuvK0gSCC1EAFdaxTzGS91ffMELdWkRHjXcMMm8RletjZOdR63+ZRo1k
         kom5hxlAccj2tblwLKc6bo18fVBnn/hLsc+LgnqKid7Gkw2+N/jPYHK1HBGTbDV7oRdz
         SKRwnBBEiVIj0wEMOInR3KWuBhKfJrZe+qs0g=
MIME-Version: 1.0
Date: Wed, 4 Nov 2009 11:53:57 -0500
Message-ID: <eb5b0ce50911040853l304afa74y2fc722f270e4ca5c@mail.gmail.com>
Subject: timeseries merge/join question
From: Calvin <calvin.lists@gmail.com>
To: mapreduce-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=0016e6dab02af2b3f004778e7439

--0016e6dab02af2b3f004778e7439
Content-Type: text/plain; charset=ISO-8859-1

Hey all,

I am trying to figure out the best way to approach some joining/merging
computation in a map-reduce / hbase framework.  I have the following large
timeseries datasets (key/value pairs keyed and sorted by time):

Events1:
t1, event1_value1
t3, event1_value2,
...

Event2:
t2, event2_value1
t3, event2_value2,
t4, event2_value3,
....

Currently, I am just storing these as flat files in HDFS but I have no
problems throwing them into HBase tables.  I am trying to do an operation
like the following: for every event in Events2, find and join with the event
that immediately precedes (timestamp <=) this event in table Events1.

This operation would result in something like:

JoinedEvents:
t2, events2_value1, events1_value1
t3, events2_value2, events1_value2
t4, events2_value3, events1_value2

etc.

What is the best way to go about this in Hadoop?

Thanks in advance for the help.

--0016e6dab02af2b3f004778e7439
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hey all,<div><br></div><div>I am trying to figure out the best way to appro=
ach some joining/merging computation in a map-reduce / hbase framework. =A0=
I have the following large timeseries datasets (key/value pairs keyed and s=
orted by time):</div>
<div><br></div><div>Events1:</div><div>t1, event1_value1</div><div>t3, even=
t1_value2,</div><div>...</div><div><br></div><div><div>Event2:</div><div>t2=
, event2_value1</div><div>t3, event2_value2,</div><div>t4, event2_value3,</=
div>
<div>....</div><div><br></div><div>Currently, I am just storing these as fl=
at files in HDFS but I have no problems throwing them into HBase tables. =
=A0I am trying to do an operation like the following: for every event in Ev=
ents2, find and join with the event that immediately=A0precedes (timestamp =
&lt;=3D)=A0this event in table Events1.</div>
<div><br></div><div>This operation would result in something like:</div><di=
v><br></div><div>JoinedEvents:</div><div>t2, events2_value1, events1_value1=
</div><div>t3, events2_value2, events1_value2</div><div>t4, events2_value3,=
 events1_value2</div>
<div><br></div><div>etc.</div><div><br></div><div>What is the best way to g=
o about this in Hadoop?</div><div><br></div><div>Thanks in advance for the =
help.</div></div>

--0016e6dab02af2b3f004778e7439--