Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type;
  b=zOPm3aDsIt2KUZ1eNJ8uniyxSLEI95JVMdXwcp76CadkukAkU3iKP1CTPiiddPeNlKAlokepeMF38sxYeQRLTS+jVFvl2Ui3IuUKgirB9gAdnvjKdSHfPIfD6izFE3uHYqvaujny8Vhcy+KgoaK2eJa4lPqI3Ue9uovN7yKMCTQ=
  ;
References: <1372826091.75664.YahooMailNeo@web120903.mail.ne1.yahoo.com>
	<CALr1C9o0-uYhSEy3pMUT8YEgt+wELD1bw8cbN1Ucu1oj8Up2Vg@mail.gmail.com>
	<1372865652.49137.YahooMailNeo@web120904.mail.ne1.yahoo.com>
	<BLU0-SMTP304C281DF47635A7181622F8F730@phx.gbl>
 <CALr1C9qj5V+Go2AUw3CZKcyOHHt6svRm_T3WrFoWusvSiLWAKg@mail.gmail.com>
Message-ID: <1373476558.93417.YahooMailNeo@web120903.mail.ne1.yahoo.com>
Date: Wed, 10 Jul 2013 10:15:58 -0700 (PDT)
From: "S. Zhou" <myxjtu@yahoo.com>
Reply-To: "S. Zhou" <myxjtu@yahoo.com>
Subject: Re: MapReduce job with mixed data sources: HBase table and HDFS files
To: "user@hbase.apache.org" <user@hbase.apache.org>
In-Reply-To: 
 <CALr1C9qj5V+Go2AUw3CZKcyOHHt6svRm_T3WrFoWusvSiLWAKg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
 boundary="357496409-1515064781-1373476558=:93417"

--357496409-1515064781-1373476558=:93417
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

Hi Azuryy, I am testing the way you suggested. Now I am facing a compilatio=
n error for the following statement:=0Aconf.set(TableInputFormat.SCAN, Tabl=
eMapReduceUtil.convertScanToString(new Scan()));=0A=0A=0AThe error is: "met=
hod convertScanToString is not visible in TableMapReduceUtil". Could u help=
? It blocks me. =0A=0A=0ABTW, I am using the HBase-server jar file version =
0.95.1-hadoop1 . I tried other versions as well like 0.94.9 and got the sam=
e error.=0A=0AThanks!=0A=0A=0A________________________________=0A From: Azu=
ryy Yu <azuryyyu@gmail.com>=0ATo: user@hbase.apache.org =0ASent: Wednesday,=
 July 3, 2013 6:02 PM=0ASubject: Re: MapReduce job with mixed data sources:=
 HBase table and HDFS files=0A =0A=0AHi,=0A1) It cannot input two different=
 cluster's data to a MR job.=0A2) If your data locates in the same cluster,=
 then:=0A=0A=A0 =A0 conf.set(TableInputFormat.SCAN,=0ATableMapReduceUtil.co=
nvertScanToString(new Scan()));=0A=A0 =A0 conf.set(TableInputFormat.INPUT_T=
ABLE, tableName);=0A=0A=A0 =A0 MultipleInputs.addInputPath(conf, new Path(i=
nput_on_hdfs),=0ATextInputFormat.class, MapperForHdfs.class);=0A=A0 =A0 Mul=
tipleInputs.addInputPath(conf, new Path(input_on_hbase),=0ATableInputFormat=
.class, MapperForHBase.class);*=0A=0A*=0Abut,=0Anew Path(input_on_hbase) ca=
n be any path, it make no sense.*=0A=0A*=0APlease refer to=0Aorg.apache.had=
oop.hbase.mapreduce.IndexBuilder for how to read table in the=0AMR job unde=
r $HBASE_HOME/src/example*=0A=0A=0A=0A*=0A=0A=0AOn Thu, Jul 4, 2013 at 5:19=
 AM, Michael Segel <michael_segel@hotmail.com>wrote:=0A=0A> You may want to=
 pull your data from your HBase first in a separate map=0A> only job and th=
en use its output along with other HDFS input.=0A> There is a significant d=
isparity between the reads from HDFS and from=0A> HBase.=0A>=0A>=0A> On Jul=
 3, 2013, at 10:34 AM, S. Zhou <myxjtu@yahoo.com> wrote:=0A>=0A> > Azuryy, =
I am looking at the MultipleInputs doc. But I could not figure=0A> out how =
to add HBase table as a Path to the input? Do you have some sample=0A> code=
? Thanks!=0A> >=0A> >=0A> >=0A> >=0A> > ________________________________=0A=
> > From: Azuryy Yu <azuryyyu@gmail.com>=0A> > To: user@hbase.apache.org; S=
. Zhou <myxjtu@yahoo.com>=0A> > Sent: Tuesday, July 2, 2013 10:06 PM=0A> > =
Subject: Re: MapReduce job with mixed data sources: HBase table and HDFS=0A=
> files=0A> >=0A> >=0A> > Hi ,=0A> >=0A> > Use MultipleInputs, which can so=
lve your problem.=0A> >=0A> >=0A> > On Wed, Jul 3, 2013 at 12:34 PM, S. Zho=
u <myxjtu@yahoo.com> wrote:=0A> >=0A> >> Hi there,=0A> >>=0A> >> I know how=
 to create MapReduce job with HBase data source only or HDFS=0A> >> file as=
 data source. Now I need to create a MapReduce job with mixed=0A> data=0A> =
>> sources, that is, this MR job need to read data from both HBase and HDFS=
=0A> >> files. Is it possible? If yes, could u share some sample code?=0A> =
>>=0A> >> Thanks!=0A> >> Senqiang=0A>=0A>
--357496409-1515064781-1373476558=:93417--