Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@hadoop.apache.org
Received-SPF: pass (nike.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type;
  b=s+jfusG8PPmNBFYvOyFIB5Kd/7HERF1yJ2R5eoaoUqR45dpWik4oH7wKprl4vm3MBLaS5IsHBZHR1b95v0E8Osi9fHsXn0S/mo414Fk5OuDjCX5diti82mcwBvhj5Y3bk7RqZO1zZTzcbLn0cvJ+dUmCrPhXeDIVnViw5KtF+0w=;
Message-ID: <297355.50875.qm@web111413.mail.gq1.yahoo.com>
References: <967240.52565.qm@web111409.mail.gq1.yahoo.com>
 <805542.37510.qm@web111408.mail.gq1.yahoo.com>
 <5D66A842901F8E41815AF6D27A28EC490A7D6E07EF@Mail-Ab02.rmg-ny.com>
Date: Thu, 15 Oct 2009 11:30:59 -0700 (PDT)
From: Something Something <luckyguy2050@yahoo.com>
Subject: Re: Question about MapReduce
To: hbase-user@hadoop.apache.org, Hadoop <general@hadoop.apache.org>
In-Reply-To: <5D66A842901F8E41815AF6D27A28EC490A7D6E07EF@Mail-Ab02.rmg-ny.com>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="0-593399548-1255631459=:50875"

--0-593399548-1255631459=:50875
Content-Type: text/plain; charset=us-ascii

1) I don't think TableInputFormat is useful in this case.  Looks like it's used for scanning columns from a single HTable.
2) TableMapReduceUtil - same problem.  Seems like this works with just one table.
3) JV recommended NLineInputFormat, but my parameters are not in a file.  They come from multiple files and are in memory.

I guess what I am looking for is something like... InMemoryInputFormat... similar to FileInputFormat & DbInputFormat.  There's no such class right now.

Worse comes to worst, I can write the parameters into a flat file, and use FileInputFormat - but that will slow down this process considerably.  Is there no other way?


________________________________
From: Mark Vigeant <mark.vigeant@riskmetrics.com>
To: "hbase-user@hadoop.apache.org" <hbase-user@hadoop.apache.org>
Sent: Thu, October 15, 2009 7:21:40 AM
Subject: RE: Question about MapReduce

There is a tableInputFormat class in org.apache.hadoop.hbase.mapreduce.TableInputFormat

Also, if you want to use TableMapReduceUtil you probably want to have your mapper function extend TableMapper.

Check out the javadocs for more info: http://hadoop.apache.org/hbase/docs/current/api/index.html


-----Original Message-----
From: Something Something [mailto:luckyguy2050@yahoo.com] 
Sent: Thursday, October 15, 2009 1:37 AM
To: general@hadoop.apache.org; hbase-user@hadoop.apache.org
Subject: Re: Question about MapReduce

If the answer is...

TableMapReduceUtil.initTableMapperJob

I apologize for the spam.  If this isn't the right way, please let me know.  Thanks.


--- On Wed, 10/14/09, Something Something <luckyguy2050@yahoo.com> wrote:

From: Something Something <luckyguy2050@yahoo.com>
Subject: Question about MapReduce
To: general@hadoop.apache.org, hbase-user@hadoop.apache.org
Date: Wednesday, October 14, 2009, 10:18 PM

I would like to start a Map-Reduce job that does not read data from an input file or from a database.  I would like to pass 3 arguments to the Mapper & Reducer to work on.  Basically, these arguments are keys on the 3 different tables on HBase.

In other words, I don't want to use FileInputFormat or DbInputFormat because everything I need is already on HBase. 

How can I do this?  Please let me know.  Thanks.


--0-593399548-1255631459=:50875--