hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amandeep Khurana <ama...@gmail.com>
Subject Re: DBInputFormat
Date Sat, 13 Feb 2010 07:24:49 GMT
You can find examples on how to use DBInputFormat on the internet. And if
you want a sample input format, just read any of the existing ones...


Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz


On Fri, Feb 12, 2010 at 10:39 PM, Gaurav Vashishth <vashgaurav@gmail.com>wrote:

>
> Ok thanks for the reply. Do you have any sample code which can demonstrate
> how to do this?
>
> -Gaurav
>
>
> Amandeep Khurana wrote:
> >
> > DBInputFormat splits the count() from the RDBMS table into the number of
> > mappers. If you want to split using your own scheme, you'll have to write
> > your own input format or tweak the existing one.
> >
> >
> > Amandeep Khurana
> > Computer Science Graduate Student
> > University of California, Santa Cruz
> >
> >
> > On Fri, Feb 12, 2010 at 12:08 PM, Stack <stack@duboce.net> wrote:
> >
> >> On Fri, Feb 12, 2010 at 4:32 AM, Gaurav Vashishth <vashgaurav@gmail.com
> >
> >> wrote:
> >> >
> >> > I have the Map Reduce function whose job is to process the database ,
> >> MySql,
> >> > and give us some output. For this purpose, I have created the map
> >> reduce
> >> > fucntion and have used the DBInputFormat, but Im confused in how the
> >> > JobTracker will produce the splits here.
> >> >
> >> > I want that first 'n' records from the database should be processed by
> >> > single map task and so on and if jobtracker splits the record and give
> >> less
> >> > than 'n' records, it would be problem.
> >> >
> >> > Is there any API for getting this done or Im missing something.
> >> >
> >>
> >> Maybe you have to write your own splitter?  One that makes sure each
> >> task has N rows?  Is there a splitter that is part of DBInputFormat?
> >> Can you look at how it works?  Maybe you can specify rows per task
> >> just with a configuration?
> >> St.Ack
> >>
> >
> >
>
> --
> View this message in context:
> http://old.nabble.com/DBInputFormat-tp27562875p27572830.html
> Sent from the HBase User mailing list archive at Nabble.com.
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message