hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arsalan Bilal <charsalanbi...@gmail.com>
Subject Re: Creation of Hfiles for multiple tables using Single Bulk Load Job?
Date Tue, 27 Sep 2011 07:45:34 GMT
Now this time, i am trying to achieve my objective by playing with
HfileOutputFormat and MultiHfileOutputFormat. But did not achieve my target.
So i need your help
I am using two
context.write(key, kv1);
context.write(key, kv2);
then it should write/generate data/Hfile on two different paths for two
different tables. I am trying to achieve this. Plz help.


On Mon, Sep 26, 2011 at 8:26 PM, Stack <stack@duboce.net> wrote:

> Can you play around w/ the class to make it work for your case?
> St.Ack
>
> On Mon, Sep 26, 2011 at 9:24 AM, Arsalan Bilal <charsalanbilal@gmail.com>
> wrote:
> > @Andrew
> >
> > Can u help me? U suggest that Try this: https://gist.github.com/1237770, i
> > saw it, How we can we use it actually?
> > i try to use it as
> job.setOutputFormatClass(MultiHFileOutputFormat.class);
> > but it generate different column Families(not correct) inside one
> > folder(recursively).  I need to generate different column families in
> > different folders.
> >  And there is also lack of configureIncrementalLoad Function in your
> pointed
> > file.
> >
> > Plz help me Andrew, Stack, i need your help.
> >
> >
> > On Mon, Sep 26, 2011 at 6:48 PM, Shuja Rehman <shujamughal@gmail.com>
> wrote:
> >
> >> @arslan,
> >> I did not get chance to look at it.
> >>
> >>
> >> On Mon, Sep 26, 2011 at 12:11 PM, Arsalan Bilal <
> charsalanbilal@gmail.com>wrote:
> >>
> >>> I am also facing same kind of problem, I am using one Map only and need
> to
> >>> generate multiple Hfiles for multiple table for insertion in Hbase?
> >>> Have you found any solution ? Can u help me to find solution of this
> one?
> >>>
> >>>
> >>> On Fri, Sep 23, 2011 at 8:11 PM, Andrew Purtell <apurtell@apache.org
> >wrote:
> >>>
> >>>> Try this: https://gist.github.com/1237770
> >>>>
> >>>> See line 135.
> >>>>
> >>>> Best regards,
> >>>>
> >>>>
> >>>>    - Andy
> >>>>
> >>>>
> >>>> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> >>>> (via Tom White)
> >>>>
> >>>>
> >>>> >________________________________
> >>>> >From: Shuja Rehman <shujamughal@gmail.com>
> >>>> >To: user@hbase.apache.org; Andrew Purtell <apurtell@apache.org>
> >>>> >Sent: Friday, September 23, 2011 8:18 AM
> >>>> >Subject: Re: Creation of Hfiles for multiple tables using Single
Bulk
> >>>> Load Job?
> >>>> >
> >>>> >
> >>>> >andrew,
> >>>> >
> >>>> >can you guide me where to change and what to change? it will help.
> >>>> >thnx
> >>>> >
> >>>> >
> >>>> >On Thu, Sep 22, 2011 at 9:35 PM, Shuja Rehman <shujamughal@gmail.com
> >
> >>>> wrote:
> >>>> >
> >>>> >Can you update it so I can use it? :)
> >>>> >>
> >>>> >>Thanks
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >>On Thu, Sep 22, 2011 at 8:40 PM, Andrew Purtell <
> apurtell@apache.org>
> >>>> wrote:
> >>>> >>
> >>>> >>> From: Shuja Rehman <shujamughal@gmail.com>
> >>>> >>>
> >>>> >>>
> >>>> >>>> I am using bulk load to insert data into hbase. Its
runs fine if
> I
> >>>> need to
> >>>> >>>> insert in one table. But Now, I have the requirement
in which I
> need
> >>>> to
> >>>> >>>> insert data into more than  one table.
> >>>> >>>
> >>>> >>>
> >>>> >>>We started some support for this here:
> >>>> https://issues.apache.org/jira/browse/HBASE-3727
> >>>> >>>
> >>>> >>>It needs to be updated for HFile v2, I think, a couple of
single
> line
> >>>> changes.
> >>>> >>>
> >>>> >>>Best regards,
> >>>> >>>
> >>>> >>>
> >>>> >>>   - Andy
> >>>> >>>
> >>>> >>>Problems worthy of attack prove their worth by hitting back.
- Piet
> >>>> Hein (via Tom White)
> >>>> >>>
> >>>> >>>
> >>>> >>>
> >>>> >>>----- Original Message -----
> >>>> >>>> From: Shuja Rehman <shujamughal@gmail.com>
> >>>> >>>> To: user@hbase.apache.org
> >>>> >>>> Cc:
> >>>> >>>> Sent: Thursday, September 22, 2011 8:01 AM
> >>>> >>>> Subject: Creation of Hfiles for multiple tables using
Single Bulk
> >>>> Load Job?
> >>>> >>>>
> >>>> >>>> Hi
> >>>> >>>>
> >>>> >>>> I am using bulk load to insert data into hbase. Its
runs fine if
> I
> >>>> need to
> >>>> >>>> insert in one table. But Now, I have the requirement
in which I
> need
> >>>> to
> >>>> >>>> insert data into more than  one table. But it seems
that using
> bulk
> >>>> load, we
> >>>> >>>> can generate hfiles for 1 table only.  So the question
is that is
> >>>> there any
> >>>> >>>> solution which enable us to generate hfiles in multiple
output
> paths
> >>>> and
> >>>> >>>> then we can load these different files in different
hbase tables.
> >>>> >>>>
> >>>> >>>> Thanks in advance
> >>>> >>>>
> >>>> >>>>
> >>>> >>>> --
> >>>> >>>> Regards
> >>>> >>>> Shuja-ur-Rehman Baig
> >>>> >>>
> >>>> >>>> <http://pk.linkedin.com/in/shujamughal>
> >>>> >>>>
> >>>> >>>
> >>>> >>
> >>>> >>
> >>>> >>--
> >>>> >>Regards
> >>>> >>Shuja-ur-Rehman Baig
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >
> >>>> >
> >>>> >--
> >>>> >Regards
> >>>> >Shuja-ur-Rehman Baig
> >>>> >
> >>>> >
> >>>> >
> >>>> >
> >>>> >
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Best Regards,
> >>> Arsalan Bilal
> >>>
> >>>
> >>>
> >>
> >>
> >> --
> >> Regards
> >> Shuja-ur-Rehman Baig
> >> <http://pk.linkedin.com/in/shujamughal>
> >>
> >>
> >
> >
> > --
> > Best Regards,
> > Arsalan Bilal
> >
>



-- 
Best Regards,
Arsalan Bilal

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message