hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From CRAIG LIU <craig8...@gmail.com>
Subject Re: data transfer from rdbms to hive
Date Fri, 02 May 2014 13:22:59 GMT
I am new in hive and here is my idea?
1. Use mysqldump to dump your data to csv file.
2. Load csv to hive temp table.
3. Create partition table.
4. Use dynamic partition, select from temp table to insert to partition
table. You can use udf to get the date from the timestamp.

Regards,
Craig
2014-5-2 下午9:11于 "Matt Tucker" <matthewtckr@gmail.com>写道:

> It sounds like you might need to export. Via sqoop using a query or view,
> as the date granularity in your MySQL table is different from the desired
> Hive table. The overall performance may be lower as MySQL must do more than
> just read rows from disk, but you may still find ways to get the data in
> parallel through Sqoop.
>
> On May 2, 2014 3:34 AM, "Hamza Asad" <hamza.asad13@gmail.com> wrote:
> >
> > Sqoop also support dynamic partitioning. I have done that. For that you
> have to enable dynamic partition i.e dynamic partition = true, in hive.
> >
> >
> > On Fri, May 2, 2014 at 12:57 PM, unmesha sreeveni <unmeshabiju@gmail.com>
> wrote:
> >>
> >>
> >> On Fri, May 2, 2014 at 9:41 AM, Shushant Arora <
> shushantarora09@gmail.com> wrote:
> >>>
> >>> Sqoop
> >>
> >>
> >> ​Hi Shushant
> >>   I dont think other ecosystem projects can help you.The only way to
> import data from relational DB is SQOOP.
> >>
> >>
> http://my.safaribooksonline.com/book/databases/9781449364618/6dot-hadoop-ecosystem-integration/integration_hive_partition_html
> >>
> >> Let me know your thoughts.
> >>
> >>
> >>
> >> --
> >> Thanks & Regards
> >>
> >> Unmesha Sreeveni U.B
> >> Hadoop, Bigdata Developer
> >> Center for Cyber Security | Amrita Vishwa Vidyapeetham
> >> http://www.unmeshasreeveni.blogspot.in/
> >>
> >>
> >
> >
> >
> > --
> > Muhammad Hamza Asad
>

Mime
View raw message