hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guillaume percoclap <percoc...@gmail.com>
Subject Re: Import only Delta from SDL DB using Sqoop
Date Wed, 09 Apr 2014 20:01:10 GMT
Hi thanks for this answer.

So should use this kind of command since my SQL table to import is based on
row ID values:

sqoop import  --check-column "PersonID" --incremental "append" --last-value
30

so it will only import new entries greather than ID 30. But If I want to
automate, of course I cannot check each day what is the current ID and
modify the script manually.
So what would be the best way to automatically only import last entries
greather than the last ID imported?
Thanks for all



2014-03-30 22:41 GMT+02:00 Peyman Mohajerian <mohajeri@gmail.com>:

> There are two ways, based on id or timestamp, here is the clear
> documentation:
>
> https://sqoop.apache.org/docs/1.4.2/SqoopUserGuide.html#_incremental_imports
>
>
> On Sun, Mar 30, 2014 at 12:58 PM, Guillaume percoclap
> <percoclap@gmail.com>wrote:
>
> > Hi
> >
> > Using tool SQOOP, how could I import only differential data generated
> since
> > the last import for SDL DB please?
> >
> > Let say I run first this command: sqoop import-all-tables --connect
> > 'jdbc:sqlserver://xxxxssxs.com ;username=user1;password=xxxxx'
> > --hive-import
> >
> > and the next day I want to import only modified/added/deleted data, is it
> > existing a specific command allowing to do that?
> >
> > Thanks in advance
> >
> > Perco
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message