hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alberto Andreotti <albertoandreo...@gmail.com>
Subject Re: Reg: when failures on writing to DB from map\reduce
Date Thu, 23 Aug 2012 15:40:18 GMT
unsubscribe

On 23 August 2012 12:36, Manoj Babu <manoj444@gmail.com> wrote:

>
> Hi All,
>
> In Sqoop:
> When exporting from HDFS to DB, If an export map task fails due to these
> or other reasons, it will cause the export job to fail. The results of a
> failed export are undefined. Each export map task operates in a separate
> transaction. Furthermore, individual map tasks commit their current
> transaction periodically. If a task fails, the current transaction will be
> rolled back. Any previously-committed transactions will remain durable in
> the database, leading to a partially-complete export.
>
> When using DBOutputFormat in custom map\reduce program:
> When emitting from multiple mappers or reducers and if any one of the
> mapper or reducer got failure only the failed transaction will be rolled
> back Any previously-committed transactions will remain durable in the
> database, leading to a partially-complete export.
>
> Is there any way to commit at last to avoid partially-complete?
>
> Cheers!
> Manoj.
>
>
>


-- 
José Pablo Alberto Andreotti.
Tel: 54 351 4730292
Móvil: 54351156526363.
MSN: albertoandreotti@gmail.com
Skype: andreottialberto

Mime
View raw message