sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Masahiro Yamaguchi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-1622) Copying from staging table should be in single transaction for pg_bulkload connector
Date Mon, 27 Oct 2014 22:18:34 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14185925#comment-14185925
] 

Masahiro Yamaguchi commented on SQOOP-1622:
-------------------------------------------

+1 (non-binding)

> If there are 4 map tasks and 1 reduce task, PGBulkloadExportReducer#reduce is called
4 times in current implementation, we have 4 transaction for copying from staging to destination
as a result.

I think map-reduce Job keeps atomicity.


> Copying from staging table should be in single transaction for pg_bulkload connector
> ------------------------------------------------------------------------------------
>
>                 Key: SQOOP-1622
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1622
>             Project: Sqoop
>          Issue Type: Improvement
>          Components: connectors/postgresql
>    Affects Versions: 1.4.5
>            Reporter: Masatake Iwasaki
>            Assignee: Masatake Iwasaki
>
> PGBulkloadExportReducer#reduce may be called per map task because the map output key
is unique ID of map task. Each reduce tasks should do copying from staging table in single
transaction.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message