sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-474) Split-by specification incorrectly triggers bounding value query
Date Mon, 02 Apr 2012 22:25:23 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13244755#comment-13244755
] 

jiraposter@reviews.apache.org commented on SQOOP-474:
-----------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4614/
-----------------------------------------------------------

Review request for Sqoop and Arvind Prabhakar.


Summary
-------

Before triggering the bounding value query construction, in addition to checking that the
user has specified a split by option, also take into account that the number of mappers is
1.


This addresses bug SQOOP-474.
    https://issues.apache.org/jira/browse/SQOOP-474


Diffs
-----

  ./src/java/org/apache/sqoop/mapreduce/DataDrivenImportJob.java 1308530 

Diff: https://reviews.apache.org/r/4614/diff


Testing
-------

Ran unit tests. Confirmed that, with the fix, the console output does not have the boundary
query in it (i.e. INFO db.DataDrivenDBInputFormat: BoundingValsQuery).


Thanks,

Kathleen


                
> Split-by specification incorrectly triggers bounding value query
> ----------------------------------------------------------------
>
>                 Key: SQOOP-474
>                 URL: https://issues.apache.org/jira/browse/SQOOP-474
>             Project: Sqoop
>          Issue Type: Bug
>          Components: build, connectors/generic
>    Affects Versions: 1.4.2-incubating
>            Reporter: Kathleen Ting
>            Assignee: Kathleen Ting
>         Attachments: SQOOP-474.patch
>
>
> To reproduce this, run an import using a query with number of mappers set to 1 and a
split-by specification. For example:
> {code}
> $ sqoop import --connect jdbc:mysql://localhost/hadoopguide --query 'SELECT A.*, B.*
FROM A JOIN B ON (A.AID = B.BID) WHERE $CONDITIONS' --split-by AID --target-dir /user/kateting/test1
--m=1
> {code}
> This import will output the following:
> {code}
> 12/04/02 13:29:59 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(AID),
MAX(AID) FROM (SELECT A.*, B.* FROM A JOIN B ON (A.AID = B.BID) WHERE  (1 = 1) ) AS t1
> {code}
> The problem is that the bounding value query construction is being triggered because
of the --split-by specification. However specifying split-by is redundant given that the number
of mappers is 1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message