[ https://issues.apache.org/jira/browse/SQOOP-474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245723#comment-13245723
]
jiraposter@reviews.apache.org commented on SQOOP-474:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4614/#review6664
-----------------------------------------------------------
Hi Kate, your patch looks good. I only want to mention that I made some change to the same
area of code at SQOOP-468, which Jarec is going to submit soon. So you will need to rebase
your patch once it is committed.
In my patch, I factored out that area of code into a separate function (buildBoundaryQuery).
To achieve what you're doing here, you can change the following line:
private String buildBoundaryQuery(String col, String query) {
if (col == null) { // change to --> if (col == null || options.getNumMappers() ==
1) {
return "";
}
...
}
I have tested this in my workspace by myself and seen no issues. Please let me know if you
have any concerns/questions.
- Cheolsoo
On 2012-04-02 22:23:54, Kathleen Ting wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4614/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-02 22:23:54)
bq.
bq.
bq. Review request for Sqoop and Arvind Prabhakar.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Before triggering the bounding value query construction, in addition to checking that
the user has specified a split by option, also take into account that the number of mappers
is 1.
bq.
bq.
bq. This addresses bug SQOOP-474.
bq. https://issues.apache.org/jira/browse/SQOOP-474
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ./src/java/org/apache/sqoop/mapreduce/DataDrivenImportJob.java 1308530
bq.
bq. Diff: https://reviews.apache.org/r/4614/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Ran unit tests. Confirmed that, with the fix, the console output does not have the boundary
query in it (i.e. INFO db.DataDrivenDBInputFormat: BoundingValsQuery).
bq.
bq.
bq. Thanks,
bq.
bq. Kathleen
bq.
bq.
> Split-by specification incorrectly triggers bounding value query
> ----------------------------------------------------------------
>
> Key: SQOOP-474
> URL: https://issues.apache.org/jira/browse/SQOOP-474
> Project: Sqoop
> Issue Type: Bug
> Components: build, connectors/generic
> Affects Versions: 1.4.2-incubating
> Reporter: Kathleen Ting
> Assignee: Kathleen Ting
> Attachments: SQOOP-474.patch
>
>
> To reproduce this, run an import using a query with number of mappers set to 1 and a
split-by specification. For example:
> {code}
> $ sqoop import --connect jdbc:mysql://localhost/hadoopguide --query 'SELECT A.*, B.*
FROM A JOIN B ON (A.AID = B.BID) WHERE $CONDITIONS' --split-by AID --target-dir /user/kateting/test1
--m=1
> {code}
> This import will output the following:
> {code}
> 12/04/02 13:29:59 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(AID),
MAX(AID) FROM (SELECT A.*, B.* FROM A JOIN B ON (A.AID = B.BID) WHERE (1 = 1) ) AS t1
> {code}
> The problem is that the bounding value query construction is being triggered because
of the --split-by specification. However specifying split-by is redundant given that the number
of mappers is 1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
|