spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gatorsmile <...@git.apache.org>
Subject [GitHub] spark pull request #13919: [SPARK-16222] [SQL] JDBC Sources - Handling illeg...
Date Mon, 27 Jun 2016 04:46:03 GMT
GitHub user gatorsmile opened a pull request:

    https://github.com/apache/spark/pull/13919

    [SPARK-16222] [SQL] JDBC Sources - Handling illegal input values for `fetchsize` and `batchsize`

    #### What changes were proposed in this pull request?
    For JDBC data sources, users can specify `batchsize` for multi-row inserts and `fetchsize`
for multi-row fetch. A few issues exist:
    
    - The property keys are case sensitive. Thus, the existing test cases for `fetchsize`
use incorrect names. Basically, the test cases are broken. 
    - No test cases exist for `batchsize`. 
    - We do not detect the illegal input values for `fetchsize` and `batchsize`. 
    
    For example, when `batchsize` is zero, we got the following exception: 
    ```
    Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure:
Lost task 0.0 in stage 0.0 (TID 0, localhost): java.lang.ArithmeticException: / by zero
    ```
    when `fetchsize` is less than zero, we got the exception from the underlying JDBC driver:
    ```
    Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure:
Lost task 0.0 in stage 0.0 (TID 0, localhost): org.h2.jdbc.JdbcSQLException: Invalid value
"-1" for parameter "rows" [90008-183]
    ```
    
    This PR fixes all the above issues, and issue the appropriate exceptions when detecting
the illegal inputs for `fetchsize` and `batchsize`. Also update the function descriptions.
    
    #### How was this patch tested?
    Test cases are fixed and added.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/gatorsmile/spark jdbcProperties

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/13919.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #13919
    
----
commit 852c0e9bd7ecebf1bce1906ad641d8f714b9fe45
Author: gatorsmile <gatorsmile@gmail.com>
Date:   2016-06-27T01:44:25Z

    batchsize and fetchsize

commit cf2ce283407c62f3fd4d64cf380ddfcf262d7ecc
Author: gatorsmile <gatorsmile@gmail.com>
Date:   2016-06-27T02:18:36Z

    fix

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message