hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Kimball (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1229) allow pig to write output into a JDBC db
Date Mon, 08 Feb 2010 19:07:28 GMT

    [ https://issues.apache.org/jira/browse/PIG-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831052#action_12831052

Aaron Kimball commented on PIG-1229:


This class looks reasonable to me. You'll probably need to format this as a patch to get it
accepted into the project though.

Is there a test plan for this code and/or unit tests?

Some database-specific things I've noticed: 
* You create a PreparedStatement, and call its executeUpdate() method several times then call
close() on the statement. This assumes you're in Auto-commit mode; I think you should configure
the commit mode explicitly when creating the connection. Also, you'll probably get a lot better
performance if you use addBatch() / executeBatch() for your batch size rather than individual
executeUpdate() statements. You should then call connection.commit() and ps.clear() rather
than closing the prepared statement and compiling a new one. 
* If user and pass are null, I think you may need to use DriverManager.getConnection(jdbcUrl)
instead of DriverManager.getConnection(jdbcUrl, null, null). Worth a unit test.
* See org.apache.hadoop.mapreduce.lib.db.DBOutputFormat in the MapReduce project for some
similar code to take inspiration from. 

> allow pig to write output into a JDBC db
> ----------------------------------------
>                 Key: PIG-1229
>                 URL: https://issues.apache.org/jira/browse/PIG-1229
>             Project: Pig
>          Issue Type: New Feature
>          Components: impl
>            Reporter: Ian Holsman
>            Priority: Minor
>         Attachments: DbStorage.java
> UDF to store data into a DB

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message