pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1229) allow pig to write output into a JDBC db
Date Thu, 01 Apr 2010 00:16:27 GMT

    [ https://issues.apache.org/jira/browse/PIG-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852190#action_12852190

Ashutosh Chauhan commented on PIG-1229:

Few suggestions:

Reading from test case, currently store statements look like:
 b = store a into 'dummy' using org.apache.pig.piggybank.storage.DBStorage('org.hsqldb.jdbcDriver','jdbc:hsqldb:file:/tmp/batchtest;hsqldb.default_table_type=cached;hsqldb.cache_rows=100','insert
into a...');
here 'dummy' is totally ignored. while this works, from a user experience following might
be better:

 b = store a into 'jdbc:hsqldb:file:/tmp/batchtest' using org.apache.pig.piggybank.storage.DBStorage('org.hsqldb.jdbcDriver','hsqldb.default_table_type=cached;hsqldb.cache_rows=100','insert
into a');
that is, have db url as store location and second param of store func as db params. you can
use setStoreLocation() to store url. Apart from more intuitive store stmt, this will also
allow you to check whether DB is reachable or not at compile time itself, instead of at runtime.
You can do that via checkOutputSpecs(). 

Doing DataType.findType() on every element of every tuple will be expensive. I am wondering
if you can get hold of schema in your store func and use that to map pig types to sql types.

All of these suggestions may come in as later patches. So, if you want to get this committed
and track these separately I think that also will work as this patch is functionally complete.

> allow pig to write output into a JDBC db
> ----------------------------------------
>                 Key: PIG-1229
>                 URL: https://issues.apache.org/jira/browse/PIG-1229
>             Project: Pig
>          Issue Type: New Feature
>          Components: impl
>            Reporter: Ian Holsman
>            Assignee: Ankur
>            Priority: Minor
>             Fix For: 0.8.0
>         Attachments: jira-1229-v2.patch
> UDF to store data into a DB

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message