hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-716) org.apache.hadoop.mapred.lib.db.DBInputformat not working with oracle
Date Wed, 08 Jul 2009 12:45:14 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728661#action_12728661
] 

Enis Soztutar commented on MAPREDUCE-716:
-----------------------------------------

Patch looks good, but a few things that needs to be clear:
- Will there be a performance impact of setting statement.setFetchSize(1) in mysql?  (knowing
little about the driver, I assume it will fetch the rows one by one, which might be a real
bottleneck )
- MAPREDUCE-359, changed the protected DBRecordReader to public. Maybe it is better we move
DBRR to a new class, so that extending it seems less awkward (extends DBInputFormat.DBRecordReader<T>).

- why statement is not closed in DBRR.close()? 


> org.apache.hadoop.mapred.lib.db.DBInputformat not working with oracle
> ---------------------------------------------------------------------
>
>                 Key: MAPREDUCE-716
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-716
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>         Environment: Java 1.6, HAdoop0.19.0, Linux..Oracle, 
>            Reporter: evanand
>            Assignee: Aaron Kimball
>         Attachments: HADOOP-5482.20-branch.patch, HADOOP-5482.patch, HADOOP-5482.trunk.patch,
MAPREDUCE-716.2.branch20.patch, MAPREDUCE-716.2.trunk.patch, MAPREDUCE-716.3.trunk.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> org.apache.hadoop.mapred.lib.db.DBInputformat not working with oracle.
> The out of the box implementation of the Hadoop is working properly with mysql/hsqldb,
but NOT with oracle.
> Reason is DBInputformat is implemented with mysql/hsqldb specific query constructs like
"LIMIT", "OFFSET".
> FIX:
> building a database provider specific logic based on the database providername (which
we can get using connection).
> I HAVE ALREADY IMPLEMENTED IT FOR ORACLE...READY TO CHECK_IN CODE

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message