hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Kimball (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-875) Make DBRecordReader execute queries lazily
Date Wed, 19 Aug 2009 00:59:14 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12744816#action_12744816
] 

Aaron Kimball commented on MAPREDUCE-875:
-----------------------------------------

The failing Sqoop tests claim that it's failing because it can't find Avro. Not sure why this
is happening -- Sqoop doesn't make use of Avro anywhere. Recycling patch status in case this
was transient. If not, do I have to put some more random libraries in ivy.xml? 

According to {{git-blame}}, this was added to the root ivy.xml earlier that day:

{code}
9e58f6fc (Sharad Agarwal     2009-08-14 05:10:40 +0000 276)     <dependency org="org.apache.hadoop"
9e58f6fc (Sharad Agarwal     2009-08-14 05:10:40 +0000 277)       name="avro"
9e58f6fc (Sharad Agarwal     2009-08-14 05:10:40 +0000 278)       rev="1.0.0"
9e58f6fc (Sharad Agarwal     2009-08-14 05:10:40 +0000 279)       conf="common->default"/>
9e58f6fc (Sharad Agarwal     2009-08-14 05:10:40 +0000 280)     <dependency org="org.codehaus.jackso
9e58f6fc (Sharad Agarwal     2009-08-14 05:10:40 +0000 281)       name="jackson-mapper-asl"
9e58f6fc (Sharad Agarwal     2009-08-14 05:10:40 +0000 282)       rev="1.0.1"
9e58f6fc (Sharad Agarwal     2009-08-14 05:10:40 +0000 283)       conf="common->default"/>
9e58f6fc (Sharad Agarwal     2009-08-14 05:10:40 +0000 284)     <dependency org="com.thoughtworks.pa
9e58f6fc (Sharad Agarwal     2009-08-14 05:10:40 +0000 285)       name="paranamer"
9e58f6fc (Sharad Agarwal     2009-08-14 05:10:40 +0000 286)       rev="1.5"
9e58f6fc (Sharad Agarwal     2009-08-14 05:10:40 +0000 287)       conf="common->default"/>
{code}

> Make DBRecordReader execute queries lazily
> ------------------------------------------
>
>                 Key: MAPREDUCE-875
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-875
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Aaron Kimball
>            Assignee: Aaron Kimball
>         Attachments: MAPREDUCE-875.patch
>
>
> DBInputFormat's DBRecordReader executes the user's SQL query in the constructor. If the
query is long-running, this can cause task timeout. The user is unable to spawn a background
thread (e.g., in a MapRunnable) to inform Hadoop of on-going progress. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message