hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12247) Replace setHTable() with initializeTable() in TableInputFormat.
Date Tue, 14 Oct 2014 21:26:33 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14171554#comment-14171554
] 

Enis Soztutar commented on HBASE-12247:
---------------------------------------

IF has an interesting lifecycle in hadoop MR. The IF is configured statically, and the configuration
is saved in the configuration object. Then MR initializes the class once, and passes the conf
back in through setConf(). This instance is used for calling getSplits(). After that, there
is no close method or anything that is called. Then for each map task, the IF is initialized
again and passed the job configuration. Then createRecordReader() is called with one of the
InputSplits that is passed. The returned record reader is closed properly. 

I think we can do this: 
 IF.setConf() will create a Connection. IF.getSplits() will close that connection at the end
in a finally close. Also the RecordReder.close() will close the connection in RR.close() method.
It is similar to how DBInputFormat works as well https://github.com/apache/hadoop-common/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/db/DBInputFormat.java





> Replace setHTable() with initializeTable() in TableInputFormat.
> ---------------------------------------------------------------
>
>                 Key: HBASE-12247
>                 URL: https://issues.apache.org/jira/browse/HBASE-12247
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 2.0.0, 0.99.2
>            Reporter: Solomon Duskis
>            Assignee: Solomon Duskis
>             Fix For: 2.0.0, 0.99.2
>
>         Attachments: HBASE-12247.patch, HBASE-12247_v2.patch
>
>
> TableInputFormat uses a new HTable().  Switch that use to a manually created ConnectionFactory.createConnection()
and call initializeTable().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message