hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-2773) HiveStorageHandler.configureTableJobProperites() should let the handler know wether it is configuration for input or output
Date Thu, 08 Mar 2012 02:14:57 GMT

    [ https://issues.apache.org/jira/browse/HIVE-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224942#comment-13224942
] 

Phabricator commented on HIVE-2773:
-----------------------------------

ashutoshc has requested changes to the revision "HIVE-2773 [jira] HiveStorageHandler.configureTableJobProperites()
should let the handler know wether it is configuration for input or output".

  Francis,
  Normally, we first apply patches to trunk and then potentially backport them to a branch.
Can, you please first generate a patch for trunk?
  Overall, patch looks good. I left couple of comments.  Also, lint is complaining for few
lines being too long. Can you take care of those one too?

INLINE COMMENTS
  ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java:698 Should this be NoSuchMethodError
?
  ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java:708 Should this be NoSuchMethodError
?

REVISION DETAIL
  https://reviews.facebook.net/D2007

BRANCH
  arcpatch-D1815

                
> HiveStorageHandler.configureTableJobProperites() should let the handler know wether it
is configuration for input or output
> ---------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-2773
>                 URL: https://issues.apache.org/jira/browse/HIVE-2773
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Francis Liu
>            Assignee: Francis Liu
>              Labels: hcatalog, storage_handler
>         Attachments: HIVE-2773.D1815.1.patch, HIVE-2773.D2007.1.patch, HIVE-2773.patch
>
>
> HiveStorageHandler.configureTableJobProperties() is called to allow the storage handler
to setup any properties that the underlying inputformat/outputformat/serde may need. But the
handler implementation does not know whether it is being called for configuring input or output.
This makes it a problem for handlers which sets an external state. In the case of HCatalog's
HBase storageHandler, whenever a write needs to be configured we create a write transaction
which needs to be committed or aborted later on. In this case configuring for both input and
output each time configureTableJobProperties() is called would not be desirable. This has
become an issue since HCatalog is dropping storageDrivers for SerDe and StorageHandler (see
HCATALOG-237).
> My proposal is to replace configureTableJobProperties() with two methods:
> configureInputJobProperties()
> configureOutputJobProperties()
> Each method will have the same signature. I cursory look at the code and I believe changes
should be straighforward also given that we are not really changing anything just splitting
responsibility. If the community is fine with this approach I will go ahead and create a aptch.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message