hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gopal V (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-8760) Pass a copy of HiveConf to hooks
Date Thu, 06 Nov 2014 20:26:34 GMT

    [ https://issues.apache.org/jira/browse/HIVE-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200852#comment-14200852
] 

Gopal V commented on HIVE-8760:
-------------------------------

Immutable structures are prudent from an API design stand-point - copying always to get to
a shared-nothing model potentially breaks anyone who relied on synchronization on that object
elsewhere.

The stability impact of copying is currently invisible and unknown, but eventually a lot of
System.identityHashcode is applied to debug those and System.err, because LOG.info() is synchronized.

The performance impact of however is well known (as quoted earlier). The core API issue over-all
for me is that we don't have immutable Conf objects - I keep hitting these {{new Configuration()}}
perf issues (track HADOOP-11223 for the impact on HDFS).

At the very least, I know the stability impact of copying in one Hook, the surface is rather
narrow for that problem to trace through (i.e "ship Hook2.java, Hook3.java etc and test them
without rebuilding all of hive").

On top of it, the biggest user of Hooks seems to be itests (which ships something like 20
single thread hooks). You'll be slowing down all of them, all the time.

> Pass a copy of HiveConf to hooks
> --------------------------------
>
>                 Key: HIVE-8760
>                 URL: https://issues.apache.org/jira/browse/HIVE-8760
>             Project: Hive
>          Issue Type: Bug
>          Components: Configuration
>    Affects Versions: 0.13.0, 0.14.0
>            Reporter: Ashutosh Chauhan
>            Assignee: Ashutosh Chauhan
>         Attachments: HIVE-8760.patch
>
>
> because hadoop's {{Configuration}} is not thread-safe



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message