hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brock Noland (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-9335) Address review items on HIVE-9257 [Spark Branch]
Date Sun, 11 Jan 2015 01:17:34 GMT

     [ https://issues.apache.org/jira/browse/HIVE-9335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Brock Noland updated HIVE-9335:
-------------------------------
    Description: 
I made a pass through HIVE-9257 and found the following issues: 


{{HashTableSinkOperator.java}}
The fields EMPTY_OBJECT_ARRAY and EMPTY_ROW_CONTAINER are no longer constants and should not
be in upper case.

{{HivePairFlatMapFunction.java}}
We share NumberFormat accross threads and it's not thread safe.

{{KryoSerializer.java}}
we eat the stack trace in deserializeJobConf

{{SparkMapRecordHandler}}
in processRow we should not be using {{StringUtils.stringifyException}} since LOG can handle
stack traces.
in close:

{noformat}
        // signal new failure to map-reduce
        LOG.error("Hit error while closing operators - failing tree");
        throw new IllegalStateException("Error while closing operators", e);
{noformat}

Should be:

{noformat}
         String msg = "Error while closing operators: " + e;
        throw new IllegalStateException(msg, e);
{noformat}

{{SparkSessionManagerImpl}} - the method {{canReuseSession}} is useless

{{GenSparkSkewJoinProcessor}}

{noformat}
+      // keep it as reference in case we need fetch work
+//        localPlan.getAliasToFetchWork().put(small_alias.toString(),
+//            new FetchWork(tblDir, tableDescList.get(small_alias)));
{noformat}

{{GenSparkWorkWalker}} trim ws

{{SparkCompiler}} remote init

{{SparkEdgeProperty}} trim ws

{{CounterStatsPublisher}} eat exception

{{Hadoop23Shims}} unused import of {{ResourceBundles}}

> Address review items on HIVE-9257 [Spark Branch]
> ------------------------------------------------
>
>                 Key: HIVE-9335
>                 URL: https://issues.apache.org/jira/browse/HIVE-9335
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>    Affects Versions: spark-branch
>            Reporter: Brock Noland
>            Assignee: Brock Noland
>         Attachments: HIVE-9335.1-spark.patch
>
>
> I made a pass through HIVE-9257 and found the following issues: 
> {{HashTableSinkOperator.java}}
> The fields EMPTY_OBJECT_ARRAY and EMPTY_ROW_CONTAINER are no longer constants and should
not be in upper case.
> {{HivePairFlatMapFunction.java}}
> We share NumberFormat accross threads and it's not thread safe.
> {{KryoSerializer.java}}
> we eat the stack trace in deserializeJobConf
> {{SparkMapRecordHandler}}
> in processRow we should not be using {{StringUtils.stringifyException}} since LOG can
handle stack traces.
> in close:
> {noformat}
>         // signal new failure to map-reduce
>         LOG.error("Hit error while closing operators - failing tree");
>         throw new IllegalStateException("Error while closing operators", e);
> {noformat}
> Should be:
> {noformat}
>          String msg = "Error while closing operators: " + e;
>         throw new IllegalStateException(msg, e);
> {noformat}
> {{SparkSessionManagerImpl}} - the method {{canReuseSession}} is useless
> {{GenSparkSkewJoinProcessor}}
> {noformat}
> +      // keep it as reference in case we need fetch work
> +//        localPlan.getAliasToFetchWork().put(small_alias.toString(),
> +//            new FetchWork(tblDir, tableDescList.get(small_alias)));
> {noformat}
> {{GenSparkWorkWalker}} trim ws
> {{SparkCompiler}} remote init
> {{SparkEdgeProperty}} trim ws
> {{CounterStatsPublisher}} eat exception
> {{Hadoop23Shims}} unused import of {{ResourceBundles}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message