hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7730) Extend ReadEntity to add accessed columns from query
Date Tue, 26 Aug 2014 15:52:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14110858#comment-14110858
] 

Hive QA commented on HIVE-7730:
-------------------------------



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12664292/HIVE-7730.003.patch

{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 6115 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_opt_vectorization
org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
{noformat}

Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/503/testReport
Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/503/console
Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-503/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12664292

> Extend ReadEntity to add accessed columns from query
> ----------------------------------------------------
>
>                 Key: HIVE-7730
>                 URL: https://issues.apache.org/jira/browse/HIVE-7730
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Xiaomeng Huang
>         Attachments: HIVE-7730.001.patch, HIVE-7730.002.patch, HIVE-7730.003.patch
>
>
> -Now what we get from HiveSemanticAnalyzerHookContextImpl is limited. If we have hook
of HiveSemanticAnalyzerHook, we may want to get more things from hookContext. (e.g. the needed
colums from query).-
> -So we should get instance of HiveSemanticAnalyzerHookContext from configuration, extends
HiveSemanticAnalyzerHookContext with a new implementation, overide the HiveSemanticAnalyzerHookContext.update()
and put what you want to the class.-
> Hive should store accessed columns to ReadEntity when we set HIVE_STATS_COLLECT_SCANCOLS(or
we can add a confVar) is true.
> Then external authorization model can get accessed columns when do authorization in compile
before execute. Maybe we will remove columnAccessInfo from BaseSemanticAnalyzer, old authorization
and AuthorizationModeV2 can get accessed columns from ReadEntity too.
> Here is the quick implement in SemanticAnalyzer.analyzeInternal() below:
> {code}   boolean isColumnInfoNeedForAuth = SessionState.get().isAuthorizationModeV2()
>         && HiveConf.getBoolVar(conf, HiveConf.ConfVars.HIVE_AUTHORIZATION_ENABLED);
>     if (isColumnInfoNeedForAuth
>         || HiveConf.getBoolVar(this.conf, HiveConf.ConfVars.HIVE_STATS_COLLECT_SCANCOLS)
== true) {
>       ColumnAccessAnalyzer columnAccessAnalyzer = new ColumnAccessAnalyzer(pCtx);
>       setColumnAccessInfo(columnAccessAnalyzer.analyzeColumnAccess()); 
>     }
>     compiler.compile(pCtx, rootTasks, inputs, outputs);
>     // TODO: 
>     // after compile, we can put accessed column list to ReadEntity getting from columnAccessInfo
if HIVE_AUTHORIZATION_ENABLED is set true
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message