Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id BECD91104C for ; Fri, 22 Aug 2014 00:12:12 +0000 (UTC) Received: (qmail 68447 invoked by uid 500); 22 Aug 2014 00:12:11 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 68367 invoked by uid 500); 22 Aug 2014 00:12:11 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 68162 invoked by uid 500); 22 Aug 2014 00:12:11 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 68138 invoked by uid 99); 22 Aug 2014 00:12:11 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 22 Aug 2014 00:12:11 +0000 Date: Fri, 22 Aug 2014 00:12:11 +0000 (UTC) From: "Szehon Ho (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HIVE-7730) Extend ReadEntity to add accessed columns from query MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-7730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14106215#comment-14106215 ] Szehon Ho commented on HIVE-7730: --------------------------------- I think its reasonable. Xiaomeng, can you put up a review request at : [https://reviews.apache.org/dashboard/|https://reviews.apache.org/dashboard/] for some comments? > Extend ReadEntity to add accessed columns from query > ---------------------------------------------------- > > Key: HIVE-7730 > URL: https://issues.apache.org/jira/browse/HIVE-7730 > Project: Hive > Issue Type: Bug > Reporter: Xiaomeng Huang > Attachments: HIVE-7730.001.patch, HIVE-7730.002.patch > > > -Now what we get from HiveSemanticAnalyzerHookContextImpl is limited. If we have hook of HiveSemanticAnalyzerHook, we may want to get more things from hookContext. (e.g. the needed colums from query).- > -So we should get instance of HiveSemanticAnalyzerHookContext from configuration, extends HiveSemanticAnalyzerHookContext with a new implementation, overide the HiveSemanticAnalyzerHookContext.update() and put what you want to the class.- > Hive should store accessed columns to ReadEntity when we set HIVE_STATS_COLLECT_SCANCOLS(or we can add a confVar) is true. > Then external authorization model can get accessed columns when do authorization in compile before execute. Maybe we will remove columnAccessInfo from BaseSemanticAnalyzer, old authorization and AuthorizationModeV2 can get accessed columns from ReadEntity too. > Here is the quick implement in SemanticAnalyzer.analyzeInternal() below: > {code} boolean isColumnInfoNeedForAuth = SessionState.get().isAuthorizationModeV2() > && HiveConf.getBoolVar(conf, HiveConf.ConfVars.HIVE_AUTHORIZATION_ENABLED); > if (isColumnInfoNeedForAuth > || HiveConf.getBoolVar(this.conf, HiveConf.ConfVars.HIVE_STATS_COLLECT_SCANCOLS) == true) { > ColumnAccessAnalyzer columnAccessAnalyzer = new ColumnAccessAnalyzer(pCtx); > setColumnAccessInfo(columnAccessAnalyzer.analyzeColumnAccess()); > } > compiler.compile(pCtx, rootTasks, inputs, outputs); > // TODO: > // after compile, we can put accessed column list to ReadEntity getting from columnAccessInfo if HIVE_AUTHORIZATION_ENABLED is set true > {code} -- This message was sent by Atlassian JIRA (v6.2#6252)