hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carl Steinbach (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1096) Hive Variables
Date Wed, 28 Jul 2010 01:12:17 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893022#action_12893022
] 

Carl Steinbach commented on HIVE-1096:
--------------------------------------

The reviewboard jiraposter is temporarily down so I'm posting this by hand. The original comments
and context are here: https://review.cloudera.org/r/229/


* trunk/conf/hive-default.xml:
Spelling: substituation

* trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/VariableSubstitution.java:
Make these variables private?

* trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/VariableSubstitution.java:
This should be static. No need to instantiate a VariableSubstitution object.

* trunk/ql/src/java/org/apache/hadoop/hive/ql/processors/AddResourceProcessor.java:
Since we want to do substitution for all commands it would probably make sense to do the substitution
in CommandProcessorFactory.get() and make CommandProcessor an abstract class with the following
implementation:

{code}
public abstract class {
  protected String command;

  CommandProcessor(String command) {
    this.command = command;
  }

  public abstract CommandProcessorResponse run();

  public String getCommand() {
    return command;
  }
}
{code}

In other words, CommandProcessorFactory would return a CommandProcessor object that has been
initialized with a substituted copy of the command.

* trunk/ql/src/java/org/apache/hadoop/hive/ql/processors/SetProcessor.java:
Replace these string literals with constants, e.g:

{code}
public static final String ENV_PREFIX = "env:";
public static final String SYSTEM_PREFIX = "system:"
public static final String HIVECONF_PREFIX = "hiveconf:"
{code}


* trunk/ql/src/java/org/apache/hadoop/hive/ql/processors/SetProcessor.java:
String propName = varname.substring(SYSTEM_PREFIX.length());


* trunk/ql/src/java/org/apache/hadoop/hive/ql/processors/SetProcessor.java:
Can we remove this special case for "silent"? In SessionState this actually maps to "hive.session.silent"
and I don't see any test cases that cover this case, i.e. that call "set silent" or "set silent=x".
It also seems that this introduces in inconsistency since "set silent" will show the value
of "hive.session.silent", but the output of "set" will not list a value for the property "silent".

Anyone know if there is any older code that depends on this behavior?


> Hive Variables
> --------------
>
>                 Key: HIVE-1096
>                 URL: https://issues.apache.org/jira/browse/HIVE-1096
>             Project: Hadoop Hive
>          Issue Type: New Feature
>          Components: Query Processor
>            Reporter: Edward Capriolo
>            Assignee: Edward Capriolo
>             Fix For: 0.6.0, 0.7.0
>
>         Attachments: 1096-9.diff, hive-1096-10-patch.txt, hive-1096-11-patch.txt, hive-1096-12.patch.txt,
hive-1096-2.diff, hive-1096-7.diff, hive-1096-8.diff, hive-1096.diff
>
>
> From mailing list:
> --Amazon Elastic MapReduce version of Hive seems to have a nice feature called "Variables."
Basically you can define a variable via command-line while invoking hive with -d DT=2009-12-09
and then refer to the variable via ${DT} within the hive queries. This could be extremely
useful. I can't seem to find this feature even on trunk. Is this feature currently anywhere
in the roadmap?--
> This could be implemented in many places.
> A simple place to put this is 
> in Driver.compile or Driver.run we can do string substitutions at that level, and further
downstream need not be effected. 
> There could be some benefits to doing this further downstream, parser,plan. but based
on the simple needs we may not need to overthink this.
> I will get started on implementing in compile unless someone wants to discuss this more.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message