hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Reuben Kuhnert (JIRA)" <>
Subject [jira] [Commented] (HIVE-12891) Hive fails when is set to a relative location
Date Mon, 25 Jan 2016 17:19:40 GMT


Reuben Kuhnert commented on HIVE-12891:

The problem that this patch addresses is that the value of '' can be set externally
to be a relative path (we're seeing this problem occur in oozie). To adjust for this issue,
the above patch uses 'Coercion' to validate/modify the value before passing it to the user.
That is, rather than simply creating a one-off coercion, I thought it would be useful in general
to have a way to hook into {{SystemVariables.substitute}} to validate or adjust the property
before returning to the user.

The template is:

public abstract class VariableCoercion {
  private final String name;

  public VariableCoercion(String name) { = name;

  public String getName() { return; }
  public abstract String getCoerced(Configuration configuration, String originalValue);
  public abstract String setCoerced(Configuration configuration, String originalValue);

where {{getCoerced}} is called on get and {{setCoerced}} is called on set (configuration is
passed in if the coerced value is context sensitive). In addition, to add other coercions,
simply subclass the above and add it here:

public class SystemVariables {
  // HERE: List of coercions:
  private static final VariableCoercionSet COERCIONS = new VariableCoercionSet()
    .add(new JavaIOTmpdirVariableCoercion());

If a coercion hook exists for a particular name (see {{VariableCoercion.getName()}}) then
it is loaded and the raw value is passed through, then returned to the user:

  public String getCoerced(Configuration configuration, String variableName, String originalValue)
    if (COERCIONS.contains(variableName)) {
      return COERCIONS.get(variableName).getCoerced(configuration, originalValue);
    } else {
      return originalValue;

> Hive fails when is set to a relative location
> ------------------------------------------------------------
>                 Key: HIVE-12891
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Reuben Kuhnert
>            Assignee: Reuben Kuhnert
>         Attachments: HIVE-12891.01.19.2016.01.patch, HIVE-12891.03.patch, HIVE-12981.01.22.2016.02.patch
> The function {{SessionState.createSessionDirs}} fails when trying to create directories
where {{}} is set to a relative location.
> {code}
> \[uber-SubtaskRunner] ERROR o.a.h.hive..ql.Driver - FAILED: IllegalArgumentException Relative path in absolute URI: file:./tmp/<user>/<guid>/hive_2015_12_11_09-12-25_352_4325234652356-1
> ...
> Minor variations:
> \[uber-SubtaskRunner] ERROR o.a.h.hive..ql.Driver - FAILED: SemanticException Exception
while processing Exception while writing out the local file 
> o.a.h.hive.ql/parse.SemanticException: Exception while processing exception while writing
out local file 
> ... 
> caused by: java.lang.IllegalArgumentException: Relative
path in absolute URI: file:./tmp/<user>/<guid>/hive_2015_12_11_09-12-25_352_4325234652356-1

> at o.a.h.fs.Path.initialize (206) 
> at o.a.h.fs.Path.<init>(197)... 
> at o.a.h.hive.ql.context.getScratchDir(267) 
> {code}

This message was sent by Atlassian JIRA

View raw message