hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Reuben Kuhnert (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-12891) Hive fails when java.io.tmpdir is set to a relative location
Date Mon, 25 Jan 2016 17:19:40 GMT

    [ https://issues.apache.org/jira/browse/HIVE-12891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15115581#comment-15115581
] 

Reuben Kuhnert commented on HIVE-12891:
---------------------------------------

The problem that this patch addresses is that the value of 'java.io.tmpdir' can be set externally
to be a relative path (we're seeing this problem occur in oozie). To adjust for this issue,
the above patch uses 'Coercion' to validate/modify the value before passing it to the user.
That is, rather than simply creating a one-off coercion, I thought it would be useful in general
to have a way to hook into {{SystemVariables.substitute}} to validate or adjust the property
before returning to the user.

The template is:

{code}
public abstract class VariableCoercion {
  private final String name;

  public VariableCoercion(String name) {
    this.name = name;
  }

  public String getName() { return this.name; }
  public abstract String getCoerced(Configuration configuration, String originalValue);
  public abstract String setCoerced(Configuration configuration, String originalValue);
}
{code}

where {{getCoerced}} is called on get and {{setCoerced}} is called on set (configuration is
passed in if the coerced value is context sensitive). In addition, to add other coercions,
simply subclass the above and add it here:

{code}
public class SystemVariables {
  ...
  // HERE: List of coercions:
  private static final VariableCoercionSet COERCIONS = new VariableCoercionSet()
    .add(new JavaIOTmpdirVariableCoercion());
{code}

If a coercion hook exists for a particular name (see {{VariableCoercion.getName()}}) then
it is loaded and the raw value is passed through, then returned to the user:

{code}
  public String getCoerced(Configuration configuration, String variableName, String originalValue)
{
    if (COERCIONS.contains(variableName)) {
      return COERCIONS.get(variableName).getCoerced(configuration, originalValue);
    } else {
      return originalValue;
    }
  }
{code}

> Hive fails when java.io.tmpdir is set to a relative location
> ------------------------------------------------------------
>
>                 Key: HIVE-12891
>                 URL: https://issues.apache.org/jira/browse/HIVE-12891
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Reuben Kuhnert
>            Assignee: Reuben Kuhnert
>         Attachments: HIVE-12891.01.19.2016.01.patch, HIVE-12891.03.patch, HIVE-12981.01.22.2016.02.patch
>
>
> The function {{SessionState.createSessionDirs}} fails when trying to create directories
where {{java.io.tmpdir}} is set to a relative location.
> {code}
> \[uber-SubtaskRunner] ERROR o.a.h.hive..ql.Driver - FAILED: IllegalArgumentException
java.net.URISyntaxException: Relative path in absolute URI: file:./tmp/<user>/<guid>/hive_2015_12_11_09-12-25_352_4325234652356-1
> ...
> Minor variations:
> \[uber-SubtaskRunner] ERROR o.a.h.hive..ql.Driver - FAILED: SemanticException Exception
while processing Exception while writing out the local file 
> o.a.h.hive.ql/parse.SemanticException: Exception while processing exception while writing
out local file 
> ... 
> caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
path in absolute URI: file:./tmp/<user>/<guid>/hive_2015_12_11_09-12-25_352_4325234652356-1

> at o.a.h.fs.Path.initialize (206) 
> at o.a.h.fs.Path.<init>(197)... 
> at o.a.h.hive.ql.context.getScratchDir(267) 
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message