hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arvind Prabhakar (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HIVE-1277) Select query with specific projection(s) fails if the local file system directory for ${hive.user.scratchdir} does not exist.
Date Wed, 24 Mar 2010 22:03:27 GMT

     [ https://issues.apache.org/jira/browse/HIVE-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Arvind Prabhakar updated HIVE-1277:

    Attachment: HIVE-1277.patch


- Modified the {{MapRedTask.execute()}} to test if the scratch directory exists.
- If the scratch directory does not exist, an attempt is made to create it using {{File.mkdirs()}}
- If the call to create does not succeed and the directory still does not exist, an exception
is raised to indicate a failure condition.
- A new unchecked exception type is introduced {{TaskExecutionException}} to be used as a
generic mechanism to signal execution failures.
- Added a package-info.java file for the package.

*Testing Done*
- Built with default Hadoop as well as version
- Running tests against Hadoop since the past hour, no failures recorded yet.

> Select query with specific projection(s) fails if the local file system directory for
${hive.user.scratchdir} does not exist.
> -----------------------------------------------------------------------------------------------------------------------------
>                 Key: HIVE-1277
>                 URL: https://issues.apache.org/jira/browse/HIVE-1277
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Query Processor
>         Environment: Mac OSX 10.6.2, Java SE 6 (1.6.0_17)
>            Reporter: Arvind Prabhakar
>            Assignee: Arvind Prabhakar
>         Attachments: HIVE-1277.patch
> When a simple select query with specified projections is processed, the {{MapRedTask}}
attempts to serialize the execution plan to a scratch directory on the local file system specified
by {{hive.user.scratchdir}} configuration property. The default value of this property points
to {{/tmp/hive-${user.name}}}. On some *nix systems, the /tmp directory gets cleaned up on
every reboot thus causing the query execution to fail.
> *Exception stacktrace*
> {quote}
> hive> select foo from dummy;
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> java.io.IOException: No such file or directory
> 	at java.io.UnixFileSystem.createFileExclusively(Native Method)
> 	at java.io.File.checkAndCreate(File.java:1704)
> 	at java.io.File.createTempFile(File.java:1792)
> 	at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:89)
>         ...
> {quote}

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message