Mailing-List: contact issues-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Wed, 28 Sep 2016 18:01:20 +0000 (UTC)
From: "Naveen Gangam (JIRA)" <jira@apache.org>
To: issues@hive.apache.org
Message-ID: <JIRA.13005596.1474077628000.689857.1475085680490@Atlassian.JIRA>
In-Reply-To: <JIRA.13005596.1474077628000@Atlassian.JIRA>
References: <JIRA.13005596.1474077628000@Atlassian.JIRA> <JIRA.13005596.1474077628455@arcas>
Subject: [jira] [Updated] (HIVE-14784) Operation logs are disabled
 automatically if the parent directory does not exist.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 28 Sep 2016 18:01:22 -0000


     [ https://issues.apache.org/jira/browse/HIVE-14784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Naveen Gangam updated HIVE-14784:
---------------------------------
    Attachment: HIVE-14784.1.patch

New patch with feedback

> Operation logs are disabled automatically if the parent directory does not exist.
> ---------------------------------------------------------------------------------
>
>                 Key: HIVE-14784
>                 URL: https://issues.apache.org/jira/browse/HIVE-14784
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2
>    Affects Versions: 1.1.0
>            Reporter: Naveen Gangam
>            Assignee: Naveen Gangam
>         Attachments: HIVE-14784.1.patch, HIVE-14784.patch
>
>
> Operation logging is disabled automatically for the query if for some reason the parent directory (named after the hive session id) that gets created when the session is established gets deleted (for any reason). For ex: if the operation logdir is /tmp which automatically can get purged at a configured interval by the OS.
> Running a query from that session leads to
> {code}
> 2016-09-15 15:09:16,723 WARN org.apache.hive.service.cli.operation.Operation: Unable to create operation log file: /tmp/hive/operation_logs/b8809985-6b38-47ec-a49b-6158a67cd9fc/d35414f7-2418-426c-8489-c6f643ca4599
> java.io.IOException: No such file or directory
> 	at java.io.UnixFileSystem.createFileExclusively(Native Method)
> 	at java.io.File.createNewFile(File.java:1012)
> 	at org.apache.hive.service.cli.operation.Operation.createOperationLog(Operation.java:195)
> 	at org.apache.hive.service.cli.operation.Operation.beforeRun(Operation.java:237)
> 	at org.apache.hive.service.cli.operation.Operation.run(Operation.java:255)
> 	at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:398)
> 	at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:385)
> 	at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:271)
> 	at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:490)
> 	at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
> 	at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
> 	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> 	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> 	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:692)
> 	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> {code}
> This later leads to errors like (more prominent when using HUE as HUE does not close hive sessions and attempts to retrieve the operations logs days after they were created).
> {code}
> WARN org.apache.hive.service.cli.thrift.ThriftCLIService: Error fetching results: 
> org.apache.hive.service.cli.HiveSQLException: Couldn't find log associated with operation handle: OperationHandle [opType=EXECUTE_STATEMENT, getHandleIdentifier()=d35414f7-2418-426c-8489-c6f643ca4599]
> 	at org.apache.hive.service.cli.operation.OperationManager.getOperationLogRowSet(OperationManager.java:259)
> 	at org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:701)
> 	at org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:451)
> 	at org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:676)
> 	at org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1553)
> 	at org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1538)
> 	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> 	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> 	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:692)
> 	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745) 
> {code}


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)