hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nemon Lou (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-9839) HiveServer2 leaks OperationHandle on async queries which fail at compile phase
Date Sun, 22 Mar 2015 03:04:10 GMT

    [ https://issues.apache.org/jira/browse/HIVE-9839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14374764#comment-14374764
] 

Nemon Lou commented on HIVE-9839:
---------------------------------

Refering to SQLOperation.java,there is no chance that a HiveSQLException throws and the asyn
background operation submits to thread pool successfully at the same time.Only line 181 and
line 244 can causing  HiveSQLException.
So the answer to "If OperationHandle has to be returned to report the error in Async mode"
is no.
{code}
 @Override
178	  public void runInternal() throws HiveSQLException {
179	    setState(OperationState.PENDING);
180	    final HiveConf opConfig = getConfigForOperation();
181	    prepare(opConfig);
182	    if (!shouldRunAsync()) {
183	      runQuery(opConfig);
184	    } else {
185	      // We'll pass ThreadLocals in the background thread from the foreground (handler)
thread
186	      final SessionState parentSessionState = SessionState.get();
187	      // ThreadLocal Hive object needs to be set in background thread.
188	      // The metastore client in Hive is associated with right user.
189	      final Hive parentHive = getSessionHive();
190	      // Current UGI will get used by metastore when metsatore is in embedded mode
191	      // So this needs to get passed to the new background thread
192	      final UserGroupInformation currentUGI = getCurrentUGI(opConfig);
193	      // Runnable impl to call runInternal asynchronously,
194	      // from a different thread
195	      Runnable backgroundOperation = new Runnable() {
196	        @Override
197	        public void run() {
198	          PrivilegedExceptionAction<Object> doAsAction = new PrivilegedExceptionAction<Object>()
{
199	            @Override
200	            public Object run() throws HiveSQLException {
201	              Hive.set(parentHive);
202	              SessionState.setCurrentSessionState(parentSessionState);
203	              // Set current OperationLog in this async thread for keeping on saving query
log.
204	              registerCurrentOperationLog();
205	              try {
206	                runQuery(opConfig);
207	              } catch (HiveSQLException e) {
208	                setOperationException(e);
209	                LOG.error("Error running hive query: ", e);
210	              } finally {
211	                unregisterOperationLog();
212	              }
213	              return null;
214	            }
215	          };
216	
217	          try {
218	            currentUGI.doAs(doAsAction);
219	          } catch (Exception e) {
220	            setOperationException(new HiveSQLException(e));
221	            LOG.error("Error running hive query as user : " + currentUGI.getShortUserName(),
e);
222	          }
223	          finally {
224	            /**
225	             * We'll cache the ThreadLocal RawStore object for this background thread
for an orderly cleanup
226	             * when this thread is garbage collected later.
227	             * @see org.apache.hive.service.server.ThreadWithGarbageCleanup#finalize()
228	             */
229	            if (ThreadWithGarbageCleanup.currentThread() instanceof ThreadWithGarbageCleanup)
{
230	              ThreadWithGarbageCleanup currentThread =
231	                  (ThreadWithGarbageCleanup) ThreadWithGarbageCleanup.currentThread();
232	              currentThread.cacheThreadLocalRawStore();
233	            }
234	          }
235	        }
236	      };
237	      try {
238	        // This submit blocks if no background threads are available to run this operation
239	        Future<?> backgroundHandle =
240	            getParentSession().getSessionManager().submitBackgroundOperation(backgroundOperation);
241	        setBackgroundHandle(backgroundHandle);
242	      } catch (RejectedExecutionException rejected) {
243	        setState(OperationState.ERROR);
244	        throw new HiveSQLException("The background threadpool cannot accept" +
245	            " new task for execution, please retry the operation", rejected);
246	      }
247	    }
248	  }
{code}



> HiveServer2 leaks OperationHandle on async queries which fail at compile phase
> ------------------------------------------------------------------------------
>
>                 Key: HIVE-9839
>                 URL: https://issues.apache.org/jira/browse/HIVE-9839
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2
>    Affects Versions: 0.14.0, 0.13.1, 1.0.0, 1.1.0
>            Reporter: Nemon Lou
>            Priority: Critical
>         Attachments: OperationHandleMonitor.java, hive-9839.patch
>
>
> Using beeline to connect to HiveServer2.And type the following:
> drop table if exists table_not_exists;
> select * from table_not_exists;
> There will be an OperationHandle object staying in HiveServer2's memory for ever even
after quit from beeline .



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message