hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joydeep Sen Sarma <>
Subject RE: thread cofinement session state
Date Tue, 10 Mar 2009 03:44:02 GMT
Min is right. this seems a little screwed up.

The Thrift Interface handler is constructed just once for the lifetime of the HiveServer.
The session object is initialized inside this constructor. The SessionState.start() is called
only once (in the constructor)

All connections/requests then go through the same handler object - but when they run in worker
threads - they don't have a current session object for that thread.

I think the solution here is to initialize TThreadPoolServer using a custom implementation
of TProcessorFactory. The getProcessor() call can return a freshly constructed handler (with
new SessionState etc.). This will work in all scenarios I think. (btw - the same thread can
serve different connections - but from reading the code - a getProcessor() call will be made
for every connection)

(see TThreadPoolServer.WorkerProcess and TThreadPoolServer.serve())

From: Prasad Chakka []
Sent: Monday, March 09, 2009 8:23 PM
Subject: Re: thread cofinement session state

I am assuming he is using the same code as MetaStore server. AFAIK, TThreadPoolServer is supposed
to use a new thread for each connection.

From: Joydeep Sen Sarma <>
Reply-To: <>
Date: Mon, 9 Mar 2009 20:16:22 -0700
To: <>
Subject: RE: thread cofinement session state

(also been reading up on this code a bit just now)

That's weird. It seems to be using TThreadPoolServer and that seems to just service all requests
from a single connection in one thread. (and uses the same processor I assume that seems to
initialize the session state in the interface constructor)

Are ur execute calls happening on the same connection?

From: Min Zhou []
Sent: Monday, March 09, 2009 8:03 PM
Subject: thread cofinement session state

Hi list,
   I found each invoke of HiveServer's execute method run in different threads. Those threads
which execute a HSQL query(not a client connection may execute several pieces of queries),
have not their own session state. when I call SessionState.get(), it will return null beacuse
session state on this thread hadnot been constructed before. see also fragment of

  public static String getRealFiles(Configuration conf) {
    // fill in local files to be added to the task environment
    SessionState ss = SessionState.get();  // return ss will get null !!!

Is it a bug?

My research interests are distributed systems, parallel computing and bytecode based virtual

View raw message