hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kristopher Glover <kglo...@appnexus.com>
Subject Re: threading with hive client
Date Thu, 15 Aug 2013 19:55:32 GMT
Interesting, I didn't realize that. If that's the case then I suppose it'd
be really bad for me to circumvent the lock by reproducing the Driver#run
method by calling Driver#compile and Driver#execute directly from within
my app. 

If that is the case why make Driver#compile and Driver#execute public
methods? There doesn't seem to be any inheritance that requires them to be
public and the fact that they are public opens up a thread safety issue.

Thanks,
Kris

On 8/15/13 1:11 PM, "Brock Noland" <brock@cloudera.com> wrote:

>The hive semantic analyzer is not fully thread safe.  We'd like to remove
>that lock but it will be a large project.
>
>Brock
>
>
>On Thu, Aug 15, 2013 at 11:12 AM, Kristopher Glover
><kglover@appnexus.com>wrote:
>
>> Hi Everyone,
>>
>> I'm experiencing a threading issue with the Hive client where I want to
>> run multiple queries on the same JVM.
>>
>>  The problem I'm having is that org.apache.hadoop.hive.ql.Driver#run
>>(line
>> 907)  has the following few lines of code :
>>
>>  synchronized (compileMonitor) {
>>
>>       ret = compile(command);
>>
>>     }
>>
>>
>> The compileMonitor is a static so it blocks all threads even though I'm
>> using different instances of the Driver class. I could explicitly call
>> Driver#compile then Driver#execute to avoid the synchronized block but I
>> don't know if it's serving a special purpose. Does anyone know why that
>> synchronized block is there and if its really necessary ?
>>
>>
>> Thanks,
>>
>> Kris
>>
>
>
>
>-- 
>Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org


Mime
View raw message