hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suyog Parlikar <suyogparli...@gmail.com>
Subject Re: Hive Concurrency support
Date Mon, 24 Aug 2015 04:46:28 GMT
Hello Elliot,

Thanks for clarification.

But I am still not able to understand the hive working.

My cluster has following properties.

Txn.manager- dummyTxnManager

Cocurrency - true

Actually I am trying to insert data into two different partitions of a
table at the same time.

When I check the locks present on the table ,it shows the shared lock.Which
does not allow the writes on the table.

So I wanted to understand that ,

Does hive execute these two insert operations sequentially or it executes
it in parallel .

Thanks,
Suyog
On Aug 23, 2015 4:23 PM, "Elliot West" <teabot@gmail.com> wrote:

> Shared locks are all that is required to insert data into transactional
> tables. Multiple clients can hold a shared lock simultaneously. Each client
> will write using uniquely assigned transaction ids so that their work is
> isolated from one another. It should actually be possible for multiple
> clients to insert into the same partition concurrently.
>
> See slide 12 in:
> http://www.slideshare.net/mobile/Hadoop_Summit/w-525210-comalley
>
> Thanks - Elliot.
>
> On Friday, 21 August 2015, Suyog Parlikar <suyogparlikar@gmail.com> wrote:
>
>> Thanks Elliot,
>>
>> For the immediate reply.
>>
>> But as per hive locking mechanism,
>> While inserting data to a partition hive acquires exclusive lock on that
>> partition and shared lock on the entire table.
>>
>> How is it possible to insert data into a different partition of the same
>> table while having shared lock on the table which does not allow write
>> operation.
>>
>> Please correct me if my understanding about the same is wrong.
>> (I am using hql inserts only for these operations)
>>
>> Thanks,
>> Suyog
>> On Aug 21, 2015 7:28 PM, "Elliot West" <teabot@gmail.com> wrote:
>>
>>> I presume you mean "into different partitions of a table at the same
>>> time"? This should be possible. It is certainly supported by the streaming
>>> API, which is probably where you want to look if you need to insert large
>>> volumes of data to multiple partitions concurrently. I can't see why it
>>> would not also be possible with HQL INSERTs.
>>>
>>> On Friday, 21 August 2015, Suyog Parlikar <suyogparlikar@gmail.com>
>>> wrote:
>>>
>>>> Can we insert data in different partitions of a table at a time.
>>>>
>>>> Waiting for inputs .
>>>>
>>>> Thanks in advance.
>>>>
>>>> - suyog
>>>>
>>>

Mime
View raw message