hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suyog Parlikar <suyogparli...@gmail.com>
Subject Re: Hive Concurrency support
Date Mon, 24 Aug 2015 15:11:33 GMT
No table is not transactional
On Aug 24, 2015 10:30 AM, "Elliot West" <teabot@gmail.com> wrote:

> Is the table configured to be transactional?
>
>
> https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-TableProperties
>
> On Monday, 24 August 2015, Suyog Parlikar <suyogparlikar@gmail.com> wrote:
>
>> Hello Elliot,
>>
>> Thanks for clarification.
>>
>> But I am still not able to understand the hive working.
>>
>> My cluster has following properties.
>>
>> Txn.manager- dummyTxnManager
>>
>> Cocurrency - true
>>
>> Actually I am trying to insert data into two different partitions of a
>> table at the same time.
>>
>> When I check the locks present on the table ,it shows the shared
>> lock.Which does not allow the writes on the table.
>>
>> So I wanted to understand that ,
>>
>> Does hive execute these two insert operations sequentially or it executes
>> it in parallel .
>>
>> Thanks,
>> Suyog
>> On Aug 23, 2015 4:23 PM, "Elliot West" <teabot@gmail.com> wrote:
>>
>>> Shared locks are all that is required to insert data into transactional
>>> tables. Multiple clients can hold a shared lock simultaneously. Each client
>>> will write using uniquely assigned transaction ids so that their work is
>>> isolated from one another. It should actually be possible for multiple
>>> clients to insert into the same partition concurrently.
>>>
>>> See slide 12 in:
>>> http://www.slideshare.net/mobile/Hadoop_Summit/w-525210-comalley
>>>
>>> Thanks - Elliot.
>>>
>>> On Friday, 21 August 2015, Suyog Parlikar <suyogparlikar@gmail.com>
>>> wrote:
>>>
>>>> Thanks Elliot,
>>>>
>>>> For the immediate reply.
>>>>
>>>> But as per hive locking mechanism,
>>>> While inserting data to a partition hive acquires exclusive lock on
>>>> that partition and shared lock on the entire table.
>>>>
>>>> How is it possible to insert data into a different partition of the
>>>> same table while having shared lock on the table which does not allow write
>>>> operation.
>>>>
>>>> Please correct me if my understanding about the same is wrong.
>>>> (I am using hql inserts only for these operations)
>>>>
>>>> Thanks,
>>>> Suyog
>>>> On Aug 21, 2015 7:28 PM, "Elliot West" <teabot@gmail.com> wrote:
>>>>
>>>>> I presume you mean "into different partitions of a table at the same
>>>>> time"? This should be possible. It is certainly supported by the streaming
>>>>> API, which is probably where you want to look if you need to insert large
>>>>> volumes of data to multiple partitions concurrently. I can't see why
it
>>>>> would not also be possible with HQL INSERTs.
>>>>>
>>>>> On Friday, 21 August 2015, Suyog Parlikar <suyogparlikar@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Can we insert data in different partitions of a table at a time.
>>>>>>
>>>>>> Waiting for inputs .
>>>>>>
>>>>>> Thanks in advance.
>>>>>>
>>>>>> - suyog
>>>>>>
>>>>>

Mime
View raw message