hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bejoy...@yahoo.com
Subject Re: Map join optimization issue
Date Fri, 15 Feb 2013 16:57:02 GMT
Hi 

In later versions of hive you actually don't need a map joint hint in your query. Just the
following would suffice the purpose

Set hive.auto.convert.join=true 

Regards 
Bejoy KS

Sent from remote device, Please excuse typos

-----Original Message-----
From: Mayuresh Kunjir <mayuresh.kunjir@gmail.com>
Date: Fri, 15 Feb 2013 10:37:52 
To: user<user@hive.apache.org>
Reply-To: user@hive.apache.org
Subject: Re: Map join optimization issue

Thanks Aniket. I actually had not specified the map-join hint though. Sorry
for providing the wrong information earlier. I had only
set hive.auto.convert.join=true before firing my join query.

~Mayuresh



On Thu, Feb 14, 2013 at 10:44 PM, Aniket Mokashi <aniket486@gmail.com>wrote:

> I think hive.mapjoin.smalltable.filesize parameter will be disregarded in
> that case.
>
>
> On Thu, Feb 14, 2013 at 7:25 AM, Mayuresh Kunjir <
> mayuresh.kunjir@gmail.com> wrote:
>
>> Yes, the hint was specified.
>> On Feb 14, 2013 3:11 AM, "Aniket Mokashi" <aniket486@gmail.com> wrote:
>>
>>> have you specified map-join hint in your query?
>>>
>>>
>>> On Thu, Feb 7, 2013 at 11:39 AM, Mayuresh Kunjir <
>>> mayuresh.kunjir@gmail.com> wrote:
>>>
>>>>
>>>> Hello all,
>>>>
>>>>
>>>> I am trying to join two tables, the smaller being of size 4GB. When I
>>>> set hive.mapjoin.smalltable.filesize parameter above 500MB, Hive tries to
>>>> perform a local task to read the smaller file. This of-course fails since
>>>> the file size is greater and the backup common join is then run. What I do
>>>> not understand is why did Hive attempt a map join when small file size was
>>>> greater than the smalltable.filesize parameter.
>>>>
>>>>
>>>> ~Mayuresh
>>>>
>>>>
>>>
>>>
>>> --
>>> "...:::Aniket:::... Quetzalco@tl"
>>>
>>
>
>
> --
> "...:::Aniket:::... Quetzalco@tl"
>

Mime
View raw message