hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xuefu Zhang <xzh...@cloudera.com>
Subject Re: Hive Mapred local task distribution
Date Sat, 06 Sep 2014 14:23:47 GMT
By "same host", don't you mean your HiveServer2 host? One solution is to
have multiple HiveServer2 instances and do load balance among them.

--Xuefu


On Fri, Sep 5, 2014 at 11:37 PM, Abhilash L L <abhilash@capillarytech.com>
wrote:

> Hello,
>
>    We are using Hive 0.11 connecting to it via Hive Thrift server 2.
>
>    A lot of our queries are launching map red local tasks, which is good
> and expected. Since we are firing queries in parallel, these tasks are all
> starting on same host and  consuming a lot of resources.
>
>    Is there a way to distribute these on different nodes ?
>
>     Or the only possibility is to do it in app layer and load balance it
> on few of the thrift servers ?
>
>     Please do let me know in case I have to share any more information
> regarding the setup etc.
>
> Regards,
> Abhilash L L
> Capillary Technologies
> M:919886208262
> abhilash@capillarytech.com | www.capillarytech.com
>
> Email from people at capillarytech.com may not represent official policy
> of  Capillary Technologies unless explicitly stated. Please see our
> Corporate-Email-Policy
> <http://support.capillary.co.in/policy-public/Corporate-Email-Policy.pdf>
> for details. Contents of this email are confidential. Please contact the
> Sender if you have received this email in error.
>
>
> Email from people at capillarytech.com may not represent official policy
> of Capillary Technologies unless explicitly stated. Please see our
> Corporate-Email-Policy for details.Contents of this email are confidential.
> Please contact the Sender if you have received this email in error.
>

Mime
View raw message