hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rohith Sharma K S <rohithsharm...@huawei.com>
Subject RE: Setting | Verifying | Hive Query Parameters from Java
Date Mon, 10 Mar 2014 02:59:32 GMT
Hi,

yarn.nodemanager.resource.memory-mb is YARN configuration. Others are client configurations.
In order to take effect, you need to set at NodeManger configurations and restart NM.

This change can be observed in RM web UI.

Thanks & Regards
Rohith Sharma K S

From: Gordon Wang [mailto:gwang@gopivotal.com]
Sent: 07 March 2014 14:45
To: user
Subject: Re: Setting | Verifying | Hive Query Parameters from Java

Hi Rinku,

I think  yarn.nodemanager.resource.memory-mb is a static configuration prop. You can not change
this dynamically.

The other MR props looks OK to me.

On Fri, Mar 7, 2014 at 2:58 PM, Garg, Rinku <Rinku.Garg@fisglobal.com<mailto:Rinku.Garg@fisglobal.com>>
wrote:
Hi Gordon,

Thanks a lot for your reply.

The properties mentioned in mail trail  are just an example. Actual properties that we want
to set are as given below:

set yarn.nodemanager.resource.memory-mb=16384;
set mapreduce.map.memory.mb=2048;
set mapreduce.reduce.memory.mb=2048;
set mapreduce.map.java.opts=-Xmx2048M;
set yarn.app.mapreduce.am.command-opts=-Xmx2048m;

Please suggest.

Thanks
Rinku Garg

From: Gordon Wang [mailto:gwang@gopivotal.com<mailto:gwang@gopivotal.com>]
Sent: Friday, March 07, 2014 11:49 AM
To: user@hive.apache.org<mailto:user@hive.apache.org>
Subject: Re: Setting | Verifying | Hive Query Parameters from Java

The 2 following props are for hive server2. I don't think you can change it in your jdbc session.
I am wondering why you need to change them in your jdbc connection.

props.setProperty(hive.server2.async.exec.threads,50);
props.setProperty(hive.server2.thrift.max.worker.threads,500);


You can set props in your jdbc connetion with HQL like "propA=valueA;"

More, your connection url is for hive server, it does not work for hive server2.
If you need to use hive server2, you have to use
jdbc:hive2://master149:10000/default


On Tue, Mar 4, 2014 at 7:43 PM, Garg, Rinku <Rinku.Garg@fisglobal.com<mailto:Rinku.Garg@fisglobal.com>>
wrote:
Hi All,

We have installed CDH4.2.0 and hive-0.10.0-cdh4.2.0. Both are working as desired.

We need to set hive configuration parameter from Java while making JDBC connection.

We have written a java program to execute queries on hive server with some configurations
properties setting dynamically . We are doing it as below

                CONNECTION_URL=jdbc:hive://master149:10000/default

Next, we are doing following method to set properties through java

props.setProperty(hive.server2.async.exec.threads,50);
props.setProperty(hive.server2.thrift.max.worker.threads,500);
props.setProperty(hive.groupby.orderby.position.alias,false);

and a hive connection is made as given below

hiveConnection = DriverManager.getConnection(connectionURL,props);

by above steps when a hive connection is made using hive-jdbc and we are getting hive query
results as desired.

QUERY:


1.       Are we doing rightly for setting up the hive properties, if yes then how can we verify
that?

2.       If the above is not the right way, then how can we achieve setting hive configuration
parameters from Java using JDBC?

Thanks
Rinku Garg

_____________
The information contained in this message is proprietary and/or confidential. If you are not
the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose,
distribute or use the message in any manner; and (iii) notify the sender immediately. In addition,
please be aware that any message addressed to our domain is subject to archiving and review
by persons other than the intended recipient. Thank you.



--
Regards
Gordon Wang
_____________
The information contained in this message is proprietary and/or confidential. If you are not
the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose,
distribute or use the message in any manner; and (iii) notify the sender immediately. In addition,
please be aware that any message addressed to our domain is subject to archiving and review
by persons other than the intended recipient. Thank you.



--
Regards
Gordon Wang

Mime
View raw message