hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sanjay Subramanian <Sanjay.Subraman...@wizecommerce.com>
Subject Re: MySQL instance on hadoop name node server in production ?
Date Wed, 27 Mar 2013 18:41:09 GMT
Thanks Nitin…the mysql replication is a good point
Regards
sanjay

From: Nitin Pawar <nitinpawar432@gmail.com<mailto:nitinpawar432@gmail.com>>
Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>" <user@hive.apache.org<mailto:user@hive.apache.org>>
Date: Wednesday, March 27, 2013 11:36 AM
To: "user@hive.apache.org<mailto:user@hive.apache.org>" <user@hive.apache.org<mailto:user@hive.apache.org>>
Subject: Re: MySQL instance on hadoop name node server in production ?

few writes understood but many reads will depends on how big your metastore will grow.

also what size of hdfs you are looking at that will play a huge factor in this cause namenode
will contain the FSImage in memory so you need a larger memory given to namenode. Potentially
if its a live cluster then I guess your namenode will be under heavy loads if you are setting
up a production system.

i would still suggest for metastore you setup a couple of VMs with mysql replication enabled.
so the reads can be distributed with load balancers etc


On Thu, Mar 28, 2013 at 12:00 AM, Sanjay Subramanian <Sanjay.Subramanian@wizecommerce.com<mailto:Sanjay.Subramanian@wizecommerce.com>>
wrote:
Thanks Nitin.
The mysql instance is for hive metastore only so few writes, many reads.
Regards
sanjay

From: Nitin Pawar <nitinpawar432@gmail.com<mailto:nitinpawar432@gmail.com>>
Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>" <user@hive.apache.org<mailto:user@hive.apache.org>>
Date: Wednesday, March 27, 2013 11:13 AM
To: "user@hive.apache.org<mailto:user@hive.apache.org>" <user@hive.apache.org<mailto:user@hive.apache.org>>
Subject: Re: MySQL instance on hadoop name node server in production ?

ideally when you setup namenode and jobtracker in any hadoop cluster on production those are
normally dedicated to that purpose only.

Depending on how frequently you are going to run queries and how much data the hdfs is going
to hold is key factor in deciding this.


On Wed, Mar 27, 2013 at 11:32 PM, Sanjay Subramanian <Sanjay.Subramanian@wizecommerce.com<mailto:Sanjay.Subramanian@wizecommerce.com>>
wrote:
Hi all

I am planning to install mysql server (as hive metastore) on the same box
as my name node.
My name node has 16GB RAM and hopefully I can get 2TB
Any problems with mysql on the dame node as name node ?

Thanks
sanjay


CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s)
and may contain confidential and privileged information. Any unauthorized review, use, disclosure
or distribution is prohibited. If you are not the intended recipient, please contact the sender
by reply email and destroy all copies of the original message along with any attachments,
from your computer system. If you are the intended recipient, please be advised that the content
of this message is subject to access, review and disclosure by the sender's Email System Administrator.




--
Nitin Pawar

CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s)
and may contain confidential and privileged information. Any unauthorized review, use, disclosure
or distribution is prohibited. If you are not the intended recipient, please contact the sender
by reply email and destroy all copies of the original message along with any attachments,
from your computer system. If you are the intended recipient, please be advised that the content
of this message is subject to access, review and disclosure by the sender's Email System Administrator.



--
Nitin Pawar

CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s)
and may contain confidential and privileged information. Any unauthorized review, use, disclosure
or distribution is prohibited. If you are not the intended recipient, please contact the sender
by reply email and destroy all copies of the original message along with any attachments,
from your computer system. If you are the intended recipient, please be advised that the content
of this message is subject to access, review and disclosure by the sender's Email System Administrator.

Mime
View raw message