Mailing-List: contact chukwa-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: chukwa-user@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns;
	h=received:user-agent:date:subject:from:to:message-id:
	thread-topic:thread-index:in-reply-to:mime-version:content-type:
	content-transfer-encoding:x-originalarrivaltime;
	b=nXUFGbUhPdHrwa3l7HyAgxx1gdb5IQpNyrxsRECUOsZKMOxvcUSkXanu519yACVN
User-Agent: Microsoft-Entourage/12.24.0.100205
Date: Mon, 15 Mar 2010 13:47:43 -0700
Subject: Re: How to set up HDFS -> MySQL from trunk?
From: Eric Yang <eyang@yahoo-inc.com>
To: <chukwa-user@hadoop.apache.org>
Message-ID: <C7C3EAFF.7116%eyang@yahoo-inc.com>
Thread-Topic: How to set up HDFS -> MySQL from trunk?
Thread-Index: AcrEgMJp7nQSI76kfU68HE+wwOWc1w==
In-Reply-To: <4B9E82E5.3060800@mustardgrain.com>
Mime-version: 1.0
Content-type: text/plain;
	charset="US-ASCII"
Content-transfer-encoding: 7bit

Hi Kirk,

I am working on a design which removes MySQL from Chukwa.  I am making this
departure from MySQL because MDL framework was for prototype purpose.  It
will not scale in production system where Chukwa could be host on large
hadoop cluster.  HICC will serve data directly from HDFS in the future.

Meanwhile, the dbAdmin.sh from Chukwa 0.3 is still compatible with trunk
version of Chukwa.  You can load ChukwaRecords using
org.apache.hadoop.chukwa.dataloader.MetricDataLoader class or mdl.sh from
Chukwa 0.3.

MetricDataLoader class will be mark as deprecated, and it will not be
supported once we make transition to Avro + Tfile.

Regards,
Eric

On 3/15/10 11:56 AM, "Kirk True" <kirk@mustardgrain.com> wrote:

> Hi all,
> 
> I recently switched to trunk as I was experiencing a lot of issues with
> 0.3.0. In 0.3.0, there was a dbAdmin.sh script that would run and try to
> stick data in MySQL from HDFS. However, that script is gone and when I
> run the system as built from trunk, nothing is ever populated in the
> database. Where are the instructions for setting up the HDFS -> MySQL
> data migration for HICC?
> 
> Thanks,
> Kirk