chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <>
Subject Re: Backfilling and retroactive table creation in MySQL database
Date Thu, 11 Jun 2009 23:07:58 GMT
TableCreator creates partition base on two parameters.  First parameter is
date, and second parameter is partition size.

For example, you could run:

        ${JAVA_HOME}/bin/java -DCLUSTER=${CLUSTER} ${JVM_OPTS}
org.apache.hadoop.chukwa.database.TableCreator 2009-06-01 7 &

To create the week partition for 2009-06-01.

The current implementation of backfilling tools does not take this into
consideration.  MetricDataLoader contains the schema of the database in
memory while it is loading data.  We could modify MetricDataLoader to keep
track of table partition to accomplish this.


On 6/11/09 3:42 PM, "Jiaqi Tan" <> wrote:

> Hi,
> I've been trying to use the MDL to retroactively load data that was
> generated at an earlier time (somewhat like backfilling, but not,
> because I'm loading my own state-machine data), but when I create new
> tables and fire up in the present time, it only creates
> tables into the future. Is there any way to create tables for time
> periods that have passed?
> Also, taking a step back, if we have support for backfilling, should
> there also be support for retroactive table creation for the database,
> for instance, for bulk-loading trace data that wasn't collected using
> the Agent+Collector framework?
> Jiaqi

View raw message