hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Elazar Leibovich <elaz...@gmail.com>
Subject Saving counters in Mapfile
Date Tue, 23 Jul 2013 21:57:54 GMT
Hi,

A common use case one want an ordered structure for, is for saving counters.

Naturally, I wanted to save the counters in a Mapfile:

    for (long ix = 0; ix < MAXVALUE; ix++) {
        mapfile.append(new Text("counter key of val " + ix), new
LongWritable(ix));
    }

This however looks a bit inefficient. We'll store two files, and an index
file. The index file will contain an offset (long) to the sequence file,
which would contain a single long.

I'd rather have only the index file, that would store the counter value
instead of offsets.

Is there a way to do that with Mapfile? Perhaps there's a better way to
save searchable counters in HDFS file?

Thanks,

Mime
View raw message