hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marko Dinic <hacker.ma...@gmail.com>
Subject Use ChainMapper with TableMapReduceUtil
Date Tue, 29 Mar 2016 07:17:28 GMT
Is it possible to chain multiple mappers when the first mapper in the chain
reads from a HBase table? That is, to use ChainMapper with
TableMapReduceUtil?

I'm trying to perform some transformations on each record retrieved from
the HBase, but I'm not able to make it work.

Here's a code snipped where are try to use the two:

    Job job = new Job(hBaseConfig, "Retrieving data from HBase");
    job.setJarByClass(DataRetrievalDriver.class);

    Scan scan = scanCreator.getScan();

    TableMapReduceUtil.initTableMapperJob(
            table,
            scan,
            DataRetrievalMapper.class,
            Text.class,
            ElementWritable.class,
            job,
            true,
            SaltTableInputFormat.class
    );

    ChainMapper.addMapper(job, TransformMapper.class, Text.class,
ElementWritable.class, Text.class, ElementWritable.class, new
Configuration());

    SequenceFileOutputFormat.setOutputPath(job, new Path("out"));

    job.setReducerClass(Reducer.class);
    job.setNumReduceTasks(1);

    job.waitForCompletion(true);

But for some reason this doesn't work.

Any help is highly appreciated.

Best regards,

-- 
Marko Dinic

Mime
View raw message