Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
MIME-Version: 1.0
Date: Tue, 29 Mar 2016 09:17:28 +0200
Message-ID: 
 <CAJLYvfb=GAvEnaOckJFCye53rJZGYKZ8LPaS4GmhMzaBxzLYoQ@mail.gmail.com>
Subject: Use ChainMapper with TableMapReduceUtil
From: Marko Dinic <hacker.marko@gmail.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=001a114283126519d3052f2ad22b

--001a114283126519d3052f2ad22b
Content-Type: text/plain; charset=UTF-8

Is it possible to chain multiple mappers when the first mapper in the chain
reads from a HBase table? That is, to use ChainMapper with
TableMapReduceUtil?

I'm trying to perform some transformations on each record retrieved from
the HBase, but I'm not able to make it work.

Here's a code snipped where are try to use the two:

    Job job = new Job(hBaseConfig, "Retrieving data from HBase");
    job.setJarByClass(DataRetrievalDriver.class);

    Scan scan = scanCreator.getScan();

    TableMapReduceUtil.initTableMapperJob(
            table,
            scan,
            DataRetrievalMapper.class,
            Text.class,
            ElementWritable.class,
            job,
            true,
            SaltTableInputFormat.class
    );

    ChainMapper.addMapper(job, TransformMapper.class, Text.class,
ElementWritable.class, Text.class, ElementWritable.class, new
Configuration());

    SequenceFileOutputFormat.setOutputPath(job, new Path("out"));

    job.setReducerClass(Reducer.class);
    job.setNumReduceTasks(1);

    job.waitForCompletion(true);

But for some reason this doesn't work.

Any help is highly appreciated.

Best regards,

-- 
Marko Dinic

--001a114283126519d3052f2ad22b
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><p style=3D"margin:0px 0px 1em;padding:0px;border:0px;font=
-size:15px;clear:both;color:rgb(34,36,38);font-family:Arial,&#39;Helvetica =
Neue&#39;,Helvetica,sans-serif;line-height:19.5px">Is it possible to chain =
multiple mappers when the first mapper in the chain reads from a HBase tabl=
e? That is, to use ChainMapper with TableMapReduceUtil?</p><p style=3D"marg=
in:0px 0px 1em;padding:0px;border:0px;font-size:15px;clear:both;color:rgb(3=
4,36,38);font-family:Arial,&#39;Helvetica Neue&#39;,Helvetica,sans-serif;li=
ne-height:19.5px">I&#39;m trying to perform some transformations on each re=
cord retrieved from the HBase, but I&#39;m not able to make it work.</p><p =
style=3D"margin:0px 0px 1em;padding:0px;border:0px;font-size:15px;clear:bot=
h;color:rgb(34,36,38);font-family:Arial,&#39;Helvetica Neue&#39;,Helvetica,=
sans-serif;line-height:19.5px">Here&#39;s a code snipped where are try to u=
se the two:</p><pre style=3D"white-space:pre-wrap;margin-top:0px;margin-bot=
tom:1em;padding:5px;border:0px;font-size:13px;overflow:auto;width:auto;max-=
height:600px;font-family:Consolas,Menlo,Monaco,&#39;Lucida Console&#39;,=
9;Liberation Mono&#39;,&#39;DejaVu Sans Mono&#39;,&#39;Bitstream Vera Sans =
Mono&#39;,&#39;Courier New&#39;,monospace,sans-serif;word-wrap:normal;color=
:rgb(34,36,38);background-color:rgb(238,238,238)"><code style=3D"margin:0px=
;padding:0px;border:0px;font-family:Consolas,Menlo,Monaco,&#39;Lucida Conso=
le&#39;,&#39;Liberation Mono&#39;,&#39;DejaVu Sans Mono&#39;,&#39;Bitstream=
 Vera Sans Mono&#39;,&#39;Courier New&#39;,monospace,sans-serif;white-space=
:inherit">    Job job =3D new Job(hBaseConfig, &quot;Retrieving data from H=
Base&quot;);
    job.setJarByClass(DataRetrievalDriver.class);

    Scan scan =3D scanCreator.getScan();

    TableMapReduceUtil.initTableMapperJob(
            table,
            scan,
            DataRetrievalMapper.class,
            Text.class,
            ElementWritable.class,
            job,
            true,
            SaltTableInputFormat.class
    );

    ChainMapper.addMapper(job, TransformMapper.class, Text.class, ElementWr=
itable.class, Text.class, ElementWritable.class, new Configuration());

    SequenceFileOutputFormat.setOutputPath(job, new Path(&quot;out&quot;));

    job.setReducerClass(Reducer.class);
    job.setNumReduceTasks(1);

    job.waitForCompletion(true);</code></pre><div style=3D"font-size:12.8px=
">But for some reason this doesn&#39;t work.</div><div style=3D"font-size:1=
2.8px"><br></div><div style=3D"font-size:12.8px">Any help is highly appreci=
ated.</div><div style=3D"font-size:12.8px"><br></div><div style=3D"font-siz=
e:12.8px">Best regards,</div><div><br></div>-- <br><div class=3D"gmail_sign=
ature">Marko Dinic</div>
</div>

--001a114283126519d3052f2ad22b--