beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "huangjianhuang (JIRA)" <>
Subject [jira] [Created] (BEAM-3002) Unable to provide a Coder for org.apache.hadoop.hbase.client.Mutation
Date Fri, 29 Sep 2017 14:19:00 GMT
huangjianhuang created BEAM-3002:

             Summary: Unable to provide a Coder for org.apache.hadoop.hbase.client.Mutation
                 Key: BEAM-3002
             Project: Beam
          Issue Type: Bug
          Components: sdk-java-core
    Affects Versions: 2.1.0
         Environment: hadoop2.8.0, hbase1.2.6
            Reporter: huangjianhuang
            Assignee: Kenneth Knowles

i write a demo with HbaseIO, and format data into Mutation to write to hbase.
The demo works fine on idea or using mvn exec:java command, but doesn't work after shade packaged
as jar (run with java -cp). 
The error message is :

Using the default output Coder from the producing PTransform failed: Unable to provide a Coder
for org.apache.hadoop.hbase.client.Mutation.
  Building a Coder using a registered CoderProvider failed.
  See suppressed exceptions for detailed failures.
        at org.apache.beam.sdk.values.PCollection.getCoder(
        at org.apache.beam.sdk.values.PCollection.finishSpecifying(
        at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifying(
        at org.apache.beam.sdk.runners.TransformHierarchy.visit(
        at org.apache.beam.sdk.Pipeline.traverseTopologically(
        at org.apache.beam.sdk.Pipeline.validate(
        at com.joe.FlinkDemoFinal.main(


And i tried to print the default coder of Mutation, on IDEA it works fine and print "HBaseMutationCoder",
but show nothing by running as jar.

And then i tried to register "HBaseMutationCoder" manully, but found that the HBaseMutationCoder
is a private class, i don't know how to register a coder for Mutation.

Part of my code:
                .apply("Hbase data format",
                        ParDo.of(new DoFn<Long, Mutation>() {
                    public void processElement(ProcessContext context) {


                        byte[] qual = Bytes.toBytes("count");

                        byte[] cf = Bytes.toBytes("cf");
                        byte[] row = Bytes.toBytes("kafka");
                        byte[] val = Bytes.toBytes(context.element().toString());
                        Mutation mutation = new Put(row).addColumn(cf, qual, val);


This message was sent by Atlassian JIRA

View raw message