hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: BigDecimalColumnInterpreter
Date Wed, 05 Sep 2012 19:22:41 GMT
And your HBase version is ?

Since you use Double.parseDouble(), looks like it would be more efficient
to develop DoubleColumnInterpreter.

On Wed, Sep 5, 2012 at 12:07 PM, Julian Wissmann
<julian.wissmann@sdace.de>wrote:

> Hi,
> the schema looks like this:
> RowKey: id,timerange_timestamp,offset (String)
> Qualifier: Offset (long)
> Timestamp: timestamp (long)
> Value:number (BigDecimal)
>
> Or as code when I read data from csv:byte[] value =
> Bytes.toBytes(BigDecimal.valueOf(Double.parseDouble(cData[2])));
>
> Cheers,
>
> Julian
>
> 2012/9/5 Ted Yu <yuzhihong@gmail.com>
>
> > You haven't told us the schema of your table yet.
> > Your table should have column whose value can be interpreted by
> > BigDecimalColumnInterpreter.
> >
> > Cheers
> >
> > On Wed, Sep 5, 2012 at 9:17 AM, Julian Wissmann <
> julian.wissmann@sdace.de
> > >wrote:
> >
> > > Hi,
> > >
> > > I am currently experimenting with the BigDecimalColumnInterpreter from
> > > https://issues.apache.org/jira/browse/HBASE-6669.
> > >
> > > I was thinking the best way for me to work with it would be to use the
> > Java
> > > class and just use that as is.
> > >
> > > Imported it into my project and tried to work with it as is, by just
> > > instantiating the ColumnInterpreter as BigDecimalColumnInterpreter.
> Okay,
> > > threw errors and also complained about not knowing where to find such a
> > > class.
> > >
> > > So I did some reading and found out, that I'd need to have an Endpoint
> > for
> > > it. So I imported AggregateImplementation and AggregateProtocol into my
> > > workspace, renamed them, and refactored them where necessary to take
> > > BigDecimal. Re-exported the jar, then and had another try.
> > >
> > > So when I call:
> > > ------
> > > final Scan scan = new Scan((metricID + "," +
> basetime_begin).getBytes(),
> > > (metricID + "," + basetime_end).getBytes());
> > > scan.addFamily(family.getBytes());
> > > final ColumnInterpreter<BigDecimal, BigDecimal> ci = new
> > > BigDecimalColumnInterpreter();
> > > Map<byte[], BigDecimal> results =
> > > table.coprocessorExec(BigDecimalProtocol.class, null, null,
> > >     new Batch.Call<BigDecimalProtocol,BigDecimal>() {
> > >       public BigDecimal call(BigDecimalProtocol instance)throws
> > > IOException{
> > >         return instance.getMax(ci, scan);
> > >       }
> > >     });
> > > ------
> > > I get errors in the log again, that it can't find
> > > BigDecimalColumnInterpreter... okay, so I tried
> > > ------
> > > Scan scan = new Scan((metricID + "," + basetime_begin).getBytes(),
> > > (metricID + "," + basetime_end).getBytes());
> > > scan.addFamily(family.getBytes());
> > > final ColumnInterpreter<BigDecimal, BigDecimal> ci = new
> > > BigDecimalColumnInterpreter();
> > > AggregationClient ag = new AggregationClient(config);
> > > BigDecimal max = ag.max(Bytes.toBytes(tableName), ci, scan);
> > > ------
> > > I don't get errors recored in the log anymore, but a load of Java error
> > > output:
> > > ------
> > >
> > > java.util.concurrent.ExecutionException:
> > > org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> > > attempts=10, exceptions:
> > > Wed Sep 05 18:13:43 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:44 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:45 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:46 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:49 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:51 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:55 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:59 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:14:07 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:14:23 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > >
> > > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
> > > at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processExecs(HConnectionManager.java:1434)
> > > at
> > org.apache.hadoop.hbase.client.HTable.coprocessorExec(HTable.java:1263)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.client.coprocessor.AggregationClient.sum(AggregationClient.java:259)
> > > at
> > >
> > >
> >
> helpers.HbaseReaderBigDecimal.getWeeksumSCAN(HbaseReaderBigDecimal.java:360)
> > > at helpers.HbaseReaderBigDecimal.main(HbaseReaderBigDecimal.java:81)
> > > Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException:
> > Failed
> > > after attempts=10, exceptions:
> > > Wed Sep 05 18:13:43 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:44 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:45 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:46 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:49 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:51 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:55 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:59 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:14:07 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:14:23 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:183)
> > > at
> > >
> org.apache.hadoop.hbase.ipc.ExecRPCInvoker.invoke(ExecRPCInvoker.java:79)
> > > at $Proxy7.getSum(Unknown Source)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.client.coprocessor.AggregationClient$4.call(AggregationClient.java:263)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.client.coprocessor.AggregationClient$4.call(AggregationClient.java:260)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$4.call(HConnectionManager.java:1422)
> > > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> > > at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> > > at
> > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> > > at
> > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> > > at java.lang.Thread.run(Thread.java:679)
> > > org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> > > attempts=10, exceptions:
> > > Wed Sep 05 18:13:43 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:44 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:45 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:46 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:49 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:51 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:55 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:13:59 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:14:07 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > > Wed Sep 05 18:14:23 CEST 2012,
> > > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@50502819,
> > > java.io.IOException:
> > > IPC server unable to read call parameters: Error in readFields
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:183)
> > > at
> > >
> org.apache.hadoop.hbase.ipc.ExecRPCInvoker.invoke(ExecRPCInvoker.java:79)
> > > at $Proxy7.getSum(Unknown Source)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.client.coprocessor.AggregationClient$4.call(AggregationClient.java:263)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.client.coprocessor.AggregationClient$4.call(AggregationClient.java:260)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$4.call(HConnectionManager.java:1422)
> > > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> > > at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> > > at
> > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> > > at
> > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> > > at java.lang.Thread.run(Thread.java:679)
> > >
> > > ------
> > >
> > > I'm not really sure about what I'm doing wrong. Does anyone have a hint
> > > towards the right direction?
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message