Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of yuzhihong@gmail.com
 designates 209.85.210.169 as permitted sender)
Subject: Re: median aggregate Was: AggregateProtocol Help
References: 
 <CALte62yus86e-mNt4VrVkXo_NqR3z6_hdoMkyEb2Kkzji86ksg@mail.gmail.com>
 <725F3D3F-AC1E-4DAD-8E79-D7CE35598C3F@gmail.com>
From: yuzhihong@gmail.com
Content-Type: text/plain;
	charset=us-ascii
In-Reply-To: <725F3D3F-AC1E-4DAD-8E79-D7CE35598C3F@gmail.com>
Message-Id: <19ED0BD2-D67D-4FE0-8A94-43917F38D847@gmail.com>
Date: Fri, 6 Jan 2012 06:37:38 -0800
To: "user@hbase.apache.org" <user@hbase.apache.org>
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (1.0)

This is a good summary.=20

Do you mind putting what you wrote below on hbase-5123 ?

Thanks


On Jan 6, 2012, at 6:22 AM, Royston Sellman <royston.sellman@googlemail.com>=
 wrote:

> Hi Ted,
>=20
> Yes, that is the use case I am thinking of.=20
>=20
> Re: 5123 I have also had some time to think about other aggregation functi=
ons (Please be aware that I am new to HBase, Coprocessors, and the Aggregati=
on Protocol and I have little knowledge of distributed numerical algorithms!=
). It seems to me the pattern in AP is to return a SINGLE value from a SINGL=
E column (CF:CQ) of a table. In future one might wish to extend AP to return=
 MULTIPLE values from MULTIPLE columns, so it is good to keep this in mind f=
or the SINGLE value/SINGLE column (SVSC) case.=20
>=20
> So, common SVSC aggregation functions (AP supported first):
> min
> max
> sum
> count
> avg (arithmetic mean)
> std
> median
> mode=20
> quantile/ntile
> mult/product
>=20
> for column values of all numeric types, returning values of that type.
>=20
> Some thoughts on the future possibilities:
> An example of a future SINGLE value MULTIPLE column use case could be weig=
hted versions of the above functions i.e. a column of weights applied to the=
 column of values then the new aggregation derived.
> (note: there is a very good description of Weighted Median in the R langua=
ge documentation:
> http://www.stat.ucl.ac.be/ISdidactique/Rhelp/library/R.basic/html/weighted=
.median.html)
>=20
> An example of future MULTIPLE value SINGLE column could be range: return a=
ll rows with a column value between two values. Maybe this is a bad example b=
ecause there could be better HBase ways to do it with filters/scans at a hig=
her level. Perhaps binning is a better example? i.e. return an array contain=
ing values derived from applying one of the SVSC functions to a binned colum=
n e.g:
> int bins =3D 100;
> aClient.sum(table, ci, scan, bins); =3D> {12.3, 14.5...}
> Another example (common in several programming languages) is to map an arb=
itrary function over a column and return the new vector. Of course, again th=
is may be a bad example in the case of long HBase columns but it seems like a=
n appropriate thing to do with coprocessors.
>=20
> MULTIPLE value MULTIPLE column examples are common in spatial data process=
ing but I see there has been a lot of spatial/GIS discussion around HBase wh=
ich I have not read yet. So I'll keep quiet for now.
>=20
> I hope these thoughts strike a balance between my (special interest) use c=
ase of statistical/spatial functions on tables and general purpose (but copr=
ocessor enabled/regionserver distributed) HBase.
>=20
> Best regards,
> Royston
>=20
>=20
> On 6 Jan 2012, at 03:31, Ted Yu wrote:
>=20
>> Royston:
>> For the median aggregate, is the following what you're looking for ?
>> Find the median among the values of all the keyvalue for cf:qualifier
>> column.
>>=20
>> There is a well known distributed method of computing median that involve=
s
>> multiple roundtrips (to the region servers).
>>=20
>> Just want to confirm the use case.
>>=20
>> Thanks
>>=20
>> On Wed, Jan 4, 2012 at 10:57 AM, Royston Sellman <
>> royston.sellman@googlemail.com> wrote:
>>=20
>>> Great ideas. Thanks.
>>>=20
>>> w.r.t. 5123: I'll think about it for a day or two then make some comment=
s.
>>>=20
>>> 5122 is very desirable.
>>>=20
>>> Best Regards,
>>> Royston
>>>=20
>>> On 4 Jan 2012, at 15:01, Ted Yu wrote:
>>>=20
>>>> I can see room for improvement w.r.t. ColumnInterpreters
>>>> I logged two JIRAs:
>>>> https://issues.apache.org/jira/browse/HBASE-5122 is for loading
>>>> ColumnInterpreters dynamically
>>>>=20
>>>> https://issues.apache.org/jira/browse/HBASE-5123 is for adding more
>>>> aggregation functions.
>>>>=20
>>>> Royston:
>>>> Feel free to elaborate on 5213 and explain what Mult aggregate should d=
o.
>>>>=20
>>>> Cheers
>>>>=20
>>>> On Wed, Jan 4, 2012 at 3:43 AM, Royston Sellman <
>>>> royston.sellman@googlemail.com> wrote:
>>>>=20
>>>>> Ted, Himanshu and Gary,
>>>>>=20
>>>>> It works now! I re-created my HBase table to contain Bytes.toBytes(Lon=
g)
>>>>> values and that fixed it.
>>>>>=20
>>>>> For the time being we can convert everything to Longs and work with
>>> that,
>>>>> but we will probably write our own ColumnInterpreters soon for our dat=
a
>>>>> types, so thanks for the pointer to HBASE-4946. There are also
>>> Functions we
>>>>> need (e.g. Median, Weighted Median, Mult) which might best be placed i=
n
>>> the
>>>>> Aggregations Protocol. We'll be sure to discuss this with you when we
>>> start.
>>>>>=20
>>>>> Meanwhile, thanks again for all your help!
>>>>>=20
>>>>> Royston
>>>>>=20
>>>>>=20
>>>>> On 3 Jan 2012, at 18:58, Ted Yu wrote:
>>>>>=20
>>>>>> I like long messages :-) because they provide more clues.
>>>>>>=20
>>>>>> For part 1, you don't have to call Bytes.toxxx as long as the
>>> interpreter
>>>>>> uses method consistent with the way you write values into HBase table=
s.
>>>>>>=20
>>>>>> For part 2, HBASE-4946 is related.
>>>>>> Basically you need to place the jar containing your coprocessor and
>>>>>> interpreter code on hdfs so that you can load it into your HBase tabl=
e.
>>>>>> Look at this for details:
>>>>>> https://issues.apache.org/jira/browse/HBASE-4554
>>>>>>=20
>>>>>> Cheers
>>>>>>=20
>>>>>> On Tue, Jan 3, 2012 at 10:42 AM, Royston Sellman <
>>>>>> royston.sellman@googlemail.com> wrote:
>>>>>>=20
>>>>>>> Hi Ted,
>>>>>>>=20
>>>>>>> PART 1
>>>>>>> =3D=3D=3D=3D=3D
>>>>>>> Thanks for the hint. I think maybe you have given me some inspiratio=
n!
>>>>>>>=20
>>>>>>> It looks like getValue will return null if the table value is not th=
e
>>>>>>> length
>>>>>>> of a long. When we created our table (batch loading CSVs using the
>>>>>>> SampleUploader example) we simply have this as our put():
>>>>>>> put.add(family, Bytes.toBytes("advanceKWh"), advanceKWh);
>>>>>>> [note we do no Bytes.toxxx casts to the advanceKWh value. The values=

>>>>> look
>>>>>>> OK
>>>>>>> from HBase shell though :-)]
>>>>>>>=20
>>>>>>> but I looked at TestAggregateProtocol.java again and I see there put=
s
>>>>> like:
>>>>>>> p2.add(TEST_FAMILY, Bytes.add(TEST_MULTI_CQ, Bytes.toBytes(l)),
>>>>>>> Bytes.toBytes(l * 10));
>>>>>>>=20
>>>>>>> So my hypothesis is that we need to do something like:
>>>>>>> Long l =3D new Long(1);
>>>>>>> put.add(family, Bytes.toBytes("advanceKWh"), Bytes.toBytes(l *
>>>>>>> advanceKWh));
>>>>>>> when we create the table.
>>>>>>>=20
>>>>>>> Do you think my hypothesis is correct? Did we build our table
>>>>> incorrectly
>>>>>>> for reading longs from it?
>>>>>>>=20
>>>>>>> PART 2
>>>>>>> =3D=3D=3D=3D=3D
>>>>>>> Anyway we will obviously need to make our own interpreters. but we
>>>>> failed
>>>>>>> at
>>>>>>> this task so far:
>>>>>>> In order to implement our own ColumnInterpretter, we first attempted=

>>>>> simply
>>>>>>> extending the LongColumnInterpreter and passing that as a parameter t=
o
>>>>>>> aClient.sum().
>>>>>>> import
>>> org.apache.hadoop.hbase.client.coprocessor.LongColumnInterpreter;
>>>>>>>=20
>>>>>>> public class LCI extends LongColumnInterpreter {
>>>>>>> public Long add(Long a, Long b) {
>>>>>>> System.out.print("LCI.add("+a+","+b+")");
>>>>>>> Long c =3D super.add(a,b);
>>>>>>> System.out.println(" -> "+c);
>>>>>>> return c;
>>>>>>> }
>>>>>>> };
>>>>>>> ....
>>>>>>> final ColumnInterpreter<Long, Long> ci =3D new LCI();
>>>>>>> long sum=3D-1;
>>>>>>> try {
>>>>>>> sum =3D aClient.sum(EDRP_TABLE, ci, scan);
>>>>>>> ...
>>>>>>> However, we received class not found errors in our regionserver log
>>>>> when we
>>>>>>> ran this code
>>>>>>> Clearly we are missing something
>>>>>>> We've started looking at modifying the HBase source and rebuilding
>>> that
>>>>>>> But I think this is not the way we should be working
>>>>>>> It seems that we should be implementing the ColumnInterpreter
>>> interface
>>>>> and
>>>>>>> passing an instance of that implementation class too as a parameter.=

>>> Is
>>>>>>> this correct?  Could you provide us with an example?  Any ideas why w=
e
>>>>>>> might
>>>>>>> be getting class not found exception from the regionserver?
>>>>>>>=20
>>>>>>> Sorry for the long message!
>>>>>>>=20
>>>>>>> Royston (and Tom)
>>>>>>>=20
>>>>>>>=20
>>>>>>> -----Original Message-----
>>>>>>> From: Ted Yu [mailto:yuzhihong@gmail.com]
>>>>>>> Sent: 03 January 2012 18:00
>>>>>>> To: user@hbase.apache.org
>>>>>>> Subject: Re: AggregateProtocol Help
>>>>>>>=20
>>>>>>> My previous email might not be hitting the root cause.
>>>>>>> I think the following method in LCI may be giving you the null:
>>>>>>>=20
>>>>>>> public Long getValue(byte[] colFamily, byte[] colQualifier, KeyValue=

>>> kv)
>>>>>>>  throws IOException {
>>>>>>> if (kv =3D=3D null || kv.getValueLength() !=3D Bytes.SIZEOF_LONG)
>>>>>>>  return null;
>>>>>>> return Bytes.toLong(kv.getBuffer(), kv.getValueOffset());
>>>>>>>=20
>>>>>>> Look at the if statement above carefully.
>>>>>>> If it doesn't match how you store values in HBase, feel free to
>>> subclass
>>>>>>> LongColumnInterpreter and provide the correct interpretation.
>>>>>>>=20
>>>>>>> BTW you don't need to restart cluster just because you need to use
>>> your
>>>>> own
>>>>>>> interpreter :-)
>>>>>>>=20
>>>>>>> On Tue, Jan 3, 2012 at 9:48 AM, Royston Sellman <
>>>>>>> royston.sellman@googlemail.com> wrote:
>>>>>>>=20
>>>>>>>> Hi Ted,
>>>>>>>>=20
>>>>>>>> Here is the output. As you can see aClient is not nul:
>>>>>>>>=20
>>>>>>>> AggregationClient aClient =3D new AggregationClient(conf);
>>>>>>>> System.err.println("aClient: "+aClient);
>>>>>>>>=20
>>>>>>>> <<<    aClient:
>>>>>>>> org.apache.hadoop.hbase.client.coprocessor.AggregationClient@28787c=
16
>>>>>>>>=20
>>>>>>>> It will take us a little while to add log code to LCI... we have to=

>>>>>>>> edit the source, rebuild 0.92, redistribute round our cluster,
>>> restart
>>>>>>>> ;) We'll get back to you when this is done.
>>>>>>>>=20
>>>>>>>> Royston
>>>>>>>>=20
>>>>>>>> -----Original Message-----
>>>>>>>> From: Ted Yu [mailto:yuzhihong@gmail.com]
>>>>>>>> Sent: 03 January 2012 17:10
>>>>>>>> To: user@hbase.apache.org
>>>>>>>> Subject: Re: AggregateProtocol Help
>>>>>>>>=20
>>>>>>>> Royston:
>>>>>>>> Thanks for your effort trying to hunt down the problem.
>>>>>>>>=20
>>>>>>>> Can you add a log after this line to see if aClient is null ?
>>>>>>>>           AggregationClient aClient =3D new AggregationClient(conf)=
;
>>>>>>>>=20
>>>>>>>> I was looking at LongColumnInterpreter.add() which is called by
>>>>>>>> aClient.sum()
>>>>>>>> Can you add a few log statements in LongColumnInterpreter.add() to
>>> see
>>>>>>>> what parameters are passed to it ?
>>>>>>>>=20
>>>>>>>> Cheers
>>>>>>>>=20
>>>>>>>> On Tue, Jan 3, 2012 at 8:32 AM, Royston Sellman <
>>>>>>>> royston.sellman@googlemail.com> wrote:
>>>>>>>>=20
>>>>>>>>> Hi Ted, Himanshu, Gary,
>>>>>>>>>=20
>>>>>>>>> Thanks again for your attention. I experimented with a shorter tab=
le
>>>>>>>>> and it looks like the timeout error was spurious...
>>>>>>>>>=20
>>>>>>>>> With the shorter table I now get an NPE when I call
>>>>>>>>> AggregationClient.sum().
>>>>>>>>> Here's the code snippet:
>>>>>>>>>=20
>>>>>>>>>            // Test the table
>>>>>>>>>            HTable table =3D new HTable(EDRP_TABLE);
>>>>>>>>>            Get get =3D new Get(Bytes.toBytes("row-aa"));
>>>>>>>>>            get.addColumn(Bytes.toBytes("EDRP"),
>>>>>>>>> Bytes.toBytes("advanceKWh"));
>>>>>>>>>            Result result =3D table.get(get);
>>>>>>>>>            byte [] val =3D result.getValue(Bytes.toBytes("EDRP"),
>>>>>>>>> Bytes.toBytes("advanceKWh"));
>>>>>>>>>            System.out.println("Row aa =3D " +
>>>>>>>>> Bytes.toString(val));
>>>>>>>>>=20
>>>>>>>>>            AggregationClient aClient =3D new
>>> AggregationClient(conf);
>>>>>>>>>            Scan scan =3D new Scan();
>>>>>>>>>             scan.addColumn(EDRP_FAMILY, EDRP_QUALIFIER);
>>>>>>>>>            scan.setStartRow(Bytes.toBytes("row-ab"));
>>>>>>>>>            scan.setStopRow(Bytes.toBytes("row-az"));
>>>>>>>>>            System.out.println(Bytes.toString(EDRP_FAMILY) + ":"
>>>>>>>>> + Bytes.toString(EDRP_QUALIFIER));
>>>>>>>>>             final ColumnInterpreter<Long, Long> ci =3D new
>>>>>>>>> LongColumnInterpreter();
>>>>>>>>>             long sum=3D-1;
>>>>>>>>>            try {
>>>>>>>>>                    sum =3D aClient.sum(EDRP_TABLE, ci, scan);
>>>>>>>>>            } catch (Throwable e) {
>>>>>>>>>                    // TODO Auto-generated catch block
>>>>>>>>>                    e.printStackTrace();
>>>>>>>>>            }
>>>>>>>>>            System.out.println(sum);
>>>>>>>>>=20
>>>>>>>>> The first part is just to check that my table is OK. It prints the=

>>>>>>>>> correct value for row aa. Then I check CF:CQ is correct. Then I ge=
t
>>>>>>>>> the -1 that sum was defined as. Then the NPE. Here is the run
>>> output:
>>>>>>>>>=20
>>>>>>>>> Row aa =3D 3.0
>>>>>>>>> EDRP:advanceKWh
>>>>>>>>> -1
>>>>>>>>> java.lang.NullPointerException
>>>>>>>>> at EDRPAggregator.testSumWithValidRange(EDRPAggregator.java:66)
>>>>>>>>> at EDRPAggregator.main(EDRPAggregator.java:96)
>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at=

>>>>>>>>>=20
>>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorIm=
pl
>>>>>>>>> .j
>>>>>>>>> ava:39
>>>>>>>>> )
>>>>>>>>> at
>>>>>>>>>=20
>>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAc=
ce
>>>>>>>>> ss
>>>>>>>>> orImpl
>>>>>>>>> .java:25)
>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>>>>>=20
>>>>>>>>> Line 66 is the line:
>>>>>>>>>                    sum =3D aClient.sum(EDRP_TABLE, ci, scan);
>>>>>>>>>=20
>>>>>>>>> But I'm not sure that is really the line generating the NPE becaus=
e
>>>>>>>>> I get the -1 output (same result when line 66 is not wrapped in
>>>>>>>>> try-catch
>>>>>>>> BTW).
>>>>>>>>> Anyway, -1 is clearly not the correct value for the sum.
>>>>>>>>>=20
>>>>>>>>> [Note: I'm setting a start and end row on the scan because I saw a=

>>>>>>>>> comment in AggregationClient.java saying they must be set. However=
,
>>>>>>>>> I think this comment is obsolete as the test code in
>>>>>>>>> TestAggregateProtocol.java sometimes does not include start & end
>>>>>>>>> rows. The run is exactly the same without the start & stop rows ]
>>>>>>>>>=20
>>>>>>>>> We are struggling with Aggregations not least because
>>>>>>>>> TestAggregateProtocol.java is the only source of example code we
>>>>>>>>> could find and it is not ideal because it is designed to work with=

>>>>>>>>> MiniCluster rather than a real cluster like ours. Is there any oth=
er
>>>>>>>>> example code in existence?
>>>>>>>>> I think that would really help us.
>>>>>>>>>=20
>>>>>>>>> Meanwhile, thanks for your patience with our questions. We are
>>>>>>>>> really impressed with the AggregationProtocol
>>>>>>>>> architecture/functionality and very keen to use it.
>>>>>>>>>=20
>>>>>>>>> Royston (and Tom)
>>>>>>>>>=20
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Gary Helmling [mailto:ghelmling@gmail.com]
>>>>>>>>> Sent: 02 January 2012 06:23
>>>>>>>>> To: user@hbase.apache.org
>>>>>>>>> Subject: Re: AggregateProtocol Help
>>>>>>>>>=20
>>>>>>>>> Hi Royston,
>>>>>>>>>=20
>>>>>>>>> Try increasing the value set for hbase.rpc.timeout (by default 60
>>>>>>>> seconds).
>>>>>>>>> Add something like this to hbase-site.xml:
>>>>>>>>>=20
>>>>>>>>> <property>
>>>>>>>>> <name>hbase.rpc.timeout</name>
>>>>>>>>> <value>180000</value>
>>>>>>>>> </property>
>>>>>>>>>=20
>>>>>>>>> This would increase the timeout value to 3 minutes, for example.
>>>>>>>>>=20
>>>>>>>>> But as Ted and Himanshu mention, 557k rows is not a large table an=
d
>>>>>>>>> should not be taking very long to process, unless these happen to b=
e
>>>>>>>>> very, very wide rows.  It sounds like there could be something els=
e
>>>>>>>>> going on.  Does debug level logging show anything else in the regi=
on
>>>>>>>> server log?
>>>>>>>>>=20
>>>>>>>>> --gh
>>>>>>>>>=20
>>>>>>>>> On Sun, Jan 1, 2012 at 5:53 PM, Ted Yu <yuzhihong@gmail.com> wrote=
:
>>>>>>>>>> Thanks for the reminder Himanshu.
>>>>>>>>>>=20
>>>>>>>>>> Royston:
>>>>>>>>>> =46rom this blog you can get some history on this subject:
>>>>>>>>>> http://zhihongyu.blogspot.com/2011/03/genericizing-endpointcoproc=
e
>>>>>>>>>> ss
>>>>>>>>>> or
>>>>>>>>>> .html
>>>>>>>>>>=20
>>>>>>>>>>=20
>>>>>>>>>> On Sun, Jan 1, 2012 at 5:18 PM, Himanshu Vashishtha
>>>>>>>>>> <hvashish@cs.ualberta.ca
>>>>>>>>>>> wrote:
>>>>>>>>>>=20
>>>>>>>>>>> Hello Royston,
>>>>>>>>>>>=20
>>>>>>>>>>> Sorry to hear that you are getting trouble while using
>>>>>>>>>>> Aggregation functionalities.
>>>>>>>>>>>=20
>>>>>>>>>>> 557k rows seems to be a small table and a SocketTimeout does not=

>>>>>>>>>>> seem to be an ok response.
>>>>>>>>>>> It will be good to know the region distribution as such. (how
>>>>>>>>>>> many
>>>>>>>>> regions?
>>>>>>>>>>> Is it a full table scan?)
>>>>>>>>>>>=20
>>>>>>>>>>> You are using the sum function; how are you using the
>>>>>>>> ColumnInterpreter.
>>>>>>>>>>> Can you enable the log level to debug to see why the RS is takin=
g
>>>>>>>>>>> that long to respond (more than 113 sec).
>>>>>>>>>>> The 0 return value is the default result.
>>>>>>>>>>>=20
>>>>>>>>>>> Thanks for trying this out.
>>>>>>>>>>>=20
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Himanshu
>>>>>>>>>>>=20
>>>>>>>>>>> On Sun, Jan 1, 2012 at 12:26 PM, Royston Sellman <
>>>>>>>>>>> royston.sellman@googlemail.com> wrote:
>>>>>>>>>>>=20
>>>>>>>>>>>> Hi Ted,
>>>>>>>>>>>>=20
>>>>>>>>>>>> I think 0 is the only value we ever see (I'll check tomorrow:
>>>>>>>>>>>> the server is down right now). Our table has 557,000 rows. I'll=

>>>>>>>>>>>> try a much shorter table tomorrow.
>>>>>>>>>>>>=20
>>>>>>>>>>>> Yes, we have RS running on the NN, but it's a test cluster and
>>>>>>>>>>>> we are
>>>>>>>>>>> used
>>>>>>>>>>>> to it :)
>>>>>>>>>>>>=20
>>>>>>>>>>>> Do you think using AggregationProtocol is the best strategy for=

>>>>>>>>>>>> the case where we want to use basic SQL-style functions like
>>>>>>>>>>>> SUM, AVG, STD, MIN, MAX? Do you think there is a better strateg=
y?
>>>>>>>>>>>>=20
>>>>>>>>>>>> Many thanks,
>>>>>>>>>>>> Royston
>>>>>>>>>>>>=20
>>>>>>>>>>>>=20
>>>>>>>>>>>> On 1 Jan 2012, at 17:58, Ted Yu wrote:
>>>>>>>>>>>>=20
>>>>>>>>>>>>> Royston:
>>>>>>>>>>>>> Happy New Year to you too.
>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>> java.net.SocketTimeoutException: Call to
>>>>>>>>>>>>>>> namenode/10.0.0.235:60020
>>>>>>>>>>> failed
>>>>>>>>>>>> on
>>>>>>>>>>>>>=20
>>>>>>>>>>>>> It seems the namenode above actually refers to a region server=
.
>>>>>>>>>>>>> This
>>>>>>>>>>> is a
>>>>>>>>>>>>> little bit confusing :-)
>>>>>>>>>>>>>=20
>>>>>>>>>>>>> The sum value below is 0.
>>>>>>>>>>>>> Have you ever seen a value greater than 0 ?
>>>>>>>>>>>>>=20
>>>>>>>>>>>>> How many rows are there in this CF:CQ ?
>>>>>>>>>>>>> The timeout was reported earlier by other people where
>>>>>>>>>>>>> there're many
>>>>>>>>>>> rows
>>>>>>>>>>>>> in the table.
>>>>>>>>>>>>>=20
>>>>>>>>>>>>> There is a JIRA to provide streaming support for coprocessor
>>>>>>>>>>>>> but the development there has stalled.
>>>>>>>>>>>>>=20
>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>=20
>>>>>>>>>>>>> On Sun, Jan 1, 2012 at 9:35 AM, Royston Sellman <
>>>>>>>>>>>>> royston.sellman@googlemail.com> wrote:
>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> Hi Gary and Ted,
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> Royston (Tom's colleague) here. Back onto this after the
>>>>>>>>>>>>>> Christmas/New
>>>>>>>>>>>> Year
>>>>>>>>>>>>>> break.
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> Many thanks for your help so far. We enabled our database
>>>>>>>>>>>>>> via your hbase-site.xml mod and were able to move on. to
>>>>>>>>>>>>>> other errors. But I
>>>>>>>>>>>> think
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>> are now actually getting an aggregation partially calculated
>>>>>>>>>>>>>> on our
>>>>>>>>>>>> table
>>>>>>>>>>>>>> (this feels like progress). The details:
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> On running our client we now get this exception:
>>>>>>>>>>>>>> 11/12/31 17:51:09 WARN
>>>>>>>>>>>>>> client.HConnectionManager$HConnectionImplementation: Error
>>>>>>>>>>>>>> executing
>>>>>>>>>>> for
>>>>>>>>>>>>>> row
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> java.util.concurrent.ExecutionException:
>>>>>>>>>>>>>> org.apache.hadoop.hbase.client.RetriesExhaustedException:
>>>>>>>>>>>>>> Failed after attempts=3D10, exceptions:
>>>>>>>>>>>>>> Sat Dec 31 17:41:30 GMT 2011,
>>>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1fc4f0f8,
>>>>>>>>>>>>>> java.net.SocketTimeoutException: Call to
>>>>>>>>>>>>>> namenode/10.0.0.235:60020
>>>>>>>>>>> failed
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>> socket timeout exception: java.net.SocketTimeoutException:
>>>>>>>>>>>>>> 60000
>>>>>>>>>>> millis
>>>>>>>>>>>>>> timeout while waiting for channel to be ready for read. ch :
>>>>>>>>>>>>>> java.nio.channels.SocketChannel[connected
>>>>>>>>>>>>>> local=3D/10.0.0.235:59999 remote=3Dnamenode/10.0.0.235:60020]=

>>>>>>>>>>>>>> (8 more of these, making for 10 tries) Sat Dec 31 17:51:09
>>>>>>>>>>>>>> GMT 2011,
>>>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1fc4f0f8,
>>>>>>>>>>>>>> java.net.SocketTimeoutException: Call to
>>>>>>>>>>>>>> namenode/10.0.0.235:60020
>>>>>>>>>>> failed
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>> socket timeout exception: java.net.SocketTimeoutException:
>>>>>>>>>>>>>> 60000
>>>>>>>>>>> millis
>>>>>>>>>>>>>> timeout while waiting for channel to be ready for read. ch :
>>>>>>>>>>>>>> java.nio.channels.SocketChannel[connected
>>>>>>>>>>>>>> local=3D/10.0.0.235:59364 remote=3Dnamenode/10.0.0.235:60020]=

>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>   at
>>>>>>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java=
:
>>>>>>>>>>>>>> 22
>>>>>>>>>>>>>> 2)
>>>>>>>>>>>>>>   at
>>>>>>> java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>>>>>>>>>>>>>   at
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>=20
>>>>>>>>>>>=20
>>>>>>>>>=20
>>>>>>>>>=20
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>=20
>>>>>>>=20
>>>>>=20
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementat=
ion.
>>>>>>>>>>>>>> processExecs(HConnectionManager.java:1465)
>>>>>>>>>>>>>>   at
>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.client.HTable.coprocessorExec(HTable.jav=
a
>>>>>>>>>>> :1
>>>>>>>>>>> 55
>>>>>>>>>>> 5)
>>>>>>>>>>>>>>   at
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.client.coprocessor.AggregationClient.sum=
(
>>>>>>>>>>> Ag
>>>>>>>>>>> gr
>>>>>>>>>>> egation
>>>>>>>>>>>>>> Client.java:229)
>>>>>>>>>>>>>>   at
>>>>>>>>>>>>>> EDRPAggregator.testSumWithValidRange(EDRPAggregator.java:51)
>>>>>>>>>>>>>>   at EDRPAggregator.main(EDRPAggregator.java:77)
>>>>>>>>>>>>>>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>>>>>>>>>>>> Method)
>>>>>>>>>>>>>>   at
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>=20
>>>>>>>>>>>=20
>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl=
.
>>>>>>>>>>> java:39
>>>>>>>>>>>>>> )
>>>>>>>>>>>>>>   at
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>=20
>>>>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethod=
A
>>>>>>>>>>> cc
>>>>>>>>>>> es
>>>>>>>>>>> sorImpl
>>>>>>>>>>>>>> .java:25)
>>>>>>>>>>>>>>   at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>>>>>>>>>   at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> Looking at the log (.regionserver-namenode.log) I see this
>>>>>>>>>>>>>> debug
>>>>>>>>>>>> message:
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> 2011-12-31 17:42:23,472 DEBUG
>>>>>>>>>>>>>> org.apache.hadoop.hbase.coprocessor.AggregateImplementation:
>>>>>>>>>>>>>> Sum from
>>>>>>>>>>>> this
>>>>>>>>>>>>>> region is
>>>>>>>>>>> EDRPTestTbl,,1324485124322.7b9ee0d113db9b24ea9fdde90702d006.:
>>>>>>>>>>>> 0
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> Where the sum value looks reasonable which makes me think
>>>>>>>>>>>>>> the sum of a CF:CQ worked. But I never see this value on
>>>>>>> stdout.
>>>>>>>>>>>>>> Then I see this warning:
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> 2011-12-31 17:42:23,476 WARN org.apache.hadoop.ipc.HBaseServe=
r:
>>>>>>>>>>>>>> (responseTooSlow):
>>>>>>>>>>>>>> {"processingtimems":113146,"call":"execCoprocess$
>>>>>>>>>>>>>> 2011-12-31 17:42:23,511 WARN org.apache.hadoop.ipc.HBaseServe=
r:
>>>>>>>>>>>>>> IPC
>>>>>>>>>>>> Server
>>>>>>>>>>>>>> Responder, call execCoprocessor([B@4b22fad6, getSum(org.$
>>>>>>>>>>>>>> 2011-12-31 17:42:23,515 WARN org.apache.hadoop.ipc.HBaseServe=
r:
>>>>>>>>>>>>>> IPC
>>>>>>>>>>>> Server
>>>>>>>>>>>>>> handler 1 on 60020 caught: java.nio.channels.ClosedChann$
>>>>>>>>>>>>>>   at
>>>>>>>>>>>>>>=20
>>>>>>>>>>> sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.j=
a
>>>>>>>>>>> va
>>>>>>>>>>> :1
>>>>>>>>>>> 33)
>>>>>>>>>>>>>>   at
>>>>>>>>>>> sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>>>>>>>>>>>>>>   at
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer=
.
>>>>>>>>>>> ja
>>>>>>>>>>> va
>>>>>>>>>>> :1651)
>>>>>>>>>>>>>>   at
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processRespons=
e
>>>>>>>>>>> (H
>>>>>>>>>>> Ba
>>>>>>>>>>> seServe
>>>>>>>>>>>>>> r.java:924)
>>>>>>>>>>>>>>   at
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBas=
e
>>>>>>>>>>> Se
>>>>>>>>>>> rv
>>>>>>>>>>> er.java
>>>>>>>>>>>>>> :1003)
>>>>>>>>>>>>>>   at
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady=
(
>>>>>>>>>>> HB
>>>>>>>>>>> as
>>>>>>>>>>> eServer
>>>>>>>>>>>>>> .java:409)
>>>>>>>>>>>>>>   at
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>=20
>>>>>>>>>>>=20
>>>>>>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java=
:
>>>>>>>>>>> 1345)
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> Have we missed out some step in the HBase/RegionServerconfig?=

>>>>>>>>>>>>>> Or is
>>>>>>>>>>> our
>>>>>>>>>>>>>> client code still deficient?
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> Can you offer any suggestions? Is there any example code for
>>>>>>>>>>>>>> the new Aggregations stuff.
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> Thanks and Happy New Year to you guys,
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> Royston (and Tom).
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> (HBase 0.92, Hadoop 1.0)
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>> From: Gary Helmling [mailto:ghelmling@gmail.com]
>>>>>>>>>>>>>> Sent: 23 December 2011 18:06
>>>>>>>>>>>>>> To: user@hbase.apache.org
>>>>>>>>>>>>>> Subject: Re: AggregateProtocol Help
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> Hi Tom,
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> The test code is not really the best guide for configuration.=

>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> To enable the AggregateProtocol on all of your tables, add
>>>>>>>>>>>>>> this to the hbase-site.xml for the servers in your cluster:
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> <property>
>>>>>>>>>>>>>> <name>hbase.coprocessor.user.region.classes</name>
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>=20
>>>>>>>>>>> <value>org.apache.hadoop.hbase.coprocessor.AggregateImplementati=
o
>>>>>>>>>>> n<
>>>>>>>>>>> /v
>>>>>>>>>>> alue>
>>>>>>>>>>>>>> </property>
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> If you only want to use the aggregate functions on a
>>>>>>>>>>>>>> specific table
>>>>>>>>>>> (or
>>>>>>>>>>>>>> tables), then you can enable that individually for the table
>>>>>>>>>>>>>> from the
>>>>>>>>>>>>>> shell:
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> 1) disable the table
>>>>>>>>>>>>>> hbase> disable 'EDRP7'
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> 2) add the coprocessor
>>>>>>>>>>>>>> hbase> alter 'EDRP7', METHOD =3D> 'table_att',
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>=20
>>>>>>>>>>> 'coprocessor'=3D>'|org.apache.hadoop.hbase.coprocessor.Aggregate=
Imp
>>>>>>>>>>> le
>>>>>>>>>>> me
>>>>>>>>>>> ntation
>>>>>>>>>>>>>> ||'
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> (Note that the pipes in the value string are required)
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> 3) re-enable the table
>>>>>>>>>>>>>> hbase> enable 'EDRP7'
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> Either way should work.  With the second approach you will
>>>>>>>>>>>>>> see the coprocessor listed when you describe the table from
>>>>>>>>>>>>>> the shell, as Ted mentioned.  With the first approach you
>>>>>>>>>>>>>> will not, but it should be
>>>>>>>>>>>> loaded
>>>>>>>>>>>>>> all the same.
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> --gh
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>> On Fri, Dec 23, 2011 at 7:04 AM, Ted Yu
>>>>>>>>>>>>>> <yuzhihong@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>>>>>> I don't know why you chose HBaseTestingUtility to create
>>>>>>>>>>>>>>> the
>>>>>>>>> table.
>>>>>>>>>>>>>>> I guess you followed test code example.
>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>> At least you should pass the conf to this ctor:
>>>>>>>>>>>>>>> public HBaseTestingUtility(Configuration conf) {
>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>> If coprocessor was installed correctly, you should see
>>>>>>>>>>>>>>> something like(from
>>>>>>>>>>>>>>> HBASE-5070):
>>>>>>>>>>>>>>> coprocessor$1 =3D>
>>>>>>>>>>>>>>>=20
>>>>>>>>>=20
>>> '|org.apache.hadoop.hbase.constraint.ConstraintProcessor|1073741823|'
>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>> On Fri, Dec 23, 2011 at 3:02 AM, Tom Wilcox
>>>>>>>>>>>>>>> <Tom.Wilcox@cse.org.uk>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>> I am not sure how we load the AggregateImplementation into
>>>>>>>>>>>>>>>> the
>>>>>>>>>>> table.
>>>>>>>>>>>>>>>> When we are creating a table, we use the same functions as
>>>>>>>>>>>>>>>> the test
>>>>>>>>>>> as
>>>>>>>>>>>>>> follows...
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>> ...
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>> conf.set(CoprocessorHost.REGION_COPROCESSOR_CONF_KEY,
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>>=20
>>>>>>> "org.apache.hadoop.hbase.coprocessor.AggregateImplementation"
>>>>>>>>>>>>>>>>> );
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>>         // Utility.CreateHBaseTable(conf,
>>>>>>>>>>>>>>>>> otherArgs[1],
>>>>>>>>>>>>>>>> otherArgs[2],
>>>>>>>>>>>>>>>>> true);
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>>         HBaseTestingUtility util =3D new
>>>>>>>>>>>>>>>>> HBaseTestingUtility();
>>>>>>>>>>>>>>>>>         HTable table =3D util.createTable(EDRP_TABLE,
>>>>>>>>>>>>>>>>> EDRP_FAMILY);
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>>         AggregationClient aClient =3D new
>>>>>>>>>>>>>>>>> AggregationClient(conf);
>>>>>>>>>>>>>>>> ...
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>> Running DESCRIBE on a table produced shows the following
>>>>>>>> output:
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>> hbase(main):002:0> describe 'EDRP7'
>>>>>>>>>>>>>>>> DESCRIPTION
>>>>>>>>>>>>>>>>                                       ENABLED {NAME =3D>
>>>>>>>>>>>>>>>> 'EDRP7', FAMILIES =3D> [{NAME =3D> 'advanceKWh', BLOOMFILTE=
R
>>>>>>>>>>>>>>>> =3D> 'NONE', REPLICATION_SCOPE =3D> '0', VERSIONS =3D>  tru=
e
>>>>>>>>>>>>>>>> '3', COMPRESSION =3D> 'NONE', MIN_VERSIONS =3D> '0', TTL =3D=
>
>>>>>>>>>>>>>>>> '2147483647', BLOCKSIZE =3D> '65536', IN_MEMORY =3D> 'false=
',
>>>>>>>>>>>>>>>> B LOCKCACHE =3D> 'true'}]}
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>> We are using the tip of 0.92 (cloned from the Git repo).
>>>>>>>>>>>>>>>> See the version string below:
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>> hbase(main):005:0> version 0.92.0, r1208286, Thu Dec 15
>>>>>>>>>>>>>>>> 13:16:03 GMT 2011
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>> We would really appreciate an example of how to create a
>>>>>>>>>>>>>>>> table that is enabled to handle Aggregation).
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>>>>>> From: Ted Yu [yuzhihong@gmail.com]
>>>>>>>>>>>>>>>> Sent: 22 December 2011 17:03
>>>>>>>>>>>>>>>> To: user@hbase.apache.org
>>>>>>>>>>>>>>>> Subject: Re: AggregateProtocol Help
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>> Have you loaded AggregateImplementation into your table ?
>>>>>>>>>>>>>>>> Can you show us the contents of the following command in
>>>>>>>>>>>>>>>> hbase
>>>>>>>>>>> shell:
>>>>>>>>>>>>>>>> describe 'your-table'
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>> BTW are you using the tip of 0.92 ?
>>>>>>>>>>>>>>>> HBASE-4946 would be of help for dynamically loaded
>>>>>>>>>>>>>>>> coprocessors
>>>>>>>>>>> which
>>>>>>>>>>>>>>>> you might use in the future.
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>> On Thu, Dec 22, 2011 at 8:09 AM, Tom Wilcox
>>>>>>>>>>>>>>>> <Tom.Wilcox@cse.org.uk>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>> We are trying to use the aggregation functionality in
>>>>>>>>>>>>>>>>> HBase
>>>>>>>>>>>>>>>>> 0.92 and we have managed to get the test code working
>>>>>>>>>>>>>>>>> using the
>>>>>>>>>>> following
>>>>>>>>>>>>>> command:
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>> java -classpath junit-4.10.jar:build/*:$HBASELIBS/*
>>>>>>>>>>>>>>>>> org.junit.runner.JUnitCore
>>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>> Closer inspection of this test class has revealed that it
>>>>>>>>>>>>>>>>> uses a mini DFS cluster to populate and run the tests.
>>>>>>>>>>>>>>>>> These tests return
>>>>>>>>>>>>>> successfully.
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>> However, when we attempt to run similar code on our
>>>>>>>>>>>>>>>>> development HDFS cluster we experience the following error=
:
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>> 11/12/22 15:46:28 WARN
>>>>>>>>>>>>>>>>> client.HConnectionManager$HConnectionImplementation:
>>>>>>>>>>>>>>>>> Error executing for
>>>>>>>>>>>>>>>> row
>>>>>>>>>>>>>>>>> java.util.concurrent.ExecutionException:
>>>>>>>>>>>>>>>>>=20
>>>>>>> org.apache.hadoop.hbase.ipc.HBaseRPC$UnknownProtocolException:
>>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.HBaseRPC$UnknownProtocolExcep
>>>>>>>>>>>>>>>>> ti
>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>> : No
>>>>>>>>>>>>>>>> matching
>>>>>>>>>>>>>>>>> handler for protocol
>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.coprocessor.AggregateProtocol
>>>>>>>>>>>>>>>>> in region
>>>>>>>>>>>> EDRPTestTbl,,1324485124322.7b9ee0d113db9b24ea9fdde90702d006.
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.regionserver.HRegion.exec(HRegion
>>>>>>>>>>>>>>>>> .j
>>>>>>>>>>>>>>>>> av
>>>>>>>>>>>>>>>>> a:4010
>>>>>>>>>>>>>>>>> )
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.execCoprocess=
o
>>>>>>>>>>> r(
>>>>>>>>>>> HR
>>>>>>>>>>>>>>>> egionServer.java:3040)
>>>>>>>>>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>>>>>>>>>>>>>> Method)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>=20
>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl=
.
>>>>>>>>>>>>>>>> java:39)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethod=
A
>>>>>>>>>>> cc
>>>>>>>>>>> es
>>>>>>>>>>>>>>>> sorImpl.java:25)
>>>>>>>>>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(Writab=
l
>>>>>>>>>>> eR
>>>>>>>>>>> pc
>>>>>>>>>>>>>>>> Engine.java:364)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>=20
>>>>>>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java=
:
>>>>>>>>>>>>>>>> 1325)
>>>>>>>>>>>>>>>>> [sshexec]
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.
>>>>>>>>>>>>>>>>> ja
>>>>>>>>>>>>>>>>> va
>>>>>>>>>>>>>>>>> :222)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImp=
l
>>>>>>>>>>> em
>>>>>>>>>>> en
>>>>>>>>>>>>>>>> tation.processExecs(HConnectionManager.java:1465)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>> org.apache.hadoop.hbase.client.HTable.coprocessorExec(HTable.
>>>>>>>>>>>>>>>>> java:1
>>>>>>>>>>>>>>>>> 555)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.client.coprocessor.AggregationClient.sum=
(
>>>>>>>>>>> Ag
>>>>>>>>>>> gr
>>>>>>>>>>>>>>>> egationClient.java:229)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> EDRPAggregator.testSumWithValidRange(EDRPAggregator.java:5=
1)
>>>>>>>>>>>>>>>>> at EDRPAggregator.main(EDRPAggregator.java:77)
>>>>>>>>>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>>>>>>>>>>>>>> Method)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>=20
>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl=
.
>>>>>>>>>>>>>>>> java:39)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethod=
A
>>>>>>>>>>> cc
>>>>>>>>>>> es
>>>>>>>>>>>>>>>> sorImpl.java:25)
>>>>>>>>>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>>>>>>>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>>>>>>>>>>>>> Caused by:
>>>>>>>>>>>>>>>>>=20
>>>>>>> org.apache.hadoop.hbase.ipc.HBaseRPC$UnknownProtocolException:
>>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.HBaseRPC$UnknownProtocolExcep
>>>>>>>>>>>>>>>>> ti
>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>> : No
>>>>>>>>>>>>>>>> matching
>>>>>>>>>>>>>>>>> handler for protocol
>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.coprocessor.AggregateProtocol
>>>>>>>>>>>>>>>>> in region
>>>>>>>>>>>> EDRPTestTbl,,1324485124322.7b9ee0d113db9b24ea9fdde90702d006.
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.regionserver.HRegion.exec(HRegion
>>>>>>>>>>>>>>>>> .j
>>>>>>>>>>>>>>>>> av
>>>>>>>>>>>>>>>>> a:4010
>>>>>>>>>>>>>>>>> )
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.execCoprocess=
o
>>>>>>>>>>> r(
>>>>>>>>>>> HR
>>>>>>>>>>>>>>>> egionServer.java:3040)
>>>>>>>>>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>>>>>>>>>>>>>> Method)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>=20
>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl=
.
>>>>>>>>>>>>>>>> java:39)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethod=
A
>>>>>>>>>>> cc
>>>>>>>>>>> es
>>>>>>>>>>>>>>>> sorImpl.java:25)
>>>>>>>>>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(Writab=
l
>>>>>>>>>>> eR
>>>>>>>>>>> pc
>>>>>>>>>>>>>>>> Engine.java:364)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>=20
>>>>>>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java=
:
>>>>>>>>>>>>>>>> 1325)
>>>>>>>>>>>>>>>>> [sshexec]
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Na
>>>>>>>>>>>>>>>>> ti
>>>>>>>>>>>>>>>>> ve
>>>>>>>>>>>>>>>>> Method)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeCons=
t
>>>>>>>>>>> ru
>>>>>>>>>>> ct
>>>>>>>>>>>>>>>> orAccessorImpl.java:39)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Delega=
t
>>>>>>>>>>> in
>>>>>>>>>>> gC
>>>>>>>>>>>>>>>> onstructorAccessorImpl.java:27)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> java.lang.reflect.Constructor.newInstance(Constructor.java=
:
>>>>>>>>>>>>>>>>> 51
>>>>>>>>>>>>>>>>> 3)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteExcep=
t
>>>>>>>>>>> io
>>>>>>>>>>> n(
>>>>>>>>>>>>>>>> RemoteExceptionHandler.java:96)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImp=
l
>>>>>>>>>>> em
>>>>>>>>>>> en
>>>>>>>>>>>>>>>> tation.translateException(HConnectionManager.java:1651)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImp=
l
>>>>>>>>>>> em
>>>>>>>>>>> en
>>>>>>>>>>>>>>>> tation.getRegionServerWithRetries(HConnectionManager.java:
>>>>>>>>>>>>>>>> 13
>>>>>>>>>>>>>>>> 27
>>>>>>>>>>>>>>>> )
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker.invoke(ExecRPC
>>>>>>>>>>>>>>>>> In
>>>>>>>>>>>>>>>>> vo
>>>>>>>>>>>>>>>>> ker.ja
>>>>>>>>>>>>>>>>> va:79)
>>>>>>>>>>>>>>>>> at $Proxy3.getSum(Unknown Source)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.client.coprocessor.AggregationClient$4.c=
a
>>>>>>>>>>> ll
>>>>>>>>>>> (A
>>>>>>>>>>>>>>>> ggregationClient.java:233)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.client.coprocessor.AggregationClient$4.c=
a
>>>>>>>>>>> ll
>>>>>>>>>>> (A
>>>>>>>>>>>>>>>> ggregationClient.java:230)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImp=
l
>>>>>>>>>>> em
>>>>>>>>>>> en
>>>>>>>>>>>>>>>> tation$4.call(HConnectionManager.java:1453)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.
>>>>>>>>>>>>>>>>> ja
>>>>>>>>>>>>>>>>> va
>>>>>>>>>>>>>>>>> :303)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoo=
l
>>>>>>>>>>> Ex
>>>>>>>>>>> ec
>>>>>>>>>>>>>>>> utor.java:886)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExe=
c
>>>>>>>>>>> ut
>>>>>>>>>>> or
>>>>>>>>>>>>>>>> .java:908)
>>>>>>>>>>>>>>>>> at java.lang.Thread.run(Thread.java:662)
>>>>>>>>>>>>>>>>>=20
>>>>>>> org.apache.hadoop.hbase.ipc.HBaseRPC$UnknownProtocolException:
>>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.HBaseRPC$UnknownProtocolExcep
>>>>>>>>>>>>>>>>> ti
>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>> : No
>>>>>>>>>>>>>>>> matching
>>>>>>>>>>>>>>>>> handler for protocol
>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.coprocessor.AggregateProtocol
>>>>>>>>>>>>>>>>> in region
>>>>>>>>>>>> EDRPTestTbl,,1324485124322.7b9ee0d113db9b24ea9fdde90702d006.
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.regionserver.HRegion.exec(HRegion
>>>>>>>>>>>>>>>>> .j
>>>>>>>>>>>>>>>>> av
>>>>>>>>>>>>>>>>> a:4010
>>>>>>>>>>>>>>>>> )
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.execCoprocess=
o
>>>>>>>>>>> r(
>>>>>>>>>>> HR
>>>>>>>>>>>>>>>> egionServer.java:3040)
>>>>>>>>>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>>>>>>>>>>>>>> Method)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>=20
>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl=
.
>>>>>>>>>>>>>>>> java:39)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethod=
A
>>>>>>>>>>> cc
>>>>>>>>>>> es
>>>>>>>>>>>>>>>> sorImpl.java:25)
>>>>>>>>>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(Writab=
l
>>>>>>>>>>> eR
>>>>>>>>>>> pc
>>>>>>>>>>>>>>>> Engine.java:364)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>=20
>>>>>>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java=
:
>>>>>>>>>>>>>>>> 1325)
>>>>>>>>>>>>>>>>> [sshexec]
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Na
>>>>>>>>>>>>>>>>> ti
>>>>>>>>>>>>>>>>> ve
>>>>>>>>>>>>>>>>> Method)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeCons=
t
>>>>>>>>>>> ru
>>>>>>>>>>> ct
>>>>>>>>>>>>>>>> orAccessorImpl.java:39)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Delega=
t
>>>>>>>>>>> in
>>>>>>>>>>> gC
>>>>>>>>>>>>>>>> onstructorAccessorImpl.java:27)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> java.lang.reflect.Constructor.newInstance(Constructor.java=
:
>>>>>>>>>>>>>>>>> 51
>>>>>>>>>>>>>>>>> 3)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteExcep=
t
>>>>>>>>>>> io
>>>>>>>>>>> n(
>>>>>>>>>>>>>>>> RemoteExceptionHandler.java:96)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImp=
l
>>>>>>>>>>> em
>>>>>>>>>>> en
>>>>>>>>>>>>>>>> tation.translateException(HConnectionManager.java:1651)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImp=
l
>>>>>>>>>>> em
>>>>>>>>>>> en
>>>>>>>>>>>>>>>> tation.getRegionServerWithRetries(HConnectionManager.java:
>>>>>>>>>>>>>>>> 13
>>>>>>>>>>>>>>>> 27
>>>>>>>>>>>>>>>> )
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker.invoke(ExecRPC
>>>>>>>>>>>>>>>>> In
>>>>>>>>>>>>>>>>> vo
>>>>>>>>>>>>>>>>> ker.ja
>>>>>>>>>>>>>>>>> va:79)
>>>>>>>>>>>>>>>>> at $Proxy3.getSum(Unknown Source)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.client.coprocessor.AggregationClient$4.c=
a
>>>>>>>>>>> ll
>>>>>>>>>>> (A
>>>>>>>>>>>>>>>> ggregationClient.java:233)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.client.coprocessor.AggregationClient$4.c=
a
>>>>>>>>>>> ll
>>>>>>>>>>> (A
>>>>>>>>>>>>>>>> ggregationClient.java:230)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImp=
l
>>>>>>>>>>> em
>>>>>>>>>>> en
>>>>>>>>>>>>>>>> tation$4.call(HConnectionManager.java:1453)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.
>>>>>>>>>>>>>>>>> ja
>>>>>>>>>>>>>>>>> va
>>>>>>>>>>>>>>>>> :303)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoo=
l
>>>>>>>>>>> Ex
>>>>>>>>>>> ec
>>>>>>>>>>>>>>>> utor.java:886)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExe=
c
>>>>>>>>>>> ut
>>>>>>>>>>> or
>>>>>>>>>>>>>>>> .java:908)
>>>>>>>>>>>>>>>>> at java.lang.Thread.run(Thread.java:662)
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>> The source for our class is:
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>> import static org.junit.Assert.assertEquals;
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>> import java.io.IOException;
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>> import org.apache.hadoop.conf.Configuration;
>>>>>>>>>>>>>>>>> import org.apache.hadoop.hbase.HBaseConfiguration;
>>>>>>>>>>>>>>>>> import org.apache.hadoop.hbase.HBaseTestingUtility;
>>>>>>>>>>>>>>>>> import org.apache.hadoop.hbase.HConstants;
>>>>>>>>>>>>>>>>> import org.apache.hadoop.hbase.HTableDescriptor;
>>>>>>>>>>>>>>>>> import org.apache.hadoop.hbase.client.HTable;
>>>>>>>>>>>>>>>>> import org.apache.hadoop.hbase.client.Put;
>>>>>>>>>>>>>>>>> import org.apache.hadoop.hbase.client.Scan;
>>>>>>>>>>>>>>>>> import
>>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.client.coprocessor.AggregationCli
>>>>>>>>>>>>>>>>> en
>>>>>>>>>>>>>>>>> t;
>>>>>>>>>>>>>>>>> import
>>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.client.coprocessor.LongColumnInte
>>>>>>>>>>>>>>>>> rp re ter; import org.apache.hadoop.hbase.util.Bytes;
>>>>>>>>>>>>>>>>> import org.apache.hadoop.util.GenericOptionsParser;
>>>>>>>>>>>>>>>>> import
>>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.coprocessor.ColumnInterpreter;
>>>>>>>>>>>>>>>>> import
>>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.coprocessor.CoprocessorHost;
>>>>>>>>>>>>>>>>> import org.junit.Test;
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>> public class EDRPAggregator {
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>>  // private static final byte[] EDRP_FAMILY =3D
>>>>>>>>>>>>>>>>> Bytes.toBytes("EDRP");
>>>>>>>>>>>>>>>>>  // private static final byte[] EDRP_QUALIFIER =3D
>>>>>>>>>>>>>>>>> Bytes.toBytes("advanceKWh");
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>>  private static byte[] ROW =3D Bytes.toBytes("testRow");
>>>>>>>>>>>>>>>>>  private static final int ROWSIZE =3D 20;
>>>>>>>>>>>>>>>>>  private static byte[][] ROWS =3D makeN(ROW, ROWSIZE);
>>>>>>>>>>>>>>>>>  private static final byte[] TEST_QUALIFIER =3D
>>>>>>>>>>>>>>>>> Bytes.toBytes("TestQualifier");
>>>>>>>>>>>>>>>>>  private static final byte[] TEST_MULTI_CQ =3D
>>>>>>>>>>>>>>>>> Bytes.toBytes("TestMultiCQ");
>>>>>>>>>>>>>>>>>  private static final int rowSeperator1 =3D 5;
>>>>>>>>>>>>>>>>>    private static final int rowSeperator2 =3D 12;
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>>  public static void
>>>>>>>>>>>>>>>>> testSumWithValidRange(Configuration
>>>>>>>>>>>>>>>>> conf,
>>>>>>>>>>>>>>>>>                String[] otherArgs) throws Throwable {
>>>>>>>>>>>>>>>>>         byte[] EDRP_TABLE =3D Bytes.toBytes(otherArgs[1]);=

>>>>>>>>>>>>>>>>>         byte[] EDRP_FAMILY =3D
>>>>>>>>>>>>>>>>> Bytes.toBytes(otherArgs[2]);
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>> conf.set(CoprocessorHost.REGION_COPROCESSOR_CONF_KEY,
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>>=20
>>>>>>> "org.apache.hadoop.hbase.coprocessor.AggregateImplementation"
>>>>>>>>>>>>>>>>> );
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>>         // Utility.CreateHBaseTable(conf,
>>>>>>>>>>>>>>>>> otherArgs[1],
>>>>>>>>>>>>>>>> otherArgs[2],
>>>>>>>>>>>>>>>>> true);
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>>         HBaseTestingUtility util =3D new
>>>>>>>>>>>>>>>>> HBaseTestingUtility();
>>>>>>>>>>>>>>>>>         HTable table =3D util.createTable(EDRP_TABLE,
>>>>>>>>>>>>>>>>> EDRP_FAMILY);
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>>         AggregationClient aClient =3D new
>>>>>>>>>>>>>>>>> AggregationClient(conf);
>>>>>>>>>>>>>>>>>         Scan scan =3D new Scan();
>>>>>>>>>>>>>>>>>         scan.addColumn(EDRP_TABLE, EDRP_FAMILY);
>>>>>>>>>>>>>>>>>         final ColumnInterpreter<Long, Long> ci =3D new
>>>>>>>>>>>>>>>>> LongColumnInterpreter();
>>>>>>>>>>>>>>>>>         long sum =3D
>>>>>>>>>>>>>>>>> aClient.sum(Bytes.toBytes(otherArgs[0]),
>>>>>>>>>>>>>>>>> ci,
>>>>>>>>>>>>>>>> scan);
>>>>>>>>>>>>>>>>>         System.out.println(sum);
>>>>>>>>>>>>>>>>>  }
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>>  /**
>>>>>>>>>>>>>>>>>  * Main entry point.
>>>>>>>>>>>>>>>>>  *
>>>>>>>>>>>>>>>>>   * @param argsThe
>>>>>>>>>>>>>>>>>  *            command line parameters.
>>>>>>>>>>>>>>>>>  * @throws Exception
>>>>>>>>>>>>>>>>>  *             When running the job fails.
>>>>>>>>>>>>>>>>>  */
>>>>>>>>>>>>>>>>>  public static void main(String[] args) throws
>>>>>>>>>>>>>>>>> Exception {
>>>>>>>>>>>>>>>>>         Configuration conf =3D
>>>>>>>>>>>>>>>>> HBaseConfiguration.create();
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>>         String[] otherArgs =3D new
>>>>>>>>>>>>>>>>> GenericOptionsParser(conf,
>>>>>>>>>>>>>>>>> args)
>>>>>>>>>>>>>>>>>                      .getRemainingArgs();
>>>>>>>>>>>>>>>>>         if (otherArgs.length !=3D 3) {
>>>>>>>>>>>>>>>>>                System.err
>>>>>>>>>>>>>>>>>                             .println("Wrong number of
>>>>>>>>>>>>>>>>> arguments: " + otherArgs.length);
>>>>>>>>>>>>>>>>>                System.err.println("Usage: " +
>>>>>>>>>>>>>>>>> "<tablename> <colfam> <qualifier>");
>>>>>>>>>>>>>>>>>                System.exit(-1);
>>>>>>>>>>>>>>>>>         }
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>>         try {
>>>>>>>>>>>>>>>>>                testSumWithValidRange(conf, otherArgs);
>>>>>>>>>>>>>>>>>         } catch (Throwable e) {
>>>>>>>>>>>>>>>>>                e.printStackTrace();
>>>>>>>>>>>>>>>>>         }
>>>>>>>>>>>>>>>>>  }
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>>  /**
>>>>>>>>>>>>>>>>>  * an infrastructure method to prepare rows for the
>>>>>>>>> testtable.
>>>>>>>>>>>>>>>>>  *
>>>>>>>>>>>>>>>>>   * @param base
>>>>>>>>>>>>>>>>>  * @param n
>>>>>>>>>>>>>>>>>  * @return
>>>>>>>>>>>>>>>>>  */
>>>>>>>>>>>>>>>>>  private static byte[][] makeN(byte[] base, int n) {
>>>>>>>>>>>>>>>>>         byte[][] ret =3D new byte[n][];
>>>>>>>>>>>>>>>>>         for (int i =3D 0; i < n; i++) {
>>>>>>>>>>>>>>>>>                ret[i] =3D Bytes.add(base,
>>>>>>>>>>>>>>>>> Bytes.toBytes(i));
>>>>>>>>>>>>>>>>>         }
>>>>>>>>>>>>>>>>>         return ret;
>>>>>>>>>>>>>>>>>  }
>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>> Please can you suggest what might be causing and/or how
>>>>>>>>>>>>>>>>> we might fix this UnknownProtocolException?
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>> Also, does anyone have any working examples using the
>>>>>>>>>>>>>>>>> aggregation
>>>>>>>>>>>>>>>> protocol
>>>>>>>>>>>>>>>>> other than the test code?
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Tom
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>>>=20
>>>>>>>>>>>>=20
>>>>>>>>>>>>=20
>>>>>>>>>>>=20
>>>>>>>>>=20
>>>>>>>>>=20
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>=20
>>>>>>>=20
>>>>>=20
>>>>>=20
>>>=20
>>>=20
>=20