hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From anil gupta <anilgupt...@gmail.com>
Subject Re: BigDecimalColumnInterpreter
Date Wed, 03 Oct 2012 18:26:33 GMT
Glad to know that it works on your cluster. After running the TestUtility
on my cluster i was pretty sure that you missed some steps in deploying it.
:)

On Wed, Oct 3, 2012 at 9:26 AM, Julian Wissmann <julian.wissmann@sdace.de>wrote:

> Hi, thank you Anil. I realized I had put the jar only on the master and
> expected stuff to work from there. I now deployed it on all the
> regionservers and loaded AggregateImplementation on the regions. It works
> now. Thank you!
>
> 2012/9/29 Robertis Tongbram <rtongbram@gmail.com>
>
> > For pushing jars to the region servers , i have a small script.
> >
> > <code>
> >
> > #!/usr/bin/expect -f
> >
> >
> > for {set i 1} {$i < 6} {incr i 1} {
> >
> >
> >     spawn scp target/lib.jar root@region-$i:/opt/hbase/hbase-0.92.1/lib/
> >     expect {
> >         -re ".*es.*o.*" {
> >             exp_send "yes\r"
> >                 exp_continue
> >         }
> >         -re ".*sword.*" {
> >             exp_send "p4sSw0rd\r"
> >         }
> >     }
> >     interact
> >
> > }
> >
> > </code>
> >
> > I have my region servers as region-1, region-2 and so on, default
> > installation is at /opt/hbase/hbase-0.92.1/
> >  lib.jar is the final jar file that has the classes.
> >
> > I put this script where the pom file is ( i have a maven project).
> >
> >
> >
> >
> > Hope this helps.
> >
> >
> > Thanks,
> > Robertis
> >
> >
> >
> > On Sat, Sep 29, 2012 at 8:35 AM, anil gupta <anilgupta84@gmail.com>
> wrote:
> >
> > > This ColumnInterpreter is required at the server side. You need to put
> > the
> > > jar into the classpath of regionservers. HBase picks up all the jar
> under
> > > $hbase_home/lib folder. So you can put the jar containing the BDCI
> class
> > in
> > > this folder. Please make sure that in your client side you are
> > referencing
> > > the correct BDCI class(the full name of class at server side should
> match
> > > with the ColumnInterpreter class you reference on client side). Once
> you
> > > put the jar into lib folder then you will need to restart the cluster
> > once
> > > so that the regionserver will the load the jar containing BDCI.
> > >
> > > Basically you just need to put the utility class(which i sent a couple
> of
> > > weeks ago) into a jar, copy the jar into lib folder, restart the
> cluster,
> > > and then run the utility. It should work.
> > >
> > > Hope it's clear this time.
> > >
> > > Thanks,
> > > Anil Gupta
> > >
> > > On Sat, Sep 29, 2012 at 8:03 AM, Julian Wissmann
> > > <julian.wissmann@sdace.de>wrote:
> > >
> > > > Interesting. Did you put just the ColumnInterpreter into the jar?
> > > >
> > > > 2012/9/21 Anil Gupta <anilgupta84@gmail.com>
> > > >
> > > > > I was able to run it successfully once I copied the jar into lib
> > > folders
> > > > > of region server and hamster and restarted the cluster.
> > > > > Did you copied the jar into lib folders of region servers and
> hmaster
> > > and
> > > > > restarted the cluster. If yes, can you share the logs of
> regionserver
> > > > > hosting the region test table?
> > > > >
> > > > > Best Regards,
> > > > > Anil
> > > > >
> > > > > On Sep 21, 2012, at 4:14 AM, Julian Wissmann <
> > julian.wissmann@sdace.de
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Anil,
> > > > > >
> > > > > > found some time to test it, today. Same error as you get, same
> > error
> > > as
> > > > > > always ;-)
> > > > > >
> > > > > > Julian
> > > > > >
> > > > > > 2012/9/20 anil gupta <anilgupta84@gmail.com>
> > > > > >
> > > > > >> Hi Julian,
> > > > > >>
> > > > > >> I am curious to know whether you got the opportunity to
test
> BDCI
> > > > > utility i
> > > > > >> sent last week along with some suggestions on using it?
Did it
> run
> > > > > >> successfully?
> > > > > >>
> > > > > >> I will try to have a look at your unit test over weekend.
> > > > > >> Thanks,
> > > > > >> Anil Gupta
> > > > > >>
> > > > > >> On Thu, Sep 20, 2012 at 10:29 AM, Julian Wissmann
> > > > > >> <julian.wissmann@sdace.de>wrote:
> > > > > >>
> > > > > >>> Hi,
> > > > > >>>
> > > > > >>> as I've also mentioned in the JIRA Issue:
> > > > > >>>
> > > > > >>> I've written a Test, but have problems with Medium Tests
> > requiring
> > > > > >>> MiniDFSCluster:
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> -------------------------------------------------------------------------------
> > > > > >>> Test set:
> > org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> -------------------------------------------------------------------------------
> > > > > >>> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time
elapsed:
> > > 3.239
> > > > > >>> sec <<< FAILURE!
> > > > > >>> org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol
Time
> > > > elapsed:
> > > > > >>> 0.001 sec <<< ERROR!
> > > > > >>> java.lang.NullPointerException
> > > > > >>> at
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422)
> > > > > >>> at
> > > > >
> org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280)
> > > > > >>> at
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:433)
> > > > > >>> at
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:653)
> > > > > >>> at
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:603)
> > > > > >>> at
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:590)
> > > > > >>> at
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol.setupBeforeClass(TestAggregateProtocol.java:77)
> > > > > >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
> > > > > >>> at
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > > > > >>> at
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > > > > >>> at java.lang.reflect.Method.invoke(Method.java:597)
> > > > > >>> at
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> > > > > >>> at
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> > > > > >>> at
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> > > > > >>> at
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
> > > > > >>> at
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> > > > > >>> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> > > > > >>> at org.junit.runners.Suite.runChild(Suite.java:128)
> > > > > >>> at org.junit.runners.Suite.runChild(Suite.java:24)
> > > > > >>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> > > > > >>> at
> > > > > >>
> > > >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> > > > > >>> at
> > > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> > > > > >>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> > > > > >>> at
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > > > >>> at
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > > > >>> at java.lang.Thread.run(Thread.java:662)
> > > > > >>>
> > > > > >>> Therefor I have not been able to do a run of the test,
yet.
> > However
> > > > if
> > > > > >>> someone would be willing to run it I can just attach
it to the
> > > issue.
> > > > > >>>
> > > > > >>> Regards,
> > > > > >>> Julian
> > > > > >>>
> > > > > >>>> Hi Ted,
> > > > > >>>>
> > > > > >>>> I've been unable to squeeze time for writing unit
test for
> BDCI
> > > due
> > > > to
> > > > > >> my
> > > > > >>>> tight schedule lately. I will try to submit the
patch with
> unit
> > > test
> > > > > >> next
> > > > > >>>> week.
> > > > > >>>>
> > > > > >>>> Thanks,
> > > > > >>>> Anil
> > > > > >>>>
> > > > > >>>> On Sat, Sep 15, 2012 at 7:56 AM, Ted Yu <yuzhihong@gmail.com>
> > > > wrote:
> > > > > >>>>
> > > > > >>>>> Anil:
> > > > > >>>>> Thanks for the sharing.
> > > > > >>>>> If you add a test to HBASE-6669 showing use
of
> > > > > >>> BigDecimalColumnInterpreter,
> > > > > >>>>> your patch would be more easily acceptable.
> > > > > >>>>>
> > > > > >>>>> Cheers
> > > > > >>>>>
> > > > > >>>>> On Fri, Sep 14, 2012 at 3:17 PM, anil gupta
<
> > > anilgupta84@gmail.com
> > > > >
> > > > > >>> wrote:
> > > > > >>>>>
> > > > > >>>>>> Hi Guys,
> > > > > >>>>>>
> > > > > >>>>>> The regionservers need to have the jar which
contains the
> > > > > >>>>>> BigDecimalColumnInterpreter class in their
classpath. I was
> > > > > >>> successfully
> > > > > >>>>>> able to run the utility class once i copied
the jar across
> the
> > > > > >> cluster
> > > > > >>>>> and
> > > > > >>>>>> rebooted the cluster. Also, please specify
the following
> > > property
> > > > in
> > > > > >>>>>> hbase-site.xml to load the co-processor:
> > > > > >>>>>> <property>
> > > > > >>>>>>        <name>hbase.coprocessor.region.classes</name>
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.coprocessor.AggregateImplementation</value>
> > > > > >>>>>> </property>
> > > > > >>>>>>
> > > > > >>>>>> Let me know once you guys try this out.
> > > > > >>>>>>
> > > > > >>>>>> Thanks,
> > > > > >>>>>> Anil Gupta
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>>
> > > > > >>>>
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > > >>
> > > > > >> --
> > > > > >> Thanks & Regards,
> > > > > >> Anil Gupta
> > > > > >>
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Thanks & Regards,
> > > Anil Gupta
> > >
> >
>



-- 
Thanks & Regards,
Anil Gupta

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message