incubator-hcatalog-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Petr Babkin <pbab...@revelytix.com>
Subject Re: ERROR 1070: Could not resolve org.apache.hcatalog.pig.HCatLoader
Date Fri, 24 Aug 2012 16:58:08 GMT
Thank you, Mr. Gates!

Now pig doesn't complain about hive-site.xml anymore, however it gets me
the same error:
*ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2998: Unhandled internal
error. com/facebook/fb303/FacebookService$Iface*
The log file says:
java.lang.NoClassDefFoundError: com/facebook/fb303/FacebookService$Iface
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
        at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
        at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
        at
org.apache.hcatalog.pig.PigHCatUtil.createHiveMetaClient(PigHCatUtil.java:114)
        at
org.apache.hcatalog.pig.PigHCatUtil.getTable(PigHCatUtil.java:150)
        at org.apache.hcatalog.pig.HCatLoader.getSchema(HCatLoader.java:151)
        at
org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:150)
        at
org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:109)
        at
org.apache.pig.newplan.logical.visitor.LineageFindRelVisitor.visit(LineageFindRelVisitor.java:100)
        at
org.apache.pig.newplan.logical.relational.LOLoad.accept(LOLoad.java:218)
        at
org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
        at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
        at
org.apache.pig.newplan.logical.visitor.CastLineageSetter.<init>(CastLineageSetter.java:57)
        at org.apache.pig.PigServer$Graph.compile(PigServer.java:1679)
        at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1610)
        at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
        at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
        at
org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942)
        at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
        at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
        at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
        at org.apache.pig.Main.run(Main.java:495)
        at org.apache.pig.Main.main(Main.java:111)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.ClassNotFoundException:
com.facebook.fb303.FacebookService$Iface
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
        ... 38 more

It's quite surprising how I'm running into these issue having launched HDP
from Hortonworks' pre-configured VM. I just can't imagine a place where I
could have messed up with the configuration.

Regards,
Petr Babkin.

On Fri, Aug 24, 2012 at 12:44 PM, Alan Gates <gates@hortonworks.com> wrote:

> Also your PIG_CLASSPATH has to point to the directory that hive-site.xml
> is in, not to the hive-site.xml file itself.
>
> Alan.
>
> On Aug 24, 2012, at 9:10 AM, Petr Babkin wrote:
>
> > Thanks, Aniket and Vandana,
> >
> > after adding the jars to PIG_CLASSPATH, I'm getting:
> >
> > WARN  org.apache.hadoop.hive.conf.HiveConf - hive-site.xml not found on
> CLASSPATH
> > ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2998: Unhandled internal
> error. com/facebook/fb303/FacebookService$Iface
> >
> > though my PIG_CLASSPATH has /etc/hive/conf.dist/hive-site.xml added.
> >
> > Vandana, could you please be more specific on what HCatalog
> configuration needs to be done?
> >
> > Many thanks!
> >
> > Regards,
> > Petr Babkin
> >
> > On Thu, Aug 23, 2012 at 7:02 PM, Aniket Mokashi <aniket486@gmail.com>
> wrote:
> > 1. Make sure HCatalog jar is in the class path (named hcatalog*.jar)
> > 2. Make sure your pig is running with
> -Dudf.import.list=org.apache.hcatalog.pig. This is how pig finds the
> package prefix for HCatLoader
> >
> > Workaround for 2 is- in your pig script- replace HCatLoader by
> org.apache.hcatalog.pig.HCatLoader.
> >
> > I assume HDP is doing all that already. So, I suspect something else is
> responsible here.
> >
> > Thanks,
> > Aniket
> >
> > On Thu, Aug 23, 2012 at 3:47 PM, Vandana Ayyalasomayajula <
> avandana@yahoo-inc.com> wrote:
> > Hi Petr,
> >
> > I am thinking that this issue is related to class path. You need to have
> the HCatalog, hive jars in your PIG_CLASSPATH env variable.
> >
> > From a previous post in the user list, the following this are required
> in the class path.
> >
> > Minimum set-
> > /share/hcatalog/hcatalog*.jar:
> > /share/hcatalog/lib/hive-metastore*.jar:
> > /share/hcatalog/lib/thrift-fb303-*.jar:
> > /share/hcatalog/lib/libthrift*.jar:
> > /share/hcatalog/lib/hive-exec-*.jar
> > +HCatalog configuration (hive-site et al)…
> >
> > Thanks
> > Vandana
> >
> >
> > On Aug 23, 2012, at 12:49 PM, Petr Babkin wrote:
> >
> >> I am getting the following error when trying to load the table from
> HCatalog into pig:
> >>
> >> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve
> org.apache.hcatalog.pig.HCatLoader using imports: [,
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
> >>
> >> This is all running on HDP, so I am surprised why the resolution fails.
> Googling around this issue didn't bring any result.
> >>
> >> Thanks.
> >>
> >> --
> >> Regards,
> >> Petr Babkin
> >> Ontologist, Revelytix Inc.
> >>
> >
> > Vandana Ayyalasomayajula
> >
> >
> >
> >
> >
> >
> > --
> > "...:::Aniket:::... Quetzalco@tl"
> >
> >
> >
> > --
> > Regards,
> > Petr Babkin
> > Ontologist, Revelytix Inc.
> >
>
>


-- 
Regards,
Petr Babkin
Ontologist, Revelytix Inc.

Mime
View raw message