incubator-hcatalog-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Travis Crawford <traviscrawf...@gmail.com>
Subject Re: Cannot access hive tables with pig 0.10
Date Fri, 19 Oct 2012 14:19:38 GMT
Looking at the log we see these lines showing a connection to a remote Hive
metastore:

"""
2012-10-19 12:34:19,233 [main] INFO  hive.metastore - Trying to connect to
metastore with URI thrift://appserver.hadoop.processing.com:10002
2012-10-19 12:34:19,270 [main] INFO  hive.metastore - Connected to
metastore.
"""

But later we see an in-process HiveMetaStore starting up:

"""
2012-10-19 12:35:00,997 [Thread-10] INFO
org.apache.hadoop.hive.metastore.HiveMetaStore - 0: Opening raw store with
implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
2012-10-19 12:35:01,082 [Thread-10] INFO
org.apache.hadoop.hive.metastore.ObjectStore - ObjectStore, initialize
called
"""

This seems like a hive-site.xml configuration issue. You'll likely want to
have a version of that file just for clients, with all the DB stuff
removed, and hive.metastore.uris set pointing at the HiveMetaStore thrift
service.

Also, I'd suggest getting the smallest possible script in a file to test.
Looks like it would be something like:

"""
register /hive/jars;
register /hcat/jars;
A = load 'processingpipeline.pagetime' using
org.apache.hcatalog.pig.HCatLoader();
describe A;
"""

That's enough to query the HiveMetaStore for the schema and display it,
which should help us troubleshoot your issue.

--travis



On Fri, Oct 19, 2012 at 6:14 AM, martin goodson <martingoodson@gmail.com>wrote:

> I am having problems using  pig-0.10.0  to access hive tables.  I've
> enclosed an example below and the stack trace. Essentially - no job seems
> to start and there is no meaningful error message:
>
> 2012-10-19 12:35:04,892 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Failed!
> 2012-10-19 12:35:04,894 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 1066: Unable to open iterator for alias ll
> Details at logfile: /YYYY/pig_1350650023453.log
>
> This works fine on Pig 0.8.1.
>
> Environment:
> Cloudera CDH3
> hadoop 0.20.2+923.418-1~maverick-cdh3
> hive 0.7.1+42.56-2~maverick-cdh3
> pig 0.8.1+28.39-1~maverick-cdh3
>
> Very grateful for any help
> Martin
>
>
> grunt> A = load 'processingpipeline.pagetime' using
> org.apache.hcatalog.pig.HCatLoader;
> B = FILTER A BY tracking_id=='customer' AND  record_date=='2012-09-10' ;
> A = load 'processingpipeline.pagetime' using
> org.apache.hcatalog.pig.HCatLoader;
> 2012-10-19 12:34:19,233 [main] INFO  hive.metastore - Trying to connect to
> metastore with URI thrift://appserver.hadoop.processing.com:10002
> 2012-10-19 12:34:19,270 [main] INFO  hive.metastore - Connected to
> metastore.
> grunt> B = FILTER A BY tracking_id=='customer' AND
> record_date=='2012-09-10' ;
> grunt>
>
> grunt> ll = limit B 10;
> ll = limit B 10;
> grunt> dump ll;
> dump ll;
> 2012-10-19 12:34:43,132 [main] INFO
> org.apache.pig.tools.pigstats.ScriptState - Pig features used in the
> script: FILTER,LIMIT
> 2012-10-19 12:34:43,312 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler -
> File concatenation threshold: 100 optimistic? false
> 2012-10-19 12:34:43,353 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size before optimization: 1
> 2012-10-19 12:34:43,353 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size after optimization: 1
> 2012-10-19 12:34:43,463 [main] INFO
> org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added
> to the job
> 2012-10-19 12:34:43,478 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
> 2012-10-19 12:34:43,518 [main] INFO  hive.metastore - Trying to connect to
> metastore with URI thrift://appserver.hadoop.processing.com:10002
> 2012-10-19 12:34:43,519 [main] INFO  hive.metastore - Connected to
> metastore.
> 2012-10-19 12:34:43,688 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - creating jar file Job4341676236883270366.jar
> 2012-10-19 12:34:59,989 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - jar file Job4341676236883270366.jar created
> 2012-10-19 12:35:00,010 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Setting up single store job
> 2012-10-19 12:35:00,137 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 1 map-reduce job(s) waiting for submission.
> 2012-10-19 12:35:00,637 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 0% complete
> 2012-10-19 12:35:00,997 [Thread-10] INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore - 0: Opening raw store with
> implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
> 2012-10-19 12:35:01,082 [Thread-10] INFO
> org.apache.hadoop.hive.metastore.ObjectStore - ObjectStore, initialize
> called
> 2012-10-19 12:35:01,549 [Thread-10] ERROR DataNucleus.Plugin - Bundle
> "org.eclipse.jdt.core" requires "org.eclipse.core.resources" but it cannot
> be resolved.
> 2012-10-19 12:35:01,549 [Thread-10] ERROR DataNucleus.Plugin - Bundle
> "org.eclipse.jdt.core" requires "org.eclipse.core.runtime" but it cannot be
> resolved.
> 2012-10-19 12:35:01,549 [Thread-10] ERROR DataNucleus.Plugin - Bundle
> "org.eclipse.jdt.core" requires "org.eclipse.text" but it cannot be
> resolved.
> 2012-10-19 12:35:01,727 [Thread-10] INFO  DataNucleus.Persistence -
> Property datanucleus.cache.level2 unknown - will be ignored
> 2012-10-19 12:35:01,739 [Thread-10] INFO  DataNucleus.Persistence -
> Property javax.jdo.option.NonTransactionalRead unknown - will be ignored
> 2012-10-19 12:35:01,739 [Thread-10] INFO  DataNucleus.Persistence -
> ================= Persistence Configuration ===============
> 2012-10-19 12:35:01,743 [Thread-10] INFO  DataNucleus.Persistence -
> DataNucleus Persistence Factory - Vendor: "DataNucleus"  Version:
> "2.0.3-ZD5977-CDH5293"
> 2012-10-19 12:35:01,743 [Thread-10] INFO  DataNucleus.Persistence -
> DataNucleus Persistence Factory initialised for datastore URL="jdbc:mysql://
> appserver.hadoop.processing.com:3306/hive?createDatabaseIfNotExist=true"
> driver="com.mysql.jdbc.Driver" userName="user"
> 2012-10-19 12:35:01,743 [Thread-10] INFO  DataNucleus.Persistence -
> ===========================================================
> 2012-10-19 12:35:02,507 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Creating table `DELETEME1350650102494`
> 2012-10-19 12:35:02,562 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Schema Name could not be determined for this datastore
> 2012-10-19 12:35:02,562 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Dropping table `DELETEME1350650102494`
> 2012-10-19 12:35:02,568 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Initialising Catalog "hive", Schema "" using "None" auto-start option
> 2012-10-19 12:35:02,568 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Catalog "hive", Schema "" initialised - managing 0 classes
> 2012-10-19 12:35:02,699 [Thread-10] INFO
> org.apache.hadoop.hive.metastore.ObjectStore - Setting MetaStore object pin
> classes with
> hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
> 2012-10-19 12:35:02,900 [Thread-10] INFO  DataNucleus.MetaData -
> Registering listener for metadata initialisation
> 2012-10-19 12:35:03,072 [Thread-10] INFO
> org.apache.hadoop.hive.metastore.ObjectStore - Initialized ObjectStore
> 2012-10-19 12:35:03,343 [Thread-10] WARN  DataNucleus.MetaData - MetaData
> Parser encountered an error in file
> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
> at line 11, column 6 : cvc-elt.1: Cannot find the declaration of element
> 'jdo'. - Please check your specification of DTD and the validity of the
> MetaData XML that you have specified.
> 2012-10-19 12:35:03,467 [Thread-10] WARN  DataNucleus.MetaData - MetaData
> Parser encountered an error in file
> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
> at line 312, column 13 : The content of element type "class" must match
> "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)".
> - Please check your specification of DTD and the validity of the MetaData
> XML that you have specified.
> 2012-10-19 12:35:03,471 [Thread-10] WARN  DataNucleus.MetaData - MetaData
> Parser encountered an error in file
> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
> at line 359, column 13 : The content of element type "class" must match
> "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)".
> - Please check your specification of DTD and the validity of the MetaData
> XML that you have specified.
> 2012-10-19 12:35:03,473 [Thread-10] WARN  DataNucleus.MetaData - MetaData
> Parser encountered an error in file
> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
> at line 381, column 13 : The content of element type "class" must match
> "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)".
> - Please check your specification of DTD and the validity of the MetaData
> XML that you have specified.
> 2012-10-19 12:35:03,476 [Thread-10] WARN  DataNucleus.MetaData - MetaData
> Parser encountered an error in file
> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
> at line 416, column 13 : The content of element type "class" must match
> "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)".
> - Please check your specification of DTD and the validity of the MetaData
> XML that you have specified.
> 2012-10-19 12:35:03,487 [Thread-10] WARN  DataNucleus.MetaData - MetaData
> Parser encountered an error in file
> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
> at line 453, column 13 : The content of element type "class" must match
> "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)".
> - Please check your specification of DTD and the validity of the MetaData
> XML that you have specified.
> 2012-10-19 12:35:03,491 [Thread-10] WARN  DataNucleus.MetaData - MetaData
> Parser encountered an error in file
> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
> at line 494, column 13 : The content of element type "class" must match
> "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)".
> - Please check your specification of DTD and the validity of the MetaData
> XML that you have specified.
> 2012-10-19 12:35:03,510 [Thread-10] WARN  DataNucleus.MetaData - MetaData
> Parser encountered an error in file
> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
> at line 535, column 13 : The content of element type "class" must match
> "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)".
> - Please check your specification of DTD and the validity of the MetaData
> XML that you have specified.
> 2012-10-19 12:35:03,514 [Thread-10] WARN  DataNucleus.MetaData - MetaData
> Parser encountered an error in file
> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
> at line 576, column 13 : The content of element type "class" must match
> "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)".
> - Please check your specification of DTD and the validity of the MetaData
> XML that you have specified.
> 2012-10-19 12:35:03,518 [Thread-10] WARN  DataNucleus.MetaData - MetaData
> Parser encountered an error in file
> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
> at line 621, column 13 : The content of element type "class" must match
> "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)".
> - Please check your specification of DTD and the validity of the MetaData
> XML that you have specified.
> 2012-10-19 12:35:03,533 [Thread-10] WARN  DataNucleus.MetaData - MetaData
> Parser encountered an error in file
> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
> at line 666, column 13 : The content of element type "class" must match
> "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)".
> - Please check your specification of DTD and the validity of the MetaData
> XML that you have specified.
> 2012-10-19 12:35:03,883 [Thread-10] INFO  DataNucleus.Persistence -
> Managing Persistence of Class :
> org.apache.hadoop.hive.metastore.model.MDatabase [Table : `DBS`,
> InheritanceStrategy : new-table]
> 2012-10-19 12:35:03,965 [Thread-10] INFO  DataNucleus.Persistence -
> Managing Persistence of Field :
> org.apache.hadoop.hive.metastore.model.MDatabase.parameters [Table :
> `DATABASE_PARAMS`]
> 2012-10-19 12:35:04,057 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 2 index(es) for table `DBS`
> 2012-10-19 12:35:04,069 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 0 foreign key(s) for table `DBS`
> 2012-10-19 12:35:04,071 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 2 unique key(s) for table `DBS`
> 2012-10-19 12:35:04,073 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 2 index(es) for table `DATABASE_PARAMS`
> 2012-10-19 12:35:04,086 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 1 foreign key(s) for table `DATABASE_PARAMS`
> 2012-10-19 12:35:04,088 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 1 unique key(s) for table `DATABASE_PARAMS`
> 2012-10-19 12:35:04,267 [Thread-10] INFO  DataNucleus.MetaData - Listener
> found initialisation for persistable class
> org.apache.hadoop.hive.metastore.model.MDatabase
> 2012-10-19 12:35:04,431 [Thread-10] INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore - 0: get_table :
> db=processingpipeline tbl=pagetime
> 2012-10-19 12:35:04,437 [Thread-10] INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit - ugi=martin
> ip=unknown-ip-addr    cmd=get_table : db=processingpipeline tbl=pagetime
> 2012-10-19 12:35:04,449 [Thread-10] INFO  DataNucleus.Datastore - The
> class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as
> "embedded-only" so does not have its own datastore table.
> 2012-10-19 12:35:04,449 [Thread-10] INFO  DataNucleus.Persistence -
> Managing Persistence of Class :
> org.apache.hadoop.hive.metastore.model.MSerDeInfo [Table : `SERDES`,
> InheritanceStrategy : new-table]
> 2012-10-19 12:35:04,449 [Thread-10] INFO  DataNucleus.Datastore - The
> class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as
> "embedded-only" so does not have its own datastore table.
> 2012-10-19 12:35:04,449 [Thread-10] INFO  DataNucleus.Persistence -
> Managing Persistence of Class :
> org.apache.hadoop.hive.metastore.model.MStorageDescriptor [Table : `SDS`,
> InheritanceStrategy : new-table]
> 2012-10-19 12:35:04,450 [Thread-10] INFO  DataNucleus.Persistence -
> Managing Persistence of Class :
> org.apache.hadoop.hive.metastore.model.MTable [Table : `TBLS`,
> InheritanceStrategy : new-table]
> 2012-10-19 12:35:04,450 [Thread-10] INFO  DataNucleus.Persistence -
> Managing Persistence of Field :
> org.apache.hadoop.hive.metastore.model.MSerDeInfo.parameters [Table :
> `SERDE_PARAMS`]
> 2012-10-19 12:35:04,451 [Thread-10] INFO  DataNucleus.Persistence -
> Managing Persistence of Field :
> org.apache.hadoop.hive.metastore.model.MTable.parameters [Table :
> `TABLE_PARAMS`]
> 2012-10-19 12:35:04,455 [Thread-10] INFO  DataNucleus.Persistence -
> Managing Persistence of Field :
> org.apache.hadoop.hive.metastore.model.MTable.partitionKeys [Table :
> `PARTITION_KEYS`]
> 2012-10-19 12:35:04,466 [Thread-10] INFO  DataNucleus.Persistence -
> Managing Persistence of Field :
> org.apache.hadoop.hive.metastore.model.MStorageDescriptor.bucketCols [Table
> : `BUCKETING_COLS`]
> 2012-10-19 12:35:04,466 [Thread-10] INFO  DataNucleus.Persistence -
> Managing Persistence of Field :
> org.apache.hadoop.hive.metastore.model.MStorageDescriptor.cols [Table :
> `COLUMNS`]
> 2012-10-19 12:35:04,476 [Thread-10] INFO  DataNucleus.Persistence -
> Managing Persistence of Field :
> org.apache.hadoop.hive.metastore.model.MStorageDescriptor.parameters [Table
> : `SD_PARAMS`]
> 2012-10-19 12:35:04,477 [Thread-10] INFO  DataNucleus.Persistence -
> Managing Persistence of Field :
> org.apache.hadoop.hive.metastore.model.MStorageDescriptor.sortCols [Table :
> `SORT_COLS`]
> 2012-10-19 12:35:04,644 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 1 index(es) for table `SERDES`
> 2012-10-19 12:35:04,645 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 0 foreign key(s) for table `SERDES`
> 2012-10-19 12:35:04,646 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 1 unique key(s) for table `SERDES`
> 2012-10-19 12:35:04,648 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 4 index(es) for table `TBLS`
> 2012-10-19 12:35:04,656 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 2 foreign key(s) for table `TBLS`
> 2012-10-19 12:35:04,658 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 2 unique key(s) for table `TBLS`
> 2012-10-19 12:35:04,668 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 2 index(es) for table `SDS`
> 2012-10-19 12:35:04,670 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 1 foreign key(s) for table `SDS`
> 2012-10-19 12:35:04,672 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 1 unique key(s) for table `SDS`
> 2012-10-19 12:35:04,674 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 2 index(es) for table `SORT_COLS`
> 2012-10-19 12:35:04,676 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 1 foreign key(s) for table `SORT_COLS`
> 2012-10-19 12:35:04,684 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 1 unique key(s) for table `SORT_COLS`
> 2012-10-19 12:35:04,686 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 2 index(es) for table `PARTITION_KEYS`
> 2012-10-19 12:35:04,687 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 1 foreign key(s) for table `PARTITION_KEYS`
> 2012-10-19 12:35:04,689 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 1 unique key(s) for table `PARTITION_KEYS`
> 2012-10-19 12:35:04,690 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 2 index(es) for table `BUCKETING_COLS`
> 2012-10-19 12:35:04,698 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 1 foreign key(s) for table `BUCKETING_COLS`
> 2012-10-19 12:35:04,700 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 1 unique key(s) for table `BUCKETING_COLS`
> 2012-10-19 12:35:04,707 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 2 index(es) for table `TABLE_PARAMS`
> 2012-10-19 12:35:04,709 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 1 foreign key(s) for table `TABLE_PARAMS`
> 2012-10-19 12:35:04,717 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 1 unique key(s) for table `TABLE_PARAMS`
> 2012-10-19 12:35:04,719 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 2 index(es) for table `SD_PARAMS`
> 2012-10-19 12:35:04,720 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 1 foreign key(s) for table `SD_PARAMS`
> 2012-10-19 12:35:04,722 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 1 unique key(s) for table `SD_PARAMS`
> 2012-10-19 12:35:04,723 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 2 index(es) for table `SERDE_PARAMS`
> 2012-10-19 12:35:04,724 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 1 foreign key(s) for table `SERDE_PARAMS`
> 2012-10-19 12:35:04,793 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 1 unique key(s) for table `SERDE_PARAMS`
> 2012-10-19 12:35:04,795 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 2 index(es) for table `COLUMNS`
> 2012-10-19 12:35:04,796 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 1 foreign key(s) for table `COLUMNS`
> 2012-10-19 12:35:04,798 [Thread-10] INFO  DataNucleus.Datastore.Schema -
> Validating 1 unique key(s) for table `COLUMNS`
> 2012-10-19 12:35:04,813 [Thread-10] INFO  DataNucleus.MetaData - Listener
> found initialisation for persistable class
> org.apache.hadoop.hive.metastore.model.MSerDeInfo
> 2012-10-19 12:35:04,814 [Thread-10] INFO  DataNucleus.MetaData - Listener
> found initialisation for persistable class
> org.apache.hadoop.hive.metastore.model.MStorageDescriptor
> 2012-10-19 12:35:04,814 [Thread-10] INFO  DataNucleus.MetaData - Listener
> found initialisation for persistable class
> org.apache.hadoop.hive.metastore.model.MTable
> 2012-10-19 12:35:04,842 [Thread-10] INFO  DataNucleus.MetaData - Listener
> found initialisation for persistable class
> org.apache.hadoop.hive.metastore.model.MFieldSchema
> 2012-10-19 12:35:04,867 [Thread-10] INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore - 0:
> get_partitions_by_filter : db=processingpipeline tbl=pagetime
> 2012-10-19 12:35:04,867 [Thread-10] INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit - ugi=martin
> ip=unknown-ip-addr    cmd=get_partitions_by_filter : db=processingpipeline
> tbl=pagetime
> 2012-10-19 12:35:04,890 [Thread-10] INFO
> org.apache.hadoop.mapred.JobClient - Cleaning up the staging area hdfs://
> namenode.hadoop.processing.com:54310/mnt/hadoop/tmp/mapred/staging/martin/.staging/job_201210181624_0767
> 2012-10-19 12:35:04,890 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 100% complete
> 2012-10-19 12:35:04,890 [main] ERROR
> org.apache.pig.tools.pigstats.PigStatsUtil - 0 map reduce job(s) failed!
> 2012-10-19 12:35:04,892 [main] INFO
> org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
>
> HadoopVersion    PigVersion    UserId    StartedAt    FinishedAt
> Features
> 0.20.2-cdh3u5    0.10.0-SNAPSHOT    martin    2012-10-19 12:34:43
> 2012-10-19 12:35:04    FILTER,LIMIT
>
> Failed!
>
> Failed Jobs:
> JobId    Alias    Feature    Message    Outputs
>
> Input(s):
>
> Output(s):
>
> Counters:
> Total records written : 0
> Total bytes written : 0
> Spillable Memory Manager spill count : 0
> Total bags proactively spilled: 0
> Total records proactively spilled: 0
>
> Job DAG:
> null
>
>
> 2012-10-19 12:35:04,892 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Failed!
> 2012-10-19 12:35:04,894 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 1066: Unable to open iterator for alias ll
> Details at logfile: /YYYY/pig_1350650023453.log
> grunt>
>
>
>
> $> cat /YYYYYY//pig_1350650023453.log
>
> Pig Stack Trace
> ---------------
> ERROR 1066: Unable to open iterator for alias ll
>
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
> open iterator for alias ll
>     at org.apache.pig.PigServer.openIterator(PigServer.java:857)
>     at
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:682)
>     at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
>     at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
>     at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>     at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
>     at org.apache.pig.Main.run(Main.java:490)
>     at org.apache.pig.Main.main(Main.java:111)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     at java.lang.reflect.Method.invoke(Method.java:597)
>     at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
> Caused by: java.io.IOException: Couldn't retrieve job.
>     at org.apache.pig.PigServer.store(PigServer.java:921)
>     at org.apache.pig.PigServer.openIterator(PigServer.java:832)
>     ... 12 more
>
> ================================================================================
>
>
>

Mime
View raw message