incubator-hcatalog-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From martin goodson <martingood...@gmail.com>
Subject Re: Cannot access hive tables with pig 0.10
Date Tue, 23 Oct 2012 09:36:30 GMT
Great - thank you very much.

Now working fine.



On Fri, Oct 19, 2012 at 3:19 PM, Travis Crawford
<traviscrawford@gmail.com>wrote:

> Looking at the log we see these lines showing a connection to a remote
> Hive metastore:
>
> """
> 2012-10-19 12:34:19,233 [main] INFO  hive.metastore - Trying to connect to
> metastore with URI thrift://appserver.hadoop.processing.com:10002
> 2012-10-19 12:34:19,270 [main] INFO  hive.metastore - Connected to
> metastore.
> """
>
> But later we see an in-process HiveMetaStore starting up:
>
> """
> 2012-10-19 12:35:00,997 [Thread-10] INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore - 0: Opening raw store with
> implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
> 2012-10-19 12:35:01,082 [Thread-10] INFO
> org.apache.hadoop.hive.metastore.ObjectStore - ObjectStore, initialize
> called
> """
>
> This seems like a hive-site.xml configuration issue. You'll likely want to
> have a version of that file just for clients, with all the DB stuff
> removed, and hive.metastore.uris set pointing at the HiveMetaStore thrift
> service.
>
> Also, I'd suggest getting the smallest possible script in a file to test.
> Looks like it would be something like:
>
> """
> register /hive/jars;
> register /hcat/jars;
> A = load 'processingpipeline.pagetime' using
> org.apache.hcatalog.pig.HCatLoader();
> describe A;
> """
>
> That's enough to query the HiveMetaStore for the schema and display it,
> which should help us troubleshoot your issue.
>
> --travis
>
>
>
> On Fri, Oct 19, 2012 at 6:14 AM, martin goodson <martingoodson@gmail.com>wrote:
>
>> I am having problems using  pig-0.10.0  to access hive tables.  I've
>> enclosed an example below and the stack trace. Essentially - no job seems
>> to start and there is no meaningful error message:
>>
>> 2012-10-19 12:35:04,892 [main] INFO
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
>> - Failed!
>> 2012-10-19 12:35:04,894 [main] ERROR org.apache.pig.tools.grunt.Grunt -
>> ERROR 1066: Unable to open iterator for alias ll
>> Details at logfile: /YYYY/pig_1350650023453.log
>>
>> This works fine on Pig 0.8.1.
>>
>> Environment:
>> Cloudera CDH3
>> hadoop 0.20.2+923.418-1~maverick-cdh3
>> hive 0.7.1+42.56-2~maverick-cdh3
>> pig 0.8.1+28.39-1~maverick-cdh3
>>
>> Very grateful for any help
>> Martin
>>
>>
>> grunt> A = load 'processingpipeline.pagetime' using
>> org.apache.hcatalog.pig.HCatLoader;
>> B = FILTER A BY tracking_id=='customer' AND  record_date=='2012-09-10' ;
>> A = load 'processingpipeline.pagetime' using
>> org.apache.hcatalog.pig.HCatLoader;
>> 2012-10-19 12:34:19,233 [main] INFO  hive.metastore - Trying to connect
>> to metastore with URI thrift://appserver.hadoop.processing.com:10002
>> 2012-10-19 12:34:19,270 [main] INFO  hive.metastore - Connected to
>> metastore.
>> grunt> B = FILTER A BY tracking_id=='customer' AND
>> record_date=='2012-09-10' ;
>> grunt>
>>
>> grunt> ll = limit B 10;
>> ll = limit B 10;
>> grunt> dump ll;
>> dump ll;
>> 2012-10-19 12:34:43,132 [main] INFO
>> org.apache.pig.tools.pigstats.ScriptState - Pig features used in the
>> script: FILTER,LIMIT
>> 2012-10-19 12:34:43,312 [main] INFO
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler -
>> File concatenation threshold: 100 optimistic? false
>> 2012-10-19 12:34:43,353 [main] INFO
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
>> - MR plan size before optimization: 1
>> 2012-10-19 12:34:43,353 [main] INFO
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
>> - MR plan size after optimization: 1
>> 2012-10-19 12:34:43,463 [main] INFO
>> org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added
>> to the job
>> 2012-10-19 12:34:43,478 [main] INFO
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
>> - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
>> 2012-10-19 12:34:43,518 [main] INFO  hive.metastore - Trying to connect
>> to metastore with URI thrift://appserver.hadoop.processing.com:10002
>> 2012-10-19 12:34:43,519 [main] INFO  hive.metastore - Connected to
>> metastore.
>> 2012-10-19 12:34:43,688 [main] INFO
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
>> - creating jar file Job4341676236883270366.jar
>> 2012-10-19 12:34:59,989 [main] INFO
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
>> - jar file Job4341676236883270366.jar created
>> 2012-10-19 12:35:00,010 [main] INFO
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
>> - Setting up single store job
>> 2012-10-19 12:35:00,137 [main] INFO
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
>> - 1 map-reduce job(s) waiting for submission.
>> 2012-10-19 12:35:00,637 [main] INFO
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
>> - 0% complete
>> 2012-10-19 12:35:00,997 [Thread-10] INFO
>> org.apache.hadoop.hive.metastore.HiveMetaStore - 0: Opening raw store with
>> implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
>> 2012-10-19 12:35:01,082 [Thread-10] INFO
>> org.apache.hadoop.hive.metastore.ObjectStore - ObjectStore, initialize
>> called
>> 2012-10-19 12:35:01,549 [Thread-10] ERROR DataNucleus.Plugin - Bundle
>> "org.eclipse.jdt.core" requires "org.eclipse.core.resources" but it cannot
>> be resolved.
>> 2012-10-19 12:35:01,549 [Thread-10] ERROR DataNucleus.Plugin - Bundle
>> "org.eclipse.jdt.core" requires "org.eclipse.core.runtime" but it cannot be
>> resolved.
>> 2012-10-19 12:35:01,549 [Thread-10] ERROR DataNucleus.Plugin - Bundle
>> "org.eclipse.jdt.core" requires "org.eclipse.text" but it cannot be
>> resolved.
>> 2012-10-19 12:35:01,727 [Thread-10] INFO  DataNucleus.Persistence -
>> Property datanucleus.cache.level2 unknown - will be ignored
>> 2012-10-19 12:35:01,739 [Thread-10] INFO  DataNucleus.Persistence -
>> Property javax.jdo.option.NonTransactionalRead unknown - will be ignored
>> 2012-10-19 12:35:01,739 [Thread-10] INFO  DataNucleus.Persistence -
>> ================= Persistence Configuration ===============
>> 2012-10-19 12:35:01,743 [Thread-10] INFO  DataNucleus.Persistence -
>> DataNucleus Persistence Factory - Vendor: "DataNucleus"  Version:
>> "2.0.3-ZD5977-CDH5293"
>> 2012-10-19 12:35:01,743 [Thread-10] INFO  DataNucleus.Persistence -
>> DataNucleus Persistence Factory initialised for datastore URL="jdbc:mysql://
>> appserver.hadoop.processing.com:3306/hive?createDatabaseIfNotExist=true"
>> driver="com.mysql.jdbc.Driver" userName="user"
>> 2012-10-19 12:35:01,743 [Thread-10] INFO  DataNucleus.Persistence -
>> ===========================================================
>> 2012-10-19 12:35:02,507 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Creating table `DELETEME1350650102494`
>> 2012-10-19 12:35:02,562 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Schema Name could not be determined for this datastore
>> 2012-10-19 12:35:02,562 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Dropping table `DELETEME1350650102494`
>> 2012-10-19 12:35:02,568 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Initialising Catalog "hive", Schema "" using "None" auto-start option
>> 2012-10-19 12:35:02,568 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Catalog "hive", Schema "" initialised - managing 0 classes
>> 2012-10-19 12:35:02,699 [Thread-10] INFO
>> org.apache.hadoop.hive.metastore.ObjectStore - Setting MetaStore object pin
>> classes with
>> hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
>> 2012-10-19 12:35:02,900 [Thread-10] INFO  DataNucleus.MetaData -
>> Registering listener for metadata initialisation
>> 2012-10-19 12:35:03,072 [Thread-10] INFO
>> org.apache.hadoop.hive.metastore.ObjectStore - Initialized ObjectStore
>> 2012-10-19 12:35:03,343 [Thread-10] WARN  DataNucleus.MetaData - MetaData
>> Parser encountered an error in file
>> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
>> at line 11, column 6 : cvc-elt.1: Cannot find the declaration of element
>> 'jdo'. - Please check your specification of DTD and the validity of the
>> MetaData XML that you have specified.
>> 2012-10-19 12:35:03,467 [Thread-10] WARN  DataNucleus.MetaData - MetaData
>> Parser encountered an error in file
>> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
>> at line 312, column 13 : The content of element type "class" must match
>> "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)".
>> - Please check your specification of DTD and the validity of the MetaData
>> XML that you have specified.
>> 2012-10-19 12:35:03,471 [Thread-10] WARN  DataNucleus.MetaData - MetaData
>> Parser encountered an error in file
>> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
>> at line 359, column 13 : The content of element type "class" must match
>> "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)".
>> - Please check your specification of DTD and the validity of the MetaData
>> XML that you have specified.
>> 2012-10-19 12:35:03,473 [Thread-10] WARN  DataNucleus.MetaData - MetaData
>> Parser encountered an error in file
>> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
>> at line 381, column 13 : The content of element type "class" must match
>> "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)".
>> - Please check your specification of DTD and the validity of the MetaData
>> XML that you have specified.
>> 2012-10-19 12:35:03,476 [Thread-10] WARN  DataNucleus.MetaData - MetaData
>> Parser encountered an error in file
>> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
>> at line 416, column 13 : The content of element type "class" must match
>> "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)".
>> - Please check your specification of DTD and the validity of the MetaData
>> XML that you have specified.
>> 2012-10-19 12:35:03,487 [Thread-10] WARN  DataNucleus.MetaData - MetaData
>> Parser encountered an error in file
>> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
>> at line 453, column 13 : The content of element type "class" must match
>> "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)".
>> - Please check your specification of DTD and the validity of the MetaData
>> XML that you have specified.
>> 2012-10-19 12:35:03,491 [Thread-10] WARN  DataNucleus.MetaData - MetaData
>> Parser encountered an error in file
>> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
>> at line 494, column 13 : The content of element type "class" must match
>> "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)".
>> - Please check your specification of DTD and the validity of the MetaData
>> XML that you have specified.
>> 2012-10-19 12:35:03,510 [Thread-10] WARN  DataNucleus.MetaData - MetaData
>> Parser encountered an error in file
>> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
>> at line 535, column 13 : The content of element type "class" must match
>> "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)".
>> - Please check your specification of DTD and the validity of the MetaData
>> XML that you have specified.
>> 2012-10-19 12:35:03,514 [Thread-10] WARN  DataNucleus.MetaData - MetaData
>> Parser encountered an error in file
>> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
>> at line 576, column 13 : The content of element type "class" must match
>> "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)".
>> - Please check your specification of DTD and the validity of the MetaData
>> XML that you have specified.
>> 2012-10-19 12:35:03,518 [Thread-10] WARN  DataNucleus.MetaData - MetaData
>> Parser encountered an error in file
>> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
>> at line 621, column 13 : The content of element type "class" must match
>> "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)".
>> - Please check your specification of DTD and the validity of the MetaData
>> XML that you have specified.
>> 2012-10-19 12:35:03,533 [Thread-10] WARN  DataNucleus.MetaData - MetaData
>> Parser encountered an error in file
>> "jar:file:/usr/lib/hive/lib/hive-metastore-0.7.1-cdh3u5.jar!/package.jdo"
>> at line 666, column 13 : The content of element type "class" must match
>> "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)".
>> - Please check your specification of DTD and the validity of the MetaData
>> XML that you have specified.
>> 2012-10-19 12:35:03,883 [Thread-10] INFO  DataNucleus.Persistence -
>> Managing Persistence of Class :
>> org.apache.hadoop.hive.metastore.model.MDatabase [Table : `DBS`,
>> InheritanceStrategy : new-table]
>> 2012-10-19 12:35:03,965 [Thread-10] INFO  DataNucleus.Persistence -
>> Managing Persistence of Field :
>> org.apache.hadoop.hive.metastore.model.MDatabase.parameters [Table :
>> `DATABASE_PARAMS`]
>> 2012-10-19 12:35:04,057 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 2 index(es) for table `DBS`
>> 2012-10-19 12:35:04,069 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 0 foreign key(s) for table `DBS`
>> 2012-10-19 12:35:04,071 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 2 unique key(s) for table `DBS`
>> 2012-10-19 12:35:04,073 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 2 index(es) for table `DATABASE_PARAMS`
>> 2012-10-19 12:35:04,086 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 1 foreign key(s) for table `DATABASE_PARAMS`
>> 2012-10-19 12:35:04,088 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 1 unique key(s) for table `DATABASE_PARAMS`
>> 2012-10-19 12:35:04,267 [Thread-10] INFO  DataNucleus.MetaData - Listener
>> found initialisation for persistable class
>> org.apache.hadoop.hive.metastore.model.MDatabase
>> 2012-10-19 12:35:04,431 [Thread-10] INFO
>> org.apache.hadoop.hive.metastore.HiveMetaStore - 0: get_table :
>> db=processingpipeline tbl=pagetime
>> 2012-10-19 12:35:04,437 [Thread-10] INFO
>> org.apache.hadoop.hive.metastore.HiveMetaStore.audit - ugi=martin
>> ip=unknown-ip-addr    cmd=get_table : db=processingpipeline tbl=pagetime
>> 2012-10-19 12:35:04,449 [Thread-10] INFO  DataNucleus.Datastore - The
>> class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as
>> "embedded-only" so does not have its own datastore table.
>> 2012-10-19 12:35:04,449 [Thread-10] INFO  DataNucleus.Persistence -
>> Managing Persistence of Class :
>> org.apache.hadoop.hive.metastore.model.MSerDeInfo [Table : `SERDES`,
>> InheritanceStrategy : new-table]
>> 2012-10-19 12:35:04,449 [Thread-10] INFO  DataNucleus.Datastore - The
>> class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as
>> "embedded-only" so does not have its own datastore table.
>> 2012-10-19 12:35:04,449 [Thread-10] INFO  DataNucleus.Persistence -
>> Managing Persistence of Class :
>> org.apache.hadoop.hive.metastore.model.MStorageDescriptor [Table : `SDS`,
>> InheritanceStrategy : new-table]
>> 2012-10-19 12:35:04,450 [Thread-10] INFO  DataNucleus.Persistence -
>> Managing Persistence of Class :
>> org.apache.hadoop.hive.metastore.model.MTable [Table : `TBLS`,
>> InheritanceStrategy : new-table]
>> 2012-10-19 12:35:04,450 [Thread-10] INFO  DataNucleus.Persistence -
>> Managing Persistence of Field :
>> org.apache.hadoop.hive.metastore.model.MSerDeInfo.parameters [Table :
>> `SERDE_PARAMS`]
>> 2012-10-19 12:35:04,451 [Thread-10] INFO  DataNucleus.Persistence -
>> Managing Persistence of Field :
>> org.apache.hadoop.hive.metastore.model.MTable.parameters [Table :
>> `TABLE_PARAMS`]
>> 2012-10-19 12:35:04,455 [Thread-10] INFO  DataNucleus.Persistence -
>> Managing Persistence of Field :
>> org.apache.hadoop.hive.metastore.model.MTable.partitionKeys [Table :
>> `PARTITION_KEYS`]
>> 2012-10-19 12:35:04,466 [Thread-10] INFO  DataNucleus.Persistence -
>> Managing Persistence of Field :
>> org.apache.hadoop.hive.metastore.model.MStorageDescriptor.bucketCols [Table
>> : `BUCKETING_COLS`]
>> 2012-10-19 12:35:04,466 [Thread-10] INFO  DataNucleus.Persistence -
>> Managing Persistence of Field :
>> org.apache.hadoop.hive.metastore.model.MStorageDescriptor.cols [Table :
>> `COLUMNS`]
>> 2012-10-19 12:35:04,476 [Thread-10] INFO  DataNucleus.Persistence -
>> Managing Persistence of Field :
>> org.apache.hadoop.hive.metastore.model.MStorageDescriptor.parameters [Table
>> : `SD_PARAMS`]
>> 2012-10-19 12:35:04,477 [Thread-10] INFO  DataNucleus.Persistence -
>> Managing Persistence of Field :
>> org.apache.hadoop.hive.metastore.model.MStorageDescriptor.sortCols [Table :
>> `SORT_COLS`]
>> 2012-10-19 12:35:04,644 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 1 index(es) for table `SERDES`
>> 2012-10-19 12:35:04,645 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 0 foreign key(s) for table `SERDES`
>> 2012-10-19 12:35:04,646 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 1 unique key(s) for table `SERDES`
>> 2012-10-19 12:35:04,648 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 4 index(es) for table `TBLS`
>> 2012-10-19 12:35:04,656 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 2 foreign key(s) for table `TBLS`
>> 2012-10-19 12:35:04,658 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 2 unique key(s) for table `TBLS`
>> 2012-10-19 12:35:04,668 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 2 index(es) for table `SDS`
>> 2012-10-19 12:35:04,670 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 1 foreign key(s) for table `SDS`
>> 2012-10-19 12:35:04,672 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 1 unique key(s) for table `SDS`
>> 2012-10-19 12:35:04,674 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 2 index(es) for table `SORT_COLS`
>> 2012-10-19 12:35:04,676 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 1 foreign key(s) for table `SORT_COLS`
>> 2012-10-19 12:35:04,684 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 1 unique key(s) for table `SORT_COLS`
>> 2012-10-19 12:35:04,686 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 2 index(es) for table `PARTITION_KEYS`
>> 2012-10-19 12:35:04,687 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 1 foreign key(s) for table `PARTITION_KEYS`
>> 2012-10-19 12:35:04,689 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 1 unique key(s) for table `PARTITION_KEYS`
>> 2012-10-19 12:35:04,690 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 2 index(es) for table `BUCKETING_COLS`
>> 2012-10-19 12:35:04,698 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 1 foreign key(s) for table `BUCKETING_COLS`
>> 2012-10-19 12:35:04,700 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 1 unique key(s) for table `BUCKETING_COLS`
>> 2012-10-19 12:35:04,707 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 2 index(es) for table `TABLE_PARAMS`
>> 2012-10-19 12:35:04,709 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 1 foreign key(s) for table `TABLE_PARAMS`
>> 2012-10-19 12:35:04,717 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 1 unique key(s) for table `TABLE_PARAMS`
>> 2012-10-19 12:35:04,719 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 2 index(es) for table `SD_PARAMS`
>> 2012-10-19 12:35:04,720 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 1 foreign key(s) for table `SD_PARAMS`
>> 2012-10-19 12:35:04,722 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 1 unique key(s) for table `SD_PARAMS`
>> 2012-10-19 12:35:04,723 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 2 index(es) for table `SERDE_PARAMS`
>> 2012-10-19 12:35:04,724 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 1 foreign key(s) for table `SERDE_PARAMS`
>> 2012-10-19 12:35:04,793 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 1 unique key(s) for table `SERDE_PARAMS`
>> 2012-10-19 12:35:04,795 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 2 index(es) for table `COLUMNS`
>> 2012-10-19 12:35:04,796 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 1 foreign key(s) for table `COLUMNS`
>> 2012-10-19 12:35:04,798 [Thread-10] INFO  DataNucleus.Datastore.Schema -
>> Validating 1 unique key(s) for table `COLUMNS`
>> 2012-10-19 12:35:04,813 [Thread-10] INFO  DataNucleus.MetaData - Listener
>> found initialisation for persistable class
>> org.apache.hadoop.hive.metastore.model.MSerDeInfo
>> 2012-10-19 12:35:04,814 [Thread-10] INFO  DataNucleus.MetaData - Listener
>> found initialisation for persistable class
>> org.apache.hadoop.hive.metastore.model.MStorageDescriptor
>> 2012-10-19 12:35:04,814 [Thread-10] INFO  DataNucleus.MetaData - Listener
>> found initialisation for persistable class
>> org.apache.hadoop.hive.metastore.model.MTable
>> 2012-10-19 12:35:04,842 [Thread-10] INFO  DataNucleus.MetaData - Listener
>> found initialisation for persistable class
>> org.apache.hadoop.hive.metastore.model.MFieldSchema
>> 2012-10-19 12:35:04,867 [Thread-10] INFO
>> org.apache.hadoop.hive.metastore.HiveMetaStore - 0:
>> get_partitions_by_filter : db=processingpipeline tbl=pagetime
>> 2012-10-19 12:35:04,867 [Thread-10] INFO
>> org.apache.hadoop.hive.metastore.HiveMetaStore.audit - ugi=martin
>> ip=unknown-ip-addr    cmd=get_partitions_by_filter : db=processingpipeline
>> tbl=pagetime
>> 2012-10-19 12:35:04,890 [Thread-10] INFO
>> org.apache.hadoop.mapred.JobClient - Cleaning up the staging area hdfs://
>> namenode.hadoop.processing.com:54310/mnt/hadoop/tmp/mapred/staging/martin/.staging/job_201210181624_0767
>> 2012-10-19 12:35:04,890 [main] INFO
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
>> - 100% complete
>> 2012-10-19 12:35:04,890 [main] ERROR
>> org.apache.pig.tools.pigstats.PigStatsUtil - 0 map reduce job(s) failed!
>> 2012-10-19 12:35:04,892 [main] INFO
>> org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
>>
>> HadoopVersion    PigVersion    UserId    StartedAt    FinishedAt
>> Features
>> 0.20.2-cdh3u5    0.10.0-SNAPSHOT    martin    2012-10-19 12:34:43
>> 2012-10-19 12:35:04    FILTER,LIMIT
>>
>> Failed!
>>
>> Failed Jobs:
>> JobId    Alias    Feature    Message    Outputs
>>
>> Input(s):
>>
>> Output(s):
>>
>> Counters:
>> Total records written : 0
>> Total bytes written : 0
>> Spillable Memory Manager spill count : 0
>> Total bags proactively spilled: 0
>> Total records proactively spilled: 0
>>
>> Job DAG:
>> null
>>
>>
>> 2012-10-19 12:35:04,892 [main] INFO
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
>> - Failed!
>> 2012-10-19 12:35:04,894 [main] ERROR org.apache.pig.tools.grunt.Grunt -
>> ERROR 1066: Unable to open iterator for alias ll
>> Details at logfile: /YYYY/pig_1350650023453.log
>> grunt>
>>
>>
>>
>> $> cat /YYYYYY//pig_1350650023453.log
>>
>> Pig Stack Trace
>> ---------------
>> ERROR 1066: Unable to open iterator for alias ll
>>
>> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
>> open iterator for alias ll
>>     at org.apache.pig.PigServer.openIterator(PigServer.java:857)
>>     at
>> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:682)
>>     at
>> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
>>     at
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
>>     at
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>>     at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
>>     at org.apache.pig.Main.run(Main.java:490)
>>     at org.apache.pig.Main.main(Main.java:111)
>>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>     at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>     at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>     at java.lang.reflect.Method.invoke(Method.java:597)
>>     at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
>> Caused by: java.io.IOException: Couldn't retrieve job.
>>     at org.apache.pig.PigServer.store(PigServer.java:921)
>>     at org.apache.pig.PigServer.openIterator(PigServer.java:832)
>>     ... 12 more
>>
>> ================================================================================
>>
>>
>>
>

Mime
View raw message