chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhang xiaolong (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CHUKWA-673) pig script error
Date Thu, 06 Dec 2012 06:28:58 GMT

    [ https://issues.apache.org/jira/browse/CHUKWA-673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511145#comment-13511145
] 

zhang xiaolong commented on CHUKWA-673:
---------------------------------------

Eric,
My configuration in chukwa-agen-conf.xml is:
   <property>
      <name>chukwaAgent.tags</name>
      <value>cluster="demo"</value>
      <description>The cluster's name for this agent</description>
  </property>
The cluster's name is not empty.
But when I use Hbase to get some rows in 'SystemMetrics', there are some rows lack value of
'tags:cluster'.And if the 'tags:cluster' is empty, the "ConcatBuffer = foreach CleanseBuffer
generate CONCAT(CONCAT($0, '-'), $1) as rowId,... "
in script may cause error.



Xiaolong Zhang

From: Eric Yang (JIRA)
Date: 2012-12-06 13:16
To: 54zhangxiaolong
Subject: [jira] [Comment Edited] (CHUKWA-673) pig script error

    [ https://issues.apache.org/jira/browse/CHUKWA-673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511120#comment-13511120
] 

Eric Yang edited comment on CHUKWA-673 at 12/6/12 5:15 AM:
-----------------------------------------------------------

Xiaolong, -loadKey option can be expressed as -loadKey or -loadKey=true.  -loadKey true is
actually wrong.  Ctag can be missing, if 

{code}
  <property>
    <name>chukwaAgent.tags</name>
    <value>cluster="${TODO_CLUSTER_NAME}"</value>
    <description>The cluster's name for this agent</description>
  </property>
{code}

If TODO_CLUSTER_NAME is empty.

Can you confirm this is what happened?  I can't reproduce problem described here.
                
      was (Author: eyang):
    Xiaolong, -loadKey option can be expressed as -loadKey or -loadKey=true.  -loadKey true
is actually wrong.  Ctag can be missing, if 

{code}
  <property>
    <name>chukwaAgent.tags</name>
    <value>cluster="${TODO_CLUSTER_NAME}"</value>
    <description>The cluster's name for this agent</description>
  </property>
{code}

If TODO_CLUSTER_NAME is empty.
                  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

                
> pig script error 
> -----------------
>
>                 Key: CHUKWA-673
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-673
>             Project: Chukwa
>          Issue Type: Bug
>          Components: Scripts
>    Affects Versions: 0.5.0
>         Environment: Hadoop1.03+Hbase0.92.0+Chukwa0.50+Java6
>            Reporter: zhang xiaolong
>
> Pig script ClusterSummary.pig may have some problems.
> "... network:TxErrors network:TxPackets','-loadKey -gt $START -caster Utf8StorageConverter')
AS ...." in ClusterSummary.pig didn't piont out "-loadKey (true/false)",and should be "-loadKey
true".
> And many rows stored in "hbase://SystemMetrics" didn't have value of "tags:cluster".
The script use rowkey and "tags:cluster" to generate rowId. This may cause problems. I suggest
use rowkey and "XX:ctags" to generate rowId. 
> -------------------------------------------------------------------------------------
> When I run Pig script ClusterSummary.pig , got error:
>  Unable to recreate exception from backed error: java.io.IOException: java.lang.IllegalArgumentException:
Row key is invalid
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:465)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.processOnePackageOutput(PigGenericMapReduce.java:428)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:408)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:262)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:417)
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>         at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: java.lang.IllegalArgumentException: Row key is invalid
>         at org.apache.hadoop.hbase.client.Put.<init>(Put.java:93)
>         at org.apache.hadoop.hbase.client.Put.<init>(Put.java:72)
>         at org.apache.hadoop.hbase.client.Put.<init>(Put.java:63)
>         at org.apache.pig.backend.hadoop.hbase.HBaseStorage.createPut(HBaseStorage.java:712)
>         at org.apache.pig.backend.hadoop.hbase.HBaseStorage.putNext(HBaseStorage.java:660)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98)
>         at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:587)
>         at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:463)
>         ... 11 more
> -------------------------------------------------------------------------------------

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message