chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <ey...@yahoo-inc.com>
Subject Re: questions about pig
Date Wed, 05 Jan 2011 21:33:28 GMT
I am also getting the same problem on my cluster.  I will have a patch for the empty row key
problem soon.

Regards,
Eric

On 1/4/11 8:42 PM, "Eric Yang" <eric818@gmail.com> wrote:

It looks like HbaseStorage is intentional to make empty row key
invalid.  Nothing can be done at script side to skip this.  You should
be able to fetch the empty row with:

get 'SystemMetrics, ''

in hbase shell.  If something responded, then you need to delete this row.

deleteall 'SystemMetrics', ''

Question is, how do you end up with a empty row key?  I can figure out
how this is possible if the metrics are streamed by using
SystemMetrics Adaptor.  Any idea?

regards,
Eric

On Tue, Jan 4, 2011 at 6:05 PM, Ariel Rabkin <asrabkin@gmail.com> wrote:
> Hm.
>
> Table is biggish; awk. to scan by hand. Can we modify the script to
> ignore empty rows?
>
> --Ari
>
> On Tue, Jan 4, 2011 at 8:35 PM, Eric Yang <eyang@yahoo-inc.com> wrote:
>> This looks like the row key is empty after parsing.  What does the row key look like
in SystemMetrics table?
>> The expected format is:
>>
>> 1234567890000-hostname
>>
>> Make sure there is no empty row key in SystemMetrics table.
>>
>> Regards,
>> Eric
>>
>> On 1/4/11 5:09 PM, "Ariel Rabkin" <asrabkin@gmail.com> wrote:
>>
>> So I have pig+hbase running. Thanks so much!
>>
>> But now I get the following error, from the System Metrics aggregation:
>>
>> java.io.IOException: java.lang.IllegalArgumentException: Row key is invalid
>>        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.runPipeline(PigMapReduce.java:438)
>>        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.processOnePackageOutput(PigMapReduce.java:401)
>>        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:381)
>>        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:251)
>>        at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
>>        at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>>        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>>        at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216)
>> Caused by: java.lang.IllegalArgumentException: Row key is invalid
>>        at org.apache.hadoop.hbase.client.Put.<init>(Put.java:79)
>>        at org.apache.hadoop.hbase.client.Put.<init>(Put.java:69)
>>        at org.apache.pig.backend.hadoop.hbase.HBaseStorage.putNext(HBaseStorage.java:355)
>>        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:138)
>>        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:97)
>>        at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:508)
>>        at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
>>        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.runPipeline(PigMapReduce.java:436)
>>        ... 7 more
>>
>>
>> Thoughts?
>>
>>
>>
>> --
>> Ari Rabkin asrabkin@gmail.com
>> UC Berkeley Computer Science Department
>>
>>
>
>
>
> --
> Ari Rabkin asrabkin@gmail.com
> UC Berkeley Computer Science Department
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message