hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11523) JSON serialization of PE Options is broke
Date Wed, 16 Jul 2014 17:36:06 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14063765#comment-14063765
] 

stack commented on HBASE-11523:
-------------------------------

This 'trivial change in pe' by me changing TestOptions data members from public to package
private is what broke the serialization.

{code}
96681210 ยป stack 
2014-06-26 HBASE-11415 [PE] Dump config before running test		547	    boolean nomapred = false;
548	    boolean filterAll = false;
549	    int startRow = 0;
550	    float size = 1.0f;
551	    int perClientRunRows = DEFAULT_ROWS_PER_GB;
552	    int numClientThreads = 1;
553	    int totalRows = DEFAULT_ROWS_PER_GB;
554	    float sampleRate = 1.0f;
555	    double traceRate = 0.0;
556	    String tableName = TABLE_NAME;
557	    boolean flushCommits = true;
558	    boolean writeToWAL = true;
559	    boolean autoFlush = false;
560	    boolean oneCon = false;
561	    boolean useTags = false;
562	    int noOfTags = 1;
563	    boolean reportLatency = false;
564	    int multiGet = 0;
{code}

I did it so could get insight on the configs PE was running with.  I wanted to use JSON to
print out the configs so didn't have to dick around w/ toString adding stuff as new configs
were added.  I added getters so could control what JSON emitted (private stuff would be ignored).

Sorry about that.  Thanks for the help [~ndimiduk]

I suppose could go back to all public but then can't hide TestOptions data members.... which
is probably not going to be needed.

Yeah, PE is getting unwieldy.  Start thinking about breaking it out?  A module of its own
with tests...



> JSON serialization of PE Options is broke
> -----------------------------------------
>
>                 Key: HBASE-11523
>                 URL: https://issues.apache.org/jira/browse/HBASE-11523
>             Project: HBase
>          Issue Type: Bug
>            Reporter: stack
>            Assignee: stack
>             Fix For: 0.99.0, 2.0.0
>
>         Attachments: 11520v3.txt, 11523v2.txt, HBASE-11523.00.nd.patch
>
>
> I see this when I try to run a PE MR job on master:
> {code}
> 4/07/15 22:02:27 INFO mapreduce.Job: Task Id : attempt_1405482830657_0004_m_000015_2,
Status : FAILED
> Error: org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field
"blockEncoding" (Class org.apache.hadoop.hbase.PerformanceEvaluation$TestOptions), not marked
as ignorable
>  at [Source: java.io.StringReader@41c7d592; line: 1, column: 37] (through reference chain:
org.apache.hadoop.hbase.TestOptions["blockEncoding"])
> 	at org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53)
> 	at org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:246)
> 	at org.codehaus.jackson.map.deser.StdDeserializer.reportUnknownProperty(StdDeserializer.java:604)
> 	at org.codehaus.jackson.map.deser.StdDeserializer.handleUnknownProperty(StdDeserializer.java:590)
> 	at org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:689)
> 	at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:514)
> 	at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:350)
> 	at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2402)
> 	at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1602)
> 	at org.apache.hadoop.hbase.PerformanceEvaluation$EvaluationMapTask.map(PerformanceEvaluation.java:255)
> 	at org.apache.hadoop.hbase.PerformanceEvaluation$EvaluationMapTask.map(PerformanceEvaluation.java:210)
> 	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
> 	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
> 	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
> {code}
> The JSON serialization of PE Options does not seem to be working.  If I add a setter,
it does work so unless I hear otherwise, seems like adding a setter for each PE option is
the way to go (I like the new JSON serialization).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message