hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2432) enhance hbase.util.Bytes.toBytes() with length limit
Date Mon, 12 Apr 2010 15:26:42 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12856014#action_12856014
] 

stack commented on HBASE-2432:
------------------------------

What are you trying to achieve Ted?  You want to make a row that is < 32k? (You realize
that maximum row size in 32k in hbase?).

Regards your patch, you are cutting the byte array at an arbitrary point.  Do you want to
do that?  What if your cut happens in the middle of a multibyte character.  You'll have difficulty
making a String of it out on the other side (presuming you want to do such a thing which seems
likely given your source is a String).

> enhance hbase.util.Bytes.toBytes() with length limit
> ----------------------------------------------------
>
>                 Key: HBASE-2432
>                 URL: https://issues.apache.org/jira/browse/HBASE-2432
>             Project: Hadoop HBase
>          Issue Type: Improvement
>          Components: util
>    Affects Versions: 0.20.1
>            Reporter: Ted Yu
>         Attachments: HBASE-2432.patch, HBASE-2432.patch
>
>
> The following stack trace is seen in our hadoop log:
> java.lang.IllegalArgumentException: Row > 32767
> 	at org.apache.hadoop.hbase.KeyValue.createByteArray(KeyValue.java:437)
> 	at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:405)
> 	at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:374)
> 	at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:353)
> 	at org.apache.hadoop.hbase.client.Put.add(Put.java:137)
> 	at org.apache.hadoop.hbase.client.Put.add(Put.java:108)
> 	at org.apache.nutch.scoring.webgraph.ScoreUpdater$ScoreUpdaterReducer.reduce(ScoreUpdater.java:170)
> 	at org.apache.nutch.scoring.webgraph.ScoreUpdater$ScoreUpdaterReducer.reduce(ScoreUpdater.java:127)
> 	at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> 	at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:563)
> 	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Bytes.toBytes(Float.valueOf(score).toString()) may return an array longer than 32767
bytes.
> We should enhance Bytes.toBytes() to include length limit:
>   public static byte[] toBytes(String s, int length) 
> String.getBytes() doesn't have length limit

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message