Return-Path: X-Original-To: apmail-accumulo-commits-archive@www.apache.org Delivered-To: apmail-accumulo-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 526F1F118 for ; Sun, 5 May 2013 02:45:54 +0000 (UTC) Received: (qmail 97506 invoked by uid 500); 5 May 2013 02:45:52 -0000 Delivered-To: apmail-accumulo-commits-archive@accumulo.apache.org Received: (qmail 97465 invoked by uid 500); 5 May 2013 02:45:52 -0000 Mailing-List: contact commits-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@accumulo.apache.org Delivered-To: mailing list commits@accumulo.apache.org Received: (qmail 97456 invoked by uid 99); 5 May 2013 02:45:52 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 05 May 2013 02:45:52 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 05 May 2013 02:45:42 +0000 Received: from eris.apache.org (localhost [127.0.0.1]) by eris.apache.org (Postfix) with ESMTP id 4454C2388847; Sun, 5 May 2013 02:45:20 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1479229 - in /accumulo/trunk: core/src/main/java/org/apache/accumulo/core/client/lexicoder/ core/src/main/java/org/apache/accumulo/core/client/lexicoder/util/ core/src/main/java/org/apache/accumulo/core/util/ core/src/test/java/org/apache/... Date: Sun, 05 May 2013 02:45:18 -0000 To: commits@accumulo.apache.org From: elserj@apache.org X-Mailer: svnmailer-1.0.8-patched Message-Id: <20130505024520.4454C2388847@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: elserj Date: Sun May 5 02:45:17 2013 New Revision: 1479229 URL: http://svn.apache.org/r1479229 Log: ACCUMULO-1336 Apply patch from Corey Nolet to add Typo "lexicoders" to Accumulo Added: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/BigIntegerLexicoder.java accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/BytesLexicoder.java accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/DateLexicoder.java accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/DoubleLexicoder.java accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/Encoder.java accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/IntegerLexicoder.java accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/Lexicoder.java accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/ListLexicoder.java accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/LongLexicoder.java accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/PairLexicoder.java accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/ReverseLexicoder.java accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/StringLexicoder.java accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/TextLexicoder.java accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/UIntegerLexicoder.java accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/ULongLexicoder.java accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/UUIDLexicoder.java accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/util/ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/util/ByteUtils.java accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/util/FixedByteArrayOutputStream.java accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/ accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/BigIntegerLexicoderTest.java accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/DoubleLexicoderTest.java accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/IntegerLexicoderTest.java accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/LexicoderTest.java accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/ListLexicoderTest.java accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/LongLexicoderTest.java accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/PairLexicoderTest.java accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/ReverseLexicoderTest.java accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/UIntegerLexicoderTest.java accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/ULongLexicoderTest.java accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/UUIDLexicoderTest.java Modified: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/util/Pair.java accumulo/trunk/docs/src/main/latex/accumulo_user_manual/chapters/table_design.tex Added: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/BigIntegerLexicoder.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/BigIntegerLexicoder.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/BigIntegerLexicoder.java (added) +++ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/BigIntegerLexicoder.java Sun May 5 02:45:17 2013 @@ -0,0 +1,83 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +import java.io.ByteArrayInputStream; +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; +import java.math.BigInteger; + +import org.apache.accumulo.core.iterators.ValueFormatException; +import org.apache.accumulo.core.client.lexicoder.util.FixedByteArrayOutputStream; + +/** + * + */ +public class BigIntegerLexicoder implements Lexicoder { + + @Override + public byte[] encode(BigInteger v) { + + try { + byte[] bytes = v.toByteArray(); + + byte[] ret = new byte[4 + bytes.length]; + + DataOutputStream dos = new DataOutputStream(new FixedByteArrayOutputStream(ret)); + + // flip the sign bit + bytes[0] = (byte) (0x80 ^ bytes[0]); + + int len = bytes.length; + if (v.signum() < 0) + len = -len; + + len = len ^ 0x80000000; + + dos.writeInt(len); + dos.write(bytes); + + return ret; + } catch (IOException ioe) { + throw new RuntimeException(ioe); + } + + + } + + @Override + public BigInteger decode(byte[] b) throws ValueFormatException { + + try { + DataInputStream dis = new DataInputStream(new ByteArrayInputStream(b)); + int len = dis.readInt(); + len = len ^ 0x80000000; + len = Math.abs(len); + + byte[] bytes = new byte[len]; + dis.readFully(bytes); + + bytes[0] = (byte) (0x80 ^ bytes[0]); + + return new BigInteger(bytes); + } catch (IOException ioe) { + throw new RuntimeException(ioe); + } + } + +} Added: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/BytesLexicoder.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/BytesLexicoder.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/BytesLexicoder.java (added) +++ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/BytesLexicoder.java Sun May 5 02:45:17 2013 @@ -0,0 +1,34 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +/** + * + */ +public class BytesLexicoder implements Lexicoder { + + @Override + public byte[] encode(byte[] data) { + return data; + } + + @Override + public byte[] decode(byte[] data) { + return data; + } + +} Added: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/DateLexicoder.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/DateLexicoder.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/DateLexicoder.java (added) +++ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/DateLexicoder.java Sun May 5 02:45:17 2013 @@ -0,0 +1,38 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +import java.util.Date; + +/** + * + */ +public class DateLexicoder implements Lexicoder { + + private ULongLexicoder longEncoder = new ULongLexicoder(); + + @Override + public byte[] encode(Date data) { + return longEncoder.encode(data.getTime()); + } + + @Override + public Date decode(byte[] data) { + return new Date(longEncoder.decode(data)); + } + +} Added: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/DoubleLexicoder.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/DoubleLexicoder.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/DoubleLexicoder.java (added) +++ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/DoubleLexicoder.java Sun May 5 02:45:17 2013 @@ -0,0 +1,47 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +/** + * + */ +public class DoubleLexicoder implements Lexicoder { + + private ULongLexicoder longEncoder = new ULongLexicoder(); + + @Override + public byte[] encode(Double d) { + long l = Double.doubleToRawLongBits(d); + if (l < 0) + l = ~l; + else + l = l ^ 0x8000000000000000l; + + return longEncoder.encode(l); + } + + @Override + public Double decode(byte[] data) { + long l = longEncoder.decode(data); + if (l < 0) + l = l ^ 0x8000000000000000l; + else + l = ~l; + return Double.longBitsToDouble(l); + } + +} Added: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/Encoder.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/Encoder.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/Encoder.java (added) +++ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/Encoder.java Sun May 5 02:45:17 2013 @@ -0,0 +1,24 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +/** + * + */ +public interface Encoder extends org.apache.accumulo.core.iterators.TypedValueCombiner.Encoder { + +} Added: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/IntegerLexicoder.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/IntegerLexicoder.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/IntegerLexicoder.java (added) +++ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/IntegerLexicoder.java Sun May 5 02:45:17 2013 @@ -0,0 +1,36 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +/** + * + */ +public class IntegerLexicoder implements Lexicoder { + + private UIntegerLexicoder uil = new UIntegerLexicoder(); + + @Override + public byte[] encode(Integer i) { + return uil.encode(i ^ 0x80000000); + } + + @Override + public Integer decode(byte[] data) { + return uil.decode(data) ^ 0x80000000; + } + +} Added: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/Lexicoder.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/Lexicoder.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/Lexicoder.java (added) +++ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/Lexicoder.java Sun May 5 02:45:17 2013 @@ -0,0 +1,24 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +/** + * An encoder that produces byte arrays in which the lexicographic sort order corresponds to the types sort order. + */ +public interface Lexicoder extends Encoder { + +} Added: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/ListLexicoder.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/ListLexicoder.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/ListLexicoder.java (added) +++ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/ListLexicoder.java Sun May 5 02:45:17 2013 @@ -0,0 +1,60 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +import static org.apache.accumulo.core.client.lexicoder.util.ByteUtils.concat; +import static org.apache.accumulo.core.client.lexicoder.util.ByteUtils.escape; +import static org.apache.accumulo.core.client.lexicoder.util.ByteUtils.split; +import static org.apache.accumulo.core.client.lexicoder.util.ByteUtils.unescape; + +import java.util.ArrayList; +import java.util.List; + +public class ListLexicoder implements Lexicoder> { + + private Lexicoder lexicoder; + + public ListLexicoder(Lexicoder lexicoder) { + this.lexicoder = lexicoder; + } + + @Override + public byte[] encode(List v) { + byte[][] encElements = new byte[v.size()][]; + + int index = 0; + for (LT element : v) { + encElements[index++] = escape(lexicoder.encode(element)); + } + + return concat(encElements); + } + + @Override + public List decode(byte[] b) { + + byte[][] escapedElements = split(b); + ArrayList ret = new ArrayList(escapedElements.length); + + for (byte[] escapedElement : escapedElements) { + ret.add(lexicoder.decode(unescape(escapedElement))); + } + + return ret; + } + +} Added: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/LongLexicoder.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/LongLexicoder.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/LongLexicoder.java (added) +++ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/LongLexicoder.java Sun May 5 02:45:17 2013 @@ -0,0 +1,33 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +/** + * Signed long encoder. The lexicographic encoding sorts Long.MIN_VALUE first and Long.MAX_VALUE last. The lexicographic encoding sorts -2l before -1l. It + * corresponds to the sort order of Long. + */ +public class LongLexicoder extends ULongLexicoder { + @Override + public byte[] encode(Long l) { + return super.encode(l ^ 0x8000000000000000l); + } + + @Override + public Long decode(byte[] data) { + return super.decode(data) ^ 0x8000000000000000l; + } +} Added: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/PairLexicoder.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/PairLexicoder.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/PairLexicoder.java (added) +++ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/PairLexicoder.java Sun May 5 02:45:17 2013 @@ -0,0 +1,52 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +import static org.apache.accumulo.core.client.lexicoder.util.ByteUtils.concat; +import static org.apache.accumulo.core.client.lexicoder.util.ByteUtils.escape; +import static org.apache.accumulo.core.client.lexicoder.util.ByteUtils.split; +import static org.apache.accumulo.core.client.lexicoder.util.ByteUtils.unescape; + +import org.apache.accumulo.core.util.Pair; + +public class PairLexicoder implements Lexicoder> { + + private Lexicoder firstLexicoder; + private Lexicoder secondLexicoder; + + public PairLexicoder(Lexicoder firstLexicoder, Lexicoder secondLexicoder) { + this.firstLexicoder = firstLexicoder; + this.secondLexicoder = secondLexicoder; + } + + @Override + public byte[] encode(Pair data) { + return concat(escape(firstLexicoder.encode(data.getFirst())), escape(secondLexicoder.encode(data.getSecond()))); + } + + @Override + public Pair decode(byte[] data) { + + byte[][] fields = split(data); + if (fields.length != 2) { + throw new RuntimeException("Data does not have 2 fields, it has " + fields.length); + } + + return new Pair(firstLexicoder.decode(unescape(fields[0])), secondLexicoder.decode(unescape(fields[1]))); + } + +} Added: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/ReverseLexicoder.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/ReverseLexicoder.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/ReverseLexicoder.java (added) +++ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/ReverseLexicoder.java Sun May 5 02:45:17 2013 @@ -0,0 +1,52 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +import static org.apache.accumulo.core.client.lexicoder.util.ByteUtils.escape; +import static org.apache.accumulo.core.client.lexicoder.util.ByteUtils.unescape; + +public class ReverseLexicoder implements Lexicoder { + + private Lexicoder lexicoder; + + public ReverseLexicoder(Lexicoder lexicoder) { + this.lexicoder = lexicoder; + } + + @Override + public byte[] encode(T data) { + byte[] bytes = escape(lexicoder.encode(data)); + byte[] ret = new byte[bytes.length + 1]; + + for (int i = 0; i < bytes.length; i++) + ret[i] = (byte) (0xff - (0xff & bytes[i])); + + ret[bytes.length] = (byte) 0xff; + + return ret; + } + + @Override + public T decode(byte[] data) { + byte ret[] = new byte[data.length - 1]; + + for (int i = 0; i < ret.length; i++) + ret[i] = (byte) (0xff - (0xff & data[i])); + + return lexicoder.decode(unescape(ret)); + } +} Added: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/StringLexicoder.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/StringLexicoder.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/StringLexicoder.java (added) +++ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/StringLexicoder.java Sun May 5 02:45:17 2013 @@ -0,0 +1,41 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +import java.io.UnsupportedEncodingException; + +public class StringLexicoder implements Lexicoder { + + @Override + public byte[] encode(String data) { + try { + return data.getBytes("UTF-8"); + } catch (UnsupportedEncodingException e) { + throw new RuntimeException(e); + } + } + + @Override + public String decode(byte[] data) { + try { + return new String(data, "UTF-8"); + } catch (UnsupportedEncodingException e) { + throw new RuntimeException(e); + } + } + +} Added: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/TextLexicoder.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/TextLexicoder.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/TextLexicoder.java (added) +++ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/TextLexicoder.java Sun May 5 02:45:17 2013 @@ -0,0 +1,34 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +import org.apache.accumulo.core.util.TextUtil; +import org.apache.hadoop.io.Text; + +public class TextLexicoder implements Lexicoder { + + @Override + public byte[] encode(Text data) { + return TextUtil.getBytes(data); + } + + @Override + public Text decode(byte[] data) { + return new Text(data); + } + +} Added: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/UIntegerLexicoder.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/UIntegerLexicoder.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/UIntegerLexicoder.java (added) +++ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/UIntegerLexicoder.java Sun May 5 02:45:17 2013 @@ -0,0 +1,72 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +/** + * + */ +public class UIntegerLexicoder implements Lexicoder { + + @Override + public byte[] encode(Integer i) { + int shift = 56; + int index; + int prefix = i < 0 ? 0xff : 0x00; + + for (index = 0; index < 4; index++) { + if (((i >>> shift) & 0xff) != prefix) + break; + + shift -= 8; + } + + byte ret[] = new byte[5 - index]; + ret[0] = (byte) (4 - index); + for (index = 1; index < ret.length; index++) { + ret[index] = (byte) (i >>> shift); + shift -= 8; + } + + if (i < 0) + ret[0] = (byte) (8 - ret[0]); + + return ret; + + } + + @Override + public Integer decode(byte[] data) { + + if (data[0] < 0 || data[0] > 8) + throw new IllegalArgumentException("Unexpected length " + (0xff & data[0])); + + int i = 0; + int shift = 0; + + for (int idx = data.length - 1; idx >= 1; idx--) { + i += (data[idx] & 0xffl) << shift; + shift += 8; + } + + // fill in 0xff prefix + if (data[0] > 4) + i |= -1 << ((8 - data[0]) << 3); + + return i; + } + +} Added: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/ULongLexicoder.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/ULongLexicoder.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/ULongLexicoder.java (added) +++ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/ULongLexicoder.java Sun May 5 02:45:17 2013 @@ -0,0 +1,72 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +/** + * Unsigned long encoder. The lexicographic encoding sorts first 0l and -1l last. This encoding does not correspond to the sort of Long because it does not + * consider the sign bit. If Java had an unsigned long type, this encoder would correspond to its sort order. + */ +public class ULongLexicoder implements Lexicoder { + + @Override + public byte[] encode(Long l) { + int shift = 56; + int index; + int prefix = l < 0 ? 0xff : 0x00; + + for (index = 0; index < 8; index++) { + if (((l >>> shift) & 0xff) != prefix) + break; + + shift -= 8; + } + + byte ret[] = new byte[9 - index]; + ret[0] = (byte) (8 - index); + for (index = 1; index < ret.length; index++) { + ret[index] = (byte) (l >>> shift); + shift -= 8; + } + + if (l < 0) + ret[0] = (byte) (16 - ret[0]); + + return ret; + + } + + @Override + public Long decode(byte[] data) { + + long l = 0; + int shift = 0; + + if (data[0] < 0 || data[0] > 16) + throw new IllegalArgumentException("Unexpected length " + (0xff & data[0])); + + for (int i = data.length - 1; i >= 1; i--) { + l += (data[i] & 0xffl) << shift; + shift += 8; + } + + // fill in 0xff prefix + if (data[0] > 8) + l |= -1l << ((16 - data[0]) << 3); + + return l; + } +} Added: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/UUIDLexicoder.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/UUIDLexicoder.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/UUIDLexicoder.java (added) +++ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/UUIDLexicoder.java Sun May 5 02:45:17 2013 @@ -0,0 +1,57 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +import java.io.ByteArrayInputStream; +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; +import java.util.UUID; + +import org.apache.accumulo.core.iterators.ValueFormatException; +import org.apache.accumulo.core.client.lexicoder.util.FixedByteArrayOutputStream; + +public class UUIDLexicoder implements Lexicoder { + + @Override + public byte[] encode(UUID uuid) { + try { + byte ret[] = new byte[16]; + DataOutputStream out = new DataOutputStream(new FixedByteArrayOutputStream(ret)); + + out.writeLong(uuid.getMostSignificantBits() ^ 0x8000000000000000l); + out.writeLong(uuid.getLeastSignificantBits() ^ 0x8000000000000000l); + + out.close(); + + return ret; + } catch (IOException e) { + throw new RuntimeException(e); + } + } + + @Override + public UUID decode(byte[] b) throws ValueFormatException { + try { + DataInputStream in = new DataInputStream(new ByteArrayInputStream(b)); + return new UUID(in.readLong() ^ 0x8000000000000000l, in.readLong() ^ 0x8000000000000000l); + } catch (IOException e) { + throw new RuntimeException(e); + } + } + +} Added: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/util/ByteUtils.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/util/ByteUtils.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/util/ByteUtils.java (added) +++ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/util/ByteUtils.java Sun May 5 02:45:17 2013 @@ -0,0 +1,116 @@ +package org.apache.accumulo.core.client.lexicoder.util; + +import java.util.ArrayList; + +public class ByteUtils { + + /** + * Escapes 0x00 with 0x01 0x01 and 0x01 with 0x01 0x02 + * + * @param in + * @return + */ + public static byte[] escape(byte[] in) { + int escapeCount = 0; + for (int i = 0; i < in.length; i++) { + if (in[i] == 0x00 || in[i] == 0x01) { + escapeCount++; + } + } + + if (escapeCount == 0) + return in; + + byte ret[] = new byte[escapeCount + in.length]; + int index = 0; + + for (int i = 0; i < in.length; i++) { + switch (in[i]) { + case 0x00: + ret[index++] = 0x01; + ret[index++] = 0x01; + break; + case 0x01: + ret[index++] = 0x01; + ret[index++] = 0x02; + break; + default: + ret[index++] = in[i]; + } + } + + return ret; + } + + public static byte[] unescape(byte[] in) { + int escapeCount = 0; + for (int i = 0; i < in.length; i++) { + if (in[i] == 0x01) { + escapeCount++; + i++; + } + } + + if (escapeCount == 0) + return in; + + byte ret[] = new byte[in.length - escapeCount]; + + int index = 0; + for (int i = 0; i < in.length; i++) { + if (in[i] == 0x01) { + i++; + ret[index++] = (byte) (in[i] - 1); + } else { + ret[index++] = in[i]; + } + + } + + return ret; + } + + public static byte[][] split(byte[] data) { + ArrayList offsets = new ArrayList(); + + for (int i = 0; i < data.length; i++) { + if (data[i] == 0x00) { + offsets.add(i); + } + } + + offsets.add(data.length); + + byte[][] ret = new byte[offsets.size()][]; + + int index = 0; + for (int i = 0; i < offsets.size(); i++) { + ret[i] = new byte[offsets.get(i) - index]; + System.arraycopy(data, index, ret[i], 0, ret[i].length); + index = offsets.get(i) + 1; + } + + return ret; + } + + public static byte[] concat(byte[]... fields) { + int len = 0; + for (byte[] field : fields) { + len += field.length; + } + + byte ret[] = new byte[len + fields.length - 1]; + int index = 0; + + for (byte[] field : fields) { + System.arraycopy(field, 0, ret, index, field.length); + index += field.length; + if (index < ret.length) + ret[index++] = 0x00; + } + + return ret; + } + + +} Added: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/util/FixedByteArrayOutputStream.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/util/FixedByteArrayOutputStream.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/util/FixedByteArrayOutputStream.java (added) +++ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/lexicoder/util/FixedByteArrayOutputStream.java Sun May 5 02:45:17 2013 @@ -0,0 +1,42 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder.util; + +import java.io.IOException; +import java.io.OutputStream; + +public class FixedByteArrayOutputStream extends OutputStream { + + private int i; + byte out[]; + + public FixedByteArrayOutputStream(byte out[]) { + this.out = out; + } + + @Override + public void write(int b) throws IOException { + out[i++] = (byte) b; + } + + @Override + public void write(byte b[], int off, int len) throws IOException { + System.arraycopy(b, off, out, i, len); + i += len; + } + +} Modified: accumulo/trunk/core/src/main/java/org/apache/accumulo/core/util/Pair.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/main/java/org/apache/accumulo/core/util/Pair.java?rev=1479229&r1=1479228&r2=1479229&view=diff ============================================================================== --- accumulo/trunk/core/src/main/java/org/apache/accumulo/core/util/Pair.java (original) +++ accumulo/trunk/core/src/main/java/org/apache/accumulo/core/util/Pair.java Sun May 5 02:45:17 2013 @@ -16,7 +16,7 @@ */ package org.apache.accumulo.core.util; -public class Pair { +public class Pair implements Comparable> { A first; B second; @@ -63,4 +63,15 @@ public class Pair { public String toString() { return "(" + first + "," + second + ")"; } + + + @Override + public int compareTo(Pair abPair) { + int cmp = ((Comparable) first).compareTo(abPair.first); + if (cmp == 0) { + cmp = ((Comparable) second).compareTo(abPair.second); + } + + return cmp; + } } Added: accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/BigIntegerLexicoderTest.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/BigIntegerLexicoderTest.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/BigIntegerLexicoderTest.java (added) +++ accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/BigIntegerLexicoderTest.java Sun May 5 02:45:17 2013 @@ -0,0 +1,35 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +import java.math.BigInteger; + +/** + * + */ +public class BigIntegerLexicoderTest extends LexicoderTest { + public void testSortOrder() { + assertSortOrder(new BigIntegerLexicoder(), new BigInteger("-1"), new BigInteger("0"), new BigInteger("1"), new BigInteger("-257"), new BigInteger("-256"), + new BigInteger("-255"), new BigInteger("255"), new BigInteger("256"), new BigInteger("257"), new BigInteger("65534"), new BigInteger("65535"), + new BigInteger("65536"), new BigInteger("65537"), new BigInteger("-65534"), new BigInteger("-65535"), new BigInteger("-65536"), + new BigInteger("-65537"), new BigInteger("2147483648"), new BigInteger("2147483647"), new BigInteger("2147483649"), new BigInteger("-2147483648"), + new BigInteger("-2147483647"), new BigInteger("-2147483649"), new BigInteger("32768"), new BigInteger("32769"), new BigInteger("32767"), + new BigInteger("-32768"), new BigInteger("-32769"), new BigInteger("-32767"), new BigInteger("126"), new BigInteger("127"), new BigInteger("128"), + new BigInteger("129"), new BigInteger("-126"), new BigInteger("-127"), new BigInteger("-128"), new BigInteger("-129")); + + } +} Added: accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/DoubleLexicoderTest.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/DoubleLexicoderTest.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/DoubleLexicoderTest.java (added) +++ accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/DoubleLexicoderTest.java Sun May 5 02:45:17 2013 @@ -0,0 +1,30 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +/** + * + */ +public class DoubleLexicoderTest extends LexicoderTest { + public void testSortOrder() { + assertSortOrder(new DoubleLexicoder(), Double.MIN_VALUE, Double.MAX_VALUE, Double.NEGATIVE_INFINITY, Double.POSITIVE_INFINITY, 0.0, 0.01, 0.001, 1.0, -1.0, + -1.1, -1.01, Math.nextUp(Double.NEGATIVE_INFINITY), Math.nextAfter(0.0, Double.NEGATIVE_INFINITY), + Math.nextAfter(Double.MAX_VALUE, Double.NEGATIVE_INFINITY), Math.pow(10.0, 30.0) * -1.0, Math.pow(10.0, 30.0), Math.pow(10.0, -30.0) * -1.0, + Math.pow(10.0, -30.0)); + + } +} Added: accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/IntegerLexicoderTest.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/IntegerLexicoderTest.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/IntegerLexicoderTest.java (added) +++ accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/IntegerLexicoderTest.java Sun May 5 02:45:17 2013 @@ -0,0 +1,24 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +public class IntegerLexicoderTest extends LexicoderTest { + public void testSortOrder() { + assertSortOrder(new IntegerLexicoder(), Integer.MIN_VALUE, 0xff123456, 0xffff3456, 0xffffff56, -1, 0, 1, 0x12, 0x1234, 0x123456, 0x1234678, + Integer.MAX_VALUE); + } +} Added: accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/LexicoderTest.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/LexicoderTest.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/LexicoderTest.java (added) +++ accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/LexicoderTest.java Sun May 5 02:45:17 2013 @@ -0,0 +1,65 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.Comparator; +import java.util.List; + +import junit.framework.TestCase; + +import org.apache.accumulo.core.client.lexicoder.Lexicoder; +import org.apache.accumulo.core.util.TextUtil; +import org.apache.hadoop.io.Text; + +public abstract class LexicoderTest extends TestCase { + + void assertEqualsB(byte[] ba1, byte[] ba2) { + assertEquals(new Text(ba2), new Text(ba1)); + } + + public > void assertSortOrder(Lexicoder lexicoder, Comparator comp, T... data) { + List list = new ArrayList(); + List encList = new ArrayList(); + + for (T d : data) { + list.add(d); + encList.add(new Text(lexicoder.encode(d))); + } + + if (comp != null) + Collections.sort(list, comp); + else + Collections.sort(list); + + Collections.sort(encList); + + List decodedList = new ArrayList(); + + for (Text t : encList) { + decodedList.add(lexicoder.decode(TextUtil.getBytes(t))); + } + + assertEquals(list, decodedList); + } + + public > void assertSortOrder(Lexicoder lexicoder, T... data) { + assertSortOrder(lexicoder, null, data); + } + +} Added: accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/ListLexicoderTest.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/ListLexicoderTest.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/ListLexicoderTest.java (added) +++ accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/ListLexicoderTest.java Sun May 5 02:45:17 2013 @@ -0,0 +1,74 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +import java.util.ArrayList; +import java.util.List; +import java.util.TreeSet; + +import org.apache.accumulo.core.util.TextUtil; +import org.apache.hadoop.io.Text; + +public class ListLexicoderTest extends LexicoderTest { + public void testSortOrder() { + List data1 = new ArrayList(); + data1.add(1l); + data1.add(2l); + + List data2 = new ArrayList(); + data2.add(1l); + + List data3 = new ArrayList(); + data3.add(1l); + data3.add(3l); + + List data4 = new ArrayList(); + data4.add(1l); + data4.add(2l); + data4.add(3l); + + List data5 = new ArrayList(); + data5.add(2l); + data5.add(1l); + + List> data = new ArrayList>(); + + // add list in expected sort order + data.add(data2); + data.add(data1); + data.add(data4); + data.add(data3); + data.add(data5); + + TreeSet sortedEnc = new TreeSet(); + + ListLexicoder listLexicoder = new ListLexicoder(new LongLexicoder()); + + for (List list : data) { + sortedEnc.add(new Text(listLexicoder.encode(list))); + } + + List> unenc = new ArrayList>(); + + for (Text enc : sortedEnc) { + unenc.add(listLexicoder.decode(TextUtil.getBytes(enc))); + } + + assertEquals(data, unenc); + + } +} Added: accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/LongLexicoderTest.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/LongLexicoderTest.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/LongLexicoderTest.java (added) +++ accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/LongLexicoderTest.java Sun May 5 02:45:17 2013 @@ -0,0 +1,26 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +public class LongLexicoderTest extends LexicoderTest { + public void testSortOrder() { + + assertSortOrder(new LongLexicoder(), Long.MIN_VALUE, 0xff1234567890abcdl, 0xffff1234567890abl, 0xffffff567890abcdl, 0xffffffff7890abcdl, + 0xffffffffff90abcdl, 0xffffffffffffabcdl, 0xffffffffffffffcdl, -1l, 0l, 0x01l, 0x1234l, 0x123456l, 0x12345678l, 0x1234567890l, 0x1234567890abl, + 0x1234567890abcdl, 0x1234567890abcdefl, Long.MAX_VALUE); + } +} Added: accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/PairLexicoderTest.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/PairLexicoderTest.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/PairLexicoderTest.java (added) +++ accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/PairLexicoderTest.java Sun May 5 02:45:17 2013 @@ -0,0 +1,36 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +import org.apache.accumulo.core.util.Pair; + +/** + * + */ +public class PairLexicoderTest extends LexicoderTest { + public void testSortOrder() { + PairLexicoder plexc = new PairLexicoder(new StringLexicoder(), new StringLexicoder()); + + assertSortOrder(plexc, new Pair("a", "b"), new Pair("a", "bc"), new Pair("a", "c"), new Pair( + "ab", "c"), new Pair("ab", ""), new Pair("ab", "d"), new Pair("b", "f"), new Pair("b", "a")); + + PairLexicoder plexc2 = new PairLexicoder(new LongLexicoder(), new StringLexicoder()); + + assertSortOrder(plexc2, new Pair(0x100l, "a"), new Pair(0x100l, "ab"), new Pair(0xf0l, "a"), new Pair( + 0xf0l, "ab")); + } +} Added: accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/ReverseLexicoderTest.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/ReverseLexicoderTest.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/ReverseLexicoderTest.java (added) +++ accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/ReverseLexicoderTest.java Sun May 5 02:45:17 2013 @@ -0,0 +1,62 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +import java.io.UnsupportedEncodingException; +import java.util.Collections; +import java.util.Comparator; +import java.util.Date; + +import org.junit.Test; + +public class ReverseLexicoderTest extends LexicoderTest { + public void testSortOrder() { + Comparator comp = Collections.reverseOrder(); + assertSortOrder(new ReverseLexicoder(new LongLexicoder()), comp, Long.MIN_VALUE, 0xff1234567890abcdl, 0xffff1234567890abl, 0xffffff567890abcdl, + 0xffffffff7890abcdl, 0xffffffffff90abcdl, 0xffffffffffffabcdl, 0xffffffffffffffcdl, -1l, 0l, 0x01l, 0x1234l, 0x123456l, 0x12345678l, 0x1234567890l, + 0x1234567890abl, 0x1234567890abcdl, 0x1234567890abcdefl, Long.MAX_VALUE); + + Comparator comp2 = Collections.reverseOrder(); + assertSortOrder(new ReverseLexicoder(new StringLexicoder()), comp2, "a", "aa", "ab", "b", "aab"); + + } + + /** + * Just a simple test verifying reverse indexed dates + * + * @throws UnsupportedEncodingException + */ + @Test + public void testReverseSortDates() throws UnsupportedEncodingException { + + ReverseLexicoder revLex = new ReverseLexicoder(new DateLexicoder()); + + Date date1 = new Date(); + Date date2 = new Date(System.currentTimeMillis() + 10000); + Date date3 = new Date(System.currentTimeMillis() + 500); + + Comparator comparator = Collections.reverseOrder(); + assertSortOrder(revLex, comparator, date1, date2, date3); + + // truncate date to hours + long time = System.currentTimeMillis() - (System.currentTimeMillis() % 3600000); + Date date = new Date(time); + + System.out.println(date); + + } +} Added: accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/UIntegerLexicoderTest.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/UIntegerLexicoderTest.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/UIntegerLexicoderTest.java (added) +++ accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/UIntegerLexicoderTest.java Sun May 5 02:45:17 2013 @@ -0,0 +1,33 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +public class UIntegerLexicoderTest extends LexicoderTest { + public void testEncoding() { + UIntegerLexicoder uil = new UIntegerLexicoder(); + + assertEqualsB(uil.encode(0), new byte[] {0x00}); + assertEqualsB(uil.encode(0x01), new byte[] {0x01, 0x01}); + assertEqualsB(uil.encode(0x0102), new byte[] {0x02, 0x01, 0x02}); + assertEqualsB(uil.encode(0x010203), new byte[] {0x03, 0x01, 0x02, 0x03}); + assertEqualsB(uil.encode(0x01020304), new byte[] {0x04, 0x01, 0x02, 0x03, 0x04}); + assertEqualsB(uil.encode(0xff020304), new byte[] {0x05, 0x02, 0x03, 0x04}); + assertEqualsB(uil.encode(0xffff0304), new byte[] {0x06, 0x03, 0x04}); + assertEqualsB(uil.encode(0xffffff04), new byte[] {0x07, 0x04}); + assertEqualsB(uil.encode(-1), new byte[] {0x08}); + } +} Added: accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/ULongLexicoderTest.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/ULongLexicoderTest.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/ULongLexicoderTest.java (added) +++ accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/ULongLexicoderTest.java Sun May 5 02:45:17 2013 @@ -0,0 +1,51 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +public class ULongLexicoderTest extends LexicoderTest { + + public void testEncoding() { + ULongLexicoder ull = new ULongLexicoder(); + + assertEqualsB(ull.encode(0l), new byte[] {0x00}); + assertEqualsB(ull.encode(0x01l), new byte[] {0x01, 0x01}); + assertEqualsB(ull.encode(0x1234l), new byte[] {0x02, 0x12, 0x34}); + assertEqualsB(ull.encode(0x123456l), new byte[] {0x03, 0x12, 0x34, 0x56}); + assertEqualsB(ull.encode(0x12345678l), new byte[] {0x04, 0x12, 0x34, 0x56, 0x78}); + assertEqualsB(ull.encode(0x1234567890l), new byte[] {0x05, 0x12, 0x34, 0x56, 0x78, (byte) 0x90}); + assertEqualsB(ull.encode(0x1234567890abl), new byte[] {0x06, 0x12, 0x34, 0x56, 0x78, (byte) 0x90, (byte) 0xab}); + assertEqualsB(ull.encode(0x1234567890abcdl), new byte[] {0x07, 0x12, 0x34, 0x56, 0x78, (byte) 0x90, (byte) 0xab, (byte) 0xcd}); + assertEqualsB(ull.encode(0x1234567890abcdefl), new byte[] {0x08, 0x12, 0x34, 0x56, 0x78, (byte) 0x90, (byte) 0xab, (byte) 0xcd, (byte) 0xef}); + + assertEqualsB(ull.encode(0xff34567890abcdefl), new byte[] {0x09, 0x34, 0x56, 0x78, (byte) 0x90, (byte) 0xab, (byte) 0xcd, (byte) 0xef}); + assertEqualsB(ull.encode(0xffff567890abcdefl), new byte[] {0x0a, 0x56, 0x78, (byte) 0x90, (byte) 0xab, (byte) 0xcd, (byte) 0xef}); + assertEqualsB(ull.encode(0xffffff7890abcdefl), new byte[] {0x0b, 0x78, (byte) 0x90, (byte) 0xab, (byte) 0xcd, (byte) 0xef}); + assertEqualsB(ull.encode(0xffffffff90abcdefl), new byte[] {0x0c, (byte) 0x90, (byte) 0xab, (byte) 0xcd, (byte) 0xef}); + assertEqualsB(ull.encode(0xffffffffffabcdefl), new byte[] {0x0d, (byte) 0xab, (byte) 0xcd, (byte) 0xef}); + assertEqualsB(ull.encode(0xffffffffffffcdefl), new byte[] {0x0e, (byte) 0xcd, (byte) 0xef}); + assertEqualsB(ull.encode(0xffffffffffffffefl), new byte[] {0x0f, (byte) 0xef}); + + assertEqualsB(ull.encode(-1l), new byte[] {16}); + } + + public void testSortOrder() { + // only testing non negative + assertSortOrder(new ULongLexicoder(), 0l, 0x01l, 0x1234l, 0x123456l, 0x12345678l, 0x1234567890l, 0x1234567890abl, 0x1234567890abcdl, 0x1234567890abcdefl, + Long.MAX_VALUE); + } + +} Added: accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/UUIDLexicoderTest.java URL: http://svn.apache.org/viewvc/accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/UUIDLexicoderTest.java?rev=1479229&view=auto ============================================================================== --- accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/UUIDLexicoderTest.java (added) +++ accumulo/trunk/core/src/test/java/org/apache/accumulo/core/client/lexicoder/UUIDLexicoderTest.java Sun May 5 02:45:17 2013 @@ -0,0 +1,37 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.client.lexicoder; + +import java.util.ArrayList; +import java.util.UUID; + +public class UUIDLexicoderTest extends LexicoderTest { + public void testSortOrder() { + + assertSortOrder(new UUIDLexicoder(), UUID.randomUUID(), UUID.randomUUID(), UUID.randomUUID(), UUID.randomUUID(), UUID.randomUUID(), UUID.randomUUID()); + + ArrayList uuids = new ArrayList(); + + for (long ms = -260l; ms < 260l; ms++) { + for (long ls = -2l; ls < 2; ls++) { + uuids.add(new UUID(ms, ls)); + } + } + + assertSortOrder(new UUIDLexicoder(), uuids.toArray(new UUID[0])); + } +} Modified: accumulo/trunk/docs/src/main/latex/accumulo_user_manual/chapters/table_design.tex URL: http://svn.apache.org/viewvc/accumulo/trunk/docs/src/main/latex/accumulo_user_manual/chapters/table_design.tex?rev=1479229&r1=1479228&r2=1479229&view=diff ============================================================================== --- accumulo/trunk/docs/src/main/latex/accumulo_user_manual/chapters/table_design.tex (original) +++ accumulo/trunk/docs/src/main/latex/accumulo_user_manual/chapters/table_design.tex Sun May 5 02:45:17 2013 @@ -111,6 +111,50 @@ com.google.mail_201004 Appending dates provides the additional capability of restricting a scan to a given date range. +\section{Lexicoders} +Since Keys in Accumulo are sorted lexicographically by default, it's often useful to encode +common data types into a byte format in which their sort order corresponds to the sort order +in their native form. An example of this is encoding dates and numerical data so that they can +be better seeked or searched in ranges. + +The lexicoders are a standard and extensible way of encoding Java types. Here's an example +of a lexicoder that encodes a java Date object so that it sorts lexicographically: + +\small +\begin{verbatim} +// create new date lexicoder +DateLexicoder dateEncoder = new DateLexicoder(); + +// truncate time to hours +long epoch = System.currentTimeMillis(); +Date hour = new Date(epoch - (epoch % 3600000)); + +// encode the rowId so that it is sorted lexicographically +Mutation mutation = new Mutation(dateEncoder.encode(hour)); +mutation.put(new Text("colf"), new Text("colq"), new Value(new byte[]{})); +\end{verbatim} +\normalsize + +If we want to return the most recent date first, we can reverse the sort order +with the reverse lexicoder: + +\small +\begin{verbatim} +// create new date lexicoder and reverse lexicoder +DateLexicoder dateEncoder = new DateLexicoder(); +ReverseLexicoder reverseEncoder = new ReverseLexicoder(dateEncoder); + +// truncate date to hours +long epoch = System.currentTimeMillis(); +Date hour = new Date(epoch - (epoch % 3600000)); + +// encode the rowId so that it sorts in reverse lexicographic order +Mutation mutation = new Mutation(reverseEncoder.encode(hour)); +mutation.put(new Text("colf"), new Text("colq"), new Value(new byte[]{})); +\end{verbatim} +\normalsize + + \section{Indexing} In order to support lookups via more than one attribute of an entity, additional indexes can be built. However, because Accumulo tables can support any number of