Mailing-List: contact dev-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@accumulo.apache.org
Received-SPF: pass (athena.apache.org: domain of josh.elser@gmail.com
 designates 209.85.220.169 as permitted sender)
Message-ID: <508F3408.5070203@gmail.com>
Date: Mon, 29 Oct 2012 21:57:28 -0400
From: Josh Elser <josh.elser@gmail.com>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64;
 rv:16.0) Gecko/20121010 Thunderbird/16.0.1
MIME-Version: 1.0
To: dev@accumulo.apache.org
Subject: Re: Setting Charset in getBytes() call.
References: 
 <CAOiJXP4Nenbv7dMkzTGaOeHguPJEHV7hc2CV7BGZWUfovmaPZQ@mail.gmail.com>
 <CAFHxAsOEJ=Qx4yjvNDPhXEWEEXo4eXegdfHh8v4nho_mzLwG2A@mail.gmail.com>
 <CAMz+Duv=-oOWpHXx1gt6sJfP6DObY+dKGKrkHu44U5n3rXezqA@mail.gmail.com>
 <CAOiJXP6_R5sEn=1V3H3JPY6ZzPt6SES62Eh32cevU91SGiP_CA@mail.gmail.com>
 <508EACFF.5000704@gmail.com>
 <CALhtWkcqDFc0PTMavb4-thSXwz3meg9cNUaxKM0=sKXwiFZpzA@mail.gmail.com>
 <CADczPYTHpZQ=mexKSX0K3LgoPMi_fmJ7jCpa-GBp7PFgHwqM-g@mail.gmail.com>
 <508EB572.2020507@gmail.com>
 <CAJRvFdq5cL2dXt_2g8nr581TqPvyj_g4fQjv3H2TcA8BnK+T7Q@mail.gmail.com>
 <CAGgKvPko6e4Y62t5Pe=UebeWm9zEaYkMOmmn=V1HwySYgWC8Gw@mail.gmail.com>
 <CADczPYR+8M9ka4yAGWC=3CBHavxDitGJdS2VRzReRNPjf0CpgQ@mail.gmail.com>
 <CALhtWkeazo98xq0CFzw5_VNWf80JWrKb4cymt3yWRzTBntbDFQ@mail.gmail.com>
 <CAOiJXP4Rzrw38AwonsvPSfMczH5h2QPtzp0dz6twDqyaKYdE5A@mail.gmail.com>
 <CAGgKvPmF71OZS3BebkCLXKbwXeDJUF6tB+xD24T3QPMLh3u-Xg@mail.gmail.com>
 <508F236F.6070108@gmail.com>
 <CALhtWkddTBxaOMBuiNhOD53q0ZDe7-3iHfZBQA2XX8fkLO6+VA@mail.gmail.com>
In-Reply-To: 
 <CALhtWkddTBxaOMBuiNhOD53q0ZDe7-3iHfZBQA2XX8fkLO6+VA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

I'm saying that I don't know of anything in the core API which performs 
a getBytes() on the data itself. Accumulo itself is agnostic dealing 
only in byte[]. I think we're saying the same thing..

On 10/29/2012 8:54 PM, Benson Margulies wrote:
> On Mon, Oct 29, 2012 at 8:46 PM, Josh Elser <josh.elser@gmail.com> wrote:
>> +1 Mike.
>>
>> 1. It would be hard for me to believe Key/Value are ever handled internally
>> in terms of Strings, but, if such a case does exist, it would be extremely
>> prudent to fix.
>>
>> 2. FWIW, the Shell does use ISO-8859-1 as its charset which is referenced by
>> other commands [1,2]. It would be good to double check all of the other
>> commands.
>
> I'm a bit lost. Any possible Java String can be rendered in UTF-8. So,
> if you are calling String.getBytes to turn a string into some bytes
> for some purpose, I think you need UTF-8.
>
> On the other hand, as Mike pointed out, new String(somebytes, "utf-8")
> will destroy data for some byte values that are not, in fact, UTF-8.
> By why would Accumulo ever need to string-ify some array of bytes of
> uncertain parentage?
>
>
>>
>> [1]
>> https://github.com/apache/accumulo/blob/trunk/core/src/main/java/org/apache/accumulo/core/util/shell/Shell.java
>> [2]
>> https://github.com/apache/accumulo/blob/trunk/core/src/main/java/org/apache/accumulo/core/util/shell/commands/InsertCommand.java
>>
>>
>> On 10/29/2012 8:27 PM, Michael Flester wrote:
>>>
>>> I agree with Benson entirely with one caveat. It seems to me that there
>>> might be two categories of things being discussed
>>>
>>>     1. User data (keys and values)
>>>     2. Ancillary things needed for operation of Accumulo (passwords).
>>>
>>> These could well be considered separately. Trying to do anything with
>>> keys and values other than treating them as bytes all of the time
>>> I find quite scary.
>>>
>>> And if this is only being done to satisfy pmd or findbugs, those tools
>>> can be convinced to modify their reporting about this issue.
>>>
>>