Mailing-List: contact dev-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@accumulo.apache.org
Received-SPF: pass (athena.apache.org: domain of bimargulies@gmail.com
 designates 209.85.223.169 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <50906F21.9040901@gmail.com>
References: 
 <CAOiJXP4Nenbv7dMkzTGaOeHguPJEHV7hc2CV7BGZWUfovmaPZQ@mail.gmail.com>
	<CADczPYTHpZQ=mexKSX0K3LgoPMi_fmJ7jCpa-GBp7PFgHwqM-g@mail.gmail.com>
	<508EB572.2020507@gmail.com>
	<CAJRvFdq5cL2dXt_2g8nr581TqPvyj_g4fQjv3H2TcA8BnK+T7Q@mail.gmail.com>
	<CAGgKvPko6e4Y62t5Pe=UebeWm9zEaYkMOmmn=V1HwySYgWC8Gw@mail.gmail.com>
	<CADczPYR+8M9ka4yAGWC=3CBHavxDitGJdS2VRzReRNPjf0CpgQ@mail.gmail.com>
	<CALhtWkeazo98xq0CFzw5_VNWf80JWrKb4cymt3yWRzTBntbDFQ@mail.gmail.com>
	<CAOiJXP4Rzrw38AwonsvPSfMczH5h2QPtzp0dz6twDqyaKYdE5A@mail.gmail.com>
	<CAGgKvPmF71OZS3BebkCLXKbwXeDJUF6tB+xD24T3QPMLh3u-Xg@mail.gmail.com>
	<508F236F.6070108@gmail.com>
	<CALhtWkddTBxaOMBuiNhOD53q0ZDe7-3iHfZBQA2XX8fkLO6+VA@mail.gmail.com>
	<508F3408.5070203@gmail.com>
	<CADczPYR1zGTde3DSak=R8rkM3U6rsCHc45UP+xm1jqc-BDbCpA@mail.gmail.com>
	<CAOiJXP4U0KwFdzHsWJiz4Byi60SFws4WV4LMw0GSz348J1y2iA@mail.gmail.com>
	<5090546F.5040009@gmail.com>
	<CAOiJXP6FRX2paVUi-8r7zYByoHYesLEK_WbQmYq8i85AmxBbwQ@mail.gmail.com>
	<50906F21.9040901@gmail.com>
Date: Tue, 30 Oct 2012 20:31:51 -0400
Message-ID: 
 <CALhtWkeQT87JrWw=BnvhM2OgEB8OCeh3ZYWOXjRj_Ao4vgiOkg@mail.gmail.com>
Subject: Re: Setting Charset in getBytes() call.
From: Benson Margulies <bimargulies@gmail.com>
To: dev@accumulo.apache.org
Content-Type: text/plain; charset=UTF-8

On Tue, Oct 30, 2012 at 8:21 PM, Josh Elser <josh.elser@gmail.com> wrote:
> On 10/30/2012 7:47 PM, David Medinets wrote:
>>>
>>> My issue with this is that you have now hard-coded the fact that everyone
>>> else is going to use UTF-8.
>>
>>
>> Who is everyone else? I agree that I have hard-coded the use of UTF-8.
>> On the other hand, I've merely codified an existing practice. Thus the
>> issue is now exposed, the places the convention is used are defined.
>> Once a consensus is reached, we can implement it with confidence.
>
>
> "Everyone else" is everyone who builds Accumulo since you committed your
> changes and uses it. Ignoring that, forcing a single charset isn't the big
> issue here (as we've *all* agreed that UTF-8 should not cause any
> data-correctness issues) so for now I'll just drop it as it's just creating
> confusion.
>
> My issue is *how* you implemented the default charset. We already have 3
> people (Marc, Bill and myself) who have stated that we believe inline
> charset declaration is not the correct implementation and that using the JVM
> property is the better implementation.
>
> I'd encourage others to weigh in to form a complete consensus and shift the
> discussion to that implementation if needed.
>
>>
>>> way to fix the problem. I still contest that setting the desired encoding
>>> (via the appropriate JVM property like Bill Slacum initial suggested) is
>>> the
>>> proper way to address the issue.
>>
>>
>> It is easy to do both. Create a ByteEncodingInitializer (or somesuch)
>> class that reads the JVM property and defines a globally used Charset.
>> The find those utf8 definitions and usages and replace them with the
>> globally-defined value.
>
>
> Again, by setting the 'file.encoding' JVM parameter, such a class is
> unnecessary because it should be handled internal to Java. For Oracle/Sun
> JDK and OpenJDK, setting the "file.encoding" parameter at run time will use
> the provided charset you wanted without actually changing any code.

If Accumulo was only a pile of servers, you could do this. You could
say that part of the configuration process for the servers is to
specify the desired encoding to file.encoding, and your shell scripts
could set UTF-8 by default.

But Accumulo is *not* just a pile of servers. Setting file.encoding
effects the entire JVM. A webapp that uses Accumulo now would need to
have the entire servlet container have a particular setting of
file.encoding. This just does not work in the wild. Even without the
servlet container issue, a user of Accumulo may need to plug it into
an existing code base that has other reasons to set file.encoding, and
will not like it when Accumulo starts to corrupt his or her string
data.