Return-Path: X-Original-To: apmail-incubator-empire-db-user-archive@minotaur.apache.org Delivered-To: apmail-incubator-empire-db-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9CB2870FD for ; Wed, 9 Nov 2011 19:46:17 +0000 (UTC) Received: (qmail 94570 invoked by uid 500); 9 Nov 2011 19:46:17 -0000 Delivered-To: apmail-incubator-empire-db-user-archive@incubator.apache.org Received: (qmail 94551 invoked by uid 500); 9 Nov 2011 19:46:17 -0000 Mailing-List: contact empire-db-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: empire-db-user@incubator.apache.org Delivered-To: mailing list empire-db-user@incubator.apache.org Received: (qmail 94543 invoked by uid 99); 9 Nov 2011 19:46:17 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 09 Nov 2011 19:46:17 +0000 X-ASF-Spam-Status: No, hits=1.8 required=5.0 tests=FREEMAIL_FROM,FREEMAIL_REPLY,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of francisdb@gmail.com designates 209.85.220.175 as permitted sender) Received: from [209.85.220.175] (HELO mail-vx0-f175.google.com) (209.85.220.175) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 09 Nov 2011 19:46:12 +0000 Received: by vcbfl17 with SMTP id fl17so1787338vcb.6 for ; Wed, 09 Nov 2011 11:45:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=l2kvi2F+3dNpx5cWntyIIbh6R7FpXsBwSEOVH/bZNuY=; b=bmz3UHF4/i6x2fayFS8z/L5WD9MEp7ZzflpZmjbLGoMkxWu5KUtd+/j1hOaxlZPu0k KTWfdZFBexJQHxB9KnsMEwVxrHV8ZD3z1SzveL4BBDx9HcYP7+fMeWXRbBlgwRdS6wEw vaHGfixqhvc0swGUqluartie34/ePMQt+nMO8= Received: by 10.52.69.70 with SMTP id c6mr6917453vdu.65.1320867952062; Wed, 09 Nov 2011 11:45:52 -0800 (PST) MIME-Version: 1.0 Received: by 10.52.183.166 with HTTP; Wed, 9 Nov 2011 11:45:31 -0800 (PST) In-Reply-To: References: From: Francis De Brabandere Date: Wed, 9 Nov 2011 20:45:31 +0100 Message-ID: Subject: Re: Inserting Chinese character record to DB To: empire-db-user@incubator.apache.org Cc: =?ISO-8859-1?Q?Rainer_D=F6bele?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I created an issue for this: https://issues.apache.org/jira/browse/EMPIREDB-122 please continue the discussion overthere On Wed, Nov 9, 2011 at 8:38 PM, Francis De Brabandere wrote: > Hi Ben, > > Good news, this can be fixed and has to be fixed on our side. In > sqlserver you have to prepend your string with N for unicode, for > example: > INSERT INTO DEPARTMENTS( NAME, BUSINESS_UNIT, UPDATE_TIMESTAMP) VALUES > ( N'junit', N'=E4=B8=AD=E6=96=87', '2011-11-09 20:28:40.718') > > Change DBDatabaseDriver line 664 to the following to fix the issue: > valBuf.append("N'"); > > Rainer, how would we fix this? getValueString() in DBDatabaseDriver is > final. We could do this by default or have an option to enable the > N's. > > Cheers, > Francis > > On Wed, Nov 9, 2011 at 4:57 AM, Ben Ng wrote: >> Hi Francis, >> >> Attached with a project which can reproduce my issue. Thanks >> >> Cheers, >> Ben >> >> On Wed, Nov 9, 2011 at 5:07 AM, Francis De Brabandere >> wrote: >>> >>> Hi Ben, >>> >>> Any chance you could set up a very small project to reproduce this >>> encoding issue? >>> Something like this: >>> >>> https://svn.apache.org/repos/asf/incubator/empire-db/tags/apache-empire= -db-2.2.0-incubating/empire-db/src/test/java/org/apache/empire/db/hsql/DBDa= tabaseDriverHSqlTest.java >>> >>> Cheers, >>> Francis >>> >>> >>> >>> On Tue, Nov 8, 2011 at 2:50 AM, Ben Ng wrote: >>> > Dear Francis, >>> > >>> > I am now using MS SQL Server 2008. I have tried two JDBC drivers. Bot= h >>> > are >>> > fail to store Chinese character without prepared statements. >>> > 1) net.sourceforge.jtds.jdbc.Driver (jtds-1.2.jar) >>> > 2) com.microsoft.sqlserver.jdbc.SQLServerDriver (sqljdbc4.jar latest >>> > version) >>> > >>> > Collation of the database is SQL_Latin1_General_CP1_CI_AS and the fie= ld >>> > is a >>> > NVARCHAR. >>> > Hope that those information can be helpful to you. Thanks >>> > >>> > Best wishes, >>> > Ben >>> > >>> > >>> > On Mon, Nov 7, 2011 at 10:34 PM, Francis De Brabandere >>> > >>> > wrote: >>> >> >>> >> Ben, could you tell us what database and driver version you are usin= g? >>> >> >>> >> I'll try to create a unit/integration test for this. >>> >> >>> >> Cheers, >>> >> F >>> >> >>> >> On Mon, Nov 7, 2011 at 3:25 PM, Rainer D=C3=B6bele >>> >> wrote: >>> >> > Hi Francis, >>> >> > >>> >> > I agree but we need a test case for this. >>> >> > It may as well depend on the JDBC-Driver used - in fact I don't kn= ow. >>> >> > Also my Chinese is a little rusted ;-) >>> >> > >>> >> > Regards >>> >> > Rainer >>> >> > >>> >> >> from: Francis De Brabandere [mailto:francisdb@gmail.com] >>> >> >> to: empire-db-user@incubator.apache.org >>> >> >> re: Re: Inserting Chinese character record to DB >>> >> >> >>> >> >> I think this still is something we should fix for non-prepared >>> >> >> statements >>> >> >> >>> >> >> On Mon, Nov 7, 2011 at 9:41 AM, Ben Ng wrote: >>> >> >> > Hi Rainer, >>> >> >> > >>> >> >> > It seems that my problem have been solved when I enable prepare= d >>> >> >> statement. >>> >> >> > Thank you for your help. >>> >> >> > >>> >> >> > Regards, >>> >> >> > Ben >>> >> >> > >>> >> >> > On Mon, Nov 7, 2011 at 4:19 PM, Rainer D=C3=B6bele >>> >> >> wrote: >>> >> >> >> >>> >> >> >> Hi Ben, >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> I don't know for sure but it may be a good idea to tell Empire= -db >>> >> >> >> to >>> >> >> >> use prepared statements. >>> >> >> >> >>> >> >> >> Simply write: >>> >> >> >> >>> >> >> >> db.setPreparedStatementsEnabled(true); >>> >> >> >> >>> >> >> >> in your initialization code. >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> Please let us know whether this works and tell us which DBMS a= nd >>> >> >> >> JDBC-Driver you are using. >>> >> >> >> >>> >> >> >> Regards >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> Rainer >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> from: Ben Ng [mailto:cnj125@gmail.com] >>> >> >> >> to: empire-db-user@incubator.apache.org >>> >> >> >> re: Re: Inserting Chinese character record to DB >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> Hi Empire-DB team, >>> >> >> >> >>> >> >> >> I am trying to insert a record with Chinese character from my = JSP >>> >> >> >> page. >>> >> >> >> The encoding is set to UTF-8 already and I am trying to insert >>> >> >> >> the >>> >> >> >> record by the code below where "remarks" is a string contain >>> >> >> >> Chinese >>> >> >> characters. >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> DBRecord rec =3D new DBRecord(); >>> >> >> >> rec.create(db.REQUESTS); >>> >> >> >> rec.setValue(db.REQUESTS.REMARKS, remarks); rec.update(conn); >>> >> >> >> >>> >> >> >> After inserting the record, the data of the field REMARKS beco= me >>> >> >> >> "???". >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> I tried to use traditional method(PreparedStatement) to insert >>> >> >> >> this >>> >> >> >> record by using the same db and connection. >>> >> >> >> The data can be inserted correctly. >>> >> >> >> Is there any step I have missed in my code so it can support >>> >> >> >> UTF-8? >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> I really appreciate for what your team have done. Thanks a lot= . >>> >> >> >> >>> >> >> >> Thanks & Regards, >>> >> >> >> Ben >>> >> >> > >>> >> >> >>> >> >> >>> >> >> >>> >> >> -- >>> >> >> http://www.somatik.be >>> >> >> Microsoft gives you windows, Linux gives you the whole house. >>> >> > >>> >> >>> >> >>> >> >>> >> -- >>> >> http://www.somatik.be >>> >> Microsoft gives you windows, Linux gives you the whole house. >>> > >>> > >>> >>> >>> >>> -- >>> http://www.somatik.be >>> Microsoft gives you windows, Linux gives you the whole house. >> >> > > > > -- > http://www.somatik.be > Microsoft gives you windows, Linux gives you the whole house. > --=20 http://www.somatik.be Microsoft gives you windows, Linux gives you the whole house.