hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yan xiaohui <l0h...@gmail.com>
Subject Re: Chinese characters replaced with "??" when use map-reduce script write by java.
Date Sun, 25 Oct 2009 06:14:10 GMT
It works,thank you for your help!

On Sun, Oct 25, 2009 at 1:28 PM, 김영우 <warwithin@gmail.com> wrote:

> Hi yan,
>
> Would you like to modify your query like below and then, please let me know
> it works or not.
>
> from test select transform( zh ) using  'java jar *-Dfile.encoding=UTF-8*Test.jar';
>
> Thanks
>
> Youngwoo
>
> 2009/10/25 yan xiaohui <l0he1g@gmail.com>
>
> I have checked out that both "select zh" and "select *" got the right
>> result.I think it's not the point.
>>
>> The problem is "select transform( zh ) using  'java jar Test.jar'",when *you
>> use a java program as map-reduce script.*
>>
>> You see,I have to  write my program by java,because my program involve
>> some java lib file.
>>
>> 2009/10/25 Zheng Shao <zshao9@gmail.com>
>>
>> Can you try "select zh from mytable"?
>>>
>>> In hive, "Select *" is done differently than other queries.
>>>
>>>
>>>
>>> On 10/24/09, yan xiaohui <l0he1g@gmail.com> wrote:
>>> > I write a test program to output chinese characters from hive table by
>>> java:
>>> > Test.java:
>>> > import java.util.Scanner;
>>> >
>>> > public class Test {
>>> >     public static void main( String[] args ) {
>>> >         // TODO Auto-generated method stub
>>> >         Scanner scan = new Scanner( System.in );
>>> >             while( scan.hasNext() ){
>>> >                 String s = scan.nextLine();
>>> >                 System.out.println( s );
>>> >             }
>>> >     }
>>> > }
>>> > and then:
>>> > hive:>add file Test.jar;
>>> > hive:>create table test(zh string);
>>> > hive:>load data local inpath 'test.txt' into test;
>>> > hive:>from test select transform( zh ) using  'java jar Test.jar';
>>> >
>>> > the output like this:
>>> > ?????? ?????? ?????? ?????? ?????? ??????
>>> > ??????
>>> > ?????? VANCL ??????
>>> >
>>> > if I do:
>>> > hive:>select * from test;
>>> > or
>>> > $cat test.txt |java -jar Test.jar.Both the outputs are correct:
>>> > 官方网 指环王 指环 中文 官方网站 官方
>>> > 农场
>>> > 桌面 VANCL 新闻
>>> >
>>> > I have tried python script instead,there is no problem.But I have to
>>> use
>>> > java for some reason.
>>> >
>>> > I' m sure  my java code encoded by utf-8.
>>> > So,It seems neigther the problem of jar file nor hive,if that where is
>>> the
>>> > cause?
>>> > Thanks.
>>> >
>>>
>>> --
>>> Sent from Gmail for mobile | mobile.google.com
>>>
>>> Yours,
>>> Zheng
>>>
>>
>>
>

Mime
View raw message