flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marina <ppi...@yahoo.com>
Subject Re: Unicode data handling with flume
Date Tue, 12 May 2015 12:26:06 GMT
Did you try specifying encoding for your source, for example like this:a1.sources.r1.inputCharset
= ISO8859-1 << whatever charset you need>>?
Marina

      From: "Vishwakarma, Chhaya" <Chhaya.Vishwakarma@Teradata.com>
 To: "user@flume.apache.org" <user@flume.apache.org> 
 Sent: Tuesday, May 12, 2015 4:20 AM
 Subject: Unicode data handling with flume
   
 <!--#yiv0710838864 _filtered #yiv0710838864 {font-family:Helvetica;panose-1:2 11 6 4 2
2 2 2 2 4;} _filtered #yiv0710838864 {font-family:Helvetica;panose-1:2 11 6 4 2 2 2 2 2 4;}
_filtered #yiv0710838864 {font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;} _filtered #yiv0710838864
{font-family:Tahoma;panose-1:2 11 6 4 3 5 4 4 2 4;} _filtered #yiv0710838864 {font-family:Consolas;panose-1:2
11 6 9 2 2 4 3 2 4;}#yiv0710838864 #yiv0710838864 p.yiv0710838864MsoNormal, #yiv0710838864
li.yiv0710838864MsoNormal, #yiv0710838864 div.yiv0710838864MsoNormal {margin:0in;margin-bottom:.0001pt;font-size:11.0pt;font-family:"Calibri",
"sans-serif";}#yiv0710838864 a:link, #yiv0710838864 span.yiv0710838864MsoHyperlink {color:blue;text-decoration:underline;}#yiv0710838864
a:visited, #yiv0710838864 span.yiv0710838864MsoHyperlinkFollowed {color:purple;text-decoration:underline;}#yiv0710838864
p {margin-right:0in;margin-left:0in;font-size:12.0pt;font-family:"Times New Roman", "serif";}#yiv0710838864
code {font-family:"Courier New";}#yiv0710838864 pre {margin:0in;margin-bottom:.0001pt;font-size:10.0pt;font-family:"Courier
New";}#yiv0710838864 span.yiv0710838864EmailStyle17 {font-family:"Calibri", "sans-serif";color:windowtext;}#yiv0710838864
span.yiv0710838864HTMLPreformattedChar {font-family:"Courier New";}#yiv0710838864 span.yiv0710838864apple-converted-space
{}#yiv0710838864 .yiv0710838864MsoChpDefault {font-family:"Calibri", "sans-serif";} _filtered
#yiv0710838864 {margin:1.0in 1.0in 1.0in 1.0in;}#yiv0710838864 div.yiv0710838864WordSection1
{}-->Hi all, I'm trying to put a CSV file into HDFS using flume, file contains some unicode
characters also. Once the file is there in HDFS I tried to view the content, but unable to
see the records properly. File content Name    age  sal    msg    Abc     21 
1200    Lukè éxample àpple    Xyz     23  1400    er stîget ûf mit grôzer
Output in console I did hdfs dfs -get /flume/events/csv/events.1234567 Below is the output
Name,age,sal,msg    Abc,21,1200,Luk��xample��pple    Xyz,23,1400,er st�get�f
mit gr�zer Does flume supports Unicode characters? If not how it can be handled? Flume version
is 1.4.0    

  
Mime
View raw message