hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raj Hadoop <hadoop...@yahoo.com>
Subject Re: How to load /t /n file to Hive
Date Fri, 20 Sep 2013 22:53:19 GMT
Hi Gabo,

Are you suggesting to use java.net.URLEncoder ? Can you be more specific ? I have lot of fields
in the file which are not only URL related but some text fields which has new line characters.

Thanks,
Raj


________________________________
 From: Gabriel Eisbruch <gabrieleisbruch@gmail.com>
To: "user@hive.apache.org" <user@hive.apache.org>; Raj Hadoop <hadoopraj@yahoo.com>

Sent: Friday, September 20, 2013 4:43 PM
Subject: Re: How to load /t /n file to Hive
 


Hi 
 One way that we used to solve that problem it's to transform the data when you are creating/loading
it, for example we've applied UrlEncode to each field on create time.

Thanks,
Gabo.



2013/9/20 Raj Hadoop <hadoopraj@yahoo.com>

Hi Nitin,
> 
>Thanks for the reply. I have a huge file in unix.
> 
>As per the file definition, the file is a tab separated file of fields. But I am sure
that within some field's I have some new line character. 
> 
>How should I find a record? It is a huge file. Is there some command?
> 
>Thanks,
> 
>
>
>From: Nitin Pawar <nitinpawar432@gmail.com>
>To: "user@hive.apache.org" <user@hive.apache.org>; Raj Hadoop <hadoopraj@yahoo.com>

>Sent: Friday, September 20, 2013 3:15 PM
>Subject: Re: How to load /t /n file to Hive
>
>
>
>If your data contains new line chars, its better you write a custom map reduce job and
convert the data into a single line removing all unwanted chars in column separator as well
just having single new line char per line 
>
>
>
>On Sat, Sep 21, 2013 at 12:38 AM, Raj Hadoop <hadoopraj@yahoo.com> wrote:
>
>Please note that there is an escape chacter in the fields where the /t and /n are present.
>>
>>
>>
>>From: Raj Hadoop <hadoopraj@yahoo.com>
>>To: Hive <user@hive.apache.org> 
>>Sent: Friday, September 20, 2013 3:04 PM
>>Subject: How to load /t /n file to Hive
>>
>>
>>
>>Hi,
>> 
>>I have a file which is delimted by a tab. Also, there are some fields in the file
which has a tab /t character and a new line /n character in some fields.
>> 
>>Is there any way to load this file using Hive load command? Or do i have to use a
Custom Map Reduce (custom) Input format with java ? Please advise.
>> 
>>Thanks,
>>Raj
>>
>>
>
>
>
>-- 
>Nitin Pawar
>
>
>
Mime
View raw message