hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From karthik maddala <karthikmaddal...@gmail.com>
Subject Re: Which SerDe for Custom Binary Data.
Date Fri, 13 Mar 2015 18:13:01 GMT
Currently we have data in NFS and we have proprietery tools to access the
data.
We are planning to move the data into HDFS and use HiveQL for accessing the
data and run batch jobs.
So looking out for the custom SerDe(assuming the existing SerDe's will not
be able to read the underlying data)  to read the data using hive.
On Fri, Mar 13, 2015 at 10:33 AM, Mich Talebzadeh <mich@peridale.co.uk>
wrote:

> Hive as I use it is particularly useful for getting data out of relational
> tables and more importantly query that data using HiveQL (a variation of
> transact sql)
>
> .
>
>
>
> If your data is in binary format and assuming that you manage to store it
> in HDFS, how are you intending to access the data. At the consumer level
> what tools are you going to use? Do you a propriety tool with the correct
> drivers to access the data?
>
>
>
> HTH
>
>
>
> Mich Talebzadeh
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> *Publications due shortly:*
>
> *Creating in-memory Data Grid for Trading Systems with Oracle TimesTen and
> Coherence Cache*
>
>
>
> NOTE: The information in this email is proprietary and confidential. This
> message is for the designated recipient only, if you are not the intended
> recipient, you should destroy it immediately. Any information in this
> message shall not be understood as given or endorsed by Peridale Ltd, its
> subsidiaries or their employees, unless expressly so stated. It is the
> responsibility of the recipient to ensure that this email is virus free,
> therefore neither Peridale Ltd, its subsidiaries nor their employees accept
> any responsibility.
>
>
>
> *From:* karthik maddala [mailto:karthikmaddala18@gmail.com]
> *Sent:* 13 March 2015 15:56
> *To:* user@hive.apache.org
> *Subject:* Which SerDe for Custom Binary Data.
>
>
>
>
>
>
>
> I want to set up a DW based on Hive. However, my data does not come as
> handy csv files but  as binary files in a proprietary format.
>
>
>
> The binary file  consists of  serialized data using C language.
>
>
>
>
>
> Could you please suggest which input format to be used and how to write a
> custom SerDe for the above mentioned data.
>
>
>
>
>
> Thanks,
>
> Karthik Maddala
>
>
>
>
>

Mime
View raw message