hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mahender Sarangam <>
Subject RE: Building Rule Engine/ Rule Transformation
Date Mon, 30 Nov 2015 05:25:58 GMT
We are not expert in java programing, we work .NET code. But there is no support for .NET UDF.

Subject: Re: Building Rule Engine/ Rule Transformation
Date: Sun, 29 Nov 2015 11:33:05 +0100

Why not implement Hive UDF in Java?
On 28 Nov 2015, at 21:26, Mahender Sarangam <> wrote:

 Hi team,
We need expert input to discuss how to implement Rule engine in hive. Do you have any references
available to implement rule in hive/pig.

We are migrating our Stored Procedures into Multiple Hive query but it is becoming complex
in maintenance,  Hive is not Procedural Language, so we could not write IF ELSE logic or any
kind of procedural language. Can any one suggest us which HDP technology can be helpful for
Procedural language replacement, we are thinking of PIG, Can it  becomes best example to perform
rule/data transformation.
Our data is in Structure format table with around 250 columns, We are having Rules like Update
Columns based on lookup table values, delete rows which doesn't satisfy the condition, update
statement include update of multiple columns like case etc and some date conversions columns.
Suggest us best way to implement this rule engine.  previously we have used our SQL Server
for Rule engine, now we have migrated application to Big Data, we are looking for any references
available to perform this rule transformation.
Some of our Finding are 

Make use of Hive StreamWriting PIG.
We are .NET developers, Can we think of writing .EXE application and stream row wise data
to .EXE and apply rules on top of  each row. Will it be nicer solution or Is it better to
implement in PIG, but implementing doesn't fetch me much benefit when compared with Hive.
Can you please comment on above approaches please.
View raw message