hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mahender Sarangam <mahender.bigd...@outlook.com>
Subject Building Rule Engine/ Rule Transformation
Date Sat, 28 Nov 2015 20:26:55 GMT
 Hi team,
 
We need expert input to discuss how to implement Rule engine in hive. Do you have any references
available to implement rule in hive/pig.

 
We are migrating our Stored Procedures into Multiple Hive query but it is becoming complex
in maintenance,  Hive is not Procedural Language, so we could not write IF ELSE logic or any
kind of procedural language. Can any one suggest us which HDP technology can be helpful for
Procedural language replacement, we are thinking of PIG, Can it  becomes best example to perform
rule/data transformation.
 
Our data is in Structure format table with around 250 columns, We are having Rules like Update
Columns based on lookup table values, delete rows which doesn't satisfy the condition, update
statement include update of multiple columns like case etc and some date conversions columns.
Suggest us best way to implement this rule engine.  previously we have used our SQL Server
for Rule engine, now we have migrated application to Big Data, we are looking for any references
available to perform this rule transformation.
 
 
Some of our Finding are 

Make use of Hive StreamWriting PIG.
 
We are .NET developers, Can we think of writing .EXE application and stream row wise data
to .EXE and apply rules on top of  each row. Will it be nicer solution or Is it better to
implement in PIG, but implementing doesn't fetch me much benefit when compared with Hive.
Can you please comment on above approaches please.
 
Thanks,
Mahender  		 	   		  
Mime
View raw message