hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joel D <games2013....@gmail.com>
Subject Re: Business Rules Engine for Hive
Date Mon, 16 Apr 2018 21:15:56 GMT
The business rules we've here are currently embedded in hive code. They
range from basic standardization using case blocks to complex multi-column
validation.

Thanks.

On Mon, Apr 16, 2018 at 5:03 PM Jörn Franke <jornfranke@gmail.com> wrote:

> The question is what do your rules do? Do you need to maintain a factbase
> or do they just check data quality within certain tables?
>
> On 16. Apr 2018, at 22:28, Joel D <games2013.sam@gmail.com> wrote:
>
> Ok.
>
> Rough ideas:
> To keep the business logic outside code, I was thinking to give a custom
> UI.
>
> Next read from UI data and build UDFs using the rules defined outside the
> UDF.
>
> 1 UDF per data object.
>
> Not sure these are just thoughts.
>
> On Mon, Apr 16, 2018 at 1:40 PM Jörn Franke <jornfranke@gmail.com> wrote:
>
>> I would not use Drools with Spark, it does not scale to the distributed
>> setting.
>>
>> You could translate the rules to hive queries but this would not be
>> exactly the same thing.
>>
>> > On 16. Apr 2018, at 17:59, Joel D <games2013.sam@gmail.com> wrote:
>> >
>> > Hi,
>> >
>> > Any suggestions on how to implement Business Rules Engine with Hive
>> ETLs?
>> >
>> > For spark based Etl jobs, I was exploring Drools but not sure about
>> Hive.
>> >
>> > Thanks.
>>
>

Mime
View raw message