gobblin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jbaranick@gmail.com"<jbaran...@gmail.com>
Subject Re: Need help on Gobblin
Date Sat, 11 Nov 2017 09:32:21 GMT


On 2017-11-11 01:24, Mohan <mohandoss.tr@gmail.com> wrote: 
> Is there any option to check source input records  counts and writer count
> using property or any other logic in Gobblin ?
> 

Yes.  Look at "RowLevelPolicy" and "TaskLevelPolicy".  Their config keys =
are "qualitychecker.row.policies", "qualitychecker.row.policy.types", =
"qualitychecker.task.policies", and "qualitychecker.task.policy.types".  =
A couple examples are: "RowCountPolicy" and "AvroHeaderTimestampPolicy".

Mime
View raw message