apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Munagala Ramanath <amberar...@yahoo.com>
Subject Re: How to specify more than one fields as a dedup keyExpression
Date Wed, 25 Oct 2017 22:43:35 GMT
 The mapping: tuple -> dedup-key needs to be 1-1; if multiple tuples are mapped to the same
dedup key you'll see problems like this. In your case multiple tuples can be mapped to the
same value of "id + id1". For example, all tuples with (id, id1) being any of these pairs
will all map to a value of 3: (0, 3), (1, 2), (2, 1), (3, 0).
A simple way to get a unique dedep key is to convert all of your dedup fields to strings and
catenate them. So in your event/tuple class, define a getDedupKey() method and within it,
compute this string and return it. Then, you can usethe expression {$}.getDedupKey(). Something
along those lines should work.
Ram
 
    On Wednesday, October 25, 2017, 12:21:59 PM PDT, Vivek Bhide <vivek.bhide@target.com>
wrote:  
 
 Thank you everyone for reply. Solution that chinmay suggested is working but
then I see one more discrepancy. 

After adding more that 1 fields as a dedup key, my expectation was to have
the dedup decision made on combination of these 2 keys. I did run the test
case multiple times with BoundedDedupOperator and what i found is, events
are marked as Duplicate but if search for the corresponding Unique entry in
sysout then that entry is no where to be found. Its not happening for all
but for most of entires marked as Duplicate

Is my expectation of dedup behavior is correct and is this a right way to
validate if its working as expected?
Dedup_test_case_output.txt
<http://apache-apex-users-list.78494.x6.nabble.com/file/t127/Dedup_test_case_output.txt> 

Sample entries :

Present as Unique and Duplicate : 
Duplicate: TestEvent [id=75, id1=64, eventTime=Wed Oct 25 12:09:44 PDT 2017]
Unique: TestEvent [id=75, id1=64, eventTime=Wed Oct 25 12:09:18 PDT 2017]

Only present at Duplicate : 
Duplicate: TestEvent [id=23, id1=77, eventTime=Wed Oct 25 12:09:04 PDT 2017]
Duplicate: TestEvent [id=44, id1=63, eventTime=Wed Oct 25 12:09:40 PDT 2017]




--
Sent from: http://apache-apex-users-list.78494.x6.nabble.com/

Mime
View raw message