apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bhupesh Chawda <bhup...@datatorrent.com>
Subject Re: BoundedDedupOperator failing with java.lang.IllegalArgumentException: bucket conflict
Date Fri, 09 Jun 2017 05:40:03 GMT
Hi Vivek,

Did the exception occur immediately, or after some records were processed?
Can you send the full container log for this operator.
Please also share the properties file that you used for configuration.

~ Bhupesh


_______________________________________________________

Bhupesh Chawda

E: bhupesh@datatorrent.com | Twitter: @bhupeshsc

www.datatorrent.com  |  apex.apache.org



On Fri, Jun 9, 2017 at 4:55 AM, Vivek Bhide <bhide.vivek@gmail.com> wrote:

> Hi,
>
> I am using the BoundedDedupOperator and with default value of numBuckets
> (46340) the container is failing with below bucket conflict exception
>
> 2017-06-08 17:52:10,140 INFO  stram.StreamingContainerParent
> (StreamingContainerParent.java:log(170)) - child msg: Stopped running due
> to
> an exception. java.lang.IllegalArgumentException: bucket conflict -27877
> 27877
>         at
> org.apache.apex.malhar.lib.state.managed.AbstractManagedStateImpl.
> handleBucketConflict(AbstractManagedStateImpl.java:320)
>         at
> org.apache.apex.malhar.lib.state.managed.AbstractManagedStateImpl.
> prepareBucket(AbstractManagedStateImpl.java:269)
>         at
> org.apache.apex.malhar.lib.state.managed.AbstractManagedStateImpl.
> getValueFromBucketAsync(AbstractManagedStateImpl.java:305)
>         at
> org.apache.apex.malhar.lib.state.managed.ManagedTimeStateImpl.getAsync(
> ManagedTimeStateImpl.java:76)
>         at
> org.apache.apex.malhar.lib.dedup.BoundedDedupOperator.
> getAsyncManagedState(BoundedDedupOperator.java:151)
>         at
> org.apache.apex.malhar.lib.dedup.AbstractDeduper.
> processTuple(AbstractDeduper.java:163)
>         at
> org.apache.apex.malhar.lib.dedup.BoundedDedupOperator$1.
> process(BoundedDedupOperator.java:91)
>         at com.datatorrent.api.DefaultInputPort.put(
> DefaultInputPort.java:79)
>         at
> com.datatorrent.stram.stream.BufferServerSubscriber$BufferReservoir.sweep(
> BufferServerSubscriber.java:288)
>         at com.datatorrent.stram.engine.GenericNode.run(GenericNode.
> java:269)
>         at
> com.datatorrent.stram.engine.StreamingContainer$2.run(
> StreamingContainer.java:1428)
>
>
> I tried changing the numBucket value to something else but it still fails
> with same error. I was planning to use it with just 1 bucket but I think
> that will slow the operator too much since the unique keys in our
> applications are very high
>
> Has anybody else encountered the same problem?
>
> Regards
> Vivek
>
>
>
> --
> View this message in context: http://apache-apex-users-list.
> 78494.x6.nabble.com/BoundedDedupOperator-failing-with-java-lang-
> IllegalArgumentException-bucket-conflict-tp1698.html
> Sent from the Apache Apex Users list mailing list archive at Nabble.com.
>

Mime
View raw message