apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chandni Singh <chan...@datatorrent.com>
Subject Re: Reg. partitioning operator
Date Fri, 06 Nov 2015 01:14:01 GMT
Hi Venkatesh,

      property>
              <name>dt.application.appname.operator.reader.prop.
PARTITIONER</name>
              <value>com.datatorrent.lib.partitioner.
StatelessPartitioner:4</value>
       </property>

In the above config, PARTITIONER should be an attribute:
      property>
              <name>dt.application.appname.operator.reader.attr.
PARTITIONER</name>
              <value>com.datatorrent.common.partitioner.StatelessPartitioner
:4</value>
       </property>

I believe with release 3.0 the StatelessPartitioner was moved to Apex
common package so you should be using that as well.

Chandni



On Thu, Nov 5, 2015 at 5:03 PM, Kottapalli, Venkatesh <
VKottapalli@directv.com> wrote:

> Hi,
>
>                 I have tried partitioning block reader by setting the
> following properties.
>
> <property>
>               <name>dt.application.appname.operator.reader.attr.MEMORY_MB
>               </name>
>               <value>24576</value>
>        </property>
>
>        property>
>
> <name>dt.application.appname.operator.reader.prop.PARTITIONER</name>
>
> <value>com.datatorrent.lib.partitioner.StatelessPartitioner:4</value>
>        </property>
>
> <!- Only Downstream operator of reader, It has a port 'input' listening on
> output port of reader - This will not unify data -->
>        <property>
>
> <name>dt.application.appname.operator.console.port.input.attr.PARTITION_PARALLEL</name>
>               <value>true</value>
>        </property>
>
> This is reading data into a single operator and not partitioning reader
> operator, do I need to add something else for the partitioning to be
> enabled. BlockReader has not implemented Partitioner since it is already
> implemented in one of its parent classes. Please suggest.
>
> Regards,
> Venkatesh.
>
>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message