My bad !!! Yea you are right.But now the problem is when I get DataFrame by using below code and get JavaRDD from it , its number of partitions is 1 and its of type MapPartitionsRDD.
String sql = "select simulationUUID,stockReturn from STOCKSIMULATIONRETURNSVAL where businessDate = ? and symbol = ?";
DataFrame df =jic.fromCache(PARTITIONED_CACHE_NAME).sql(sql, businessDate,stock);
This is causing performance issues for me as I have only 1 partition and my reduceByKey method is not performing in desired way.
On Friday, April 29, 2016 6:11 PM, Vladimir Ozerov <email@example.com> wrote:
I see method "getPartitions" in IgniteRDD, not "getNumPartitions". Please confirm that we are talking about the same thing.
Anyway, logic of this method is extremely straightforward - it simply call Ignite.affinity("name_of_your_cache").partitions() method, so it should return actual number of partitions.
"getPartitions" returns array, could you please show is printed to the console from your code?