My bad !!! Yea you are right.But now the problem is when I get DataFrame by using below code and get JavaRDD from it , its number of partitions is 1 and its of type  MapPartitionsRDD.

String sql = "select simulationUUID,stockReturn from STOCKSIMULATIONRETURNSVAL where businessDate = ? and symbol = ?";
DataFrame df =jic.fromCache(PARTITIONED_CACHE_NAME).sql(sql, businessDate,stock);
 df.javaRDD();

This is causing performance issues for me as I have only 1 partition and my reduceByKey method is not performing in desired way.

Regards,
Vij



On Friday, April 29, 2016 6:11 PM, Vladimir Ozerov <vozerov@gridgain.com> wrote:


Hi Vij,

I see method "getPartitions" in IgniteRDD, not "getNumPartitions". Please confirm that we are talking about the same thing.

Anyway, logic of this method is extremely straightforward - it simply call Ignite.affinity("name_of_your_cache").partitions() method, so it should return actual number of partitions. 
"getPartitions" returns array, could you please show is printed to the console from your code?


Vladimir.

On Fri, Apr 29, 2016 at 3:10 PM, vijayendra bhati <veejayendraa@yahoo.com> wrote:
Yes its Spark RDD's standard method, but it has been overridden in IgniteRDD.

Regards,
Vij


On Friday, April 29, 2016 5:25 PM, Vladimir Ozerov <vozerov@gridgain.com> wrote:


Hi Vij,

I am not quite uderstand where does method "getNumPartitions" came from. Is it on standard Spark API? I do not see it on org.apache.spark.api.java.JavaRDD class.

Vladimir.

On Fri, Apr 29, 2016 at 7:50 AM, vijayendra bhati <veejayendraa@yahoo.com> wrote:
Hi Val,

I am creating DataFrame using below code - 

public DataFrame getStockSimulationReturnsDataFrame(LocalDate businessDate,String stock){
/*
* If we use sql query , we are assuming that data is in cache.
* */
String sql = "select simulationUUID,stockReturn from STOCKSIMULATIONRETURNSVAL where businessDate = ? and symbol = ?";
DataFrame df =jic.fromCache(PARTITIONED_CACHE_NAME).sql(sql, businessDate,stock);
return df;
}


And to check partitions I am doing - 

private JavaRDD<Row> getSimulationsForStock(String stock,LocalDate businessDate)
{
DataFrame df =  StockSimulationsReaderFactory.getStockSimulationStore(jsc, businessDate, businessDate).getStockSimulationReturnsDataFrame(businessDate, stock);
System.out.println("^^^^^^^^"+df.javaRDD().getNumPartitions());
return df.javaRDD();
}

Regards,
Vij


On Friday, April 29, 2016 3:09 AM, vkulichenko <valentin.kulichenko@gmail.com> wrote:


Hi Vij,


How do you check the number of partitions and what are you trying to
achieve? Can you show the code?

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Number-of-partitions-of-IgniteRDD-tp4644p4671.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.