ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vijayendra bhati <veejayend...@yahoo.com>
Subject Re: Number of partitions of IgniteRDD
Date Fri, 29 Apr 2016 12:10:24 GMT
Yes its Spark RDD's standard method, but it has been overridden in IgniteRDD.

    On Friday, April 29, 2016 5:25 PM, Vladimir Ozerov <vozerov@gridgain.com> wrote:

 Hi Vij,
I am not quite uderstand where does method "getNumPartitions" came from. Is it on standard
Spark API? I do not see it on org.apache.spark.api.java.JavaRDD class.
On Fri, Apr 29, 2016 at 7:50 AM, vijayendra bhati <veejayendraa@yahoo.com> wrote:

Hi Val,
I am creating DataFrame using below code - 
 public DataFrame getStockSimulationReturnsDataFrame(LocalDate businessDate,String stock){
/*  * If we use sql query , we are assuming that data is in cache.  * */ String sql = "select
simulationUUID,stockReturn from STOCKSIMULATIONRETURNSVAL where businessDate = ? and symbol
= ?"; DataFrame df =jic.fromCache(PARTITIONED_CACHE_NAME).sql(sql, businessDate,stock); return
df; }

And to check partitions I am doing - 
private JavaRDD<Row> getSimulationsForStock(String stock,LocalDate businessDate) { DataFrame
df =  StockSimulationsReaderFactory.getStockSimulationStore(jsc, businessDate, businessDate).getStockSimulationReturnsDataFrame(businessDate,
stock); System.out.println("^^^^^^^^"+df.javaRDD().getNumPartitions()); return df.javaRDD();

    On Friday, April 29, 2016 3:09 AM, vkulichenko <valentin.kulichenko@gmail.com> wrote:

 Hi Vij,

How do you check the number of partitions and what are you trying to
achieve? Can you show the code?


View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Number-of-partitions-of-IgniteRDD-tp4644p4671.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.


View raw message