ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: Number of partitions of IgniteRDD
Date Fri, 29 Apr 2016 12:41:46 GMT
Hi Vij,

I see method "getPartitions" in IgniteRDD, not "getNumPartitions". Please
confirm that we are talking about the same thing.

Anyway, logic of this method is extremely straightforward - it simply call
Ignite.affinity("name_of_your_cache").partitions() method, so it should
return actual number of partitions.
"getPartitions" returns array, could you please show is printed to the
console from your code?


Vladimir.

On Fri, Apr 29, 2016 at 3:10 PM, vijayendra bhati <veejayendraa@yahoo.com>
wrote:

> Yes its Spark RDD's standard method, but it has been overridden in
> IgniteRDD.
>
> Regards,
> Vij
>
>
> On Friday, April 29, 2016 5:25 PM, Vladimir Ozerov <vozerov@gridgain.com>
> wrote:
>
>
> Hi Vij,
>
> I am not quite uderstand where does method "getNumPartitions" came from.
> Is it on standard Spark API? I do not see it on*
> org.apache.spark.api.java.JavaRDD* class.
>
> Vladimir.
>
> On Fri, Apr 29, 2016 at 7:50 AM, vijayendra bhati <veejayendraa@yahoo.com>
> wrote:
>
> Hi Val,
>
> I am creating DataFrame using below code -
>
> * public DataFrame getStockSimulationReturnsDataFrame(LocalDate
> businessDate,String stock){*
> * /**
> * * If we use sql query , we are assuming that data is in cache.*
> * * */*
> * String sql = "select simulationUUID,stockReturn from
> STOCKSIMULATIONRETURNSVAL where businessDate = ? and symbol = ?";*
> * DataFrame df =jic.fromCache(PARTITIONED_CACHE_NAME).sql(sql,
> businessDate,stock);*
> * return df;*
> * }*
>
>
> And to check partitions I am doing -
>
> private JavaRDD<Row> getSimulationsForStock(String stock,LocalDate
> businessDate)
> {
> DataFrame df =  StockSimulationsReaderFactory.getStockSimulationStore(jsc,
> businessDate,
> businessDate).getStockSimulationReturnsDataFrame(businessDate, stock);
> System.out.println("^^^^^^^^"+df.javaRDD().getNumPartitions());
> return df.javaRDD();
> }
>
> Regards,
> Vij
>
>
> On Friday, April 29, 2016 3:09 AM, vkulichenko <
> valentin.kulichenko@gmail.com> wrote:
>
>
> Hi Vij,
>
>
> How do you check the number of partitions and what are you trying to
> achieve? Can you show the code?
>
> -Val
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Number-of-partitions-of-IgniteRDD-tp4644p4671.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
>
>
>
>
>
>

Mime
View raw message