ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Evgenii Zhuravlev <e.zhuravlev...@gmail.com>
Subject Re: Failed to parse query: exception with Scala
Date Thu, 09 Nov 2017 13:22:38 GMT
Hey,

Just checked SharedRDDExample, replaced all Integer mentions to the String
to try to reproduce the behavior you described. In my case, it shows normal
count(in this example it's 10).

As for me, it looks like you have problems in your code. Please share a
small reproducer for this problem and community will help.

Regards,
Evgenii




2017-11-09 9:52 GMT+03:00 future expert <futureexpert2019@gmail.com>:

> I was able to resolve the *"1%lo"* exception by setting export
> _JAVA_OPTIONS=-Djava.net.preferIPv4Stack=true and there are no errors
> while saving and retrieving the shared-rdd from another Spark App.
>
> However, I am seeing that the pair Ignite RDD [String, String] always has
> a count of 1. The jsonRDD used to save to cache has many records and has
> count > 10,000.
>
> for example, *transformedValues.take(5).foreach(println)* prints only 1
> pair and *transformedValues.count()* is always equal to 1. In the visor,
> I see that this RDD has only 0.14 as Totalsize (Total size Min/Avg/Max
>   | 0.14 / 0.00 / 0.14 )
>
> This is not the case with Integer pair RDDs and works as expected.
>
> Does anybody the possible reason for this strange behavior of String pair
> Ignite RDDs?
>
>
>
>
>
> On Wed, Nov 8, 2017 at 3:22 PM, future expert <futureexpert2019@gmail.com>
> wrote:
>
>> I think I found something interesting. I see the below when i print the
>> current topology from visor.
>>
>> [image: Inline image 1]
>>
>> The "1%lo" string that is under the Int./Ext. IPs is the same as the
>> string that is included in the below exception when *sharedRDDConsumer.first()
>> or **sharedRDDConsumer.take(5).foreach(println) *is executed*. *
>>
>> java.lang.NumberFormatException: For input string: *"1%lo"*
>> at java.lang.NumberFormatException.forInputString(NumberFormatE
>> xception.java:65)
>>
>> Is this a possible bug?
>>
>>
>>>
>>>
>>>
>>> On Wed, Nov 8, 2017 at 11:00 AM, future expert <
>>> futureexpert2019@gmail.com> wrote:
>>>
>>>> I am just trying to retrieve the already cached shared-rdd(
>>>> sharedRDDProducer) back from cache. I have also tried like below too
>>>> without success.
>>>>
>>>> *val *sharedRDDConsumer*: IgniteRDD[String, String] =
>>>> igniteContext.fromCache[String, String]("*sharedRDDProducer*")*
>>>>
>>>> I am getting the same error even when running the below example program
>>>>
>>>>     Exception   : class javax.cache.CacheException
>>>>     Message     : class org.apache.ignite.internal.pro
>>>> cessors.query.IgniteSQLException: Failed to parse query: select _val
>>>> from String
>>>>
>>>> Also, not sure why the retrieved RDD count() always shows 1 instead of
>>>> the actual recordcount!
>>>>
>>>> Thanks.
>>>>
>>>>
>>>> On Wed, Nov 8, 2017 at 8:45 AM, Evgenii Zhuravlev <
>>>> e.zhuravlev.wk@gmail.com> wrote:
>>>>
>>>>> I don't really get, what you trying to do here:
>>>>>
>>>>> val sharedRDDConsumer = igniteContext.fromCache("sharedRDDProducer")
>>>>>
>>>>> it looks like a mistake
>>>>>
>>>>> Here is example of using ignite sql from spark in java:
>>>>>
>>>>> https://github.com/apache/ignite/blob/master/examples/src/ma
>>>>> in/spark/org/apache/ignite/examples/spark/SharedRDDExample.java
>>>>>
>>>>> the same for scala:
>>>>>
>>>>> https://github.com/apache/ignite/blob/master/examples/src/ma
>>>>> in/scala/org/apache/ignite/scalar/examples/spark/ScalarShare
>>>>> dRDDExample.scala
>>>>>
>>>>> 2017-11-08 19:02 GMT+03:00 future expert <futureexpert2019@gmail.com>:
>>>>>
>>>>>> Thanks. I currently do not have indexed types for cache
>>>>>> "sharedRDDProducer" as i currently add it as below.
>>>>>>
>>>>>> *val sharedRDDProducer: IgniteRDD[String, String] =
>>>>>> igniteContext.fromCache[String, String]("sharedRDDProducer")*
>>>>>> *sharedRDDProducer.savePairs(jsonRdd)*
>>>>>>
>>>>>> Is the indexed types needed for sharedRDDProducer as well? If so,
>>>>>> how can I add it?
>>>>>>
>>>>>> Also, I am getting the below exception with all the different types
>>>>>> of datasets when trying to do a *sharedRDDConsumer.first() or **sharedRDDConsumer.take(5).foreach(println).
>>>>>> *I think that something is wrong with the saved sharedRDDProducer.
Could
>>>>>> it be an Ignite version issue?
>>>>>>
>>>>>> java.lang.NumberFormatException: For input string: "1%lo"
>>>>>> at java.lang.NumberFormatException.forInputString(NumberFormatE
>>>>>> xception.java:65)
>>>>>>
>>>>>>
>>>>>> I tried the below example using [Int, Int] rdd as well but the SQL
>>>>>> part at the end is giving the same exception. Do you have a working
SQL
>>>>>> query sample in scala using [string, string] pair rdd? Thanks.
>>>>>>
>>>>>> https://github.com/apache/ignite/blob/master/examples/src/ma
>>>>>> in/scala/org/apache/ignite/scalar/examples/spark/ScalarShare
>>>>>> dRDDExample.scala
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Nov 8, 2017 at 6:27 AM, ezhuravlev <e.zhuravlev.wk@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Do you have indexed types for cache "sharedRDDProducer"?
>>>>>>>
>>>>>>> like
>>>>>>>
>>>>>>> cacheCfg.setIndexedTypes(String.class, String.class);
>>>>>>>
>>>>>>> Evgenii
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message