ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shawn Du" <shawn...@neulion.com.cn>
Subject 答复: 答复: 答复: query binaryobject cache in zeppelin
Date Fri, 16 Dec 2016 02:44:54 GMT
Hi,

 

Blow SQL reports error

select * from onlinesessioncount_by_appversion_childapptypeid_countryid_networktype_parentapptypeid_regionid_sessiontimelevel_usertype
limit 10;

But  SQL without asterisk is ok:

select timestamp from onlinesessioncount_by_appversion_childapptypeid_countryid_networktype_parentapptypeid_regionid_sessiontimelevel_usertype
limit 10;

 

Zeppelin complains: 

Failed resolve class for ID: -869249411

 

Ignite server report below errors:

 

[02:37:29,212][SEVERE][pub-#118%null%][GridJobWorker] Failed to execute job [jobId=300f0750951-7c383220-1fe9-47b9-a101-aa273e72f0ba,
ses=GridJobSessionImpl [ses=GridTaskSessionImpl [taskName=o.a.i.i.processors.cache.query.jdbc.GridCacheQueryJdbcTask,
dep=LocalDeployment [super=GridDeployment [ts=1481854956302, depMode=SHARED, clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6,
clsLdrId=2e030750951-c9dac7eb-827c-4bb5-99d1-c347192977e0, userVer=0, loc=true, sampleClsName=java.lang.String,
pendingUndeploy=false, undeployed=false, usage=0]], taskClsName=o.a.i.i.processors.cache.query.jdbc.GridCacheQueryJdbcTask,
sesId=200f0750951-7c383220-1fe9-47b9-a101-aa273e72f0ba, startTime=1481855849160, endTime=9223372036854775807,
taskNodeId=7c383220-1fe9-47b9-a101-aa273e72f0ba, clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6,
closed=false, cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false, internal=false,
subjId=7c383220-1fe9-47b9-a101-aa273e72f0ba, mapFut=IgniteFuture [orig=GridFutureAdapter [resFlag=0,
res=null, startTime=1481855849202, endTime=0, ignoreInterrupts=false, state=INIT]]], jobId=300f0750951-7c383220-1fe9-47b9-a101-aa273e72f0ba]]

class org.apache.ignite.binary.BinaryObjectException: Failed resolve class for ID: -869249411

        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:696)

        at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1491)

        at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1450)

        at org.apache.ignite.internal.binary.BinaryObjectImpl.deserializeValue(BinaryObjectImpl.java:637)

        at org.apache.ignite.internal.binary.BinaryObjectImpl.value(BinaryObjectImpl.java:142)

        at org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinary(CacheObjectContext.java:272)

        at org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinariesIfNeeded(CacheObjectContext.java:185)

        at org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinariesIfNeeded(CacheObjectContext.java:169)

        at org.apache.ignite.internal.processors.cache.GridCacheContext.unwrapBinariesIfNeeded(GridCacheContext.java:1749)

        at org.apache.ignite.internal.processors.query.GridQueryCacheObjectsIterator.next(GridQueryCacheObjectsIterator.java:64)

        at org.apache.ignite.internal.processors.query.GridQueryCacheObjectsIterator.next(GridQueryCacheObjectsIterator.java:29)

        at org.apache.ignite.internal.processors.cache.query.jdbc.GridCacheQueryJdbcTask$JdbcDriverJob.execute(GridCacheQueryJdbcTask.java:308)

        at org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:556)

        at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6564)

        at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:550)

        at org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:479)

        at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)

        at org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1180)

        at org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1894)

        at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1082)

        at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:710)

        at org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:102)

        at org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:673)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:745)

Caused by: class org.apache.ignite.IgniteCheckedException: Class definition was not found
at marshaller cache and local file. [id=-869249411, file=/home/ec2-user/apache-ignite-fabric/work/marshaller/-869249411.classname]

        at org.apache.ignite.internal.MarshallerContextImpl.className(MarshallerContextImpl.java:218)

        at org.apache.ignite.internal.MarshallerContextAdapter.getClass(MarshallerContextAdapter.java:174)

        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:680)

        ... 25 more

 

This is cache configuration:

 

Cache 'onlineviewcount_by_appversion_childapptypeid_countryid_networktype_parentapptypeid_regionid_streamtype_usertype_viewtimelevel(@c13)':

+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+

| Name(@)                     | onlineviewcount_by_appversion_childapptypeid_countryid_networktype_parentapptypeid_regionid_streamtype_usertype_viewtimelevel(@c13)
|

| Nodes                       | 1                                                        
                                                                          |

| Total size Min/Avg/Max      | 191225 / 191225.00 / 191225                              
                                                                          |

|   Heap size Min/Avg/Max     | 191225 / 191225.00 / 191225                              
                                                                          |

|   Off-heap size Min/Avg/Max | 0 / 0.00 / 0                                             
                                                                          |

+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+

 

Nodes for: onlineviewcount_by_appversion_childapptypeid_countryid_networktype_parentapptypeid_regionid_streamtype_usertype_viewtimelevel(@c13)

+===============================================================================================================+

|       Node ID8(@), IP       | CPUs | Heap Used | CPU Load |   Up Time    |         Size
        | Hi/Mi/Rd/Wr |

+===============================================================================================================+

| C9DAC7EB(@n0), 172.31.33.77 | 8    | 8.45 %    | 0.20 %   | 00:18:55:843 | Total: 191225
       | Hi: 0       |

|                             |      |           |          |              |   Heap: 191225
      | Mi: 0       |

|                             |      |           |          |              |   Off-Heap: 0
       | Rd: 0       |

|                             |      |           |          |              |   Off-Heap Memory:
0 | Wr: 0       |

+---------------------------------------------------------------------------------------------------------------+

'Hi' - Number of cache hits.

'Mi' - Number of cache misses.

'Rd' - number of cache reads.

'Wr' - Number of cache writes.

 

Aggregated queries metrics:

  Minimum execution time: 00:00:00:000

  Maximum execution time: 00:00:00:000

  Average execution time: 00:00:00:000

  Total number of executions: 0

  Total number of failures:   0

 

Cache configuration:

+==========================================================================================================+

|                   Name                    |                            Value           
                 |

+==========================================================================================================+

| Mode                                      | PARTITIONED                                
                 |

| Atomicity Mode                            | ATOMIC                                     
                 |

| Atomic Write Ordering Mode                | PRIMARY                                    
                 |

| Statistic Enabled                         | off                                        
                 |

| Management Enabled                        | off                                        
                 |

| Time To Live Eager Flag                   | true                                       
                 |

| Write Synchronization Mode                | PRIMARY_SYNC                               
                 |

| Swap Enabled                              | off                                        
                 |

| Invalidate                                | off                                        
                 |

| Start Size                                | 10000                                      
                 |

| Affinity Function                         | o.a.i.cache.affinity.rendezvous.RendezvousAffinityFunction
  |

| Affinity Backups                          | 0                                          
                 |

| Affinity Partitions                       | 1024                                       
                 |

| Affinity Exclude Neighbors                | false                                      
                 |

| Affinity Mapper                           | o.a.i.i.processors.cache.CacheDefaultBinaryAffinityKeyMapper
|

| Rebalance Mode                            | ASYNC                                      
                 |

| Rebalance Batch Size                      | 524288                                     
                 |

| Rebalance Thread Pool size                | 2                                          
                 |

| Rebalance Timeout                         | 10000                                      
                 |

| Rebalance Delay                           | 0                                          
                 |

| Time Between Rebalance Messages           | 0                                          
                 |

| Eviction Policy Enabled                   | off                                        
                 |

| Eviction Policy                           | <n/a>                                
                       |

| Eviction Policy Max Size                  | <n/a>                                
                       |

| Eviction Filter                           | <n/a>                                
                       |

| Eviction Key Buffer Size                  | 1024                                       
                 |

| Eviction Synchronized                     | off                                        
                 |

| Eviction Overflow Ratio                   | 10.0                                       
                 |

| Synchronous Eviction Timeout              | 10000                                      
                 |

| Synchronous Eviction Concurrency Level    | 4                                          
                 |

| Near Cache Enabled                        | off                                        
                 |

| Near Start Size                           | 0                                          
                 |

| Near Eviction Policy                      | <n/a>                                
                       |

| Near Eviction Policy Max Size             | <n/a>                                
                       |

| Default Lock Timeout                      | 0                                          
                 |

| Metadata type count                       | 1                                          
                 |

| Cache Interceptor                         | <n/a>                                
                       |

| Store Enabled                             | on                                         
                 |

| Store Class                               | com.neulion.qos.ignite.store.MetricCacheStore
               |

| Store Factory Class                       | com.neulion.qos.ignite.store.MetricCacheStoreFactory
        |

| Store Keep Binary                         | true                                       
                 |

| Store Read Through                        | off                                        
                 |

| Store Write Through                       | on                                         
                 |

| Write-Behind Enabled                      | on                                         
                 |

| Write-Behind Flush Size                   | 10240                                      
                 |

| Write-Behind Frequency                    | 5000                                       
                 |

| Write-Behind Flush Threads Count          | 1                                          
                 |

| Write-Behind Batch Size                   | 512                                        
                 |

| Concurrent Asynchronous Operations Number | 500                                        
                 |

| Memory Mode                               | ONHEAP_TIERED                              
                 |

| Off-Heap Size                             | <n/a>                                
                       |

| Loader Factory Class Name                 | <n/a>                                
                       |

| Writer Factory Class Name                 | <n/a>                                
                       |

| Expiry Policy Factory Class Name          | javax.cache.configuration.FactoryBuilder$SingletonFactory
   |

| Query Execution Time Threshold            | 3000                                       
                 |

| Query Schema Name                         |                                            
                 |

| Query Escaped Names                       | off                                        
                 |

| Query Onheap Cache Size                   | 10240                                      
                 |

| Query SQL functions                       | <n/a>                                
                       |

| Query Indexed Types                       | <n/a>                                
                       |

+----------------------------------------------------------------------------------------------------------+

 

 

Thanks

Shawn

 

 

发件人: Andrey Mashenkov [mailto:andrey.mashenkov@gmail.com] 
发送时间: 2016年12月15日 18:21
收件人: user@ignite.apache.org
主题: Re: 答复: 答复: query binaryobject cache in zeppelin

 

Hi Shawn,

 

I've got Zeppelin sources, rewrite test query () with arterisk (org.apache.zeppelin.ignite.IgniteSqlInterpreterTest).
It works fine for me as with ignite 1.7 as 1.8.

Would you please share sql query that failed, jdbc connection string that used by zeppelin,
ignite cache configuration and also ignite logs if possible?

 

On Tue, Dec 13, 2016 at 3:34 AM, Shawn Du <shawn.du@neulion.com.cn <mailto:shawn.du@neulion.com.cn>
> wrote:

 

Hi Andrey,

 

I had code to add entities to configuration. It seems that it doesn’t support using Asterisk
in select clause.

If I use column names, zeppelin can show the data.

 

Also some background about the testing:

 

Using ignite 1.8 and zeppelin 0.6.2.  in order to make zeppelin work with ignite 1.8, I had
to build from source and change pom.xml to use ignite 1.8.0.

All Seems good. 

 

Thanks

Shawn

 

发件人: Andrey Mashenkov [mailto:andrey.mashenkov@gmail.com <mailto:andrey.mashenkov@gmail.com>
] 
发送时间: 2016年12月12日 18:54
收件人: user@ignite.apache.org <mailto:user@ignite.apache.org> 
主题: Re: 答复: query binaryobject cache in zeppelin

 

Hi Shawn,

 

Looks strange that Query Indexed Types  is "n/a"

Are you forget to add query entities to configuration? I can't see in your code: cacheCfg.setQueryEntities(Arrays.asList(entity))

 

 

On Mon, Dec 12, 2016 at 1:41 PM, Shawn Du <shawn.du@neulion.com.cn <mailto:shawn.du@neulion.com.cn>
> wrote:

Hi,

 

Just this is part of my cache configuration.  See the red part. Query schema Name is empty,
it is  so-called *table name*?

 

|Store Write Through                         | off                                       
                  |

| Write-Behind Enabled                      | off                                        
                 |

| Write-Behind Flush Size                   | 10240                                      
                 |

| Write-Behind Frequency                    | 5000                                       
                 |

| Write-Behind Flush Threads Count          | 1                                          
                 |

| Write-Behind Batch Size                   | 512                                        
                 |

| Concurrent Asynchronous Operations Number | 500                                        
                 |

| Memory Mode                               | ONHEAP_TIERED                              
                 |

| Off-Heap Size                             | <n/a>                                
                       |

| Loader Factory Class Name                 | <n/a>                                
                       |

| Writer Factory Class Name                 | <n/a>                                
                       |

| Expiry Policy Factory Class Name          | javax.cache.configuration.FactoryBuilder$SingletonFactory
   |

| Query Execution Time Threshold            | 3000                                       
                 |

| Query Schema Name                         |                                            
                 |

| Query Escaped Names                       | off                                        
                 |

| Query Onheap Cache Size                   | 10240                                      
                 |

| Query SQL functions                       | <n/a>                                
                       |

| Query Indexed Types                       | <n/a>                                
                       |

+----------------------------------------------------------------------------------------------------------+

 

 

I create queryEntity by the following code,  ColumnScheme is my own class, just contains information
column name/type and ensureIndex or not.

 

Is the entity value type is “Table Name”?  I set it the same with the cache name.  I try
to use this as the table name in SQL, still not work.  Please help. Thanks.

 

QueryEntity entity = new QueryEntity();
entity.setKeyType(keyType);
entity.setValueType(valueType);
LinkedHashMap<String, String> fields = new LinkedHashMap<>();
List<QueryIndex> indexes = new ArrayList<>();
for (ColumnScheme columnScheme : columns)
{
    fields.put(columnScheme.getName(), columnScheme.getType());
    if (columnScheme.isEnsureIndex())
    {
        indexes.add(new QueryIndex(columnScheme.getName()));
    }
}
entity.setIndexes(indexes);
entity.setFields(fields);

 

Shawn

 

发件人: Andrey Mashenkov [mailto:amashenkov@gridgain.com <mailto:amashenkov@gridgain.com>
] 
发送时间: 2016年12月12日 17:24
收件人: user@ignite.apache.org <mailto:user@ignite.apache.org> 
主题: Re: query binaryobject cache in zeppelin

 

Hi Shawn,

 

Classes that you want to use in queries should be set via setQueryEntities.

 

*Table name* in Ignite has name of type. E.g if you want to get some record of class "my.org.Person"
you should use "Person" as table name: Select * from Person. 

To make cross cache query you should use full table name as "cache_name".class_name: Select
... from Person, "other_cache".Org Where ...

 

For JDK classes like java.lang.Integer, table name will be "Integer"

 

On Mon, Dec 12, 2016 at 12:09 PM, Shawn Du <shawn.du@neulion.com.cn <mailto:shawn.du@neulion.com.cn>
> wrote:

Hi,

 

Today, I had a try of zeppelin. After setup a zeppelin node and began to issue SQL for query,
I don’t know how to input the *table name* of SQL.

For my cache are built with BinaryObject and config with setQueryEntities.

 

It is possible to do query in zeppinlin for these caches?

 

Thanks

Shawn  

 





 

-- 

С уважением,
Машенков Андрей Владимирович
Тел. +7-921-932-61-82

Best regards,
Andrey V. Mashenkov
Cerr: +7-921-932-61-82





 

-- 

С уважением,
Машенков Андрей Владимирович
Тел. +7-921-932-61-82

Best regards,
Andrey V. Mashenkov
Cerr: +7-921-932-61-82


Mime
View raw message