ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ray <ray...@cisco.com>
Subject Different behavior when saving date from Dataframe API and RDD API
Date Fri, 23 Mar 2018 07:48:01 GMT
I was trying out one of Ignite 2.4's new features - saving data from
dataframe.
But I found some inconsistency between the Dataframe API and RDD API.

This is the code from saving dataframe to Ignite.
DF.write
.format(FORMAT_IGNITE)
.mode(SaveMode.Append)
.option(OPTION_CONFIG_FILE, CONFIG)
.option(OPTION_TABLE, "table_name")
.option(OPTION_CREATE_TABLE_PRIMARY_KEY_FIELDS, "a,b,c,d")
.option(OPTION_CREATE_TABLE_PARAMETERS,
"template=partitioned,affinitykey=a")
.option(OPTION_STREAMER_ALLOW_OVERWRITE, "true")
.save()
After data finished saving, I ran this command to create an index on field
a.
CREATE INDEX IF NOT EXISTS idx ON table_name (a);
Then I run this query to see if the index is working.

explain select a from table_name where a = '303';
PLAN  SELECT
    __Z0.a AS __C0_0
FROM PUBLIC.table_name __Z0
    /* PUBLIC.AFFINITY_KEY: a = '303' */
WHERE __Z0.a = '303'

But when I try query the data I insert in the old RDD way, the result is
explain select a from table_name where a = '303';
PLAN  SELECT
    __Z0.a AS __C0_0
FROM PUBLIC.table_name __Z0
    /* PUBLIC.table_name_IDX: a = '303' */WHERE __Z0.a = '303'

The result shows with affinity key, the index created is not effective.
I tried creating index on other non affinity key field, the index is
working.
Please advise is this behavior expected?

THanks



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Mime
View raw message