ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From svonn <sveng...@posteo.de>
Subject RE: Question about data distribution
Date Sat, 27 Jan 2018 12:46:12 GMT

My class for my keys looks like this.

private String deviceId;

    private long measurementId;

    private long timestamp;

    public IgniteKey(String deviceId, long measurementId, long timestamp) {
        this.deviceId = deviceId;
        this.measurementId = measurementId;
        this.timestamp = timestamp;


One device can have multiple measurements, but any calculation only requires
other entries from the same measurement as of now, thus only the
measurementId should be relevant.

One measurement contains 100k - 200k entries in one stream, and 500-1000 in
the other stream. Both streams use the same class for keys.

Whenever a new measurementId arrives I'm doing some output on the node it's
being processed on - I've had following case:
Measurement 1 (short M1) -> node1
M2 -> node1
M3 -> node2
M4 -> node1
M5 -> node1
M6 -> node1

I expected that even M2 will already be placed on node2 - however,
performance wise, I don't think either node is close to it's limit, I'm not
sure if that also relevant.
Due to the 5min expiry policy I can end up with one node having ~1 million
cache entries while the other one has 0.

- svonn

Sent from: http://apache-ignite-users.70518.x6.nabble.com/

View raw message