ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steven van der Merwe <stevevanderme...@gmail.com>
Subject Ignite use case
Date Mon, 11 Dec 2017 14:00:27 GMT
Hi All

I am new to Ignite and I am trying to understand if Ignite will fit my use
case (I hope so). It seems to have all of the aspects I am looking for.

I am trying to do the following:

   - I am building up a realtime analytics / business dashboard based on a
   whole lot of metrics and messages in the integration space (utilising
   something like a time series database).
   - I would like to stream messages from a number of different data
   collectors (I am using queues at the moment for this task)
      - At the moment I am putting these messages into Graylog /
      Elasticsearch but I am looking to change this
   - I need to store the time series data in a DB (About 1-5 million
   messages a day)
      - I want to evict / delete the data after a month or so (if space is
      a problem)
      - I need to do some basic aggregation and CEP
   - I would like to aggregate the data over time (and thus reduce the
   granularity / size)


I guess my questions are the following:

   - Can I start using Ignite (in a single (or a few) nodes initially) as a
   SQL / KV store (I will probably persist the data to disk very soon only
   keeping a few 10-20GB in RAM and use it more like a traditional “DB”)?
      - My concern is the Ignite was not designed for this (as it seems
      mostly in memory) so I am unsure about how it will work persisting
      500-1000GB of data on a single node (I guess I am trying to use
it more as
      a hybrid in memory store)
      - The documentation seems to refer to using it as a cache in front of
      a RDBMS but I would like to just use Ignite as the only DBMS
   - Can I aggregate the data using time as a “grouping" field ?
      - I will delete the expired data from the db periodically
      - I want to do CEP on the data


Thanks for the help in advance.

Regards
Steve

Mime
View raw message