ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steven van der Merwe <stevevanderme...@gmail.com>
Subject Re: Ignite use case
Date Mon, 11 Dec 2017 17:57:35 GMT
Brilliant - Thanks for the pointers Dennis


On 11 December 2017 at 19:43:37, Denis Magda (dmagda@apache.org) wrote:

Hello Steve,

Your use case perfectly well fits Ignite’s capabilities. Please see inline
for the rest.

On Dec 11, 2017, at 6:00 AM, Steven van der Merwe <
stevevandermerwe@gmail.com> wrote:

Hi All

I am new to Ignite and I am trying to understand if Ignite will fit my use
case (I hope so). It seems to have all of the aspects I am looking for.

I am trying to do the following:

   - I am building up a realtime analytics / business dashboard based on a
   whole lot of metrics and messages in the integration space (utilising
   something like a time series database).
   - I would like to stream messages from a number of different data
   collectors (I am using queues at the moment for this task)
      - At the moment I am putting these messages into Graylog /
      Elasticsearch but I am looking to change this
   - I need to store the time series data in a DB (About 1-5 million
   messages a day)
      - I want to evict / delete the data after a month or so (if space is
      a problem)
      - I need to do some basic aggregation and CEP
   - I would like to aggregate the data over time (and thus reduce the
   granularity / size)

I guess my questions are the following:

   - Can I start using Ignite (in a single (or a few) nodes initially) as a
   SQL / KV store (I will probably persist the data to disk very soon only
   keeping a few 10-20GB in RAM and use it more like a traditional “DB”)?

Yes, the documentation section below explains how to define how much memory
to dedicate in RAM and how to enable the persistence:

   - My concern is the Ignite was not designed for this (as it seems mostly
   in memory) so I am unsure about how it will work persisting 500-1000GB of
   data on a single node (I guess I am trying to use it more as a hybrid in
   memory store)

Ignite 2.x is based on a completely new memory architecture where disk
stores a subset of data and RAM caches as much data as you can afford.

So, presently Ignite is also used as a system of records or classical DB.

   - The documentation seems to refer to using it as a cache in front of a
   RDBMS but I would like to just use Ignite as the only DBMS

What’s the documentation you’re referring to? There are many documentation
and use cases pages Ignite is used for.

For instance, the use cases on the site unveil how to use it as a DBMS:

   - Can I aggregate the data using time as a “grouping" field ?
      - I will delete the expired data from the db periodically
      - I want to do CEP on the data



Thanks for the help in advance.


View raw message