ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@apache.org>
Subject Re: Ignite use case
Date Mon, 11 Dec 2017 17:43:32 GMT
Hello Steve,

Your use case perfectly well fits Ignite’s capabilities. Please see inline for the rest.

> On Dec 11, 2017, at 6:00 AM, Steven van der Merwe <stevevandermerwe@gmail.com>
wrote:
> 
> Hi All
> 
> I am new to Ignite and I am trying to understand if Ignite will fit my use case (I hope
so). It seems to have all of the aspects I am looking for.
> 
> I am trying to do the following:
> I am building up a realtime analytics / business dashboard based on a whole lot of metrics
and messages in the integration space (utilising something like a time series database). 
> I would like to stream messages from a number of different data collectors (I am using
queues at the moment for this task)
> At the moment I am putting these messages into Graylog / Elasticsearch but I am looking
to change this
> I need to store the time series data in a DB (About 1-5 million messages a day)
> I want to evict / delete the data after a month or so (if space is a problem) 
> I need to do some basic aggregation and CEP
> I would like to aggregate the data over time (and thus reduce the granularity / size)
> 
> I guess my questions are the following:
> Can I start using Ignite (in a single (or a few) nodes initially) as a SQL / KV store
(I will probably persist the data to disk very soon only keeping a few 10-20GB in RAM and
use it more like a traditional “DB”)?
Yes, the documentation section below explains how to define how much memory to dedicate in
RAM and how to enable the persistence:
https://apacheignite.readme.io/docs/memory-configuration#section-data-regions
> My concern is the Ignite was not designed for this (as it seems mostly in memory) so
I am unsure about how it will work persisting 500-1000GB of data on a single node (I guess
I am trying to use it more as a hybrid in memory store)
Ignite 2.x is based on a completely new memory architecture where disk stores a subset of
data and RAM caches as much data as you can afford.
https://apacheignite.readme.io/docs/durable-memory

So, presently Ignite is also used as a system of records or classical DB.

> The documentation seems to refer to using it as a cache in front of a RDBMS but I would
like to just use Ignite as the only DBMS
What’s the documentation you’re referring to? There are many documentation and use cases
pages Ignite is used for.

For instance, the use cases on the site unveil how to use it as a DBMS:
https://ignite.apache.org/usecases.html
> Can I aggregate the data using time as a “grouping" field ?
> I will delete the expired data from the db periodically
> I want to do CEP on the data
> 
Yes

Denis
> Thanks for the help in advance.
> 
> Regards
> Steve


Mime
View raw message