flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Checkpointing with RocksDB as statebackend
Date Fri, 30 Jun 2017 13:37:07 GMT
That’s great to hear!

Maybe it would make sense to add these defaults to Flink, if they don’t otherwise degrade
performance.

Best,
Aljoscha

> On 29. Jun 2017, at 22:44, Vinay Patil <vinay18.patil@gmail.com> wrote:
> 
> Hi Guys,
> 
> I am able to overcome the physical memory consumption issue by setting the following
options of RocksDB:
> 
> DBOptions: 
>      along with the FLASH_SSD_OPTIONS added the following:
>      maxBackgroundCompactions(4)
>     
> ColumnFamilyOptions:
>   max_buffer_size : 512 MB
>   block_cache_size : 128 MB
>   max_write_buffer_number : 5
>   minimum_buffer_number_to_merge : 2
>   cacheIndexAndFilterBlocks : true
>   optimizeFilterForHits: true
> 
> I am going to modify these options to get the desired performance.
> 
> I have set the DbLogDir("/var/tmp/") and StatsSumpPeriodicSec but I only got the configurations
set in the log file present in var/tmp/ 
> 
> Where will I get the RocksDB statistics if I set createStatistics ?
> 
> Let me know if any other configurations will help to get better performance. Now the
physical memory is slowly getting increased and I can see the drop in the graph ( this means
flushing is taking place at regular intervals )
> 
> Regards,
> Vinay Patil
> 
> On Thu, Jun 29, 2017 at 9:13 PM, Vinay Patil <vinay18.patil@gmail.com <mailto:vinay18.patil@gmail.com>>
wrote:
> The state size is not that huge. On the Flink UI when it showed the data sent as 4GB
, the physical memory usage was close to 90GB ..
> 
> I will re-run by setting the Flushing options of RocksDB because I am facing this issue
on 1.2.0 as well.
> 
> Regards,
> Vinay Patil
> 
> On Thu, Jun 29, 2017 at 9:03 PM, Aljoscha Krettek <aljoscha@apache.org <mailto:aljoscha@apache.org>>
wrote:
> Yup, I got that. I’m just wondering whether this occurs only with enabled checkpointing
or also when checkpointing is disabled.
> 
>> On 29. Jun 2017, at 17:31, Vinay Patil <vinay18.patil@gmail.com <mailto:vinay18.patil@gmail.com>>
wrote:
>> 
>> Hi Aljoscha,
>> 
>> Yes I have tried with 1.2.1 and 1.3.0 , facing the same issue.
>> 
>> The issue is not of Heap memory , it is of the Off-Heap memory that is getting used
 ( please refer to the earlier snapshot I have attached in which the graph keeps on growing
).
>> 
>> 
>> Regards,
>> Vinay Patil
>> 
>> On Thu, Jun 29, 2017 at 8:55 PM, Aljoscha Krettek <aljoscha@apache.org <mailto:aljoscha@apache.org>>
wrote:
>> Just a quick remark: Flink 1.3.0 and 1.2.1 always use FRocksDB, you shouldn’t manually
specify that.
>> 
>>> On 29. Jun 2017, at 17:20, Vinay Patil <vinay18.patil@gmail.com <mailto:vinay18.patil@gmail.com>>
wrote:
>>> 
>>> Hi Gerry,
>>> 
>>> Even I have faced this issue on 1.3.0 even by using FRocksDB and enabling incremental
checkpointing.
>>> 
>>> You can add FRocksDB dependency as shown here : https://github.com/apache/flink/pull/3704
<https://github.com/apache/flink/pull/3704>
>>> 
>>> We will have to set some RocksDB parameters  to get this working.
>>> 
>>> @Stefan or @Stephan : can you please help in resolving this issue
>>> 
>>> Regards,
>>> Vinay Patil
>>> 
>>> On Thu, Jun 29, 2017 at 6:01 PM, gerryzhou [via Apache Flink User Mailing List
archive.] <ml+s2336050n14063h5@n4.nabble.com <mailto:ml+s2336050n14063h5@n4.nabble.com>>
wrote:
>>> Hi, Vinay, 
>>>      I observed a similar problem in flink 1.3.0 with rocksdb. I wonder how to
use FRocksDB as you mentioned above. Thanks. 
>>> 
>>> If you reply to this email, your message will be added to the discussion below:
>>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Re-Checkpointing-with-RocksDB-as-statebackend-tp11752p14063.html
<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Re-Checkpointing-with-RocksDB-as-statebackend-tp11752p14063.html>
>>> To start a new topic under Apache Flink User Mailing List archive., email ml+s2336050n1h83@n4.nabble.com
<mailto:ml%2Bs2336050n1h83@n4.nabble.com> 
>>> To unsubscribe from Apache Flink User Mailing List archive., click here <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=dmluYXkxOC5wYXRpbEBnbWFpbC5jb218MXwxODExMDE2NjAx>.
>>> NAML <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>> 
>> 
> 
> 
> 


Mime
View raw message