ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ilya Kasnacheev <ilya.kasnach...@gmail.com>
Subject Re: Improve logging of Data Region configuration
Date Tue, 31 Dec 2019 13:01:28 GMT
Hello!

I have prepared a patch, please review. It will look like this:

Data Regions Started: 4
    ^--   sysMemPlc region [type=internal, persistence=true, lazyAlloc=false,
      ...  initCfg=40MB, maxCfg=100MB, usedRam=0MB, freeRam=100%,
allocRam=100MB, allocTotal=0MB]
    ^--   default region [type=default, persistence=true, lazyAlloc=true,
      ...  initCfg=30MB, maxCfg=30MB, usedRam=0MB, freeRam=100%,
allocRam=0MB, allocTotal=0MB]
    ^--   metastoreMemPlc region [type=internal, persistence=true,
lazyAlloc=false,
      ...  initCfg=40MB, maxCfg=100MB, usedRam=0MB, freeRam=100%,
allocRam=100MB, allocTotal=0MB]
    ^--   TxLog region [type=internal, persistence=true, lazyAlloc=false,
      ...  initCfg=40MB, maxCfg=100MB, usedRam=0MB, freeRam=100%,
allocRam=100MB, allocTotal=0MB]
    ^-- Ignite persistence [used=0MB]
...
Metrics for local node (to disable set 'metricsLogFrequency' to 0)
    ^-- Node [id=9e6c8e45,
name=internal.GridNodeMetricsLogPdsSelfTest0, uptime=00:00:12.271]
    ^-- Cluster [hosts=1, CPUs=8, servers=2, clients=0, topVer=2, minorTopVer=3]
    ^-- Network [addrs=[127.0.0.1], localHost=127.0.0.1,
discoPort=47500, commPort=45010]
    ^-- CPU [CPUs=8, curLoad=2.7%, avgLoad=3.38%, GC=0%]
    ^-- Heap [used=48MB, free=99.32%, comm=564MB]
    ^-- Off-heap memory [used=0MB, free=99.96%, allocated=230MB]
    ^-- Page memory [pages=34]
    ^--   sysMemPlc region [type=internal, persistence=true, lazyAlloc=false,
      ...  initCfg=40MB, maxCfg=100MB, usedRam=0MB, freeRam=99.98%,
allocRam=100MB, allocTotal=0MB]
    ^--   default region [type=default, persistence=true, lazyAlloc=true,
      ...  initCfg=30MB, maxCfg=30MB, usedRam=0MB, freeRam=99.75%,
allocRam=30MB, allocTotal=0MB]
    ^--   metastoreMemPlc region [type=internal, persistence=true,
lazyAlloc=false,
      ...  initCfg=40MB, maxCfg=100MB, usedRam=0MB, freeRam=99.96%,
allocRam=0MB, allocTotal=0MB]
    ^--   TxLog region [type=internal, persistence=true, lazyAlloc=false,
      ...  initCfg=40MB, maxCfg=100MB, usedRam=0MB, freeRam=100%,
allocRam=100MB, allocTotal=0MB]
    ^-- Ignite persistence [used=0MB]
    ^-- Outbound messages queue [size=0]
    ^-- Public thread pool [active=0, idle=6, qSize=0]
    ^-- System thread pool [active=0, idle=8, qSize=0]
    ^-- Custom executor 0 [active=0, idle=0, qSize=0]
    ^-- Custom executor 1 [active=0, idle=0, qSize=0]


Regards,
-- 
Ilya Kasnacheev


пт, 27 дек. 2019 г. в 15:03, Ilya Kasnacheev <ilya.kasnacheev@gmail.com>:

> Hello!
>
> I have filed a ticket about this improvement, plan to start on coding.
>
> https://issues.apache.org/jira/browse/IGNITE-12505
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> чт, 26 дек. 2019 г. в 12:50, Ilya Kasnacheev <ilya.kasnacheev@gmail.com>:
>
>> Hello!
>>
>> Okay, I will mark default region.
>>
>> We already log information about internal memory regions, so removing it
>> will require a lot of consensus. However, I can clearly map them as system
>> regions when printing.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> чт, 26 дек. 2019 г. в 12:10, Ivan Pavlukhin <vololo100@gmail.com>:
>>
>>> Ilya,
>>>
>>> Indeed the matters can be improved.
>>>
>>> Is not it useful to mark what region is default? Also some doubts
>>> about internal memory regions. It is not obvious that we should print
>>> an information about them for every user. If we need to have some
>>> determinism about offheap memory than I can think about logging
>>> amounts for internal needs of total ones (a sum for all regions).
>>>
>>> вт, 24 дек. 2019 г. в 15:38, Ilya Kasnacheev <ilyak@apache.org>:
>>> >
>>> > Hello!
>>> >
>>> > It came to my attention that we output data regions' configurations
>>> twice
>>> > when starting node, but we never output list of data regions (including
>>> > system, etc) that were actually started.
>>> >
>>> > First we have IgniteConfiguration printed (quiet=false):
>>> > 2019-07-24 02:33:33.918[INFO
>>> ][Thread-139][o.a.i.i.IgniteKernal%GridNodeName
>>> > ] IgniteConfiguration [... dfltDataRegConf=DataRegionConfiguration
>>> [name=
>>> > mem_plc, maxSize=635655159808, initSize=268435456, swapPath=null,
>>> > pageEvictionMode=DISABLED, evictionThreshold=0.9,
>>> emptyPagesPoolSize=100,
>>> > metricsEnabled=true, metricsSubIntervalCount=5,
>>> metricsRateTimeInterval=1000
>>> > , persistenceEnabled=true, checkpointPageBufSize=17179869184],
>>> storagePath=/
>>> > ssd/data, checkpointFreq=30000, lockWaitTime=10000,
>>> checkpointThreads=4,
>>> > checkpointWriteOrder=SEQUENTIAL, walHistSize=2147483647,
>>> walSegments=10,
>>> > walSegmentSize=1073741824, walPath=/ssd/data/wal, walArchivePath=/sas/
>>> > wal_archive, metricsEnabled=false, walMode=LOG_ONLY, walTlbSize=131072,
>>> > walBuffSize=5242880, walFlushFreq=2000, walFsyncDelay=1000,
>>> > walRecordIterBuffSize=67108864, alwaysWriteFullPages=false,
>>> fileIOFactory=
>>> > org.apache.ignite.internal.processors.cache.persistence.file.
>>> > AsyncFileIOFactory@3612c49a, metricsSubIntervalCnt=5,
>>> > metricsRateTimeInterval=60000, walAutoArchiveAfterInactivity=-1,
>>> > writeThrottlingEnabled=false, walCompactionEnabled=true,
>>> walCompactionLevel=
>>> > 1], ...]
>>> >
>>> > Then we have all configured Data Regions printed per IGNITE-8803
>>> > (quiet=true):
>>> >  [11:30:36] Data Regions Configured:
>>> >  [11:30:36]  ^-- plcWithMetrics [initSize=256,0 MiB, maxSize=6,3 GiB,
>>> > persistence=false, lazyMemoryAllocation=true]
>>> >  [11:30:36]  ^-- plcNoMetrics [initSize=256,0 MiB, maxSize=6,3 GiB,
>>> > persistence=false, lazyMemoryAllocation=true]
>>> >
>>> > Then we print number of Data Regions that were initialized as per
>>> > IGNITE-7196, but not regions themselves (quiet=false):
>>> > Configured data regions initialized successfully [total=4]
>>> >
>>> > I propose to keep the first one (IgniteConfiguration), remove the
>>> second
>>> > one (Data Regions Configured), and promote the last one to quiet mode
>>> while
>>> > also outputting the regions themselves like this:
>>> >  [11:30:36] Data Regions Initialized Successfully: 4
>>> >  [11:30:36]  ^-- plcWithMetrics [initSize=256,0 MiB, maxSize=6,3 GiB,
>>> > persistence=true, lazyMemoryAllocation=true]
>>> >  [11:30:36]  ^-- plcNoMetrics [initSize=256,0 MiB, maxSize=6,3 GiB,
>>> > persistence=true, lazyMemoryAllocation=true]
>>> >  [11:30:36]  ^-- sysMemPlc [initSize=40,0 MiB, maxSize=100,0 MiB,
>>> > persistence=true, lazyMemoryAllocation=false]
>>> >  [11:30:36]  ^-- volatileMemPlc [initSize=40,0 MiB, maxSize=100,0 MiB,
>>> > persistence=false, lazyMemoryAllocation=true]
>>> >
>>> > (maybe it will also include information about current usage of region
>>> in
>>> > line with IGNITE-9305's "Metrics for local node"
>>> >
>>> > WDYT?
>>> >
>>> > Regards,
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Ivan Pavlukhin
>>>
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message