kudu-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Brannan <paul.bran...@thesystech.com>
Subject File descriptor limit for WAL
Date Thu, 16 Feb 2017 16:42:12 GMT
I wrote a quick script today to see how kudu behaves if I create many
tables.  After creating 334 tables, I started getting timeouts.  I see this
in the master log file:

W0216 11:37:48.961221 49810 catalog_manager.cc:2490] CreateTablet RPC for
tablet 9b259d5c5ff74f04820240f2159bc1a0 on TS
faaf4e9b6e5945d7a14953c4cc34f164 (telx-sb-dev2:7050) failed: IO error:
Couldn't create tablet metadata: Failed to write tablet metadata
9b259d5c5ff74f04820240f2159bc1a0: Call to mkstemp() failed on name template
/var/lib/kudu/tserver/tablet-meta/9b259d5c5ff74f04820240f2159bc1a0.tmp.XXXXXX:
Too many open files (error 24)

I decreased block_manager_max_open_files, but still got the same result.
Lsof shows that the open files are for the WAL:

kudu-tser 49648 kudu 1021u   REG        8,5 67108864   16385457
/var/lib/kudu/tserver/wals/62b73d1b7f7a4e61a0a30a551e66230b/wal-000000001
kudu-tser 49648 kudu 1022r   REG        8,5 67108864   16385457
/var/lib/kudu/tserver/wals/62b73d1b7f7a4e61a0a30a551e66230b/wal-000000001
kudu-tser 49648 kudu 1023u   REG        8,5 24000000   16385458
/var/lib/kudu/tserver/wals/62b73d1b7f7a4e61a0a30a551e66230b/index.000000000

The files do not get closed until the tables are deleted, even though no
running process has any of those tables open.

Is there a setting that will reduce the number of WAL files that get
created or held open at any given point in time?

Mime
View raw message