lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Will Martin <wmartin...@outlook.com>
Subject Re: XFS or EXT4 on Amazon AWS AMIs
Date Thu, 22 Dec 2016 08:03:55 GMT
I'd like to see the MongoDB report(?). ext4fs design specifications includes support for large
files via allocation placement. MongoDB, the last time I checked, does pre-allocation which
gives it the performance benefit of ext4fs multiple design factors (Block and Inode Allocation
Policy), but the disadvantage of having to rebuild when file lengths are being exceeded; at
which time the disk fragmentation may prevent ext4fs from getting the allocation pattern it
was designed for.

That design feature is going to be unavailable with Solr where ext4fs dynamic allocation features
are less deterministic. Other performance factors on ext4fs, and mutexes (even with guard
mutexes) are pretty standard patterns. The threaded calls sound like the advantages of the
allocation pattern.

Still those statements, *based on a dated reading of mine*, may be out of date with the MongoDB
report factors.

"ext4 recognizes (better than ext3, anyway) that data locality is generally a desirably quality
of a filesystem"

https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#Block_and_Inode_Allocation_Policy

For AWS AMI, is there an r4 instance type? The c3 and m3 are superseded with *4 types that
have notable improvements in IOPs and don't cost more.

http://howto.unixdev.net/Test_LVM_Trim_Ext4.html   -- not an extended performance benchmark,
but useful to validate discard/TRIM.

On 12/22/2016 1:32 AM, William Bell wrote:

So what are people recommending for SOLR on AWS on Amazon AMI - ext4 or xfs?

I saw an article about MongoDB - saying performance on Amazon was better
due to a mutex issue on ext4 files and threaded calls.

I have been using ext4 for a long time, but I am moving to r3.* instances
and TRIM / DISCARD support just appears more supported on XFS.






Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message