lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From William Bell <billnb...@gmail.com>
Subject Re: XFS or EXT4 on Amazon AWS AMIs
Date Thu, 22 Dec 2016 20:23:08 GMT
http://edgystuff.tumblr.com/post/81219256714/tips-to-check-and-improve-your-storage-io

Which specifies:  SERVER-13417
<http://t.umblr.com/redirect?z=https%3A%2F%2Fjira.mongodb.org%2Fbrowse%2FSERVER-13417&t=MDk2ZWYxNDBjZjcxNDkwODQ1YmYyNDZjZjU1ZDU0YTlhODE1NmM0MyxXcEY4cXhmTQ%3D%3D&b=t%3Ax6OZEfmA-wU2yrtBgvccww&p=http%3A%2F%2Fedgystuff.tumblr.com%2Fpost%2F81219256714%2Ftips-to-check-and-improve-your-storage-io&m=1>


You might be right on XFS... We are testing today.

On Thu, Dec 22, 2016 at 1:03 AM, Will Martin <wmartinusa@outlook.com> wrote:

> I'd like to see the MongoDB report(?). ext4fs design specifications
> includes support for large files via allocation placement. MongoDB, the
> last time I checked, does pre-allocation which gives it the performance
> benefit of ext4fs multiple design factors (Block and Inode Allocation
> Policy), but the disadvantage of having to rebuild when file lengths are
> being exceeded; at which time the disk fragmentation may prevent ext4fs
> from getting the allocation pattern it was designed for.
>
> That design feature is going to be unavailable with Solr where ext4fs
> dynamic allocation features are less deterministic. Other performance
> factors on ext4fs, and mutexes (even with guard mutexes) are pretty
> standard patterns. The threaded calls sound like the advantages of the
> allocation pattern.
>
> Still those statements, *based on a dated reading of mine*, may be out of
> date with the MongoDB report factors.
>
> "ext4 recognizes (better than ext3, anyway) that data locality is
> generally a desirably quality of a filesystem"
>
> https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#
> Block_and_Inode_Allocation_Policy
>
> For AWS AMI, is there an r4 instance type? The c3 and m3 are superseded
> with *4 types that have notable improvements in IOPs and don't cost more.
>
> http://howto.unixdev.net/Test_LVM_Trim_Ext4.html   -- not an extended
> performance benchmark, but useful to validate discard/TRIM.
>
> On 12/22/2016 1:32 AM, William Bell wrote:
>
> So what are people recommending for SOLR on AWS on Amazon AMI - ext4 or
> xfs?
>
> I saw an article about MongoDB - saying performance on Amazon was better
> due to a mutex issue on ext4 files and threaded calls.
>
> I have been using ext4 for a long time, but I am moving to r3.* instances
> and TRIM / DISCARD support just appears more supported on XFS.
>
>
>
>
>
>
>


-- 
Bill Bell
billnbell@gmail.com
cell 720-256-8076

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message