aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David McLaughlin <dmclaugh...@apache.org>
Subject Future of storage in Aurora
Date Thu, 30 Mar 2017 17:16:58 GMT
Hi all,

I'd like to start a discussion around storage in Aurora.

I think one of the biggest mistakes we made in migrating our storage to H2
was deleting the memory stores as we moved. We made a pretty big bet that
we could eventually make H2/relational databases work. I don't think that
bet has paid off and that we need to revisit the direction we're taking.

My belief is that the current H2/MyBatis approach is untenable for large
production clusters, at least without changing our current single-master
architecture. At Twitter we are already having to fight to keep GC
manageable even without DbTaskStore enabled, so I don't see a path forward
where we could eventually enable that. So far experiments with H2 off-heap
storage have provided marginal (if any) gains.

Would anyone object to restoring the in-memory stores and creating new
implementations for the missing ones (UpdateStore)? I'd even go further and
propose that we consider in-memory H2 and MyBatis a failed experiment and
we drop that storage layer completely.

Cheers,
David

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message