incubator-blur-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravikumar Govindarajan <ravikumar.govindara...@gmail.com>
Subject Transaction Log in HDFS
Date Thu, 15 Dec 2016 12:22:01 GMT
I remember seeing Blur writing transaction logs (WAL) in HDFS but is
removed as of today...

Was re-implementing WAL-log for a particular use-case in Blur. Would be
great if community experienced any issues during WAL-writes in HDFS & could
share them here...

I was looking at corner cases where WAL files could go corrupt. I guess
there should be many scenarios but just off the top of my head

   - Program crashing on OOM/Seg-fault, with open WAL streams to HDFS
   - Name-Node, Data-Node differing on length of written block-data
   - Data-Node(s) failing to ack sync calls correctly

Any insights is much appreciated

--
Ravi

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message