accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis <de...@camfex.cz>
Subject Re: Values go to a wrong table during recovery.
Date Fri, 20 Feb 2015 22:43:23 GMT
> So just to confirm, you have seen this issue appear while walogs are enabled for all tables,
yes?

Yes.

> Any chance you can share your code that writes across several BatchWriters?

The writer is:

// way faster than Connector.createMultiTableBatchWriter
class MyMultiTableBatchWriter(maxMemory: Long, maxLatency: Long,
maxWriteThreads: Int)(implicit val conn: Connector) {
  private[this] val writers = new
scala.collection.mutable.HashMap[String, BatchWriter]
  def getBatchWriter(tableName: String): BatchWriter = synchronized {
    writers.getOrElseUpdate(tableName,
conn.createBatchWriter(tableName, maxMemory, maxLatency,
maxWriteThreads))
  }
  def flush() = writers.foreach(_._2.flush())
  def close() = writers.foreach(_._2.close())
}

But the full code which creates/uses the indexes is more than 1Mb of
scala code...

On 2/20/15, Sean Busbey <busbey@cloudera.com> wrote:
> On Fri, Feb 20, 2015 at 2:33 PM, Denis <denis@camfex.cz> wrote:
>
>> > Did you have walogs laying around when you upgraded?
>>
>> In 1.4-cluster (and first time in 1.6-cluster) I had walogs enabled
>> for data tables and disabled for index tables.
>>
>> There was a bug in 1.4, if a tablet had empty walog there were some
>> startup issues (tablet remains offline or something like this), and it
>> happened often with index tables (hmm, the same tables I have this
>> problem).
>>
>> So, in 1.4-cluster I disabled walog and ran full reindex periodically.
>> After running 1.6-cluster some time I enabled walogs for all tables as
>> the new cluster have less reliable hardware, which reboots from time
>> to time.
>>
>>
> So just to confirm, you have seen this issue appear while walogs are
> enabled for all tables, yes?
>
> Any chance you can share your code that writes across several BatchWriters?
>
> --
> Sean
>

Mime
View raw message