accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [accumulo] keith-turner commented on issue #1231: Saw partial bulk import when master killed
Date Tue, 02 Jul 2019 15:46:01 GMT
keith-turner commented on issue #1231: Saw partial bulk import when master killed
URL: https://github.com/apache/accumulo/issues/1231#issuecomment-507734528
 
 
   I am still looking into this and thinking about the best course of action.  Below is some
information I have discovered while looking into this.
   
    * Hadoop has a config property `fs.automatic.close` that disables its shutdown hook if
set to `false`.  It defaults to `true`.
    * Accumulo's tablet code tries to detect io excetptions caused by shutdown during scanning
a tablet in order to prevent sending spurious error message to the client.  See [Tablet.java
line 593](https://github.com/apache/accumulo/blob/f81516e18ddd62916fa80784e45821917534f5b9/server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/Tablet.java#L593)
and [Tablet.java line 805](https://github.com/apache/accumulo/blob/f81516e18ddd62916fa80784e45821917534f5b9/server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/Tablet.java#L805).
    * The file move operation is not idempotent.  It needs to handle the case of the file
already being moved.  Need to see if old bulk import code has this issue.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message