accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Turner (JIRA)" <>
Subject [jira] [Commented] (ACCUMULO-2217) fall back to reliable compression settings on failure
Date Fri, 24 Jan 2014 19:56:39 GMT


Keith Turner commented on ACCUMULO-2217:

The reason I was asking about gzip failure is because I was trying to think through the implementation.
 Seems like we can do one of the following.

 # If an error occurs writing data, assume its compression problem and fallback to next compression
 # Run a test to see if compression algorithm is available and if not, then fallback

Option 2 seems best.  It seems like the test for gzip would always say "available" for now.
  However as [~busbey] said there may be a condition in the future where the gzip test says
"not available".  So the config "gz,none" still makes sense.

Thinking about the implementation, there is another consideration.  Merging minor compactions
reads the tablets smallest file in addition to writing a file.  If the merging minor compaction
tries to read a snappy compressed file, then it will fail if snappy is not available.   The
purpose of merging minor compaction is to not create more files per tablet than the user specified.
 I can think of the following options, are there any other options?

 # fail to compact
 # create more files than the user specified limit
 # try to find a file we can read
 # try to reassign tablet to tserver than can read file

> fall back to reliable compression settings on failure
> -----------------------------------------------------
>                 Key: ACCUMULO-2217
>                 URL:
>             Project: Accumulo
>          Issue Type: Bug
>    Affects Versions: 1.4.4, 1.5.0
>            Reporter: Adam Fuchs
> Bad compression configuration keeps tables online:
> {code}
> createtable test
> config -t test -s table.file.compression.type=snappy
> flush
> droptable test
> {code}
> The flush results in an unsatisfied link error when Snappy is not properly configured.
This keeps the tablet online. Dropping the table when the tablet is stuck online puts a FATE
operation in the queue, which backs everything up.
> Can we fall back to something that is in the java code base (i.e. gz) in the event of
failures to load other compression code?

This message was sent by Atlassian JIRA

View raw message