zookeeper-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (Jira)" <j...@apache.org>
Subject [jira] [Commented] (ZOOKEEPER-3496) Transaction larger than jute.maxbuffer makes ZooKeeper unavailable
Date Tue, 24 Sep 2019 17:46:00 GMT

    [ https://issues.apache.org/jira/browse/ZOOKEEPER-3496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937045#comment-16937045
] 

Hadoop QA commented on ZOOKEEPER-3496:
--------------------------------------

+1 overall.  GitHub Pull Request  Build
      

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 3 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs (version ) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit
warnings.

    +1 core tests.  The patch passed core unit tests.

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/4152//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/4152//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/4152//console

This message is automatically generated.

> Transaction larger than jute.maxbuffer makes ZooKeeper unavailable
> ------------------------------------------------------------------
>
>                 Key: ZOOKEEPER-3496
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3496
>             Project: ZooKeeper
>          Issue Type: Bug
>    Affects Versions: 3.5.5, 3.4.14
>            Reporter: Mohammad Arshad
>            Assignee: Mohammad Arshad
>            Priority: Critical
>              Labels: pull-request-available
>             Fix For: 3.6.0, 3.4.15, 3.5.7
>
>         Attachments: ZOOKEEPER-3496-001.patch
>
>          Time Spent: 4h 20m
>  Remaining Estimate: 0h
>
> *Problem:*
> ZooKeeper server fails to start, logs following error
> {code:java}
> Exception in thread "main" java.io.IOException: Unreasonable length = 1001025
>          at org.apache.jute.BinaryInputArchive.checkLength(BinaryInputArchive.java:127)
>          at org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:92)
> {code}
> This indicates that one of the transactions size is more than the configured 	jute.maxbuffer
values. But how transaction more than jute.maxbuffer size is allowed to write? 
> *Analysis:*
> At ZooKeeper server jute.maxbuffer specifies the maximum size of a transaction. By default
it is 1 MB at the server
> jute.maxbuffer is used for following:
> # Size sanity check of incoming request. Incoming requests size must not be more than
jute.maxbuffer
> # Size sanity check of the transaction while reading from transaction or snapshot file.
Transaction size must not be more than jute.maxbuffer+1024
> # Size sanity check of transaction while reading data from the leader. Transaction size
must not be more than jute.maxbuffer+1024
> Request size sanity check is done in the beginning of a request processing but later
request processing adds additional information into request then writes to transaction file.
This additional information size is not considered in sanity check. This is how transaction
larger than jute.maxbuffer are accepted into ZooKeeper.  
> If this additional information size is less than 1024 Bytes then it is OK as ZooKeeper
already takes care of it. 
> But if this additional information size is more than 1024 bytes it allows the request,
But while reading from transaction/snapshot file and while reading from leader it fails and
make the ZooKeeper service unavailable  
> +Example:+
> Suppose incoming request size is 1000000 Bytes
> Configured jute.maxbuffer is 1000000
> After processing the request ZooKeeper server adds 1025 more bytes
> In this case, request will be processed successfully, and 1000000+1025 bytes will be
written to transaction file
> But while reading from the transaction log 1000000+1025 bytes cannot be read as max allowed
length is 1000000(effectively 1000000+1024).
> *Solutions:*
> If incoming request size sanity check is done after populating all additional information
then this problem is solved. But doing sanity check in the later stage of request processing
will defeat the purpose of sanity check itself. So this we can not do
> Currently additional information size is constant 1024 Bytes [Code Reference|https://github.com/apache/zookeeper/blob/branch-3.5/zookeeper-jute/src/main/java/org/apache/jute/BinaryInputArchive.java#L126].
We should increase this value and make it more reasonable. I propose to make this additional
information size to same as the jute.maxbuffer. Also make additional information size configurable.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message