zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Miller, Austin" <Austin.Mil...@morganstanley.com>
Subject RE: <Unverified Sender>RE: exposing lastSend
Date Wed, 16 Jul 2014 18:36:18 GMT
´╗┐Hello,

I'm still hoping to spark some discussion about the last send value.

Given the links below...

https://issues.apache.org/jira/browse/HBASE-1316
http://www.mail-archive.com/user%40zookeeper.apache.org/msg01214.html
https://cwiki.apache.org/confluence/display/ZOOKEEPER/Troubleshooting#Troubleshooting-GCpressure

It seems the issue of thread starvation has been considered previously and is considered sufficiently
important to possibly pursue a JNI solution.

While a JNI wrapper around a C client might increase the likelihood of ZooKeeper client threads
being scheduled in the OS, it does not completely ameliorate the issue.  Resource starvation,
memory, threads, and IO, may still prevent the client from being scheduled even in a C program.

It also adds new sources of failure, more complexity, and increases difficulty of deploying
solutions.

There still seems to be a race condition, in the JNI solution, between session loss notification
and Java if a partition and pause happen simultaneously.

If it is assumed that long pauses happen for some users and the last send value is exposed,
then I maintain it is possible to use that value in ways that increase the chances that a
transaction is not committed when a session has been lost.  Some users may actually want failover
in the case of a really long pause in the JVM, as well.  From the cluster's point of view,
the app rebooted.

GC pauses are not the only things that cause resource starvation, either, and a cause agnostic
measure, like using the last send value to gate writes, is going to provide more confidence
than a JNI wrapper around C.

Austin


--------------------------------------------------------------------------------

NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained
herein are not intended to be, and do not constitute, advice within the meaning of Section
975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received
this communication in error, please destroy all electronic and paper copies and notify the
sender immediately. Mistransmission is not intended to waive confidentiality or privilege.
Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor
electronic communications. This message is subject to terms available at the following link:
http://www.morganstanley.com/disclaimers. If you cannot access these links, please notify
us by reply message and we will send the contents to you. By messaging with Morgan Stanley
you consent to the foregoing.
Mime
View raw message