hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12427) libhdfs++: Prevent Requests from holding dangling pointer to RpcEngine
Date Wed, 11 Oct 2017 18:42:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16200743#comment-16200743
] 

Hadoop QA commented on HDFS-12427:
----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 17m 16s{color} | {color:blue}
Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  0s{color} |
{color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  0s{color} | {color:red}
The patch doesn't appear to include any new or modified tests. Please justify why no new tests
are needed for this patch. Also please list what manual steps were performed to verify this
patch. {color} |
|| || || || {color:brown} HDFS-8707 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m  7s{color}
| {color:green} HDFS-8707 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m 27s{color} |
{color:green} HDFS-8707 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m 28s{color} |
{color:green} HDFS-8707 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  7m 47s{color} |
{color:green} HDFS-8707 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 27s{color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m  6s{color} |
{color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 14m  6s{color} | {color:green}
the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 14m  6s{color} | {color:green}
the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m 55s{color} |
{color:green} the patch passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 13m 55s{color} | {color:green}
the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 13m 55s{color} | {color:green}
the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  6m 56s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m  1s{color}
| {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}105m 27s{color} | {color:red}
hadoop-hdfs-native-client in the patch failed with JDK v1.7.0_151. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 27s{color}
| {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}240m 35s{color} | {color:black}
{color} |
\\
\\
|| Reason || Tests ||
| JDK v1.7.0_151 Failed CTEST tests | test_hdfs_ext_hdfspp_test_shim_static |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:3117e2a |
| JIRA Issue | HDFS-12427 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12891493/HDFS-12427.HDFS-8707.004.patch
|
| Optional Tests |  asflicense  compile  cc  mvnsite  javac  unit  |
| uname | Linux 9a5bc907f465 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | HDFS-8707 / 3f92e63 |
| Default Java | 1.7.0_151 |
| Multi-JDK versions |  /usr/lib/jvm/java-8-oracle:1.8.0_144 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_151
|
| CTEST | https://builds.apache.org/job/PreCommit-HDFS-Build/21646/artifact/patchprocess/patch-hadoop-hdfs-project_hadoop-hdfs-native-client-jdk1.7.0_151-ctest.txt
|
| unit | https://builds.apache.org/job/PreCommit-HDFS-Build/21646/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client-jdk1.7.0_151.txt
|
| JDK v1.7.0_151  Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/21646/testReport/
|
| modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
|
| Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/21646/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> libhdfs++: Prevent Requests from holding dangling pointer to RpcEngine
> ----------------------------------------------------------------------
>
>                 Key: HDFS-12427
>                 URL: https://issues.apache.org/jira/browse/HDFS-12427
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs-client
>            Reporter: James Clampffer
>            Assignee: James Clampffer
>            Priority: Critical
>         Attachments: HDFS-12427.HDFS-8707.000.patch, HDFS-12427.HDFS-8707.001.patch,
HDFS-12427.HDFS-8707.002.patch, HDFS-12427.HDFS-8707.003.patch, HDFS-12427.HDFS-8707.004.patch
>
>
> The lifetime of Request objects is tied to the worker thread(s) in the async event loop.
 In the current code there's nothing that prevents a request from outliving the RpcEngine
(bound to FileSystem) while it's waiting for IO.  If the Request, or a task that makes a new
request, outlives the RpcEngine it attempts to dereference a dangling pointer and either crashes
or continues to run with bad data.
> Proposed fix is to reference count the RpcEngine via shared_ptr so that Requests can
hold a weak_ptr to it.  When a request or RpcConnection attempting to make a request needs
something from the RpcEngine like a call id number it can promote the weak_ptr to a shared_ptr.
 If it's unable to promote because the RpcEngine has been destroyed the Request's handler
can be invoked with an appropriate error message.  A weak_ptr must be used rather than a shared_ptr
to avoid reference cycles.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message