giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koontz" <ekoo...@hiro-tan.org>
Subject Re: Review Request: GIRAPH-211: Add secure authentication to Netty IPC
Date Wed, 10 Oct 2012 17:50:01 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6609/
-----------------------------------------------------------

(Updated Oct. 10, 2012, 5:49 p.m.)


Review request for giraph.


Changes
-------

updated description


Description (updated)
-------

Passes 

mvn -Phadoop_non_secure clean verify && mvn -Phadoop_0.20.203 clean verify &&
mvn -Phadoop_1.0 clean verify && mvn -Phadoop0.23 clean verify && mvn -Phadoop_2.0.0
clean verify && mvn -Phadoop_2.0.1 clean verify && mvn -Phadoop_2.0.2 clean
verify

How to use:

use 

  -Dgiraph.useNetty=true -Dgiraph.authenticate=true 

on your job's command line to use this feature.

How to compile and test (using Hadoop 2.0.0 for example):

mvn -Phadoop_2.0.0 clean test

$HADOOP_RUNTIME/bin/hadoop jar $GIRAPH_DIR/target/giraph-0.2-SNAPSHOT-for-hadoop-2.0.0-SNAPSHOT-jar-with-dependencies.jar
org.apache.giraph.benchmark.PageRankBenchmark -Dgiraph.useNetty=true -Dgiraph.authenticate=true
-e 100 -s 100 -v -V 100 -w 6

Look for something similar to the following in the workers' syslogs:

2012-10-10 09:24:49,924 DEBUG [New I/O client worker #1-2] org.apache.giraph.comm.netty.handler.SaslClientHandler:
decode: Got a response of type SASL_TOKEN_MESSAGE_REQUEST from server:Eugenes-MacBook-Pro.local/172.16.175.1:30010
2012-10-10 09:24:49,924 DEBUG [New I/O client worker #1-2] org.apache.giraph.comm.netty.handler.SaslClientHandler:
handleUpstream: Responding to server's token of length: 40
2012-10-10 09:24:49,924 DEBUG [New I/O client worker #1-2] org.apache.giraph.comm.netty.handler.SaslClientHandler:
handleUpstream: Response to server is null: authentication should now be complete.
2012-10-10 09:24:49,925 DEBUG [New I/O client worker #1-2] org.apache.giraph.comm.netty.ByteCounter:
handleUpstream: 0 buffer size = 17, total bytes = 207
2012-10-10 09:24:49,925 DEBUG [New I/O client worker #1-2] org.apache.giraph.comm.netty.handler.SaslClientHandler:
decode: Got a response of type SASL_COMPLETE_REQUEST from server:Eugenes-MacBook-Pro.local/172.16.175.1:30010
2012-10-10 09:24:49,925 DEBUG [New I/O client worker #1-2] org.apache.giraph.comm.netty.handler.SaslClientHandler:
handleUpstream: Server has sent us the SaslComplete message. Allowing normal work to proceed.
2012-10-10 09:24:49,926 DEBUG [main] org.apache.giraph.comm.netty.NettyClient: authenticateOnChannel:
Authentication on channel: [id: 0x32486cdd, /172.16.175.1:54768 => Eugenes-MacBook-Pro.local/172.16.175.1:30010]
has completed successfully.

..

..

2012-10-10 09:24:56,844 DEBUG [New I/O server worker #1-6] org.apache.giraph.comm.netty.handler.AuthorizeServerHandler:
messageReceived: Got class org.apache.giraph.comm.requests.SendWorkerMessagesRequest
2012-10-10 09:24:56,844 DEBUG [New I/O server worker #1-6] org.apache.giraph.comm.netty.handler.AuthorizeServerHandler:
messageReceived: authenticated client: FmpvYl8xMzQ5NzE1MTIxNDc1XzAyMDU= is authorized to do
request on server.
2012-10-10 09:24:56,844 INFO [main] org.apache.giraph.comm.netty.NettyClient: waitAllRequests:
Finished all requests. MBytes/sec sent = 0.3824, MBytes/sec received = 0.0124, MBytesSent
= 0.0019, MBytesReceived = 0.0001, ave sent req MBytes = 0.0004, ave received req MBytes =
0, secs waited = 0.0040
2012-10-10 09:24:56,844 DEBUG [New I/O server worker #1-6] org.apache.giraph.comm.netty.handler.RequestServerHandler:
messageReceived: Processing client 3, requestId 101, SEND_WORKER_MESSAGES_REQUEST took 30000
ns


This addresses bug GIRAPH-211.
    https://issues.apache.org/jira/browse/GIRAPH-211


Diffs
-----

  pom.xml 51944fa 
  src/main/java/org/apache/giraph/GiraphConfiguration.java 797ba6e 
  src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java 1849504 
  src/main/java/org/apache/giraph/comm/CommunicationsInterface.java d60f512 
  src/main/java/org/apache/giraph/comm/SecureRPCCommunications.java af9720e 
  src/main/java/org/apache/giraph/comm/WorkerClient.java c3ec4fe 
  src/main/java/org/apache/giraph/comm/netty/NettyClient.java 93f5671 
  src/main/java/org/apache/giraph/comm/netty/NettyServer.java 886d68b 
  src/main/java/org/apache/giraph/comm/netty/NettyWorkerClient.java 5b71401 
  src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientServer.java 4710f3a 
  src/main/java/org/apache/giraph/comm/netty/SaslNettyClient.java PRE-CREATION 
  src/main/java/org/apache/giraph/comm/netty/SaslNettyServer.java PRE-CREATION 
  src/main/java/org/apache/giraph/comm/netty/handler/RequestEncoder.java 3174447 
  src/main/java/org/apache/giraph/comm/netty/handler/SaslClientHandler.java PRE-CREATION 
  src/main/java/org/apache/giraph/comm/netty/handler/SaslServerHandler.java PRE-CREATION 
  src/main/java/org/apache/giraph/comm/requests/RequestType.java b412aa2 
  src/main/java/org/apache/giraph/comm/requests/SaslCompleteRequest.java PRE-CREATION 
  src/main/java/org/apache/giraph/comm/requests/SaslTokenMessageRequest.java PRE-CREATION

  src/main/java/org/apache/giraph/graph/BspServiceWorker.java bb498ce 
  src/test/java/org/apache/giraph/comm/SaslConnectionTest.java PRE-CREATION 

Diff: https://reviews.apache.org/r/6609/diff/


Testing (updated)
-------

Tested as above.


Thanks,

Eugene Koontz


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message