hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anatoli Shein (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-10595) libhdfs++: Client Name Protobuf Error
Date Tue, 05 Jul 2016 17:47:11 GMT
Anatoli Shein created HDFS-10595:

             Summary: libhdfs++: Client Name Protobuf Error
                 Key: HDFS-10595
                 URL: https://issues.apache.org/jira/browse/HDFS-10595
             Project: Hadoop HDFS
          Issue Type: Sub-task
            Reporter: Anatoli Shein

When running a cat tool (/hadoop-hdfs-native-client/src/main/native/libhdfspp/examples/cat/c/cat.c)
I get the following error:

[libprotobuf ERROR google/protobuf/wire_format.cc:1053] String field contains invalid UTF-8
data when serializing a protocol buffer. Use the 'bytes' type if you intend to send raw bytes.

However it executes correctly. Looks like this error happens when trying to serialize Client
name in ClientOperationHeaderProto::SerializeWithCachedSizes (/hadoop-hdfs-native-client/target/main/native/libhdfspp/lib/proto/datatransfer.pb.cc)

Possibly the problem is caused by generating client name as a UUID in GetRandomClientName

In Java client it looks like there are two different unique client identifiers: ClientName
and ClientId:

Client name is generated as:
clientName = "DFSClient_" + dfsClientConf.getTaskId() + "_" + ThreadLocalRandom.current().nextInt()
 + "_" + Thread.currentThread().getId(); (/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java)

ClientId is generated as a UUID in (/hadoop-common/src/main/java/org/apache/hadoop/ipc/ClientId.java)

In libhdfs++ we need to possibly also have two unique client identifiers, or fix the current
client name to work without protobuf warnings/errors.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

View raw message