hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5977) Fix or suppress native-task gcc warnings
Date Thu, 14 Aug 2014 05:14:12 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096582#comment-14096582

Todd Lipcon commented on MAPREDUCE-5977:

Have you done any simple before/after performance check of the impact of -fno-strict-aliasing?
I generally support the flag, but would be good to have some assurance that it's not hurting
any hot path. If it is we can manually add 'restrict' keywords where necessary. Another option
is to fix the warnings where they show up by changing type punning to memcpy (which generally
gets optimized to simple loads/stores if it has a fixed length)

@@ -196,7 +194,6 @@ void MeasureSingleFileLz4(const string & path, CompressResult &
total, size_t bl
     startTime = t.now();
     for (int i = 0; i < times; i++) {
 //      memset(dest, 0, currentblocksize+8);
-      int osize = LZ4_uncompress(outputBuffer, dest, currentblocksize);
 //      printf("%016llx blocksize: %lu\n", bswap64(*(uint64_t*)(dest+currentblocksize)),
Removing this call seems problematic, given this is purporting to benchmark LZO decompression
and you just removed the call to decompress :)

Maybe you want to either cast the call to void (to get rid of the warning), or perform some
kind of assertion on 'osize' so it's not unused?


   while (ir->nextPartition()) {
-    const char * key, *value;
+    const char * key;
     uint32_t keyLen, valueLen;
     while (NULL != (key = ir->nextKey(keyLen))) {
-      value = ir->value(valueLen);
+      ir->value(valueLen);

Again, for a benchmark, this seem suspicious. The compiler might well elide the ir->value()
call if it figures out there is no side effect. One trick I usually use here is to do something

   int sum = 0;
   while (ir->nextPartition()) {
     const char * key, *value;
     uint32_t keyLen, valueLen;
     while (NULL != (key = ir->nextKey(keyLen))) {
       value = ir->value(valueLen);
       sum += value[0];
   // Use the result so that the value() calls don't get optimized out.
   ASSERT_NE(0xdeadbeef, sum);

(or if you don't like assert, another option is to log the result -- anything so that gcc
knows that it can't elide the actual memory accesses in the loop)

> Fix or suppress native-task gcc warnings
> ----------------------------------------
>                 Key: MAPREDUCE-5977
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5977
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: task
>            Reporter: Todd Lipcon
>            Assignee: Manu Zhang
>         Attachments: gcc_compile.log, mapreduce-5977-v2.txt, mapreduce-5977.txt
> Currently, building the native task code on gcc 4.8 has a fair number of warnings. We
should fix or suppress them so that new warnings are easier to see.

This message was sent by Atlassian JIRA

View raw message