trafodion-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hzel...@apache.org
Subject [3/4] incubator-trafodion git commit: [TRAFODION-2615] TMUDF returns wrong results with small numeric inputs
Date Wed, 17 May 2017 16:09:47 GMT
[TRAFODION-2615] TMUDF returns wrong results with small numeric inputs

The problem is that the new TINYINT type sometimes is stored in 1
byte, other times in 2 bytes. The TMUDF code makes the assumption that
we can compute the storage size from the precision. The fix is
in the binder and makes sure the storage size matches the storage size
that the TMUDF code will compute.


Project: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/commit/d4677b51
Tree: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/tree/d4677b51
Diff: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/diff/d4677b51

Branch: refs/heads/master
Commit: d4677b51d8fa4a72ad9c4cf223e90599387b3008
Parents: 2a92ad8
Author: Hans Zeller <hzeller@apache.org>
Authored: Thu May 11 18:39:37 2017 +0000
Committer: Hans Zeller <hzeller@apache.org>
Committed: Thu May 11 18:39:37 2017 +0000

----------------------------------------------------------------------
 core/sql/optimizer/BindRelExpr.cpp | 30 ++++++++++++++++++++++++++++--
 1 file changed, 28 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/d4677b51/core/sql/optimizer/BindRelExpr.cpp
----------------------------------------------------------------------
diff --git a/core/sql/optimizer/BindRelExpr.cpp b/core/sql/optimizer/BindRelExpr.cpp
index 4efe407..5492a2c 100644
--- a/core/sql/optimizer/BindRelExpr.cpp
+++ b/core/sql/optimizer/BindRelExpr.cpp
@@ -17295,10 +17295,36 @@ RelExpr *TableMappingUDF::bindNode(BindWA *bindWA)
     CollIndex numChildCols = childRetDesc->getColumnList()->entries();
     for(CollIndex j=0; j < numChildCols; j++)
     {
-      NAColumn * childCol = new (heap) NAColumn(
+      const NAType &childColType = childRetDesc->getType(j);
+      const NAType *adjustedChildColType = &childColType;
+      NAColumn * childCol = NULL;
+
+      if (childColType.getSimpleTypeName() == "NUMERIC" &&
+          getBinaryStorageSize(childColType.getPrecision()) !=
+          childColType.getNominalSize())
+        {
+          // In the optimizer code, we may choose different storage
+          // sizes for the same type, for example a NUMERIC(1,0)
+          // sometimes has one byte, sometimes 2 bytes of storage. For
+          // examples, see NumericType::synthesizeType() and CQD
+          // TRAF_CREATE_TINYINT_LITERAL.
+
+          // In the TMUDF code, on the other hand, we compute the
+          // storage size from the precision. So, make sure we follow
+          // the TMUDF rules here, when we describe its input table
+          adjustedChildColType = new(heap) SQLNumeric(
+               getBinaryStorageSize(childColType.getPrecision()),
+               childColType.getPrecision(),
+               childColType.getScale(),
+               ((const NumericType &) childColType).isSigned(),
+               childColType.supportsSQLnull(),
+               heap);
+        }
+
+      childCol = new (heap) NAColumn(
         childRetDesc->getColRefNameObj(j).getColName().data(),
         j,
-        childRetDesc->getType(j).newCopy(heap),
+        adjustedChildColType->newCopy(heap),
         heap);
       childColumns.insert(childCol);
 


Mime
View raw message