quickstep-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From zu...@apache.org
Subject [26/38] incubator-quickstep git commit: Repeated HDFS reads in TextScan, if needed.
Date Mon, 06 Mar 2017 20:18:23 GMT
Repeated HDFS reads in TextScan, if needed.


Project: http://git-wip-us.apache.org/repos/asf/incubator-quickstep/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-quickstep/commit/64bd2c9a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-quickstep/tree/64bd2c9a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-quickstep/diff/64bd2c9a

Branch: refs/heads/reorder-partitioned-hash-join
Commit: 64bd2c9ae5164dd1926366ae58ea8ac0699edf6f
Parents: e37ec54
Author: Zuyu Zhang <zuyuz@apache.org>
Authored: Sat Mar 4 03:18:40 2017 -0800
Committer: Zuyu Zhang <zuyuz@apache.org>
Committed: Sat Mar 4 03:18:40 2017 -0800

----------------------------------------------------------------------
 relational_operators/TextScanOperator.cpp | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/64bd2c9a/relational_operators/TextScanOperator.cpp
----------------------------------------------------------------------
diff --git a/relational_operators/TextScanOperator.cpp b/relational_operators/TextScanOperator.cpp
index 6333813..a133e0c 100644
--- a/relational_operators/TextScanOperator.cpp
+++ b/relational_operators/TextScanOperator.cpp
@@ -241,9 +241,8 @@ void TextScanWorkOrder::execute() {
     }
 
     bytes_read = hdfsRead(hdfs, file_handle, buffer, text_segment_size_);
-    if (bytes_read != text_segment_size_) {
-      hdfsCloseFile(hdfs, file_handle);
-      throw TextScanReadError(filename_);
+    while (bytes_read != text_segment_size_) {
+      bytes_read += hdfsRead(hdfs, file_handle, buffer + bytes_read, text_segment_size_ -
bytes_read);
     }
   }
 #endif  // QUICKSTEP_HAVE_FILE_MANAGER_HDFS


Mime
View raw message