zeppelin-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From cornead...@apache.org
Subject incubator-zeppelin git commit: ZEPPELIN-830 Improve table display to handle large data
Date Mon, 16 May 2016 15:10:39 GMT
Repository: incubator-zeppelin
Updated Branches:
  refs/heads/master 8350e7802 -> 4eccfcd52


ZEPPELIN-830 Improve table display to handle large data

### What is this PR for?
This is an improvement for table display. By using [Handsontable](https://github.com/handsontable/handsontable)
we can load and display more data in paragraphs. Tested using sample tabular data consisting
10000x80 cells

### What type of PR is it?
Improvement

### Todos
* [x] - Decimal formatting
* [x] - Rendering without angular/html directives
* [x] - UI fixes
* [x] - License doc update

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-830

### How should this be tested?
TODO : test cases to follow

### Screenshots (if appropriate)
![apr 26 2016 17 38](https://cloud.githubusercontent.com/assets/2031306/14817566/c034ab28-0bd5-11e6-8b7a-2216ceca8153.gif)

### Questions:
* Does the licenses files need update? Yes
* Is there breaking changes for older versions? No
* Does this needs documentation? Maybe

Author: Renjith Kamath <renjith.kamath@gmail.com>

Closes #858 from r-kamath/ZEPPELIN-830 and squashes the following commits:

f13ba91 [Renjith Kamath] Merge branch 'master' of https://github.com/apache/incubator-zeppelin
into ZEPPELIN-830
dd0ced7 [Renjith Kamath] ZEPPELIN-830 fix wrong variable name
4fbb22b [Renjith Kamath] Merge branch 'master' of https://github.com/apache/incubator-zeppelin
into ZEPPELIN-830
2e422ea [Renjith Kamath] ZEPPELIN-830 update licence doc
adb8380 [Renjith Kamath] ZEPPELIN-830 remove unused floatThead dep
a96c94a [Renjith Kamath] ZEPPELIN-830 fix rendering without angular/html directives
b365e7f [Renjith Kamath] ZEPPELIN-830 fix test
b87245a [Renjith Kamath] ZEPPELIN-830 fix jshint error
de6d134 [Renjith Kamath] ZEPPELIN-830 License update
7bb508e [Renjith Kamath] ZEPPELIN-830 fix number formatting for table display
4e6beb8 [Renjith Kamath] fix selenium test failure
c833f6e [Renjith Kamath] ZEPPELIN-830 Improve table display to handle large data


Project: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/commit/4eccfcd5
Tree: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/tree/4eccfcd5
Diff: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/diff/4eccfcd5

Branch: refs/heads/master
Commit: 4eccfcd5259a6e4f1df3a66927950c6ec30a37af
Parents: 8350e78
Author: Renjith Kamath <renjith.kamath@gmail.com>
Authored: Tue May 10 10:01:37 2016 +0530
Committer: Damien CORNEAU <corneadoug@gmail.com>
Committed: Mon May 16 08:10:24 2016 -0700

----------------------------------------------------------------------
 zeppelin-distribution/src/bin_license/LICENSE   |   4 +
 .../zeppelin/integration/SparkParagraphIT.java  |   5 +-
 zeppelin-web/.jshintrc                          |   3 +-
 zeppelin-web/bower.json                         |   4 +-
 .../notebook/paragraph/paragraph.controller.js  | 134 +++++--------------
 .../src/app/notebook/paragraph/paragraph.css    |  23 ++++
 zeppelin-web/src/index.html                     |  18 +--
 zeppelin-web/test/karma.conf.js                 |  14 +-
 8 files changed, 75 insertions(+), 130 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/4eccfcd5/zeppelin-distribution/src/bin_license/LICENSE
----------------------------------------------------------------------
diff --git a/zeppelin-distribution/src/bin_license/LICENSE b/zeppelin-distribution/src/bin_license/LICENSE
index 09f98ad..e18cbef 100644
--- a/zeppelin-distribution/src/bin_license/LICENSE
+++ b/zeppelin-distribution/src/bin_license/LICENSE
@@ -132,6 +132,10 @@ The text of each license is also included at licenses/LICENSE-[project]-[version
     (The MIT License) lodash v3.9.3 (https://lodash.com/) - https://github.com/lodash/lodash/blob/3.9.3/LICENSE.txt
     (The MIT License) angular-filter v0.5.4 (https://github.com/a8m/angular-filter) - https://github.com/a8m/angular-filter/blob/v0.5.4/license.md
     (The MIT License) ngToast v1.5.5 (http://tamerayd.in/ngToast/) - http://tameraydin.mit-license.org/
+    (The MIT License) Handsontable v0.24.2 (https://github.com/handsontable/handsontable)
- https://github.com/handsontable/handsontable/blob/master/LICENSE
+    (The MIT License) Zeroclipboard v2.2.0 (https://github.com/zeroclipboard/zeroclipboard)
- https://github.com/zeroclipboard/zeroclipboard/blob/v2.2.0/LICENSE
+    (The MIT License) Moment v2.9.0 (https://github.com/moment/moment) - https://github.com/moment/moment/blob/2.9.0/LICENSE
+    (The MIT License) Pikaday v1.3.2 (https://github.com/dbushell/Pikaday) - https://github.com/dbushell/Pikaday/blob/1.3.2/LICENSE
     (The MIT License) slf4j v1.7.10 (org.slf4j:slf4j-api:jar:1.7.10 - http://www.slf4j.org)
- http://www.slf4j.org/license.html
     (The MIT License) slf4j-log4j12 v1.7.10 (org.slf4j:slf4j-log4j12:jar:1.7.10 - http://www.slf4j.org)
- http://www.slf4j.org/license.html
     (The MIT License) bcprov-jdk15on v1.51 (org.bouncycastle:bcprov-jdk15on:jar:1.51 - http://www.bouncycastle.org/java.html)
- http://www.bouncycastle.org/licence.html

http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/4eccfcd5/zeppelin-server/src/test/java/org/apache/zeppelin/integration/SparkParagraphIT.java
----------------------------------------------------------------------
diff --git a/zeppelin-server/src/test/java/org/apache/zeppelin/integration/SparkParagraphIT.java
b/zeppelin-server/src/test/java/org/apache/zeppelin/integration/SparkParagraphIT.java
index eb09539..1eadd0e 100644
--- a/zeppelin-server/src/test/java/org/apache/zeppelin/integration/SparkParagraphIT.java
+++ b/zeppelin-server/src/test/java/org/apache/zeppelin/integration/SparkParagraphIT.java
@@ -167,9 +167,8 @@ public class SparkParagraphIT extends AbstractZeppelinIT {
       WebElement paragraph1Result = driver.findElement(By.xpath(
           getParagraphXPath(1) + "//div[@class=\"tableDisplay\"]"));
       collector.checkThat("Paragraph from SparkParagraphIT of testSqlSpark result: ",
-          paragraph1Result.getText().toString(), CoreMatchers.equalTo("age job marital education
balance\n" +
-              "30 unemployed married primary 1,787")
-      );
+          paragraph1Result.getText().toString(), CoreMatchers.equalTo("age\njob\nmarital\neducation\nbalance\n30"
+
+                      " unemployed married primary 1,787\nage\njob\nmarital\neducation\nbalance"));
     } catch (Exception e) {
       handleException("Exception in SparkParagraphIT while testSqlSpark", e);
     }

http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/4eccfcd5/zeppelin-web/.jshintrc
----------------------------------------------------------------------
diff --git a/zeppelin-web/.jshintrc b/zeppelin-web/.jshintrc
index d15bbb9..bdcd213 100644
--- a/zeppelin-web/.jshintrc
+++ b/zeppelin-web/.jshintrc
@@ -31,6 +31,7 @@
     "nv": false,
     "ace": false,
     "d3": false,
-    "BootstrapDialog": false
+    "BootstrapDialog": false,
+    "Handsontable": false
   }
 }

http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/4eccfcd5/zeppelin-web/bower.json
----------------------------------------------------------------------
diff --git a/zeppelin-web/bower.json b/zeppelin-web/bower.json
index 5f4f726..2000281 100644
--- a/zeppelin-web/bower.json
+++ b/zeppelin-web/bower.json
@@ -30,9 +30,7 @@
     "ngtoast": "~2.0.0",
     "ng-focus-if": "~1.0.2",
     "bootstrap3-dialog": "bootstrap-dialog#~1.34.7",
-    "floatThead": "~1.3.2",
-    "datatables.net-bs": "~1.10.11",
-    "datatables.net-buttons-bs": "~1.1.2"
+    "handsontable": "~0.24.2"
   },
   "devDependencies": {
     "angular-mocks": "1.5.0"

http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/4eccfcd5/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
----------------------------------------------------------------------
diff --git a/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js b/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
index b19003f..a886969 100644
--- a/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
+++ b/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
@@ -1219,110 +1219,42 @@ angular.module('zeppelinWebApp')
   };
 
   var setTable = function(type, data, refresh) {
-    var getTableContentFormat = function(d) {
-      if (isNaN(d)) {
-        if (d.length>'%html'.length && '%html ' === d.substring(0, '%html '.length))
{
-          return 'html';
-        } else {
-          return '';
-        }
-      } else {
-        return '';
-      }
-    };
-
-    var formatTableContent = function(d) {
-      if (isNaN(d)) {
-        var f = getTableContentFormat(d);
-        if (f !== '') {
-          return d.substring(f.length+2);
-        } else {
-          return d;
-        }
-      } else {
-        var dStr = d.toString();
-        var splitted = dStr.split('.');
-        var formatted = splitted[0].replace(/(\d)(?=(\d{3})+(?!\d))/g, '$1,');
-        if (splitted.length>1) {
-          formatted+= '.'+splitted[1];
-        }
-        return formatted;
-      }
-    };
-
-
     var renderTable = function() {
-      var html = '';
-      html += '<table class="table table-hover table-condensed">';
-      html += '  <thead>';
-      html += '    <tr style="background-color: #F6F6F6; font-weight: bold;">';
-      for (var titleIndex in $scope.paragraph.result.columnNames) {
-        html += '<th>'+$scope.paragraph.result.columnNames[titleIndex].name+'</th>';
-      }
-      html += '    </tr>';
-      html += '  </thead>';
-      html += '  <tbody>';
-      for (var r in $scope.paragraph.result.msgTable) {
-        var row = $scope.paragraph.result.msgTable[r];
-        html += '    <tr>';
-        for (var index in row) {
-          var v = row[index].value;
-          if (getTableContentFormat(v) !== 'html') {
-            v = v.replace(/[\u00A0-\u9999<>\&]/gim, function(i) {
-              return '&#'+i.charCodeAt(0)+';';
-            });
-          }
-        html += '      <td>'+formatTableContent(v)+'</td>';
+      var height = $scope.paragraph.config.graph.height;
+      angular.element('#p' + $scope.paragraph.id + '_table').css('height', height);
+      var resultRows = $scope.paragraph.result.rows;
+      var columnNames = _.pluck($scope.paragraph.result.columnNames, 'name');
+      var container = document.getElementById('p' + $scope.paragraph.id + '_table');
+
+      var handsontable = new Handsontable(container, {
+        data: resultRows,
+        colHeaders: columnNames,
+        rowHeaders: false,
+        stretchH: 'all',
+        sortIndicator: true,
+        columnSorting: true,
+        contextMenu: false,
+        manualColumnResize: true,
+        manualRowResize: true,
+        editor: false,
+        fillHandle: false,
+        disableVisualSelection: true,
+        cells: function (row, col, prop) {
+          var cellProperties = {};
+            cellProperties.renderer = function(instance, td, row, col, prop, value, cellProperties)
{
+              Handsontable.NumericCell.renderer.apply(this, arguments);
+              if (!isNaN(value)) {
+                cellProperties.type = 'numeric';
+                cellProperties.format = '0,0';
+                cellProperties.editor = false;
+                td.style.textAlign = 'left';
+              } else if (value.length > '%html'.length && '%html ' === value.substring(0,
'%html '.length)) {
+                td.innerHTML = value.substring('%html'.length);
+              }
+            };
+          return cellProperties;
         }
-        html += '    </tr>';
-      }
-      html += '  </tbody>';
-      html += '</table>';
-
-      var tableDomEl = angular.element('#p' + $scope.paragraph.id + '_table');
-      tableDomEl.html(html);
-      var oTable = tableDomEl.children(1).DataTable({
-        paging:       false,
-        info:         false,
-        autoWidth:    false,
-        lengthChange: false,
-        searching: false,
-        dom: '<>'
       });
-
-      if ($scope.paragraph.result.msgTable.length > 10000) {
-        tableDomEl.css({
-          'overflow': 'scroll',
-          'height': $scope.paragraph.config.graph.height
-        });
-      } else {
-
-        var dataTable = angular.element('#p' + $scope.paragraph.id + '_table .table');
-        dataTable.floatThead({
-          scrollContainer: function(dataTable) {
-            return tableDomEl;
-          }
-        });
-
-        dataTable.on('remove', function () {
-          dataTable.floatThead('destroy');
-        });
-
-        tableDomEl.css({
-          'position': 'relative',
-          'height': '100%'
-        });
-        tableDomEl.perfectScrollbar('destroy')
-                  .perfectScrollbar({minScrollbarLength: 20});
-
-        angular.element('.ps-scrollbar-y-rail').css('z-index', '1002');
-
-        // set table height
-        var psHeight = $scope.paragraph.config.graph.height;
-        tableDomEl.css('height', psHeight);
-        tableDomEl.perfectScrollbar('update');
-      }
-
     };
 
     var retryRenderer = function() {

http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/4eccfcd5/zeppelin-web/src/app/notebook/paragraph/paragraph.css
----------------------------------------------------------------------
diff --git a/zeppelin-web/src/app/notebook/paragraph/paragraph.css b/zeppelin-web/src/app/notebook/paragraph/paragraph.css
index 60f3d7f..f0c650f 100644
--- a/zeppelin-web/src/app/notebook/paragraph/paragraph.css
+++ b/zeppelin-web/src/app/notebook/paragraph/paragraph.css
@@ -397,6 +397,29 @@ table.dataTable.table-condensed .sorting_desc:after {
 }
 
 /*
+  Handsontable
+*/
+
+.handsontable th {
+  font-weight: bold;
+}
+
+.handsontable th, .handsontable td {
+  border-right: 0px;
+  border-left: 0px !important;
+  padding: 4px;
+}
+
+.handsontable tr:first-child th {
+  text-align: left;
+  border-top: 0px;
+  padding: 4px 0px 0px 0px;
+  border-left: 0px;
+  border-right: 0px;
+  border-bottom: 2px solid #CCC;
+}
+
+/*
   Pivot CSS
 */
 

http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/4eccfcd5/zeppelin-web/src/index.html
----------------------------------------------------------------------
diff --git a/zeppelin-web/src/index.html b/zeppelin-web/src/index.html
index 207320e..a634fbf 100644
--- a/zeppelin-web/src/index.html
+++ b/zeppelin-web/src/index.html
@@ -44,8 +44,8 @@ limitations under the License.
     <link rel="stylesheet" href="bower_components/highlightjs/styles/github.css" />
     <link rel="stylesheet" href="bower_components/ngtoast/dist/ngToast.css" />
     <link rel="stylesheet" href="bower_components/bootstrap3-dialog/dist/css/bootstrap-dialog.min.css"
/>
-    <link rel="stylesheet" href="bower_components/datatables.net-bs/css/dataTables.bootstrap.css"
/>
-    <link rel="stylesheet" href="bower_components/datatables.net-buttons-bs/css/buttons.bootstrap.css"
/>
+    <link rel="stylesheet" href="bower_components/pikaday/css/pikaday.css" />
+    <link rel="stylesheet" href="bower_components/handsontable/dist/handsontable.css"
/>
     <!-- endbower -->
     <link rel="stylesheet" href="bower_components/jquery-ui/themes/base/all.css" />
     <!-- endbuild -->
@@ -132,16 +132,10 @@ limitations under the License.
     <script src="bower_components/ngtoast/dist/ngToast.js"></script>
     <script src="bower_components/ng-focus-if/focusIf.js"></script>
     <script src="bower_components/bootstrap3-dialog/dist/js/bootstrap-dialog.min.js"></script>
-    <script src="bower_components/floatThead/dist/jquery.floatThead.js"></script>
-    <script src="bower_components/floatThead/dist/jquery.floatThead.min.js"></script>
-    <script src="bower_components/datatables.net/js/jquery.dataTables.js"></script>
-    <script src="bower_components/datatables.net-bs/js/dataTables.bootstrap.js"></script>
-    <script src="bower_components/datatables.net-buttons/js/dataTables.buttons.js"></script>
-    <script src="bower_components/datatables.net-buttons/js/buttons.colVis.js"></script>
-    <script src="bower_components/datatables.net-buttons/js/buttons.flash.js"></script>
-    <script src="bower_components/datatables.net-buttons/js/buttons.html5.js"></script>
-    <script src="bower_components/datatables.net-buttons/js/buttons.print.js"></script>
-    <script src="bower_components/datatables.net-buttons-bs/js/buttons.bootstrap.js"></script>
+    <script src="bower_components/zeroclipboard/dist/ZeroClipboard.js"></script>
+    <script src="bower_components/moment/moment.js"></script>
+    <script src="bower_components/pikaday/pikaday.js"></script>
+    <script src="bower_components/handsontable/dist/handsontable.js"></script>
     <!-- endbower -->
     <!-- endbuild -->
     <!-- build:js({.tmp,src}) scripts/scripts.js -->

http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/4eccfcd5/zeppelin-web/test/karma.conf.js
----------------------------------------------------------------------
diff --git a/zeppelin-web/test/karma.conf.js b/zeppelin-web/test/karma.conf.js
index 8049ac6..1ec6eb6 100644
--- a/zeppelin-web/test/karma.conf.js
+++ b/zeppelin-web/test/karma.conf.js
@@ -59,16 +59,10 @@ module.exports = function(config) {
       'bower_components/ngtoast/dist/ngToast.js',
       'bower_components/ng-focus-if/focusIf.js',
       'bower_components/bootstrap3-dialog/dist/js/bootstrap-dialog.min.js',
-      'bower_components/floatThead/dist/jquery.floatThead.js',
-      'bower_components/floatThead/dist/jquery.floatThead.min.js',
-      'bower_components/datatables.net/js/jquery.dataTables.js',
-      'bower_components/datatables.net-bs/js/dataTables.bootstrap.js',
-      'bower_components/datatables.net-buttons/js/dataTables.buttons.js',
-      'bower_components/datatables.net-buttons/js/buttons.colVis.js',
-      'bower_components/datatables.net-buttons/js/buttons.flash.js',
-      'bower_components/datatables.net-buttons/js/buttons.html5.js',
-      'bower_components/datatables.net-buttons/js/buttons.print.js',
-      'bower_components/datatables.net-buttons-bs/js/buttons.bootstrap.js',
+      'bower_components/zeroclipboard/dist/ZeroClipboard.js',
+      'bower_components/moment/moment.js',
+      'bower_components/pikaday/pikaday.js',
+      'bower_components/handsontable/dist/handsontable.js',
       'bower_components/angular-mocks/angular-mocks.js',
       // endbower
       'src/app/app.js',


Mime
View raw message