phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <>
Subject [jira] [Commented] (PHOENIX-3744) Support snapshot scanners for MR-based queries
Date Fri, 19 May 2017 00:19:04 GMT


Hadoop QA commented on PHOENIX-3744:

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment
  against master branch at commit 21fb0b31b46da3b7cc27265467d83a1b4cd5c5c5.
  ATTACHMENT ID: 12868832

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 49 warning messages.

    {color:red}-1 release audit{color}.  The applied patch generated 11 release audit warnings
(more than the master's current 0 warnings).

    {color:red}-1 lineLengths{color}.  The patch introduces the following lines longer than
KEY (FIELD1 ))";
+    PhoenixMapReduceUtil.setInput(job,PhoenixIndexDBWritable.class,SNAPSHOT_NAME,tableName,tmpDir,
null, FIELD1, FIELD2, FIELD3);
+    PhoenixMapReduceUtil.setInput(job,PhoenixIndexDBWritable.class,SNAPSHOT_NAME,tableName,tmpDir,
FIELD3 + " > 0001", FIELD1, FIELD2, FIELD3);
+    PhoenixMapReduceUtil.setInput(job,PhoenixIndexDBWritable.class,SNAPSHOT_NAME,tableName,tmpDir,inputQuery);
+  private void configureJob(Job job, String tableName, String inputQuery, String condition)
throws Exception {
+      ResultSet rs = DriverManager.getConnection(getUrl(), props).createStatement().executeQuery(inputQuery);
+      assertFalse("Should only have stored" + result.size() + "rows in the table for the
+  private void upsertData(PreparedStatement stmt, String field1, String field2, int field3)
throws SQLException {
+  public static class TableSnapshotMapper extends Mapper<NullWritable, PhoenixIndexDBWritable,
ImmutableBytesWritable, NullWritable> {
+        RegionScannerFactory regionScannerFactory = new NonAggregateRegionScannerFactory(context,
useNewValueColumnQualifier, encodingScheme);

     {color:red}-1 core tests{color}.  The patch failed these unit tests:

Test results:
Release audit warnings:
Javadoc warnings:
Console output:

This message is automatically generated.

> Support snapshot scanners for MR-based queries
> ----------------------------------------------
>                 Key: PHOENIX-3744
>                 URL:
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: James Taylor
>            Assignee: Akshita Malhotra
>         Attachments: PHOENIX-3744.patch
> HBase support scanning over snapshots, with a SnapshotScanner that accesses the region
directly in HDFS. We should make sure that Phoenix can support that.
> Not sure how we'd want to decide when to run a query over a snapshot. Some ideas:
> - if there's an SCN set (i.e. the query is running at a point in time in the past)
> - if the memstore is empty
> - if the query is being run at a timestamp earlier than any memstore data
> - as a config option on the table
> - as a query hint
> - based on some kind of optimizer rule (i.e. based on estimated # of bytes that will
be scanned)
> Phoenix typically runs a query at the timestamp at which it was compiled. Any data committed
after this time should not be seen while a query is running.

This message was sent by Atlassian JIRA

View raw message