phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-3744) Support snapshot scanners for MR-based queries
Date Fri, 19 May 2017 00:19:04 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016719#comment-16016719
] 

Hadoop QA commented on PHOENIX-3744:
------------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12868832/PHOENIX-3744.patch
  against master branch at commit 21fb0b31b46da3b7cc27265467d83a1b4cd5c5c5.
  ATTACHMENT ID: 12868832

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified
tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 49 warning messages.

    {color:red}-1 release audit{color}.  The applied patch generated 11 release audit warnings
(more than the master's current 0 warnings).

    {color:red}-1 lineLengths{color}.  The patch introduces the following lines longer than
100:
    +      " FIELD1 VARCHAR NOT NULL , FIELD2 VARCHAR , FIELD3 INTEGER CONSTRAINT pk PRIMARY
KEY (FIELD1 ))";
+    PhoenixMapReduceUtil.setInput(job,PhoenixIndexDBWritable.class,SNAPSHOT_NAME,tableName,tmpDir,
null, FIELD1, FIELD2, FIELD3);
+    PhoenixMapReduceUtil.setInput(job,PhoenixIndexDBWritable.class,SNAPSHOT_NAME,tableName,tmpDir,
FIELD3 + " > 0001", FIELD1, FIELD2, FIELD3);
+    PhoenixMapReduceUtil.setInput(job,PhoenixIndexDBWritable.class,SNAPSHOT_NAME,tableName,tmpDir,inputQuery);
+  private void configureJob(Job job, String tableName, String inputQuery, String condition)
throws Exception {
+      ResultSet rs = DriverManager.getConnection(getUrl(), props).createStatement().executeQuery(inputQuery);
+      assertFalse("Should only have stored" + result.size() + "rows in the table for the
timestamp!", rs.next());
+  private void upsertData(PreparedStatement stmt, String field1, String field2, int field3)
throws SQLException {
+  public static class TableSnapshotMapper extends Mapper<NullWritable, PhoenixIndexDBWritable,
ImmutableBytesWritable, NullWritable> {
+        RegionScannerFactory regionScannerFactory = new NonAggregateRegionScannerFactory(context,
useNewValueColumnQualifier, encodingScheme);

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
     ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ArrayIT

Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/880//testReport/
Release audit warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/880//artifact/patchprocess/patchReleaseAuditWarnings.txt
Javadoc warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/880//artifact/patchprocess/patchJavadocWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/880//console

This message is automatically generated.

> Support snapshot scanners for MR-based queries
> ----------------------------------------------
>
>                 Key: PHOENIX-3744
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3744
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: James Taylor
>            Assignee: Akshita Malhotra
>         Attachments: PHOENIX-3744.patch
>
>
> HBase support scanning over snapshots, with a SnapshotScanner that accesses the region
directly in HDFS. We should make sure that Phoenix can support that.
> Not sure how we'd want to decide when to run a query over a snapshot. Some ideas:
> - if there's an SCN set (i.e. the query is running at a point in time in the past)
> - if the memstore is empty
> - if the query is being run at a timestamp earlier than any memstore data
> - as a config option on the table
> - as a query hint
> - based on some kind of optimizer rule (i.e. based on estimated # of bytes that will
be scanned)
> Phoenix typically runs a query at the timestamp at which it was compiled. Any data committed
after this time should not be seen while a query is running.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message