phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-1973) Improve CsvBulkLoadTool performance by moving keyvalue construction from map phase to reduce phase
Date Fri, 12 Feb 2016 23:03:18 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145502#comment-15145502
] 

Hadoop QA commented on PHOENIX-1973:
------------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12787719/PHOENIX-1973-4.patch
  against master branch at commit 8ece81b5522df3e6bd9dfdb3112e101215bb49f1.
  ATTACHMENT ID: 12787719

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified
tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 19 warning messages.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:red}-1 lineLengths{color}.  The patch introduces the following lines longer than
100:
    +     * Find the column index which will replace the column name in the aggregated array
and will be restored in Reducer
+        Map<byte[], Integer> qualifiers = columnMap.get(Bytes.copy(cell.getFamilyArray(),
cell.getFamilyOffset(), cell.getFamilyLength()));
+            Integer result = qualifiers.get(Bytes.copy(cell.getQualifierArray(), cell.getQualifierOffset(),
cell.getQualifierLength()));
+    private void writeAggregatedRow(Context context, int tableIndex, List<KeyValue>
lkv) throws IOException, InterruptedException {
+                outputStream.write(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength());
+        context.write(new TableRowkeyPair(Integer.toString(tableIndex), outputKey), aggregatedArray);
+        if (Bytes.compareTo(cell.getQualifierArray(), cell.getQualifierOffset(), cell.getQualifierLength(),
+                QueryConstants.EMPTY_COLUMN_BYTES, 0, QueryConstants.EMPTY_COLUMN_BYTES.length)
!= 0)
+        private final Mapper<LongWritable, Text, TableRowkeyPair, ImmutableBytesWritable>.Context
context;
+            DataInputStream input = new DataInputStream(new ByteArrayInputStream(aggregatedArray.get()));

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
     ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.rpc.PhoenixServerRpcIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ArithmeticQueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SkipScanQueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TruncateFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SpillableGroupByIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.MutableIndexToolIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SpooledTmpFileDeleteIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CountDistinctCompressionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.MutableIndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.UpsertSelectAutoCommitIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.hbase.index.covered.example.EndToEndCoveredIndexingIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ArrayToStringFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.trace.PhoenixTableMetricsWriterIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CSVCommonsLoaderIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.iterate.RoundRobinResultIteratorIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TenantSpecificViewIndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.IndexExpressionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.EvaluationOfORIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DynamicUpsertIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DateTimeIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.UnionAllIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.GlobalIndexOptimizationIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.LpadFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ReverseScanIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.StddevIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.salted.SaltedTableUpsertSelectIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.StringToArrayFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ClientTimeArithmeticQueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AlterTableIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ViewIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SpooledSortMergeJoinIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.txn.TxWriteFailureIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ArrayAppendFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ArrayIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.coprocessor.StatisticsCollectionRunTrackerIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.hbase.index.balancer.IndexLoadBalancerIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.BinaryRowKeyIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AlterSessionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.QueryDatabaseMetaDataIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.iterate.PhoenixQueryTimeoutIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.DropViewIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TenantIdTypeIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SubqueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.NthValueFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.FirstValueFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ModulusExpressionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ProductMetricsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CsvBulkLoadToolIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.hadoop.hbase.regionserver.wal.WALReplayWithIndexWritesAndUncompressedWALInHBase_094_9_IT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SignFunctionEnd2EndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.IndexMetadataIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DecodeFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.trace.PhoenixTracingEndToEndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.ImmutableIndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ToCharFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AlterTableWithViewsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.iterate.RoundRobinResultIteratorWithStatsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.iterate.ScannerLeaseRenewalIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.UpgradeIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.hbase.index.FailForUnsupportedHBaseVersionsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TenantSpecificViewIndexSaltedIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.rpc.PhoenixClientRpcIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.QueryTimeoutIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AggregateQueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ReadOnlyIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CbrtFunctionEnd2EndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ExpFunctionEnd2EndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AbsFunctionEnd2EndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SaltedViewIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.HashJoinIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.LnLogFunctionEnd2EndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.tx.TxCheckpointIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SequenceIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.InstrFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ConnectionUtilIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.MD5FunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.MultiCfQueryExecIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DerivedTableIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.PowerFunctionEnd2EndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.HashJoinMoreIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ExtendedQueryExecIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ArrayFillFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.txn.RollbackIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.LastValueFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ToNumberFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.ImmutableIndexWithStatsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SkipScanAfterManualSplitIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ScanQueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.RegexpSplitFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.InMemoryOrderByIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.RTrimFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.OctetLengthFunctionEnd2EndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.tx.TransactionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.MutableIndexFailureIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.QueryWithLimitIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.OrderByIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.QueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TransactionalViewIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.hadoop.hbase.regionserver.wal.WALReplayWithIndexWritesAndCompressedWALIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SqrtFunctionEnd2EndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.UpsertValuesIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.RowValueConstructorIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.MutableIndexReplicationIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.EncodeFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.NativeHBaseTypesIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.rpc.UpdateCacheWithScnIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.monitoring.PhoenixMetricsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DynamicColumnIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.hbase.index.covered.example.EndtoEndIndexingWithCompressionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.salted.SaltedTableVarLengthRowKeyIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.HashJoinLocalIndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.UpsertBigValuesIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.execute.PartialCommitIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DisableLocalIndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.ViewIndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.salted.SaltedTableIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AutoCommitIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TopNIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.PointInTimeQueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SequenceBulkAllocationIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.VariableLengthPKIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ColumnProjectionOptimizationIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DynamicFamilyIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.SaltedIndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SubqueryUsingSortMergeJoinIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.UpsertSelectIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ArrayPrependFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.KeyOnlyIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DistinctCountIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.PhoenixRuntimeIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ExecuteStatementsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.IsNullIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CastAndCoerceIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ReadIsolationLevelIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SortMergeJoinIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.trace.PhoenixTraceReaderIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ReverseFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SortOrderIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.RegexpSubstrFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.QueryExecWithoutSCNIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.rpc.UpdateCacheIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.StatsCollectorIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ConvertTimezoneFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.GetSetByteBitFunctionEnd2EndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.QueryMoreIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.txn.MutableRollbackIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.LocalIndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.NotQueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ArraysWithNullsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.MapReduceIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.GroupByCaseIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.IndexToolIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TenantSpecificTablesDMLIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ArrayConcatFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.LikeExpressionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.UserDefinedFunctionsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CoalesceFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.StatementHintsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ParallelIteratorsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.StatsCollectorWithSplitsAndMultiCFIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DeleteIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SpooledOrderByIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CustomEntityDataIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ContextClassloaderIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CompareDecimalToLongIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.hbase.index.covered.example.FailWithoutRetriesIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.hbase.index.covered.EndToEndCoveredColumnsIndexBuilderIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.IndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TenantSpecificTablesDDLIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.StoreNullsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.FunkyNamesIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.PercentileIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.GroupByIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.RoundFloorCeilFunctionsEnd2EndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.MinMaxAggregateFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CaseStatementIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TimezoneOffsetFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ToDateFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.RegexpReplaceFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SortMergeJoinMoreIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.InListIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.MappingTableDataTypeIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CreateTableIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ServerExceptionIT

     {color:red}-1 core zombie tests{color}.  There are 1 zombie test(s): 

Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/255//testReport/
Javadoc warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/255//artifact/patchprocess/patchJavadocWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/255//console

This message is automatically generated.

> Improve CsvBulkLoadTool performance by moving keyvalue construction from map phase to
reduce phase
> --------------------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-1973
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-1973
>             Project: Phoenix
>          Issue Type: Improvement
>            Reporter: Rajeshbabu Chintaguntla
>            Assignee: Sergey Soldatov
>             Fix For: 4.7.0
>
>         Attachments: PHOENIX-1973-1.patch, PHOENIX-1973-2.patch, PHOENIX-1973-3.patch,
PHOENIX-1973-4.patch
>
>
> It's similar to HBASE-8768. Only thing is we need to write custom mapper and reducer
in Phoenix. In Map phase we just need to get row key from primary key columns and write the
full text of a line as usual(to ensure sorting). In reducer we need to get actual key values
by running upsert query.
> It's basically reduces lot of map output to write to disk and data need to be transferred
through network.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message