hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vandana Ayyalasomayajula (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-9343) Implement stateless scanner for Stargate
Date Sun, 22 Sep 2013 23:54:06 GMT

    [ https://issues.apache.org/jira/browse/HBASE-9343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13774129#comment-13774129
] 

Vandana Ayyalasomayajula commented on HBASE-9343:
-------------------------------------------------

Hi,

I am on maternity leave till 9th December, 2013. If you need any assistance please contact
olgan@yahoo-inc.com.

Thanks
Vandana

On Sep 12, 2013, at 1:59 PM, Hadoop QA (JIRA) <jira@apache.org> wrote:

   [ https://issues.apache.org/jira/browse/HBASE-9343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13765888#comment-13765888
]

Hadoop QA commented on HBASE-9343:
----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment
 http://issues.apache.org/jira/secure/attachment/12602864/HBASE-9343_trunk.02.patch
 against trunk revision .

   {color:green}+1 @author{color}.  The patch does not contain any @author tags.

   {color:green}+1 tests included{color}.  The patch appears to include 16 new or modified
tests.

   {color:green}+1 hadoop1.0{color}.  The patch compiles against the hadoop 1.0 profile.

   {color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 2.0 profile.

   {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

   {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

   {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version
1.3.9) warnings.

   {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

   {color:green}+1 lineLengths{color}.  The patch does not introduce lines longer than 100

 {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

    {color:red}-1 core tests{color}.  The patch failed these unit tests:
                      org.apache.hadoop.hbase.coprocessor.TestMasterObserver
                 org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol
                 org.apache.hadoop.hbase.mapred.TestTableInputFormat
                 org.apache.hadoop.hbase.mapreduce.TestTimeRangeMapRed
                 org.apache.hadoop.hbase.mapreduce.TestRowCounter
                 org.apache.hadoop.hbase.io.encoding.TestChangingEncoding
                 org.apache.hadoop.hbase.client.TestHTableUtil
                 org.apache.hadoop.hbase.mapreduce.TestImportTsv
                 org.apache.hadoop.hbase.coprocessor.TestOpenTableInCoprocessor
                 org.apache.hadoop.hbase.coprocessor.TestClassLoading
                 org.apache.hadoop.hbase.thrift.TestThriftServer
                 org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster
                 org.apache.hadoop.hbase.trace.TestHTraceHooks
                 org.apache.hadoop.hbase.mapreduce.TestCopyTable
                 org.apache.hadoop.hbase.mapreduce.TestImportExport
                 org.apache.hadoop.hbase.client.TestHTablePool$TestHTableThreadLocalPool
                 org.apache.hadoop.hbase.util.TestMergeTool
                 org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles
                 org.apache.hadoop.hbase.security.access.TestTablePermissions
                 org.apache.hadoop.hbase.snapshot.TestExportSnapshot
                 org.apache.hadoop.hbase.TestZooKeeper
                 org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithRemove
                 org.apache.hadoop.hbase.client.TestCloneSnapshotFromClient
                 org.apache.hadoop.hbase.security.access.TestZKPermissionsWatcher
                 org.apache.hadoop.hbase.client.TestClientTimeouts
                 org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence
                 org.apache.hadoop.hbase.master.TestMasterFailoverBalancerPersistence
                 org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan2
                 org.apache.hadoop.hbase.client.TestFromClientSideNoCodec
                 org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildOverlap
                 org.apache.hadoop.hbase.client.TestMultiParallel
                 org.apache.hadoop.hbase.mapred.TestTableMapReduce
                 org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildBase
                 org.apache.hadoop.hbase.security.access.TestAccessControlFilter
                 org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
                 org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine
                 org.apache.hadoop.hbase.regionserver.TestHRegion
                 org.apache.hadoop.hbase.client.TestTimestampsFilter
                 org.apache.hadoop.hbase.util.TestRegionSplitter
                 org.apache.hadoop.hbase.catalog.TestMetaMigrationConvertingToPB
                 org.apache.hadoop.hbase.client.TestMetaScanner
                 org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache
                 org.apache.hadoop.hbase.coprocessor.TestRegionObserverBypass
                 org.apache.hadoop.hbase.client.TestAdmin
                 org.apache.hadoop.hbase.client.TestMultipleTimestamps
                 org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster
                 org.apache.hadoop.hbase.master.handler.TestCreateTableHandler
                 org.apache.hadoop.hbase.master.TestMasterMetricsWrapper
                 org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable
                 org.apache.hadoop.hbase.TestAcidGuarantees
                 org.apache.hadoop.hbase.master.TestRollingRestart
                 org.apache.hadoop.hbase.regionserver.TestHRegionOnCluster
                 org.apache.hadoop.hbase.TestFullLogReconstruction
                 org.apache.hadoop.hbase.coprocessor.TestRegionObserverScannerOpenHook
                 org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient
                 org.apache.hadoop.hbase.coprocessor.TestBigDecimalColumnInterpreter
                 org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
                 org.apache.hadoop.hbase.mapreduce.TestWALPlayer
                 org.apache.hadoop.hbase.client.TestScannersFromClientSide
                 org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove
                 org.apache.hadoop.hbase.mapreduce.TestCellCounter
                 org.apache.hadoop.hbase.TestIOFencing
                 org.apache.hadoop.hbase.mapreduce.TestHLogRecordReader
                 org.apache.hadoop.hbase.master.TestMasterTransitions
                 org.apache.hadoop.hbase.client.TestScannerTimeout
                 org.apache.hadoop.hbase.client.TestClientScannerRPCTimeout
                 org.apache.hadoop.hbase.util.TestMergeTable
                 org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol
                 org.apache.hadoop.hbase.client.TestShell
                 org.apache.hadoop.hbase.master.TestRestartCluster
                 org.apache.hadoop.hbase.mapreduce.TestMultithreadedTableMapper
                 org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFilesSplitRecovery
                 org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort
                 org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel
                 org.apache.hadoop.hbase.client.TestSnapshotMetadata
                 org.apache.hadoop.hbase.client.TestHTablePool$TestHTableReusablePool
                 org.apache.hadoop.hbase.TestDrainingServer
                 org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential
                 org.apache.hadoop.hbase.master.TestMasterFileSystem
                 org.apache.hadoop.hbase.master.TestZKBasedOpenCloseRegion
                 org.apache.hadoop.hbase.zookeeper.TestZooKeeperACL
                 org.apache.hadoop.hbase.util.TestCoprocessorScanPolicy
                 org.apache.hadoop.hbase.master.TestOpenedRegionHandler
                 org.apache.hadoop.hbase.io.TestFileLink
                 org.apache.hadoop.hbase.master.TestMasterMetrics
                 org.apache.hadoop.hbase.client.TestHTableMultiplexer
                 org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles
                 org.apache.hadoop.hbase.master.TestMasterFailover
                 org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery
                 org.apache.hadoop.hbase.backup.TestHFileArchiving
                 org.apache.hadoop.hbase.master.TestTableLockManager
                 org.apache.hadoop.hbase.master.handler.TestTableDescriptorModification
                 org.apache.hadoop.hbase.coprocessor.TestRowProcessorEndpoint
                 org.apache.hadoop.hbase.mapreduce.TestHRegionPartitioner
                 org.apache.hadoop.hbase.client.TestHCM
                 org.apache.hadoop.hbase.master.TestMasterShutdown
                 org.apache.hadoop.hbase.client.TestSnapshotFromClient
                 org.apache.hadoop.hbase.coprocessor.TestWALObserver
                 org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient
                 org.apache.hadoop.hbase.client.TestFromClientSide
                 org.apache.hadoop.hbase.util.TestMiniClusterLoadEncoded
                 org.apache.hadoop.hbase.master.TestRegionPlacement
                 org.apache.hadoop.hbase.client.TestFromClientSide3
                 org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor
                 org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan1
                 org.apache.hadoop.hbase.security.access.TestAccessController
                 org.apache.hadoop.hbase.TestLocalHBaseCluster
                 org.apache.hadoop.hbase.catalog.TestMetaReaderEditor
                 org.apache.hadoop.hbase.snapshot.TestRestoreFlushSnapshotFromClient
                 org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint
                 org.apache.hadoop.hbase.master.TestDistributedLogSplitting
                 org.apache.hadoop.hbase.util.TestFSUtils
                 org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildHole
                 org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
                 org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface
                 org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
                 org.apache.hadoop.hbase.master.cleaner.TestHFileCleaner
                 org.apache.hadoop.hbase.master.TestMaster
                 org.apache.hadoop.hbase.io.encoding.TestLoadAndSwitchEncodeOnDisk
                 org.apache.hadoop.hbase.regionserver.wal.TestLogRolling
                 org.apache.hadoop.hbase.util.TestHBaseFsck
                 org.apache.hadoop.hbase.regionserver.TestClusterId

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/7195//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7195//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7195//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7195//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7195//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7195//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7195//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7195//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7195//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/7195//console

This message is automatically generated.

Implement stateless scanner for Stargate
----------------------------------------

               Key: HBASE-9343
               URL: https://issues.apache.org/jira/browse/HBASE-9343
           Project: HBase
        Issue Type: Improvement
        Components: REST
  Affects Versions: 0.94.11
          Reporter: Vandana Ayyalasomayajula
          Assignee: Vandana Ayyalasomayajula
          Priority: Minor
       Attachments: HBASE-9343_94.00.patch, HBASE-9343_94.01.patch, HBASE-9343_trunk.00.patch,
HBASE-9343_trunk.01.patch, HBASE-9343_trunk.01.patch, HBASE-9343_trunk.02.patch


The current scanner implementation for scanner stores state and hence not very suitable for
REST server failure scenarios. The current JIRA proposes to implement a stateless scanner.
In the first version of the patch, a new resource class "ScanResource" has been added and
all the scan parameters will be specified as query params.
The following are the scan parameters
startrow -  The start row for the scan.
endrow - The end row for the scan.
columns - The columns to scan.
starttime, endtime - To only retrieve columns within a specific range of version timestamps,both
start and end time must be specified.
maxversions  - To limit the number of versions of each column to be returned.
batchsize - To limit the maximum number of values returned for each call to next().
limit - The number of rows to return in the scan operation.
More on start row, end row and limit parameters.
1. If start row, end row and limit not specified, then the whole table will be scanned.
2. If start row and limit (say N) is specified, then the scan operation will return N rows
from the start row specified.
3. If only limit parameter is specified, then the scan operation will return N rows from the
start of the table.
4. If limit and end row are specified, then the scan operation will return N rows from start
of table till the end row. If the end row is
reached before N rows ( say M and M &lt; N ), then M rows will be returned to the user.
5. If start row, end row and limit (say N ) are specified and N &lt; number of rows between
start row and end row, then N rows from start row
will be returned to the user. If N &gt; (number of rows between start row and end row
(say M), then M number of rows will be returned to the
user.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

                
> Implement stateless scanner for Stargate
> ----------------------------------------
>
>                 Key: HBASE-9343
>                 URL: https://issues.apache.org/jira/browse/HBASE-9343
>             Project: HBase
>          Issue Type: Improvement
>          Components: REST
>    Affects Versions: 0.94.11
>            Reporter: Vandana Ayyalasomayajula
>            Assignee: Vandana Ayyalasomayajula
>            Priority: Minor
>             Fix For: 0.98.0, 0.96.1
>
>         Attachments: HBASE-9343_94.00.patch, HBASE-9343_94.01.patch, HBASE-9343_trunk.00.patch,
HBASE-9343_trunk.01.patch, HBASE-9343_trunk.01.patch, HBASE-9343_trunk.02.patch
>
>
> The current scanner implementation for scanner stores state and hence not very suitable
for REST server failure scenarios. The current JIRA proposes to implement a stateless scanner.
In the first version of the patch, a new resource class "ScanResource" has been added and
all the scan parameters will be specified as query params. 
> The following are the scan parameters
> startrow -  The start row for the scan.
> endrow - The end row for the scan.
> columns - The columns to scan. 
> starttime, endtime - To only retrieve columns within a specific range of version timestamps,both
start and end time must be specified.
> maxversions  - To limit the number of versions of each column to be returned.
> batchsize - To limit the maximum number of values returned for each call to next().
> limit - The number of rows to return in the scan operation.
>  More on start row, end row and limit parameters.
> 1. If start row, end row and limit not specified, then the whole table will be scanned.
> 2. If start row and limit (say N) is specified, then the scan operation will return N
rows from the start row specified.
> 3. If only limit parameter is specified, then the scan operation will return N rows from
the start of the table.
> 4. If limit and end row are specified, then the scan operation will return N rows from
start of table till the end row. If the end row is 
> reached before N rows ( say M and M &lt; N ), then M rows will be returned to the
user.
> 5. If start row, end row and limit (say N ) are specified and N &lt; number of rows
between start row and end row, then N rows from start row
> will be returned to the user. If N &gt; (number of rows between start row and end
row (say M), then M number of rows will be returned to the
> user.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message