hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jim Kellerman (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-469) Streamline HStore startup and compactions
Date Sun, 30 Mar 2008 07:07:24 GMT

     [ https://issues.apache.org/jira/browse/HBASE-469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jim Kellerman updated HBASE-469:
--------------------------------

    Attachment: patch.txt

HMerge, HRegionServer
- changes that reflect changes to HRegion, CompactSplitThread and Flusher methods

ServerManager
- Return zero length array to region server if it is exiting or quiesced and Master is not
yet ready to shut down.

QueueEntry
- removed. no longer used.

CompactSplitThread
- make compactionQueue a queue of HRegion.
- Add Set<HRegion> so we can quickly determine if a region is in the queue. BlockingQueue.contains()
does a linear scan of the queue.
- Add a lock and interruptPolitely methods so that compactions/splits in progress are not
interrupted.
- Don't add a region to the queue if it is already present.

Flusher
- change queue from DelayQueue to BlockingQueue, with HRegion entries instead of QueueEntry.
- Add Set<HRegion> to quickly determine if a region is already in the queue to avoid
linear scan of BlockingQueue.contains().
- Only put regions in the queue for optional cache flush if the last time they were flushed
is older than now - optionalFlushInterval.
- Only add regions to the queue if it is not already present.

HRegion
- don't request a cache flush if one has already been requested.
- In close, null out references so garbage collector will have an easier time figuring out
what can be released.
- Add setLastFlushTime so flusher can set it once it has queued an optional flush.
- Replace largestHStore with getLargestHStoreSize: returns long instead of HStoreSize object.
- Add midKey as parameter to splitRegion.
- Reorder start of splitRegion so it doesn't do any work before validating parameters.
- Remove needsSplit and compactIfNeeded - no longer needed.
- compactStores now returns midKey if split is needed.
- snapshotMemcaches now sets flushRequested to false and sets lastFlushTime to now.
- update does not request a cache flush if one has already been requested.
- Override equals and hashCode so HRegions can be stored in a HashSet.

HStore
- loadHStoreFiles now computes max sequence id and the initial size of the store.
- Add getter for family.
- Close nulls out references to for many objects which helps the garbage collector determine
if objects can be gc'd.
- internalCacheFlush updates store size, and logs both size of cache flush and resulting map
file size (with debug logging enabled).
- Remove needsCompaction and hasReferences - no longer needed.
- compact() returns midKey if store needs to be split.
- compact() does all checking before actually starting a compaction.
- If store size is greater than desiredMaxFileSize, compact returns the midKey for the store
regardless of whether a compaction was actually done.
- Added more synchronization in completeCompaction while iterating over storeFiles.
- completeCompaction computes new store size.
- New method checkSplit replaces method size. Returns midKey if store needs to be split and
can be split.

HStoreSize
- removed. No longer needed.

Memcache
- added close method which clears all the maps and should make the job easier for garbage
collector.

HBaseTestCase
- only set fs if it has not already been set by a subclass.
- Add protected method deleteLocal to clean up files created in local fs, by M/R jobs.

TestTableIndex, TestTableMapReduce
- call new deleteLocal method 

> Streamline HStore startup and compactions
> -----------------------------------------
>
>                 Key: HBASE-469
>                 URL: https://issues.apache.org/jira/browse/HBASE-469
>             Project: Hadoop HBase
>          Issue Type: Improvement
>          Components: regionserver
>    Affects Versions: 0.2.0
>            Reporter: Jim Kellerman
>            Assignee: Jim Kellerman
>             Fix For: 0.2.0
>
>         Attachments: patch.txt, patch.txt
>
>
> Several useful patches that streamline HStore startup and compactions that were a part
of the abandoned changes in HBASE-69 should be incorporated.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message