db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Knut Magne Solem (JIRA)" <j...@apache.org>
Subject [jira] Updated: (DERBY-2798) A new approach for main-memory database
Date Sun, 24 Jun 2007 19:50:25 GMT

     [ https://issues.apache.org/jira/browse/DERBY-2798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Knut Magne Solem updated DERBY-2798:
------------------------------------

    Attachment: select.png
                DERBY-2798-10.3.1.0.stat
                DERBY-2798-10.3.1.0.diff

Cleaned up the code and made it work with Derby 10.3.1.0 (unofficial). I also did a simple
benchmarks for select and update (with durability=test) on Wisconsin testdata with Java 6.0.
Hardware is P4 2.8 GHz with HT running linux.

The patch is for the unofficial 10.3.1.0 release.

> A new approach for main-memory database
> ---------------------------------------
>
>                 Key: DERBY-2798
>                 URL: https://issues.apache.org/jira/browse/DERBY-2798
>             Project: Derby
>          Issue Type: New Feature
>          Components: Store
>    Affects Versions: 10.2.2.0
>         Environment: all
>            Reporter: Knut Magne Solem
>         Attachments: Derby-10.2.2.0-memstore.diff, DERBY-2798-10.3.1.0.diff, DERBY-2798-10.3.1.0.stat,
DERBY-2798.diff, select.png, update.png
>
>
> As a part of my Masters degree I have created an extension that allows data to reside
in memory without the need to serialize it to Page-objects. This is a pretty big chunk of
code and is sort of a proof of concept of another way to make an in-memory storage mode.
> I created two new conglomerates, called MemHeap and MemSkiplist. Derby interfaces them
the same way as it does with Heap and BTree. These new conglomerates use RawStore for transaction
support and logging, but not for storage. Instead it uses a new system service I've called
MemStore. This data store only stores pointers to Slot-objects organized in arrays corresponding
to its container/conglomerate/table. A Slot-object consists mainly of a DataValueDescriptor[]-object
representing a row in a table.
>  
> So, instead of just doing dummy-IO in memory where Derby still thinks its doing real
IO and page caching, this new approach bypasses the cache and page-structure by keeping the
DataValueDescriptor[]-objects in memory without serializing them.
>  
> Manipulating operations on data in memory are done via new operation-objects (ex. MemInsertOperation,
MemInsertUndoOperation, MemDeleteOperation...) with still uses RawStore for transaction control
and persistence. Checkpointing is done by serializing the objects in MemStore fuzzily and
completely unsynchronized to disk. Recovery consists of de-serializing the objects to MemStore
before the existing REDO- and UNDO-phase of Derby recovery in RawStore will get the data transaction-consistent
by replaying or undo the new operation-objects in the log.
>  
> Locking is hard coded as row based with locking degree SERIALIZABLE.
>  
> To get Derby to use the new conglomerates I hacked the SQL-layer to create MemHeap-tables
and MemSkiplist-indexes when the table name starts with 'mem_'.
>  
> Because this is a major rewrite of the access- and storage-layer there is a lot of known
and unknown bugs and missing functionality. What is working is essentially select, insert,
update and delete on tables with one primary key.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message