river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike McGrady <mmcgr...@topiatechnology.com>
Subject Re: datastructure classes
Date Thu, 16 Dec 2010 15:04:55 GMT
Intense network systems like our clients cannot work with database calls.  They are too slow
and do not scale.  They either reach a cost or a performance ceiling.

Sent from my iPhone

Michael McGrady
Principal investigator AF081_028 SBIR
Chief Architect
Topia Technology, Inc
Work 1.253.572.9712
Cel 1.253.720.3365

On Dec 16, 2010, at 5:55 AM, Patricia Shanahan <pats@acm.org> wrote:

> MICHAEL MCGRADY wrote:
>> Patricia,
>> If you don't mind, I am going to argue for sticking to the machines,
>> but answer your question roughly at the end.
> 
> I don't mind machines, as long as they get somewhat quantified. It was
> "machines a minute" that really gave me trouble. However, there are
> machines and machines, so I would rather think in terms of transaction
> rates.
> 
>> The questions about either memory use or connections or transactions
>> are stressors leading to the question and the urgency whether or not
>> something must scale, but they have nothing or very little to do with
>> whether it will scale.
>> If we try to involve multiple machines, then we will discover
>> frequently that, even if we do not need to scale, we cannot because,
>> for example, of Amdahl's Theorem.  If we cannot go to multiple
>> machines as a practical matter, we cannot scale no matter what
>> (ceteris paribus).  If we can, then we will find, it is our
>> experience, that we can scale no matter what (ceteris paribus).
> 
> Scaling is exactly the reason why I think the current FastList-based
> design is seriously limited, and may need to be replaced by something
> that indexes its data.
> 
> Outrigger turns a JavaSpace read into a linked
> list scan of all the entries in the space for the required type. Not too
> bad if the list rarely changes and is always small enough to fit in
> cache. If it is big or changing, that is going to cause a lot of memory
> traffic.
> 
> As implemented, it cannot be spread across machines very
> effectively because the additional scanning due to one machine not knowing that another
has already found the entry would use up the added scan power.
> 
> I think to scale it is going to need some sort of indexing, and as I
> think about indexing for the sort of ad-hoc query that JavaSpaces face,
> combined with frequent change, it starts looking very like maintaining
> an indexed database table.
> 
>> However, we should be able to do, say, hundreds of millions of
>> transactions in a day in real-time critical systems such as the FAA
>> or the stock market with data affinity and integrity and all the
>> other "ilities".  If Outrigger cannot do this, it is of no interest
>> to us.
> 
> The current record for a relational database doing simple transactions
> is 30 million transactions per minute (Oracle/SPARC TPC-C). Your mileage
> may vary, but there is no inherent limit on relational database scaling
> that puts a few hundred thousand transactions per minute out of reach.
> 
> 
>> MG
>> On Dec 14, 2010, at 9:48 PM, Patricia Shanahan wrote:
>>> Please clarify "machines a minute". Can you express it e.g. in
>>> transactions per minute?
>>> Patricia
>>> Mike McGrady wrote:
>>>> Linear - 7000 machines a minute, or more. Sent from my iPhone Michael McGrady
Principal investigator AF081_028 SBIR Chief
>>>> Architect Topia Technology, Inc Work 1.253.572.9712 Cel
>>>> 1.253.720.3365 On Dec 14, 2010, at 2:04 PM, Patricia Shanahan
>>>> <pats@acm.org> wrote:
>>>>> On 12/14/2010 1:49 PM, MICHAEL MCGRADY wrote:
>>>>>> On Dec 14, 2010, at 1:40 PM, Patricia Shanahan wrote:
>>>>> ...
>>>>>>> If we made a persistent version use a relational database
>>>>>>> to represent the space,
>>>>>> This would not be usable for us.  This is too slow and does
>>>>>> not have the correct QCC features, especially scalability.
>>>>> ...
>>>>> Could you put some numbers on what sort of scalability you
>>>>> require?
>>>>> Patricia
>> Michael McGrady Chief Architect Topia Technology, Inc. Cel
>> 1.253.720.3365 Work 1.253.572.9712 extension 2037 mmcgrady@topiatechnology.com
> 
> 
> 

Mime
View raw message