hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gauthier, Alexander" <Alex.Gauth...@Teradata.com>
Subject RE: HADOOP in Production
Date Wed, 03 Oct 2012 00:15:45 GMT

From: Ted Dunning [mailto:tdunning@maprtech.com]
Sent: Tuesday, October 02, 2012 4:13 PM
To: user@hadoop.apache.org
Subject: Re: HADOOP in Production

On Tue, Oct 2, 2012 at 7:05 PM, Hank Cohen <hank.cohen@altior.com<mailto:hank.cohen@altior.com>>
There is an important difference between real time and real fast

Real time means that system response must meet a fixed schedule.
Real fast just means sooner is better.

Good thought, but real-time can also include a fixed schedule and a specified list of exceptional
conditions which would prevent meeting the schedule.

It may also include a fixed schedule that must be met some fraction of the time (usually very
near 100% of the time).

Without providing exceptions, you basically force the designer to lie about how reliable their
system is.

Real time systems always have hard schedules. The schedule could be in microseconds to control
a laser for making masks for semiconductor manufacturing, milliseconds to control the ignition
in your car or flight controls on an F-22 or  seconds for even slower moving processes.  In
real time system missing the schedule can mean that very bad things happen: planes fall from
the sky, your laser printer fries it's imaging drum, factories explode etc.

It can mean that.  But if you specify the exceptional situations you can specifically mitigate
for them.

Most transaction processing is happy with real fast
The folks doing high velocity trading are pretty close to real time but they probably will
be happy with real fast.
If real fast systems miss a schedule then someone loses money.

Yeah.  And if you talk to these guys, they know the difference and ask for real-time.

The reason that RTOS type operating systems are popular for real time applications is that
they don't allow operations to spend indeterminate amounts of time in uninterruptable states.
 Java will never qualify as a real time system because it has garbage collection and garbage
collection can lock up a system for an indefinite amount of time while it goes through marking
and counting.

You are behind the times on a few counts.

- Java's collectors don't "count".

- Java can be real-time:


- Garbage collection can be deterministic and real-time:


View raw message