db-derby-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From spykee <toma.georg...@yahoo.com>
Subject Re: Derby replication system - Need help
Date Tue, 18 Mar 2014 20:58:13 GMT

>I agree, starting 20K threads/sec does not sound like I good idea. But 
>since I'm not familiar with these other technologies that you use, I 
>don't understand why you need to fire a thread per message. Why can't 
>you have a dedicated sender thread that reads everything in your 
>queue-table and sends it (through whatever api you have to the MQ 
>system). You could either have the sender thread check the queue table 
>periodically, or have it sleep on a monitor which the trigger action 
>grabs when adding new records to the queue table.

First you answered to this question yourself( previous response). For each
call of the stored procedure a new thread will be created to execute the
stored procedure. At this I referred, and I want to avoid creating one
thread per message even if the messages are produced at different times (
e.g. message_1 at t1, and message_2 at t2, t1 < t2).

Second, I read that pooling from a table ( perform a periodically select
c1,c2.... from table_T where.... to see if something changed) is  not an
optimal choice, but I like the idea with the monitor ( same thing here, for
each notification from the db " Hey a new row was inserted, wake up and take
the message", I will execute a stored procedure -> one thread per
notification, but since this is just a notifier, less bytes, the network
load is lower, but I will have to make a new call from my monitor thread to
the database.)

Using the monitor approach I will have to make *2 steps:*
- throw a notifier message from the db to the monitor thread ( execute a
stored procedure)
- the monitor thread will then have a logic, and then query the table for
the changes and publish them to the queue
-> this approach will ALLOW me batching the messages ( this should improve
the performance)!!

With the first approach I have to make *1 step:*
- throw a message directly to the queue
-> this approach will NOT allow me batching the messages( this should
decrease the performance)

>Why can't everything from the Derby trigger to the message queue be 
>inside the same jvm?

Everything will run on the same JVM, or different JVM ( this is why I will
use a JMS queue). For the moment there will be same JVM.

I will implement both solutions :
- monitor solution
- my first solution
- compare how many messages are sent per second/minute for each solution,
and use the approach which is the best.

Kind regards,
George



--
View this message in context: http://apache-database.10148.n7.nabble.com/Re-Derby-replication-system-Need-help-tp138003p138081.html
Sent from the Apache Derby Users mailing list archive at Nabble.com.

Mime
View raw message