manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jitu <abj...@gmail.com>
Subject Re: Transaction related Information
Date Tue, 30 Sep 2014 16:55:57 GMT
Thanks Karl.

Thanks,
Jitu

On Tue, Sep 30, 2014 at 7:36 PM, Karl Wright <daddywri@gmail.com> wrote:

> Hi Jitu,
>
> Using transactions to prevent others from changing jobs or connections is
> not how we do things in ManifoldCF.  Instead, the standard flow is as
> follows:
>
> - the object is loaded, and presented to the user
> - the object is edited by the user
> - the object is saved back to the database
>
> Because the editing process may take an extended period of time, database
> transactions are ONLY appropriate to atomicize the individual operations of
> loading and saving.  Therefore there's no provision for wrapping this
> entire cycle in a single database transaction.  When ManifoldCF needs a
> model that prevents objects from being changed by someone else during one
> of these sessions, I intended to introduce a system of editing locks, which
> expire if not released for a long enough time.
>
> In your case, if what you are looking for is protection against concurrent
> editing as well as rollback, I suggest the following flow:
>
> -- Obtain a specific write lock or write locks that represents your
> combined object.  These would be locks that you define and obtain using
> ILockManager.
> -- Make the modifications you desire WITHOUT a transaction -- always being
> sure to load the original object before you modify it.
> -- If you get errors, undo what you did by saving the original object(s)
> (which you loaded before anything was changed)
> -- Release the write lock(s)
>
> Doing it this way instead of using database transactions will allow you to
> use methods like IJobManager.load and IJobManager.save without concern.
> You will also need to be careful, of course, as to what order you do things
> in.  Your only alternative is to go directly to the database tables
> themselves and write ALL your own queries, with appropriate cache
> management, and this is much harder and seems unnecessary to me.
>
> Karl
>
>
>
>
> On Tue, Sep 30, 2014 at 9:43 AM, Jitu <abjitu@gmail.com> wrote:
>
>> Hi Karl,
>>          Thanks for your continued support. That answers my problem
>> perfectly. but while doing so i have encountered new problem with caching
>> and locking. please help me out.
>>
>> try
>> {
>>   doStuff();
>> }
>> catch (MyException e)
>> {
>>   database.signalRollback();
>>   throw e;
>> }
>> ...
>> finally
>> {
>>   database.endTransaction();
>> }
>>
>> doStuff(){
>>        connection = outputManager.load(connectionName);
>>        if (connection == null) {
>>                 connection = outputManager.create();
>>                 if (connectionName != null && connectionName.length() >
0)
>>                     connection.setName(connectionName);
>>             }
>>       connection.setDescription(connectionName);
>>       outputManager.save(connection);
>> }
>>
>>
>> consider the above case where i have a top level transaction. Inside
>> which i am calling doStuff() method. which basically tries to load existing
>> connection by name (which acquires read lock by cache manager) if
>> connection exist i am updating else i am creating a new connection object
>> and saving it (which needs a write lock). While doing this i am getting
>> "Illegal lock sequence: Write lock can't be within read lock or non-ex
>> write lock" error. Am i doing it wrongly. please advice. Thanks in advance.
>>
>> Thanks,
>> Jitu
>>
>>
>> On Tue, Sep 23, 2014 at 9:47 PM, Karl Wright <daddywri@gmail.com> wrote:
>>
>>> Hi Jitu,
>>>
>>> Editing multiple connections and jobs within a single transaction is
>>> possible, with care.  The care comes in because of caching and locking.
>>>
>>> First, the general form of a transaction is as follows:
>>>
>>> database.beginTransaction();
>>> try
>>> {
>>>   ... do stuff ...
>>> }
>>> catch (MyException e)
>>> {
>>>   database.signalRollback();
>>>   throw e;
>>> }
>>> ...
>>> finally
>>> {
>>>   database.endTransaction();
>>> }
>>>
>>> It doesn't seem like you are doing it this way from your comments, but
>>> that is the only supported way to do it.  You may also see cases in some
>>> classes where exceptions where transaction deadlock is detected and retries
>>> are done.  That is also a permitted way to structure a transaction.
>>>
>>> Second, you can't always wrap arbitrary inner transactions in outer
>>> transactions.  There are rules.  These rules have to do with locking and
>>> caching.  Specifically, when saving would invalidate a cached object, and
>>> in other parts of the transaction you read that very same cached object,
>>> you may wind up with a situation where locks are thrown in an unsupportable
>>> order.  That's when you get exceptions of the kind you mention.
>>>
>>> If you include a sample transaction that fails for you, I can comment
>>> further.
>>>
>>> Karl
>>>
>>>
>>> On Tue, Sep 23, 2014 at 11:54 AM, Jitu <abjitu@gmail.com> wrote:
>>>
>>>> Thanks Karl fir your prompt response as usual. Appreciated! Let me
>>>> explain what I am trying to achieve here.
>>>>
>>>>
>>>>
>>>> I am trying to build a web client for the manifold framework based on
>>>> the theme and design in-line with a product support. I looked into the
>>>> sample example Web-UI provided with the Manifold framework and what I
>>>> understand from that design is, the commits are reserved per “tab/menu
>>>> item” for a given entity. Example: To create a job, I need to first set
up
>>>> input and output connections and refer that info in create Job action. By
>>>> this time, the DB is already populated with the connections details and
>>>> every action from the different menu items makes an individual DB commits
>>>> in separate calls. And there is no way to roll back once it is half
>>>> committed.
>>>>
>>>>
>>>>
>>>> In my design, I tried to achieve the save on all dependent tables
>>>> including the job tables and (Yes, I have few custom tables to updates as
>>>> well) related tables in single transaction. By this my flow looks something
>>>> like this:
>>>>
>>>>
>>>>
>>>> Through Create Job I like to populate Jobs, Repoconnections, schedules,
>>>> etc tables and also two of my custom tables in single transaction. And I
>>>> should able to rollback when the transaction fails.
>>>>
>>>> I tried to achieve this using the IDBInterfaces   public void
>>>> beginTransaction(int transactionType) and then followed rollback signals,
>>>> closeTransaction() etc.. I hit this issue
>>>>
>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Illegal lock
>>>> sequence: Write lock can't be within read lock or non-ex write lock
>>>>
>>>>
>>>>
>>>> Not sure whats going wrong here with my logic and hence I thought let
>>>> me get some help understanding the multi-level transaction ability support
>>>> OR if you can help on how to wire up all the individual DB actions that is
>>>> happening from multiple menu items under “Create New Job” feature from
>>>> sample UI in single transaction and ability to rollback from all tables
>>>> within single transaction, that will be highly appreciated!
>>>>
>>>>
>>>> I hope I am able to articulate what I am trying to achieve here,,,
>>>> Thanks for your help and tim
>>>>
>>>> On Tue, Sep 23, 2014 at 4:55 PM, Karl Wright <daddywri@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Jitu,
>>>>>
>>>>> To clarify further: you can create nested transactions ONLY where the
>>>>> outer-level transaction is as weak or weaker than the inner-level
>>>>> transaction.  Furthermore, you will almost certainly cause MCF performance
>>>>> to suffer badly, as well as needing to handle back-off and retry situations.
>>>>>
>>>>> In general, long transactions are a very bad idea anyhow, because they
>>>>> force single-threadedness on the entire application. Connectors, to the
>>>>> extent that they use database operations, are required to exit any
>>>>> transactions they open before returning to their callers, AND they must
not
>>>>> leave any transactions open when they use any of the I*Activity methods.
>>>>>
>>>>> There are almost always data-centric ways of structuring things so
>>>>> that long-lived transactions are unneeded.  For example, you can introduce
>>>>> a "context identifier" in your tables that will allow you to keep track
of
>>>>> what data is associated with what event.  Careful data design should
be
>>>>> sufficient to resolve your issue.  I can't say anything further because
you
>>>>> haven't provided any actual details as to what you are trying to do.
>>>>>
>>>>> Thanks,
>>>>> Karl
>>>>>
>>>>>
>>>>> On Tue, Sep 23, 2014 at 6:28 AM, Karl Wright <daddywri@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Jitu,
>>>>>>
>>>>>> I'm afraid what you are trying to do will not work with the MCF
>>>>>> architecture.  You will need to find another way.
>>>>>>
>>>>>> Karl
>>>>>>
>>>>>>
>>>>>> On Tue, Sep 23, 2014 at 4:52 AM, Jitu <abjitu@gmail.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>    I have a requirement where on page submit i need to persist
both
>>>>>>> job and repository connection.  To maintain atomicity i added
another level
>>>>>>> of transaction at the top level. But i am getting
>>>>>>> "org.apache.manifoldcf.core.interfaces.ManifoldCFException: Illegal
lock
>>>>>>> sequence: Write lock can't be within read lock or non-ex write
lock" error.
>>>>>>> when i debugged i noticed different transaction maintained in
IDBInterface.
>>>>>>> Please let me know when to use which transaction. i googled but
did not
>>>>>>> find sufficient information on below transaction levels.
>>>>>>>
>>>>>>> public static int TRANSACTION_ENCLOSING = 0;
>>>>>>>   public static int TRANSACTION_READCOMMITTED = 1;
>>>>>>>   public static int TRANSACTION_SERIALIZED = 2;
>>>>>>>   public static int TRANSACTION_REPEATABLEREAD = 3;
>>>>>>>
>>>>>>> Basically first i am trying to persist repository connection
and if
>>>>>>> i get error while persisting job then it should roll back repository
>>>>>>> connection which it persisted already.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Jitu
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message