batchee-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Struberg <strub...@yahoo.de.INVALID>
Subject Re: bye bye security service?
Date Wed, 17 Aug 2016 16:17:27 GMT
For now I just added another method SecurityService#isAuthorizedJobName(String jobName);

That allows us to implement the scan really quickly.
Our problem is that we have batch jobs started by UC4 all 15 minutes. And that summed up in
the last 2 years. Now we have over 200k JobInstance entries.

Now one could argue that this could be cleaned up in the db - and this is certainly right
;)
But I still would prefer to have getJobNames() perform really well.



Side note:

The other thing we discovered while working on this is that we currently do not lock in the
DB. 

Imo we should aim to be able to have something like start a Job with a PartitionMapper and
then have all 5 cluster nodes grab job by job and work them off.
For this to happen we need to properly detect/prevent concurrent modifications in the DB.
Happy for feedback and brainstorming ;)


LieGrue,
strub





> On Wednesday, 17 August 2016, 17:53, Romain Manni-Bucau <rmannibucau@gmail.com>
wrote:
> > 2016-08-17 17:47 GMT+02:00 Scott Kurz <skurz3@gmail.com>:
> 
>>  If preserving this ability were a goal the filtering could be done by
>>  refining/adding DB queries in the JDBC/JPA persistence and maybe the
>>  in-memory persistence could treat performance as a non-goal.
>> 
>> 
> Hi Scott,
> 
> this is pretty hard to keep it with the same API and do it correctly on the
> backend - agree the in memory impl is not concerned.
> 
> My 2cts are that if we break the API we could directly remove it since the
> security is easy to add on top of JobOperator for any user and we never get
> any feedback of a user SecurityService for now. This allows to get rid of a
> vendor SPI which didn't proove until now to be user-useful.
> 
> @Scott: do you recall any real-world concern on that area or was it an IBM
> feature inheritance?
> 
> 
> 
>>  On Wed, Aug 17, 2016 at 10:41 AM, Romain Manni-Bucau <
>>  rmannibucau@gmail.com>
>>  wrote:
>> 
>>  > Hi guys,
>>  >
>>  > Mark and Reinhard spotted a perf issue in getJobNamed() - basically we
>>  were
>>  > fetching all jobs to then filter the names.
>>  >
>>  > Origin is the ability to filter the shown jobs per *id*. This was very
>>  > flexible cause it allows to get job metadata and filter with all these
>>  > meta. In these meta you have jbatch meta (name, startedAt etc..) but 
> also
>>  > user meta since the id is accessible through the batch so it can be 
> used
>>  > for business logic as well.
>>  >
>>  > Reviwing the security service usage it seems to me we could drop it
>>  > completely since I think we should enable to secure JobOperator and
>>  nothing
>>  > under that level to give business value to the security API.
>>  >
>>  > So here is the proposal I have:
>>  >
>>  > - to not break users who could rely on that we keep current
>>  SecurityService
>>  > API for next release (R) but @Deprecate it (impl can use 
> shortcuts/flags
>>  to
>>  > deactiavte it for jobNames temporarly)
>>  > - in release R+1 we completely remove it
>>  > - *if* we get request to secure JobOperator we build a security either 
> on
>>  > top of JobOperator or directly on persistence layer (= at query level 
> and
>>  > not java level which prevents any perf optim).
>>  >
>>  > Any objection? Other idea?
>>  >
>>  > Romain Manni-Bucau
>>  > @rmannibucau <https://twitter.com/rmannibucau> |  Blog
>>  > <https://blog-rmannibucau.rhcloud.com> | Old Wordpress Blog
>>  > <http://rmannibucau.wordpress.com> | Github 
> <https://github.com/
>>  > rmannibucau> |
>>  > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Tomitriber
>>  > <http://www.tomitribe.com> | JavaEE Factory
>>  > <https://javaeefactory-rmannibucau.rhcloud.com>
>>  >
>> 
> 

Mime
View raw message