fineract-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Avik Ganguly (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FINERACT-428) Parallelization of Jobs
Date Tue, 18 Apr 2017 06:38:41 GMT

     [ https://issues.apache.org/jira/browse/FINERACT-428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Avik Ganguly updated FINERACT-428:
----------------------------------
    Description: 
For starters, it will be useful to have some technical configuration added as job parameter
for each of the below jobs, that is batch size and thread pool size for executor service so
this has a dependency on FINERACT-425. 

Add migration script to add those 2 configurations to each of the following jobs with the
value of batch size being 500 and thread pool size being 16 :-

Add Accrual Transactions
Add Periodic Accrual Transactions
Add Accrual Transactions For Loans With Income Posted As Transactions
Generate Loan Loss Provisioning
Post Interest for Savings
Update Loan Summary

This would require separation of core functionality to a separate class.

Simplified example :-

```
final ExecutorService executor = Executors.newFixedThreadPool(threadPoolSize);
        
        final LocalDate dueDate = DateUtils.getLocalDateOfTenant();
        final Collection<LoanAccountData> loansToBeRepaidData = this.loanReadPlatformService
                .retrieveLoansToBeRepaidFromAdvancePayment(dueDate);
        Iterable<List<LoanAccountData>> loansToBeRepaid = Iterables.partition(loansToBeRepaidData,
batchSize);
        final Authentication authentication = SecurityContextHolder.getContext().getAuthentication();
        List<Callable<Object>> advancePaymentPosters = new ArrayList<Callable<Object>>();
        
        for (List<LoanAccountData> subList : loansToBeRepaid) {
        	AdvancePaymentPoster poster = (AdvancePaymentPoster) this.applicationContext.getBean("advancePaymentPoster");
        	poster.setLoans(subList);
        	poster.setTenant(ThreadLocalContextUtil.getTenant());
        	poster.setAuthentication(authentication);
        	poster.setCommandService(commandService);
        	advancePaymentPosters.add(Executors.callable(poster));
        }

        try {
			List<Future<Object>> responses = executor.invokeAll(advancePaymentPosters);
for(Future f : responses) {
                    	f.get();
                    }
		} catch (InterruptedException e1) {
			e1.printStackTrace();
		}
        
        executor.shutdown(); 

 ```

  was:
For starters, it will be useful to have some technical configuration added as job parameter
for each of the below jobs, that is batch size and thread pool size for executor service so
this has a dependency on FINERACT-425. 

Add migration script to add those 2 configurations to each of the following jobs with the
value of batch size being 500 and thread pool size being 16 :-

Add Accrual Transactions
Add Periodic Accrual Transactions
Add Accrual Transactions For Loans With Income Posted As Transactions
Generate Loan Loss Provisioning
Post Interest for Savings
Update Loan Summary

This would require separation of core functionality to a separate class.

Simplified example :-

final ExecutorService executor = Executors.newFixedThreadPool(threadPoolSize);
        
        final LocalDate dueDate = DateUtils.getLocalDateOfTenant();
        final Collection<LoanAccountData> loansToBeRepaidData = this.loanReadPlatformService
                .retrieveLoansToBeRepaidFromAdvancePayment(dueDate);
        Iterable<List<LoanAccountData>> loansToBeRepaid = Iterables.partition(loansToBeRepaidData,
batchSize);
        final Authentication authentication = SecurityContextHolder.getContext().getAuthentication();
        List<Callable<Object>> advancePaymentPosters = new ArrayList<Callable<Object>>();
        
        for (List<LoanAccountData> subList : loansToBeRepaid) {
        	AdvancePaymentPoster poster = (AdvancePaymentPoster) this.applicationContext.getBean("advancePaymentPoster");
        	poster.setLoans(subList);
        	poster.setTenant(ThreadLocalContextUtil.getTenant());
        	poster.setAuthentication(authentication);
        	poster.setCommandService(commandService);
        	advancePaymentPosters.add(Executors.callable(poster));
        }

        try {
			List<Future<Object>> responses = executor.invokeAll(advancePaymentPosters);
for(Future f : responses) {
                    	f.get();
                    }
		} catch (InterruptedException e1) {
			e1.printStackTrace();
		}
        
        executor.shutdown(); 

 


> Parallelization of Jobs
> -----------------------
>
>                 Key: FINERACT-428
>                 URL: https://issues.apache.org/jira/browse/FINERACT-428
>             Project: Apache Fineract
>          Issue Type: Improvement
>          Components: Loan, Savings
>            Reporter: Avik Ganguly
>            Assignee: Markus Geiss
>              Labels: performance
>
> For starters, it will be useful to have some technical configuration added as job parameter
for each of the below jobs, that is batch size and thread pool size for executor service so
this has a dependency on FINERACT-425. 
> Add migration script to add those 2 configurations to each of the following jobs with
the value of batch size being 500 and thread pool size being 16 :-
> Add Accrual Transactions
> Add Periodic Accrual Transactions
> Add Accrual Transactions For Loans With Income Posted As Transactions
> Generate Loan Loss Provisioning
> Post Interest for Savings
> Update Loan Summary
> This would require separation of core functionality to a separate class.
> Simplified example :-
> ```
> final ExecutorService executor = Executors.newFixedThreadPool(threadPoolSize);
>         
>         final LocalDate dueDate = DateUtils.getLocalDateOfTenant();
>         final Collection<LoanAccountData> loansToBeRepaidData = this.loanReadPlatformService
>                 .retrieveLoansToBeRepaidFromAdvancePayment(dueDate);
>         Iterable<List<LoanAccountData>> loansToBeRepaid = Iterables.partition(loansToBeRepaidData,
batchSize);
>         final Authentication authentication = SecurityContextHolder.getContext().getAuthentication();
>         List<Callable<Object>> advancePaymentPosters = new ArrayList<Callable<Object>>();
>         
>         for (List<LoanAccountData> subList : loansToBeRepaid) {
>         	AdvancePaymentPoster poster = (AdvancePaymentPoster) this.applicationContext.getBean("advancePaymentPoster");
>         	poster.setLoans(subList);
>         	poster.setTenant(ThreadLocalContextUtil.getTenant());
>         	poster.setAuthentication(authentication);
>         	poster.setCommandService(commandService);
>         	advancePaymentPosters.add(Executors.callable(poster));
>         }
>         try {
> 			List<Future<Object>> responses = executor.invokeAll(advancePaymentPosters);
> for(Future f : responses) {
>                     	f.get();
>                     }
> 		} catch (InterruptedException e1) {
> 			e1.printStackTrace();
> 		}
>         
>         executor.shutdown(); 
>  ```



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message