carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ravipesala <...@git.apache.org>
Subject [GitHub] carbondata issue #978: [CARBONDATA-1109] Acquire semaphore before submit a p...
Date Fri, 02 Jun 2017 05:49:22 GMT
Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/978
  
    Not required any check, entry count check handled inside
    
    On Fri, 2 Jun 2017 at 11:13 AM, manishgupta88 <notifications@github.com>
    wrote:
    
    > *@manishgupta88* commented on this pull request.
    > ------------------------------
    >
    > In
    > processing/src/main/java/org/apache/carbondata/processing/store/CarbonFactDataHandlerColumnar.java
    > <https://github.com/apache/carbondata/pull/978#discussion_r119785140>:
    >
    > > @@ -488,13 +488,19 @@ private NodeHolder processDataRows(List<CarbonRow>
dataRows)
    >    public void finish() throws CarbonDataWriterException {
    >      // still some data is present in stores if entryCount is more
    >      // than 0
    > -    producerExecutorServiceTaskList.add(producerExecutorService
    > -        .submit(new Producer(blockletDataHolder, dataRows, ++writerTaskSequenceCounter,
true)));
    > -    blockletProcessingCount.incrementAndGet();
    > -    processedDataCount += entryCount;
    > -    closeWriterExecutionService(producerExecutorService);
    > -    processWriteTaskSubmitList(producerExecutorServiceTaskList);
    > -    processingComplete = true;
    > +    try {
    > +      semaphore.acquire();
    >
    > @watermen <https://github.com/watermen> .....here a check is required for
    > entryCount > 0.....because we need to acquire a semaphore lock only if
    > total number of rows in raw data are not exactly divisible by page
    > size...in this case only we will have some extra rows to be process by
    > finish method else addDataToStore method will handle all the rows....Please
    > refer the below code snippet....
    >
    > public void finish() throws CarbonDataWriterException {
    > // still some data is present in stores if entryCount is more
    > // than 0
    > if (this.entryCount > 0) {
    > try {
    > semaphore.acquire();
    > producerExecutorServiceTaskList.add(producerExecutorService
    > .submit(new Producer(blockletDataHolder, dataRows,
    > ++writerTaskSequenceCounter, true)));
    > blockletProcessingCount.incrementAndGet();
    > processedDataCount += entryCount;
    > } catch (InterruptedException e) {
    > LOGGER.error(e, e.getMessage());
    > throw new CarbonDataWriterException(e.getMessage(), e);
    > }
    > }
    > closeWriterExecutionService(producerExecutorService);
    > processWriteTaskSubmitList(producerExecutorServiceTaskList);
    > processingComplete = true;
    >
    > —
    > You are receiving this because you were mentioned.
    > Reply to this email directly, view it on GitHub
    > <https://github.com/apache/carbondata/pull/978#pullrequestreview-41695875>,
    > or mute the thread
    > <https://github.com/notifications/unsubscribe-auth/AHwdxhu_fSOnCMiMXsBYK45_C5Se6qJaks5r_6EWgaJpZM4NrZnd>
    > .
    >



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message