lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doron Cohen (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-3746) suggest.fst.Sort.BufferSize should not automatically fail just because of freeMemory()
Date Mon, 06 Feb 2012 06:33:59 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-3746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201073#comment-13201073
] 

Doron Cohen commented on LUCENE-3746:
-------------------------------------

Thanks Dawid! 

{quote}
it's probably a system daemon thread for sending memory threshold notifications
{quote}

Yes this makes sense. 
Still the difference between the two JDKs felt bothering.
Some more digging, and now I think it is clear. 

Here are the stack traces reported (at the end of the test) with Oracle:
{noformat}
1.  Thread[ReaderThread,5,main]
2.  Thread[main,5,main]
3.  Thread[Reference Handler,10,system]
4.  Thread[Signal Dispatcher,9,system]
5.  Thread[Finalizer,8,system]
6.  Thread[Attach Listener,5,system]
{noformat}

And with IBM JDK:
{noformat}
1.  Thread[Attach API wait loop,10,main]
2.  Thread[Finalizer thread,5,system]
3.  Thread[JIT Compilation Thread,10,system]
4.  Thread[main,5,main]
5.  Thread[Gc Slave Thread,5,system]
6.  Thread[ReaderThread,5,main]
7.  Thread[Signal Dispatcher,5,main]
8.  Thread[MemoryPoolMXBean notification dispatcher,6,main]
{noformat}

The 8th thread is the one that started only after accessing the memory management layer. The
thing is, that in the IBM JDK that thread is created in the ThreadGroup "main", while in the
Oracle JDK it is created under "system". To me the latter makes more sense. 

To be more sure I added a fake memory notification listener and check the thread in which
notification happens: 
{code}
MemoryMXBean mmxb = ManagementFactory.getMemoryMXBean();
NotificationListener listener = new NotificationListener() {
  @Override
  public void handleNotification(Notification notification, Object handback) {
    System.out.println(Thread.currentThread());
  }
};
((NotificationEmitter) mmxb).addNotificationListener(listener, null, null);
{code}

Evidently in IBM JDK the notification is in "main" group thread (also in line with the thread-group
in the original warning message which triggered this threads discussion):
{noformat}
Thread[MemoryPoolMXBean notification dispatcher,6,main]
{noformat}

While in Oracle JDK notification is in "system" group thread:
{noformat}
Thread[Low Memory Detector,9,system]
{noformat}

This also explains why the warning is reported only for IBM JDK: because the threads check
in LTC only account for the threads in the same thread-group as the one running the specific
test case. So when dispatching happens in a "system" group thread it is not sensed by that
check at all.

Ok now with mystery solved I can commit the simpler code...
                
> suggest.fst.Sort.BufferSize should not automatically fail just because of freeMemory()
> --------------------------------------------------------------------------------------
>
>                 Key: LUCENE-3746
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3746
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: modules/spellchecker
>            Reporter: Doron Cohen
>             Fix For: 3.6, 4.0
>
>         Attachments: LUCENE-3746.patch, LUCENE-3746.patch, LUCENE-3746.patch
>
>
> Follow up op dev thread: [FSTCompletionTest failure "At least 0.5MB RAM buffer is needed"
| http://markmail.org/message/d7ugfo5xof4h5jeh]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message