lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-2791) WindowsDirectory
Date Fri, 03 Dec 2010 11:39:11 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12966495#action_12966495
] 

Robert Muir commented on LUCENE-2791:
-------------------------------------

{quote}
Ahm you dont really use overlapped/async IO, ReadFile in your code only returns when read
operation is finished, so not async. Because when you do you have to pass a flag: FILE_FLAG_OVERLAPPED.
But I don't think we should really make it async for now, that is too much stuff as Lucene
does not really support it at the moment.
{quote}

Yes I do, Overlapped IO is distinct from asynchronous IO! you just use the overlapped structure
without OVERLAPPED, as i did here.

bq. Currently you don't really need the OVERLAPPED structure at all, just pass NULL: http://msdn.microsoft.com/en-us/library/aa365467(VS.85).aspx,
or is there a reason?

Yes I do use it, do you understand C ?

bq. Do we need FILE_SHARE_WRITE?

Yes

bq. You use env->GetStringUTFChars, which is not really a problem as index filenames are
always ASCII only. The default methods in the Win32 API use the system default charset (ANSI)
for the arguments to OpenFile, which links to "OpenFileA" in kernel32.dll. I would suggest
to use the real UTF16 chars (env->GetStringChars) and use OpenFileW method, which takes
wchar instead of char. The same applies to Linux Dir, but there I am not sure how encodings
are handled.

We don't open unicode files in lucene, they are all ascii. 

bq. Also the Format functions for GetLastError() results use ANSI per default. We should use
the *W method, too, and pass it to java as wchar.

I'm not concerned about this here.

bq. Exception handling: in native read, you call throwIOException, when ReadFile() returns
false. But 2 lines later you still call the method to copy the local buffer to the java byte
array. JNI docs say, that after an Exception is in the thread's local exception, you should
not call any JNI methods anymore (expect some allowed ones). Also the copy operation is not
really needed.

I don't think you know what you are talking about!

The code uses 2 techniques, alloc on the stack+SetByteArrayRegion (for small reads), and GetByteArrayElements
(for larger reads). The JDK (at least sun, etc) *always* makes a copy for the latter.

So bottom line: I disagree with you on everything you said.


> WindowsDirectory
> ----------------
>
>                 Key: LUCENE-2791
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2791
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Store
>            Reporter: Robert Muir
>         Attachments: LUCENE-2791.patch, LUCENE-2791.patch, WindowsDirectory.dll, WindowsDirectory_amd64.dll
>
>
> We can use Windows' overlapped IO to do pread() and avoid the performance problems of
SimpleFS/NIOFSDir.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message