lucene-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Will Martin" <wmartin...@gmail.com>
Subject RE: How can I make better project than Lucene?
Date Sun, 16 Nov 2014 07:04:09 GMT
Please let me know if you go the GO route, so I can choose the language I
like best for my work.  Wmartinusa   at   google online mail.

-----Original Message-----
From: swsong_dev [mailto:swsong_dev@websqrd.com] 
Sent: Saturday, November 15, 2014 6:23 PM
To: general@lucene.apache.org
Subject: Re: How can I make better project than Lucene?

Thank you for your sincere reply, Mr. McCandless.

When I posted an email in a mailing-list, I was afraid for not getting a
considerable reply, but Ií»m now so glad I might find a way.

I agree that a new search engine in Go would be competitive. I think we all
need a next generation search engine core redesigned from the start.

And, I understand Luceneí»s limitations you mentioned. They are good points
to get started.

I have been used a search engine for first 4 years and developed a search
engines for last 6 years from the bottom, and I got feedback í░Ití»s faster
than Solr in indexing and searchingí▒. (http://ddakker.tistory.com/248
<http://ddakker.tistory.com/248>)

===Result===
Data size : 529,188
Fastcat indexing time : 1m 26s
Solr3.5 indexing time : 5m 30s
Fastcat searching time : 48ms
Solr3.5 searching time : 73ms

It applied to Koreaí»s greatest shopping service(http://danawa.com/
<http://danawa.com/>) a month ago to my delight.

But my goal has been making a globally-used open source search engine.

As you suggested, now I want to make a whole-new search engine in Go.

I have made my first search engine alone, but I would not make a new search
engine alone. I want to make it with global developers together.

If you plan to make a new search engine in Go, or know someone around you,
could you help me gathering members for a new search engine, and guide us
technically(feature requirement, efficient design)?

Or if there is already a new search engine project in Go, could you let me
know?

In Korea, no one develops a search engine except people who work at a
search engine solution company, and even they are very few and do not spend
time to an open source project.

In my case, I found a tiny venture company for making time to develop an
open source search engine 4 year ago.

I want to be involved in a next-generation search engine project. I would
be happy to make a new search engine itself.

Your little help could be great for me.

Thank you.

Sang Song


> 2014. 11. 15., ┐└╚─ 8:22, Michael McCandless <lucene@mikemccandless.com>
└█╝║:
> 
> Actually I think competing projects is very healthy for open source
development.
> 
> There are many things you could explore to "contrast" with Lucene, 
> e.g. write your new search engine in Go not Java: Java has many 
> problems, maybe Go fixes them.  Go also has a low-latency garbage 
> collector in development ... and Java's GC options still can't scale 
> to the heap sizes that are practical now.
> 
> Lucene has many limitations, so your competing engine could focus on 
> them.  E.g. the "schemalessness" of Lucene has become a big problem, 
> and near impossible to fix at this point, and prevents new important 
> features like LUCENE-5879 from being possible, so you could give your 
> engine a "gentle" schema from the start.
> 
> The Lucene Filter/Query situation is a mess: one should extend the other.
> 
> Lucene has weak support for proximity queries (SpanQuery is slow and 
> does not get much attention).
> 
> Lucene is showing its age, missing some compelling features like a 
> builtin transaction log, "core" support for numerics (they are sort of 
> hacked on top), optimistic concurrency support (sequence ids, 
> versions, something), distributed support (near real time replication, 
> etc.), multi-tenancy, an example server implementation, so the search 
> servers on top of Lucene have had to fill these gaps.  Maybe you could 
> make your engine distributed from the start (Go is a great match for 
> that, from what little I know).
> 
> All 3 highlighter options have problems.
> 
> The analysis chain (attributes) is overly complex.
> 
> In your competing engine you can borrow/copy/steal from Lucene's good 
> parts to get started...
> 
> 
> Mike McCandless
> 
> http://blog.mikemccandless.com
> 
> 
> On Fri, Nov 14, 2014 at 8:43 PM, swsong_dev <swsong_dev@websqrd.com>
wrote:
>> Ií»m developing search engine, Fastcatsearch. http://github 
>> <hthttp://githubtp//github>.com/fastcatsearch/fastcatsearch
>> 
>> Lucene is widely known and famous project and I cannot beat Lucene for
now.
>> 
>> But is there any chance to beat Lucene?
>> 
>> Anything like features, performance.
>> 
>> Please, let me know what to do to make better product than Lucene.
>> 
>> Thank you.



Mime
View raw message