[CPS-devel] nxlucene persistent transport

[Jean-Marc Orliaguet]

Julien Anguenot wrote:
>> apart from that, the NXLucene server died a day ago, I'm not sure if
>> this is related to the persistent connection bug on the client, or if it
>> is due to a log rotation that failed, or an invalid query. I'm still
>> trying to figure out. But since the lucene server was restarted it has
>> worked without any problem.
>>     
>
> Did you get a core dump ?
>
>   

no, it looks more and more like a memory leak to me. the size of the 
twistd process increases too fast, it never stabilizes or decreases again.
I tried with pylucene-2.1.0, it looks much better, but still... memory 
figures go up when re-indexing a site.


>> the lucene server setup is:
>>
>> 4 CPU 2GHz 64bits
>> Red Hat Enterprise Linux AS release 4
>> gcc-3.4.6-3 (included in RedHat)
>> pylucene2 :-)
>>     
>
> Of course, Red Hat enterprise :) You shouldn't get that much gcj bugs as
> with others distros.
>
> Are you aware about the gcc 3.4.6 bug related to the 2Go PyLucene store
> limitation ?
>
> Cheers,
>
> 	J.
>   

yes and no :-)

the main issue I believe is that the server is a 64bit architecture and 
it has 4 CPUs (gcj 3.4.6 is too old for that type of architecture)

So I have just installed the ubuntu 64bit binary 
(http://downloads.osafoundation.org/PyLucene/linux/ubuntu64/) and the 
memory usage now stays at:

VIRT  RES
185m  68m

even when reindexing an entire site (before it jumps at 600m !)


also to bind twistd to a same cpu I start nxlucene with:

$ taskset 1 bin/runnxlucene &

so I believe it is one more problem solved, but all this is really edgy...

By the way, Julien are you planning to port nxlucene to JBoss, with a 
similar XML-RPC service?

Cheers
/JM