[CPS-devel] nxlucene persistent transport

[Julien Anguenot]

Salut Jean-Marc,

Jean-Marc Orliaguet wrote:
> Julien Anguenot wrote:
>> Jean-Marc Orliaguet wrote:
>>  
>>> Julien Anguenot wrote:
>>>     
>> [...]
>>  
>>> OK, the thing is that some of the connections take a lot of time. It's
>>> either 0.3s or > 5 seconds. So I wonder if I'm not going to rewrite it
>>> using a non-persistent connection. Have you experienced problems when
>>> opening a completely new connection each time?
>>>     
>>
>> This is really a lot slower if you don't use it. You can try out by
>> changing the instantiation of the xml-rpc connection stub to not use the
>> persistent transport. Try out a reindexAll with a little bit of content
[...]
>> and you'll figure out.
> when re-indexing an entire site, no problem is experienced (I actually
> re-indexed 24 sites, and the persistent connection works like a charm).

kewl !

> however in production (i.e. in our case, with front-end servers behind a
> load-balancer, virtual IPs, firewall rules, weird routing tables,
> long-lasting connections that mysteriously die after a few minutes ...)
> the persistent connection stops working after a few minutes. Eventually
> the xmlrpc client in nuxeo.lucene.catalog.py fails to connect to the
> lucene server with the message "Connection to %s FAILED.
> nuxeo.lucene.catalog won't work until the connection will be possible
> again" and the server needs to be restarted.

Argh, this is strange... Is the NXLucene server stalled in the case ? If
it's the case that's a bug... Can you check this point out please ?

> So I changed:
> 
>    def _getProxy(self):
>        if getattr(self, '_v_proxy', None) is None:
>            self._v_proxy = xmlrpclib.ServerProxy(
>                self._getServerURL(), transport=PersistentTransport())
> 
> to:
> 
>    def _getProxy(self):
>        if getattr(self, '_v_proxy', None) is None:
>            self._v_proxy = xmlrpclib.ServerProxy(
>                self._getServerURL())
> 
> and individual queries now work perfectly well too.
> 
> So probably depending on the type of method called in the catalog
> (search vs reindex) a persistent transport is better adapted in some
> cases, or more dangerous in others. Note that I didn't experience any of
> these issues in the development environment.

I think before doing this we need to check the server status.. If this
is stalled then we have an issue somewhere because it means a query's
responsible of this...

> Otherwise, everything works perfectly, it's a *huge* performance boost
> compared to ZCatalog!

eheh great ! Glad it does work fine :)

You indexing 24 CPS sites in the same NXLucene ?

Can you give us the OS against which you are running NXLucene and the
gcc version please ?

Cheers,

	J.

-- 
Julien Anguenot | Nuxeo R&D (Paris, France)
Open Source ECM - http://www.nuxeo.com
Nuxeo 5 : http://www.nuxeo.org
Mobile: +33 (0) 6 72 57 57 66

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
Url : http://lists.nuxeo.com/pipermail/cps-devel/attachments/20070430/e4a5d351/signature.pgp