[Cps4] CPSUniversal interface

[Florent Guillaume]

Let's not forget that one of the first services we have to plug is a  
global transaction manager. For many kinds of remote access, we don't  
want to have just an XMLRPC-like request/response that's its own  
transaction. We'll want to do many things and then commit/rollback.

Other comments below:


On 12 Apr 2006, at 15:48, Eric Barroca wrote:
> On 12 avr. 06, at 00:58, Ruslan Spivak wrote:
>> В Пнд, 10/04/2006 в 15:02 +0200, Eric Barroca пишет:
>>> I'd like that Ruslan initiate the design work of the CPSUniversal
>>> module that aims at providing a unified interface on top of CPS.
>>> I may be based on ICE.
>>>
>>> Goals :
>>> 	- transparent access to CPS from other languages / platforms (Java,
>>> C#, PHP) to allow development of CPS components in other languages
>>> (ex: build your website using PHP and access CPS content).
>> Let's clarify a bit here: i think you don't mean here to write in  
>> other
>> languages than python full blown Zope components, but just Ice  
>> clients
>> in different languages to connect to Ice based CPS component that
>> provides predefined(?) API allowing clients to manipulate CPS in  
>> one way
>> or another.
>> I know we have some problems with CPSRemoteController API, it became
>> blown and custom projects have its own requirements for specific
>> methods, but i believe we could after thourough thinking provide API
>> that touches many aspects of work with CPS and create Slice  
>> definition
>> that will provide API methods for other languages.
> The idea is that the component will expose a complete API on top of  
> CPS, to allow to write components in other language.
> If we have the event service plugged + a complete API, you can  
> actually write services (like portal tools) on other languages,  
> that would react to events and do stuff on objects, for example.

Yes, "complete API" is obviously a bit large. What we want is an API  
to manipulate CPS data and CPS's main services (workflow, security,  
etc).

>>> 	- allow distributed achitecture for CPS development (CPS instances
>>> can communicate)
>> What meaning do you put into this? Communication between CPSUniversal
>> components from different instances ?
> Create a way to allow different CPS intances to communicate.
> Like sending documents between two instances.

That's a separate import/export component.

>>> 	- create a way to communicate with external components like
>>> JackRabbit or Sesame using a unique way
>> If you mean here defining means of communication like Ice, then it's
>> clear how it can be unique.
>> If it's about interface that is exposed by Jackrabbit/Sesame/etc,  
>> then i
>> do not see other way as if when using Ice, for example, to describe
>> Jackrabbit/Sesame/etc interfaces in Slice IDL which will be exposed
>> after that as Ice server's methods that will be accessing
>> Jackrabbit/Sesame/etc. Of course it's our responsibility to provide
>> those methods implementation.
> That's what I mean.
>
>>> 	- offer CPS connectors/API for Java, PHP and .NET/C#
>> This is related to first goal, i guess, unless you mean something
>> different, ie provide API(defined with Slice) for manipulating CPS.
> yes
>
>>> Principles :
>>> 	- A central client service that can be used to reach external  
>>> ICE'ed
>>> components
>> can you, please, provide use case ?
> Let's say we have JackRabbit, Lucene and Sesame available through ICE.
> We need to have a central registry that list all services  
> available, no ?
> As far as I understand, ICEBox is designed for that.
>
>>> 	- A central service where components can register methods. The
>>> service expose those methods to ICE clients.
>> what do you mean here by "components" ? Can you expand, please? Is it
>> just python code that acts like Ice service and then it looks like
> components here == CPS products.
> For example, CPSWorkflow can register methods available in the API  
> exposed through ICE.
> Same for CPSDocument, CPSCore, etc.
>
>> IceBox and component implementing interface just registers in  
>> IceBox(the
>> service/component code may connect to Jabber, for example, ie no
>> dependence on CPS).
>> Or it's component that exposes its methods via Ice and also has  
>> access
>> to CPS?
> It's a service, like CPSRemoteController, where CPS components can  
> register methods available.
> I do not know if it's inside or outside CPS.

It exists inside Zope (because for instance we want to expose  
components as zope 3 utilities to be able to call them easily). But  
all such utilities are also exposed vie Ice to the outside world.  
Conversely, Ice may expose a pseudo-utility to Zope that actually  
exists somewhere else.

>> Let's start from the fact that before client and server communicate
>> invoking methods, we provide compile time(with python it's just load
>> time) contract using Slice IDL that we store in file and it should be
>> available both on client and a server. It's not xml-rpc that we add
>> method on server and client just calls it, client should also have
>> changed .ice file from server as well, to be able to call that new
>> method.
> Well, typically, the CPSRemoteController for ICE would handle  
> those .ice files.
> And maybe serve them to ICE client.
> From another point of view, we could also add to this component the  
> ability to serve the API using different protocols, like XML-RPC or  
> SOAP.

The .ice files must be treated as interfaces, and therefore part of  
the code. They must be deployed in the same way we deploy code.

>> In Ice there is "untyped invocation and dynamic dispatch"  
>> mechanism that
>> allows to live without interface definitions with Slice, but this  
>> model
>> will quickly become complex to program, as many issues developer will
>> need to care himself.
> ok.
>>> Issues :
>>> 	- Find a way to centrally manage all ICE components of our
>>> architecture (ICEBox ?)
>> Actually i think it's IceGrid + IceBox for more complex  
>> installation, by
>> this i mean Ice services spreading on different hosts or serveral  
>> IceBox
>> instances on the same host - for example, 1 instance for java  
>> services
>> and 1 for python services. IceGrid will let aggregate them into one
>> service or just monitor their status, both possibilities are useful.
> ok.
>
>> Not related to central management, but useful components:
>> Glacier2 - though it's router has special usage for bi-directional
>> communication between client and server, though it may be done  
>> manually.
> ok.
>
>> IceStorm - plugs when there is need for efficient publisher- 
>> subscriber
>> features (here at once comes in mind AFP project with cache  
>> modification
>> events sent to CNG clients)
> It need to be plugged with the event service.

The messaging bus needs to be plugged into the event service yes.  
We'll have to see if each message has its own transaction or if they  
can be aggregated.

>> One note:
>> Currently IceBox is not available for python, only C++ and Java.  
>> If this
>> becomes issue in future we can ourselves develop necessary small  
>> bridge
>> C++ code using Ice infrastructure to have IceBox for python.
> ok.
> Let's try to make it work with IceBox for now.

Or have ZeroC write it for us.

>>> 	- Find a way to bypass the zserver and allow direct invocation from
>>> ICE clients, without using a HTTP request
>> First thing that came to my mind was using Zasync like approach,  
>> ie ZEO
>> client, but i'm not sure at the moment how well it fits with what
>> Florent does on Zope side.
>> With Z3 there may be other possibilities like using Twisted and
>> publisher interface implementation, but this should be verified  
>> and we
>> are not on Z3 anyway.
> And the Z3 publisher cannot be used to access Zope3 and Zope2  
> products ?
> Florent ?

If we add more servers to Zope 2, using the current asyncore/meduse/ 
multithreaded approach is totally horrible.
Sidnei da Silva has a branch where he has done preliminary work on  
integrating the Zope 3 twisted server and publisher into Zope 2. We  
could try to merge that into Zope 2.10, this would give us the  
ability to easily add more servers using twisted's framework.

> By the way, as I got some negative feedback with the name  
> "CPSUniversal", I propose a new one "CPSRemoting".

CPSBridge?

Florent

-- 
Florent Guillaume, Nuxeo (Paris, France)   Director of R&D
+33 1 40 33 71 59   http://nuxeo.com   fg at nuxeo.com