Channel User Manual

Yigong Liu  (2/25/2006)


1. Design and Architecture
        1.1. Application domains characteristics
        1.2. Design aspects of message passing and event dispatching applications
        1.3. Solution
2. Sample applications and generic development procedures
        2.1.
Simple send/recv applications
        2.2. Generic procedures
3. Frameworks classes
        3.1. Channel centered framework classes
        3.2. Connector centered framework classes
4. Channel specialization
        4.1. Use application specific IdType
        4.2. Use application specific Routing Algorithms
5. Connector specialization
        5.1. Transport
        5.2. Filters, Translators
6. Message/event handling
        6.1. Message creation
        6.2. Message destroy
        6.3. Message forwarding and distribution
7. Install and build

1. Design and Architecture

1.1. Application domains characteristics

    Therer is a common theme in today's software applications: interactions - they are interacting with human, with the external environment and with each other. Interacting software applications are mostly asynchronous (possibly distributed), driven by events or messages. Most of them follow one of the following models. 
Large asynchronous systems involving multiple resources (processes, computers...) mostly employ distributed message passing design. Various application domains exhibit different requirements and tradeoff for performance, space, and maintainance:
event dispatching systems are applied in single threaded systems for a flexible system structure to handle asynchronous external events. The examples are :

1.2. Design aspects of message passing and event dispatching applications

Although message passing applications and event dispatching applications appear quite different, internally they share a common core. The design of most message passing and event dispatching systems can be separated into the following "orthorgonal" aspects:
An ideal solution should allow flexible mixing and matching various implementations of the above aspects to create a best-fit message passing or event dispatching facility customized to the specific application characteristics.

1.3. Solution

Channel is a C++ template framework to facilitate users to create the best-fit publish-subscribe message passing or event dispatching architecture:
  template 
<
class Id_Type,
class Id_Trait = IdTrait<Id_Type>,
class SynchPolicy = ACE_MT_SYNCH,
class AllocPolicy,
class Router = MapRouter<Id_Type, Id_Trait, SynchPolicy, AllocPolicy>
>
class Channel
By specializing the above template with appropriate message id types and routing algorithms, Channel can support table/linear message routing, hierarchical routing and (in future) associative routing (Linda style) at the demand of user, making the message passing and event dispatching as straightforward to use as STL data structures.
Channel is built on top of ACE (Adaptive Communication Environment) to achieve platform independence.


2. Sample channel based send/recv applications

2.1. Simple send/recv applications

2.2. Generic procedures

The following are the generic procedure to develop channel based applications:
<1> Partition the application into threads and/or callbacks. Clarify the messaging/event interface among threads and/or callbacks: what messages/events will be passed.
<2> Specialize channel template for the application. Choose appropriate Id_Type, Id_Trait and Routing algorithms
<3> Define the message/event ids and data structures
<4> Application logic flow:
<A> create an instance of specialized channel
<B> connect channel to remote channels for distributed applications, using proper transport mechanisms such as tcp socket, unix domain socket, or others.
<C> create channel publishers and subscribers
<D> start application logic/threads to send/receive/process messages/events

3. Frameworks classes

There are 2 co-operating sub-frameworks inside. The first is centered around class Channel for setting up publish-subscribe namespace and message/event routing. The 2nd is centered around class Connector for remote channel connection, transport, and message marshaling/demarshaling. The joint among these 2 sub-framework is class Interface which is a publisher/subscriber in Channel sub-framework and connection endpoint in Connector framework.

3.1. Channel centered framework classes

Channel centered sub-framework involves the following major classes:
For each routing algorithm, there are associated IdTrait classes: LinearIdTrait, HierarchicalIdTrait.
Use STL map as the data structure to implement a table (key matching) based (linear) routing algorithm.
Use a tree data structure similar to "trie" to implement hierarchical (partial pathname / prefix matching) routing algorithm
Channel centered framework classes are mostly platform independent. The platform dependent parts are thread synchronization primitives and memory alloc/free; which has been (or will be) wrapped in 2 template policy classes: SynchPolicy and AllocPolicy. Right now ACE's  SynchTrait is used for SynchPolicy. While instantiating Channel template for applications, user can specify ACE_MT_SYNCH or ACE_NULL_SYNCH for multithreaded or single-threaded applications.

3.2. Connector centered framework classes

Connector centered sub-framework includes the following major classes:
Application code can create binders with customized filters and translators to implement application specific policy and register it with connector for an external address.

4. Channel specialization

4.1. Use application specific IdType class

    To make sure application specific Id type working with Channel, class Id_Type and class Id_Trait should define:
    For normal systems based on table/linear routing, Channel specialization is as simple as Channel<Id_Type>.

4.2. Use application specific Routing Algorithm

To use application specific routing algorithms with Channel framework, the application router class should provide the following methods:
    Status publish_msg(IdType t, PubSub_Scope s, Source *src)
    Status unpublish_msg(IdType t, PubSub_Scope s, Source *src)
    Status subscribe_msg(IdType t, PubSub_Scope c, Destination *dest)
    Status unsubscribe_msg(IdType t, Destination *dest)
    Status route_msg(Msg *msg, Member_Type src_type, PubSub_Scope pub_scope, timeout=0).
For the performance of message/event routing inside Channel, Source and Destination will maintain its local namespace and perform id validation locally. Possibly application specific router will implement a very different namespace which may require customized version of Source and Destinations. A namspace type enum and template partial specialization is used to make sure correct versions of Router, Source and Destination are used together.

5. Connector specialization

5.1. Transport

5.2. Filters, Translators


6. Message/event handling

6.1. Message creation

Many message passing systems require that messages are dynamically  allocated and message class/structures must inherit a base class with a virtual destructor so that all messages can be cleaned up properly.
In Channel, messages creation are flexible:
Messages in Channel don't need to inherit from a common base class. Internally application messages will be contained inside the generic message holder: class Msg which will avoid message copying and destroy message correctly in the end.

6.2. Message destruction

Message destruction in channel applications are involving:
since channels publishers & subscribers may be threads or callbacks in different DLLs, messages may be created in one DLL and freed in another
Solution: use MsgFreeCallback:

6.3. Message forwarding and distribution

Since channel is pub/sub channel and there are possible multi subscribers to the same message type in different threads, we need a way to avoid copy/duplicate messages to send to different subscribers.

Solution: using reference counting for all subscribers to share the same copy of message
subscribers must obey the following convention for sharing to work:

7. Install and build

7.1. setup

   1.1 install the latest version of ACE (Adpative Communication Environment)
       The Channel lib is built on top of ACE for platform abstractions
   1.2 download the Channel lib (.tar.gz file) and unpack it
    under $CHANNEL_ROOT
   1.3 set up env
          add the following to .bashrc:
#for Channel lib
export CHANNEL_ROOT=
export CHAN_ROOT=$CHANNEL_ROOT/channel
export LD_LIBRARY_PATH=.:$CHAN_ROOT/lib:$LD_LIBRARY_PATH
export PATH=$CHAN_ROOT/bin:$PATH
#for CVM
export CVM_ROOT=$CHANNEL_ROOT/cvm
export LD_LIBRARY_PATH=.:$CVM_ROOT/lib:$LD_LIBRARY_PATH
export PATH=$CVM_ROOT/bin:$PATH

7.2. build:

    cd $CHANNEL_ROOT
    make

7.3. run samples

      - in console 1:
    cd $CHAN_ROOT/examples/hier_text_recv
    hrecv 6666
      - in console 2:
    cd $CHAN_ROOT/examples/hier_text_recv
    hsend 6666