CCTL Manual =========== CCTL is provides a set of transport layer services to CCF and facilate program development of ``group aware'' applications. CCTL implements two abstractions: 1. session 2. channel A session is a heavy weight group abstraction used to maintain membership information and a channel is a light weight group abstraction used to implement transport level services. A session can have multiple channels. A multicast group as we understand in multicast communication (e.g., IP- multicast) is a channel group in CCTL. The typical calling sequence needed for a process to join a ``multicast group'' are the following: 0. Set the environment variable "CCTL_WP_HOST_NAME" to the host running a white page server. 1. process calls cctl_init() to initialize CCTL data structures and internal CCTL send/receive threads cctl_init(debug_level); Use debug_level = 0 if you don't want debug messages. 2. create/join a session with cctl_create_session(). If you are the first one that make the cctl_create_session() call, the session will be created. Subsequent calls to cctl_create_session() will effectively JOIN the given session but will not create a new one. The complete syntax for all CCTL calls is detailed below. For example: sess_id = cctl_create_session("my-session", 0, NULL); joins the session "my-session". Return code sess_id is ZERO if the call is successful, otherwise it returns a negative value error code. (Error codes are documents in CCFcctl/include/cctl_ecode.h 3. create/join a channel within the session using cctl_create_channel(). Again, if you are the first one that make the cctl_create_channel() call, the channel will be created. Subsequent calls to cctl_create_channel() will effectively JOIN the given channel but will not create a new one. Cctl_create_channel returns a positive channel id if the call is successful. If not, it returns a negative error code. At creation, you may specify a quality of service. For example, the following call will create/join a reliable channel: chan_id = cctl_create_channel("my-channel",CCTL_RELIABLE,NULL); The following qualities of service are implemented: CCTL_UNRELIABLE, CCTL_RELIABLE and CCTL_ATOMIC CCTL_UNRELIABLE guarantees that messages are received in FIFO (not necessary consecutive order). But the messages can be lost. CCTL_RELIABLE guarantees that messages are delivered to every process in FIFO order in the channel to which the messages are sent as long as the prcess is reachable and non-faulty. CCTL_ATOMIC guarantees that messages are gloabally ordered with respect to every message sent by every process in the channel regardless of who sent the messages. It also guarantees FIFO. But it does not handle message fragmentation so you cannot send more than 8000 bytes of information. 4. Use cctl_send() to send RELIABLE messages to a channel group. Messages can be sent synchronously or asynchronously. In synchronous transmission, cctl_send() will return only when all acknowledgements are received. The usage is: int cctl_send(int chan_id, int dest, int msg_size, char *msg, int mode, int tag) The call will return 0 if it is successful, and otherwise it returns a negative valued error code. Messages can be sent with a given tag. A ZERO tag means wildcard. For example, the following call will send a message "msg" of size "sz" to every member of the channel group "chid" (channel id) synchronously: err = cctl_send(chid, CCTL_ALL, sz, msg, CCTL_SYNC, 0); 4a. We have optimize the CCTL_UNRELIABLE service to avoid one copy step. In order to use the CCTL_UNRELIABLE service, the buffer space passed to cctl_usend() must be obtained with cctl_get_buf(). So, to send unreliable messages, you need to do: char *bufptr = cctl_get_buf(MSG_SIZE); ... /* fill the buffer (upto MSG_SIZE bytes) */ ... cctl_usend(chid, dest, bufptr, MSG_SIZE, 0); You must also use a buffer obtained through cctl_get_buf() to in unreliable receive. 5. Use cctl_recv() to receive message reliably and cctl_urecv() for unreliable messages. Syntax: int cctl_recv(int chid, int *src, char *msg, int bufLen, int *msgLen, int flags, int srcTag, int tag, int *returnedTag); You can specify to receive message from a specific source with "srcTag" (use ZERO to specify from any source). If "tag" is non-zero, then cctl_recv() will receive only messages with that given tag. Cctl_recv() returns the number of bytes in the message. 6. Use cctl_leave_channel(chan_id) to leave a channel group. 7. Use cctl_leave_session(sess_id) to leave a session. ============================================================================= CCTL API (see CCFcctl/include/cctl.h) ============================================================================= /* * CCTL_INIT: It sets up cctl (open sockets, getting wp addresses, etc). * it has to be called once for each process that uses * cctl functions, before the process makes any cctl calls. */ void cctl_init(int i); /* * CCTL_CREATE_SESSION (CCTL_JOIN_SESSION): it creates session "sname". * you have to create or join a session before creating a channel. * * sname: session name * mode: private or public (not supported yet.) so use 0 * flags: not supported. Use 0. * h: callback function: not supported: use NULL */ int cctl_create_session(char *sname, int mode,int flags, void h(void *)); /* * CCTL_LEAVE_SESSION: It leaves a session. Before leaving a session, * please make sure you leave all the channels you have joined. */ int cctl_leave_session(); /* * CCTL_CREATE_CHANNEL (CCTL_JOIN_CHANNEL): it creates a channel or * joins an existing channel. A channel is a multicast pipe used * among session members (i.e., ones joined a session) to multicast * messages. * * chname: channel name. * qos: CCTL_FAST_RELIABLE, CCTL_FAST_ATOMIC, CCTL_UNRELIABLE * CCTL_FAST_RELIABLE: relaible fifo delivery * CCTL_FAST_ATOMIC: total ordering delivery * CCTL_UNRELIABLE: unreliable (real-time) delivery. * flags: CCTL_OTHERS_ONLY, CCTL_WRITE_ONLY * CCTL_OTHERS_ONLY: don't deliver my msgs to me. * CCTL_WRITE_ONLY: don't deliver any msgs to me. * h: callback functions (not implemented). * * Return value: channel id if successful, otherwise negative numbers. * error codes can be found in cctl_ecode.h. */ int cctl_create_channel(char *chname,int qos,int flags, void h(void *)); /* * CCTL_LEAVE_CHANNEL: it leaves a joined channel. * * chid: channel id. * * Return value: 0 if successful. otherwise negative numbers. * error codes can be found in cctl_ecode.h. */ int cctl_leave_channel(int chid); /*------------------------------------------------------------------------ * CCTL_RECV: it receives a channel message. * * chid: channel id * src: the session id of the sender. * msg: the message buffer. * bufLen: the size of message buffer. * msgLen: the size of the message returned if CCTL_STREAM is not set. * the total size of the message if CCTL_STREAM is set: meaning * that it is possible that if CCTL_STREAM is set, the message * can be truncated, in which case, msgLen returns the size of * the entire message. * flags: logical or of any combination of the following bits: * CCTL_ASYNC, CCTL_SYNC, CCTL_STREAM * CCTL_ASYNC: asynchronous receive: it returns even if there is * not a message in the buffer. * CCTL_SYNC: it returns only when there is the "expected" message * in the buffer. "expected" is determined by * srcTag and tag. * srcTag: the src id that the function expects to receive a message from. * if it is a positive number (other than CCTL_ALL), then it returns * only with a message from srcTag when CCTL_SYNC is set. * If you want to receive from any session member, set it to * CCTL_ALL. * tag: the tag of the message that is expected to be received. * if it is set to an integer bigger than 0 and CCTL_SYNC, it waits * for a message with the tag. * returnedTag: the tag of the message received. ------------------------------------------------------------------------*/ int cctl_recv(int chid, int *src, char *msg, int bufLen, int *msgLen, int flags, int srcTag, int tag,int *returnedTag); /* -------------------------------------------------------------------------- * CCTL_SEND: it sends a message to a channel. * * chid: channel id. * dest: the dest member id if there is one. If dest is a positive integer, * which indicates a session member id, then it sends a point to * point message to the member. Other members does not get the msg. * If the message is intended for all others, then set it to * CCTL_ALL. * msg: the message to be sent. * len: the size of message. * flags: CCTL_FLUSH: it triggers the message to be sent immediately. * otherwise: 0. * tag: the tag of the message. 0 means no tag. --------------------------------------------------------------------------*/ int cctl_send(int chid, int dest, char *msg, int len, int flags, int tag); /* * CCTL_SELECT: it allows appl. to block for multiple channels as well as * multiple sockets (TCP or UDP). * fdset and nfds are the same parameters being passed to a unix select call. * To set fdset, you use FD_SET, FD_ZERO, FD_CLR, etc. "nfd" contains the * maximum socket descriptor that you listen to those in fdset. * * chids is the set of channel ids that you listen to. chids can be set * by CHID_SET, CHID_ZERO and CHID_CLR. nchids is the maximum channel id * in chids. * * cctl_select returns the number of sockets and channels that * are currently having messages to be received. * * When cctl_select returns, fdset and chids contain the bit vectors * for the sockets and channels with messages to be read. To see whether * a socket has a message, use FD_ISSET. Likewise, to see whether * a channel has a message, use CHID_ISSET. * exmaple usage: in ~ccf/cctl/README */ int cctl_select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptionfds, int nchids, cctl_chid_set *chids, struct timeval *timeout); /* * CCTL_GET_MYID: returns my session id. */ int cctl_get_myid(); /* * CCTL_GET_CH_MBRSHIP: returns the current membership in terms of * bit vector. The postions of bits set in * mbrship_mask corresponds to the member ids * of the channel. * * chid: channel id. * mbrship_mask: the membership mask * sz: the size of membership mask provided. */ int cctl_get_ch_mbrship(int chid,int *mbrship_mask, int sz); /* * CCTL_FLUSH: send immediately all the messages in the channel. * * chid: channel id. */ int cctl_flush(int chid); /* * cctl_urecv, cctl_usend, cctl_get_buf, cctl_free_buf * * Implementing unreliable send and receive. * CCTL avoids copying by pre-allocating a buffer containing a CCTL packet * header using cctl_get_buf(). The cctl_get_buf() call returns a pointer to * the data area of the CCTL packet to the user. */ char *cctl_get_buf(int sz); int cctl_free_buf(char *buf); int cctl_urecv(int chid, int *src, char *msg, int len, int flags); int cctl_usend(int chid, int dest, char *msg, int len, int flags); /* * Making * cctl_join_session == cctl_create_session * cctl_join_channel == cctl_create_channel */ #define cctl_join_session(sname,mod,flags,h)\ cctl_create_session(sname,mod,flags,h) #define cctl_join_channel(chname,qos,flags,h)\ cctl_create_channel(chname,qos,flags,h)