% Implementation chapter

\svnInfo $Id$

\chapter{Implementation}
\label{ch:implementation}

In this chapter, we go into the implementation matter and describe solutions for
the most important tasks. The library code is rather described in the
chapter~\ref{ch:usage} showing how we should use the \vampeer{} library.

\section{Development environment}

During the design step, UML class and sequence diagrams were used by means of
the open source tool \emph{Umbrello} \cite{umbrello}, which laudably supports
diagram exports to a vector-based graphic format, namely EPS.

During development, we used \emph{EiffelStudio~5.7} \cite{eiffelstudio} and
therefore the ISE compiler. The new introduced configuration file format ECF
would be a mercy but it is quite cumbersome to integrate older libraries
because ECF is (currently) not widely accepted.

As we stumbled around a lot with networking stuff, we used \emph{Ethereal}
\cite{ethereal} (now renamed to \emph{Wireshark}) to sniff packets sent on the
network.

For testing, we used the standard Gobo tool \emph{getest}, which provides a very
easy and comfortable way to write unit tests.

\section{Used libraries}
\label{sec:impl_libraries}

\vampeer{} is of course using several software libraries. For the first, it is
obvious that we use \emph{Gobo} \cite{gobo}, in the current release~3.5. We need
it for data structures, date/time and its XML generation and parsing support.

As we are mainly dealing with remote peers, we need a networking library. We
looked for a simple solution and got to the \emph{EiffelNet} code
\cite{eiffelnet}.  Unfortunately, the library is not used by many people
resulting in an API which is not always useful for each task. Therefore, we
wrote an extension to it, see subsection~\ref{sec:impl_socketextensions}.

To support various independent tasks at the same time, we use the
\emph{EiffelThread} library \cite{eiffelthread} and therefore switch the
\texttt{multithreaded} flag on.

For the logging task, we make use of the great \emph{Log4E} tool \cite{log4e}
which is now part of the \emph{Goanna} project. It provides a synchronous
logging mechanism similar to Jakarta's well-known Log4J.

We provide ECF and XAce files for all used libraries in the \texttt{contrib/}
directory.

\section{Socket extensions}
\label{sec:impl_socketextensions}

The main problem we have with \emph{EiffelNet} is the missing support for
timeouts. Whenever we wait for network data, we do not like to wait eternally.
Generally, there are two kinds of interrupts in these situations in which we
like to quit the reading/waiting task: first, when a certain time has been
passed and second, when we get an internal request for closing the connection
such when the user application is shutting down.

Another lack in \emph{EiffelNet} is buffering. Usually, we like to assemble a
network message and send it as \emph{one} packet. The straightforward idea to
just use a string does not really work well because with binary data, we have
to append 8-, 16- or 32-bit integers (respecting the network byte order!)---this
results in code which we certainly never like to see here and there in our
application. Additionally, we sometimes have to know how large a buffer is (for
example to specify a message body length).

We therefore wrote a helper class \eclass{P2P\_SOCKET\_EXTENSIONS} which
provides exactly these features: timing and buffering. We did not choose to
create a socket heir class because the helper methods may be used for several
types of sockets: TCP and UDP. We provide methods for writing by using a buffer,
for reading and for some socket checks.

One may fill the buffer with strings and integers (from 8 to 64 bits, using
big-endian format). The buffer is just a string and may be adapted and used at
will. As soon as the buffer is sent, the buffer is emptied again.

For the read methods (returning a string or the various integer types), one has
to set a timeout first. They read and wait until either they got exactly the
expected amount of data, they pass the timeout or a given constraint has become
active. The last two cases raise an exception. Of course for strings, there is
also a method reading \emph{up to} a given data length (in many cases, we do not
know exactly how much data we are expecting).

The socket checks provide a method to wait until a connect request is
successful within the timeout and a method to check generally whether data is
available for reading.

The timeouts are implemented with the socket's non-blocking mode. So, the
socket's read command instantly raises an error when not enough data is
available. We catch it and put our thread to sleep but regularly check for new
data. When we have slept too long, we raise an exception to the user.

Because of the non-blocking mode, we get a lot of exceptions during a \vampeer's
run. So for development, we should disable exception handling for the following
exception types: 21 (I/O exception), 23 (Retrieve exception), 24 (Developer
exception) and 27 (Runtime I/O exception). Therewith, we are not bothered with
the network latencies.

The mentioned constraint is an agent that can be defined and which is called
every time before a thread waits. When the agent returns \texttt{True}, the
actual task is interrupted as described above.

Creating this socket extensions class, we have mastered a lot of networking
troubles at a single point.

\section{XML documents}
\label{sec:impl_xmldoc}

JXTA makes heavy use of XML documents. Not only every advertisement is in XML,
but also every higher level JXTA message. Therefore, we need a simple way to
parse and generate XML documents.

Figure~\ref{fig:impl_xmldocuments} shows our class structure for XML documents.
We hide the exact operation signatures for simplicity and do not list all
descendant classes.

\begin{figure}[h] \centering
\includegraphics[width=.8\textwidth]{xmldocuments}
\caption{XML document class hierarchy}
\label{fig:impl_xmldocuments}
\end{figure}

The class \eclass{P2P\_XML\_DOCUMENT} manages most handling with the \emph{Gobo
XML} interface. It is not an abstraction so that the underlying XML library could
easily be replaced; it rather provides helper methods just to simply build an
XML tree from elements, to get the content as a string and to build a tree
parsed out of a string. When parsing, we let Gobo build the full tree and
we afterwards provide callbacks for each root element and root attribute.

Its main client class is \eclass{P2P\_XML\_CACHE} (deferred) where all XML
document classes are inheriting from. It mainly declares central document
methods, such as creation, validation, output and element matching. It inherits
from \eclass{P2P\_DOCUMENT} which provides an interface for very generic
documents. The XML tree is not built until \texttt{out} is called the first
time. Further calls return a cached XML string unless any element has been
changed (\texttt{renew\_document} should always be called internally in
element setters).

To create a new XML document type, we just need to inherit from
\eclass{P2P\_XML\_CACHE}, define some setters/getters for the actual content
elements and implement \texttt{match}, \texttt{root\_element\_name},
\texttt{attribute\_handler}, \texttt{element\_handler} (to gather data
after the parsing process) and \texttt{document} (to create the XML elements).
We may also redefine \texttt{initialize} and \texttt{validate} and add
additional creation methods.

Whereas this interface is appropriate for general XML documents, we require an
additional interface for advertisements: \eclass{P2P\_ADVERTISEMENT}.  Each
advertisement has a unique ID used for the advertisements store in the discovery
service. It should also define its lifetime and the remote expiration time. For
SRDI, an advertisement should also define some elements which can be indexed.

Hence, we declare a unified interface for advertisements as shown in
figure~\ref{fig:impl_xmldocuments}. All advertisement classes (such as
\eclass{P2P\_PEER\_ADVERTISEMENT}) should effect it.


\section{Using UUID for JXTA IDs}
\label{sec:impl_uuid}

For \vampeer, we use JXTA IDs in the UUID format as it is the case in JXTA JSE.
Look at figure~\ref{fig:impl_ids} to get an overview of the ID class hierarchy.

\begin{figure}[h] \centering
\includegraphics[width=.75\textwidth]{ids}
\caption{ID class hierarchy}
\label{fig:impl_ids}
\end{figure}

\eclass{P2P\_ID} is the main interface for an ID. Such an ID can be parsed out
of and written as a URN string. It is comparable and hashable. Direct heirs that
can be instantiated are \eclass{P2P\_NULL\_ID} (for referencing no resource,
actually never used in \vampeer) and \eclass{P2P\_GENERIC\_ID} which may contain
any valid ID in any format.

We support the \texttt{uuid} format using the deferred class
\eclass{P2P\_ID\_UUID}. It parses the unique ID part and creates a byte array.
For each ID type (such as peer, peer group, codat ID\ldots), there is an
effecting class defining the byte interpretation. Figure~\ref{fig:impl_ids}
evinces only two of them. Each such type also defines the characteristic UUID
bytes, made available through the feature \texttt{uuid}\footnote{While the peer
group UUID in a peer ID is just meta information, the peer UUID is the
characteristic part.}.

\eclass{P2P\_UUID\_TOOLS} is actually not related to the ID class structure but
provides helper methods to create a random UUID and to parse/write a UUID. In
\vampeer, UUIDs are everywhere seen as \eclass{ARRAY~[NATURAL\_8]} with a
capacity of 16 bytes.

For peer group IDs, there is a special interface \eclass{P2P\_PEERGROUP\_ID}.
It is independent on the \texttt{uuid} since we use other formats as well for
peer group IDs, for example the \texttt{jxta} format for the net and the world
peer group. The \texttt{uuid} version \eclass{P2P\_PEERGROUP\_UUID} is therefore
an heir of two classes, \eclass{P2P\_PEERGROUP\_ID} and \eclass{P2P\_ID\_UUID}.

It may be confusing that the \texttt{jxta} peer group IDs also provide a
\texttt{uuid} feature. It is used to generate child peer group IDs because
children point to the parent group uuid. The following examples try to clarify
these ID relationships:\\

\noindent \emph{World peer group ID:}
{\footnotesize $\mbox{\texttt{urn:jxta:jxta-WorldGroup}}$\\
Implicit UUID:
\texttt{59616261646162614A78746150325033}\footnote{In ASCII, the 16 bytes stand
for ``YabadabaJxtaP2P3''. Actually, the last byte was an error; it should be a
``!'' but they looked up the wrong number in the ASCII table (decimal instead
of hex)\ldots}}\\

\noindent \emph{Private net peer group ID:}\\
{\footnotesize $\mbox{\texttt{urn:jxta:uuid-}}
\underbrace{\mbox{\texttt{822A7C9E6B804759870B81B10070E9C9}}}_{\mbox{Group UUID}}
\underbrace{\mbox{\texttt{59616261646162614A78746150325033}}}_{\mbox{WPG's characteristic UUID}}
\mbox{\texttt{02}}$}\\

\noindent \emph{Peer ID, member of the private net peer group:}\\
{\footnotesize $\mbox{\texttt{urn:jxta:uuid-}}
\underbrace{\mbox{\texttt{822A7C9E6B804759870B81B10070E9C9}}}_{\mbox{Group's characteristic UUID}}
\underbrace{\mbox{\texttt{2F3F01367359485B95D5C6CFA82B9775}}}_{\mbox{Peer UUID}}
\mbox{\texttt{03}}$}

\section{Threads}
\label{sec:impl_threads}

We spend an own section to the thread handling in \vampeer{} because we would
like to present which threads exist and because users have to pay attention to
some details.

Threads are an absolute must when dealing with the network. But threads are also
a source for many software bugs because we cannot rely on all contracts anymore
in a multithreaded environment. Other threads may change something in the course
of time. This is why we have to use locking mechanisms to lock other threads out
for a certain amount of time.

While we have \texttt{synchronized} environments in Java, Eiffel currently does
not support such a mechanism in the language itself\footnote{SCOOP will be a
good solution but it is still a research topic and the current version not
stable enough.}. We therefore have to use the possibilities from
\emph{EiffelThread} which provides read/write locks (for multiple readers, one
writer), mutexes (simple locking) and condition variables (for thread
synchronization).

To gain a solid peer application, we had to identify the shared objects between
the various threads and to introduce appropriate locks.  We wanted our
application also to terminate correctly which furthermore means to appropriately
destroy a lock at a single place. This took us a considerable amount of
time\ldots

In \vampeer, we are using at least four threads besides the main thread which
we will all describe now.

\paragraph{Main thread}

First, we will look at what the main thread is used for. It is the thread which
starts the platform and is in full control by the user application.

Thus, the entire platform starting is done and all the other threads are created
by the main thread. Since \vampeer{} does not provide an event loop, the user
application has to handle its main thread itself.

Usually, the main thread is also used to shutdown the platform.

\subsection{TCP Transport}
\label{sec:impl_threads_tcp}

Most of the threads are created by the TCP transport and its related classes.

\paragraph{Server thread}

The first thread created is the server thread. \eclass{P2P\_TCP\_SERVER} binds a
server port and listens for incoming connections. When a new connection is
accepted, it passes the socket to the main tcp class
(\eclass{P2P\_TCP\_TRANSPORT}) which will handle the connection with another
thread.

The server thread can be closed by calling \texttt{shutdown} (from another
thread). The socket will be cleaned up which causes the server thread to
terminate instantly.

\paragraph{Connection threads}

Each tcp connection gets its own thread. As soon as a socket is passed from an
incoming connection, or when a new socket has to be created, the connection
object \eclass{P2P\_TCP\_CONNECTION} launches a new thread.

The thread manages waiting during connection setup and waiting for incoming
messages via its socket. It is important to know that message processing (for
received messages only) is handled by the connection thread. This means, a
service should never do extensive work in a processing agent.

It is therefore not possible for a message handler to wait for another message
retrieved with the same connection. However, sending messages is allowed in such
a handler (see next paragraph).

A connection thread terminates itself after a certain time but can of course
also be destroyed by calling \texttt{close}.

\paragraph{Message queue thread}

Because services should not be affected by the slow message delivery, outgoing
messages are queued. The calling service thread returns instantly and without
result as soon as the message has been successfully stored in our message queue.

\eclass{P2P\_TCP\_TRANSPORT} manages a message queue thread which waits for
and gets triggered on new messages. It then looks up a possible existing
connection or tries to open a new one. As soon as a connection state is in a
valid mode, it sends the message and removes it from the queue. It tries several
times when the dispatching fails.

\paragraph{} Summarizing, we have two fix threads for the TCP transport plus a
thread per connection.

In suspend mode, all connections are closed and the server as well as the queue
manager thread are stopped.

\subsection{Rendezvous connection manager}
\label{sec:impl_threads_rdv}

The rendezvous service owns another thread. It is only used to stay in contact with
rendezvous servers meaning to renew connection leases. Hence, this thread is
mostly sleeping.

The thread is launched when the service is started and is stopped when the
service gets suspended. During start, we have a bootstrapping problem because
not all services are started yet and the system may only partially be
functional. We cannot fire off sending messages in this state and have to wait
until the platform is started entirely.

That is the reason why the peer group is offering a method
\texttt{when\_fully\_started} returning \texttt{True} as soon as all group
services are started successfully or \texttt{False} on a failure. The rendezvous
connection manager thread therefore calls this method before it begins to
contact a rendezvous server to get a lease.

\subsection{Discovery SRDI}
\label{sec:impl_threads_srdi}

The discovery service creates a thread to manage SRDI pushes. It regularly looks
for new local advertisements and propagates its index changes.

Like the rendezvous connection manager, it waits with index pushing until the
platform has been started fully. The thread is stopped also in suspend mode.

\section{Advertisement storage}
\label{sec:impl_adv}

Probably one of the most important part in the entire platform is the
advertisement storage. Services and user applications should not have to store
advertisements itself but should be able to access a central storage.

That is what the discovery service provides with its local query methods. We
would like to describe its advertisement storage here. The storage can be
divided into two parts, persistent and memory storage.

\subsection{Persistent storage}

When starting the platform, a configuration directory must be declared. This is
used to store the platform configuration and some advertisements.

\begin{lstlisting}[language=, caption={Persistent storage directory layout},
label=lst:impl_persistentcache]
|-- Modules/
|   |-- jxta:uuid-DEADBEEFDEAFBABAFEEDBABE000000010206.xml
|   |-- jxta:uuid-DEADBEEFDEAFBABAFEEDBABE000000020106.xml
|   |-- jxta:uuid-DEADBEEFDEAFBABAFEEDBABE000000030106.xml
|   |-- jxta:uuid-DEADBEEFDEAFBABAFEEDBABE000000060106.xml
|   |-- jxta:uuid-DEADBEEFDEAFBABAFEEDBABE000000080106.xml
|   |-- jxta:uuid-DEADBEEFDEAFBABAFEEDBABE000000090106.xml
|   `-- jxta:uuid-DEADBEEFDEAFBABAFEEDBABE0000000B0106.xml
|-- jxta:jxta-NetGroup/
|   |-- Advs/
|   |-- Peers/
|   |   `--jxta:uuid-59616261646162614E504...3D4C31803.xml
|   `-- PeerGroup.xml
|-- jxta:jxta-WorldGroup/
|   |-- Advs/
|   |-- Peers/
|   |   `--jxta:uuid-59616261646162614E504...3D4C31803.xml
|   `-- PeerGroup.xml
`-- PlatformConfig.xml
\end{lstlisting}

The file hierarchy layout is very simple as shown in
listing~\ref{lst:impl_persistentcache}. Mainly, for each peer group there exists
a directory containing its group advertisement, one directory for general and
one for the peer advertisements. Each advertisement is stored as a single XML
file with its shortened unique ID as file name. For the module implementation
advertisements, a special modules directory is maintained.

The persistent cache is only used when the platform is loaded. During the
initialization process, configuration will be read from disk or it will be
created and stored. When the configuration directory is not present, it will be
created.

All the advertisements will be loaded into memory once the discovery service has
been started. This enables the peer application to have advertisements
permanently available.

The class \eclass{P2P\_CACHE\_MANAGER} is responsible for accessing and managing
the persistent cache. As soon as a platform instance is disposable, the cache
manager is ready for access too.

However, the cache currently suffers from a drawback: It cannot handle
advertisement lifetimes and expiration times. A stored advertisement will never
be discarded because the lifetime is not part of the XML document itself and
therefore cannot be stored to disk. That is the reason why new advertisement are
never written back to the cache. Of course, the peer application may itself
write some chosen advertisements to disk using the cache manager.

The persistent cache is thus only used for really permanent advertisements. By
default, these are the module implementation, the peer group and the own peer
advertisement(s).

\subsection{LRU cache}

The memory cache is implemented with a \emph{Least Recently Used} (LRU) cache in
the class \eclass{P2P\_ADVERTISEMENTS\_LRUCACHE}. It is inspired by the JXTA
JXME project which is used for mobile applications with limited memory.

The LRU cache may have a maximal capacity so that the oldest, never used
entries are discarded. Furthermore, expired advertisements will be removed
automatically.

The discovery protocol is designed to always differ between peer, group and
other types of advertisements. We therefore have three LRU cache objects defined
in the discovery service, one for each such type.

Advertisements can be accessed by their unique ID or by a key/value search which
make use of the advertisements' \texttt{match} method. It is also possible to get
a number of random advertisements (or all of them).

Advertisements are identified by its unique ID defined in the advertisement.
When creating a new advertisement type, it is important to define a good scheme
for the unique ID. While it is not always possible to create a unique ID because
the advertisement information is too common, we have to be aware that an
advertisement cannot be stored in the cache without having a unique ID.

The cache itself provides also the possibility to extract an SRDI for all or for
advertisements as of a certain point in time. For this, it uses the
advertisements' \texttt{index\_elements} method.

\paragraph{No capacity limits}

There is a general drawback regarding the limited capacity in the cache in
combination with the minimal implementation of the endpoint router.

Because the endpoint router is not able at the moment to automatically query for
route advertisements\footnote{The router needs the recipient's route
advertisement in order to resolve the peer ID to a transport address.}, the
advertisements cache gets useless when it removes seldom used peer and route
advertisements from time to time. After all, the peer application should not
always have to check whether the recipient's route advertisement is available or
not. It should be possible to query once for a peer advertisement\footnote{The
peer advertisement contains also the peer's route advertisement.} and then to
just send messages without worrying about the route.

Therefore, we have abolished the capacity limit for the moment. Of course, the
challenge with advertisement expiration remains. The peer application has to
guarantee that advertisement updates are published regularly to the peer group.

\section{Shared creators}
\label{sec:impl_creators}

There are some important creators (also called factories) in \vampeer\@. All
creators can be accessed using the class \eclass{P2P\_CREATORS\_SHARED}:

\paragraph{ID creator}

The ID creator represented by \eclass{P2P\_ID\_CREATOR} provides globally unique
instances for the null, NPG and WPG ID and provides the possibility to parse an
ID string and to create the appropriate ID object out of it.

It knows and can handle the \texttt{jxta} and \texttt{uuid} format types. Custom
creators may be registered. Such an agent gets a generic ID and has to return a
specific ID object or \texttt{Void} if the ID is not recognized. The custom
creators can override the \vampeer's types.

\paragraph{XML document creator}

Specific XML documents are also derived with a shared creator:
\eclass{P2P\_XML\_DOCUMENT\_CREATOR}. There are mainly two methods to
create XML documents. Both expect the XML root element.

The first method \texttt{document\_from\_element} returns a
\eclass{P2P\_XML\_CACHE} object or \texttt{Void} whenever the creation was not
successful. If the document type cannot be detected, it returns a valid
\eclass{P2P\_UNKNOWN\_XML\_DOCUMENT}.

It is used by the second method \texttt{advertisement\_from\_element} which
simply tries to cast the result to an advertisement
(\eclass{P2P\_ADVERTISEMENT}).

The document creator is also useful for user services as custom document types
can be registered too, similar to the ID creator.

\paragraph{Wire message creator}

The last creator is used for the endpoint message's wire representation.
Currently, there is only one mime type used (\texttt{application/x-jxta-msg}).
Thus, the wire representation is always in binary format, handled with
\eclass{P2P\_WIRE\_MESSAGE\_BINARY}.
