% JXTA chapter

\svnInfo $Id$

\chapter{JXTA}
\label{ch:jxta}

Before we describe the \vampeer{} design, we give an introduction to the JXTA
world. We show how the JXTA protocols are designed and present a specifications
overview.

The full specification \cite{jxta_spec}, which is available online, mainly
covers the basic ideas around JXTA and specifies nearly only the messages which
go over the wire. For more semantical details, we recommend further literature
or to look at the reference implementation source code.

The paper \cite{jxta_virtualnetworks} focuses on the newer release 2 of the
JXTA's protocols and gives a good overview. The free book by Brendon Wilson
\cite{jxta_wilson} explains JXTA with many Java examples whereas
\cite{jxta_brookshier}\footnote{\url{www.samspublishing.com} provides a free
sample chapter: ``Java Implementation of JXTA Protocols''} goes more into the
details of the Java reference implementation. Unfortunately, the books are
slightly out of date.

We will first look at peer groups, define JXTA IDs and advertisements and then
introduce the different services and protocols available. Afterwards, we present
the overall \ptp{} network infrastructure. We do not remain only at the JXTA
specification level but go further and show a few design ideas used in JXTA JSE,
the Java reference implementation\footnote{JXTA JSE is currently available in
version 2.4.1\@, see \cite{jxta_platform}. The next release 2.5 follows in March
2007.}.

\section{Peer groups}
\label{sec:jxta_peergroups}

A peer group is a compound of peers agreeing to run the same set of
services\footnote{A service is a set of features following a specification
either made by the JXTA project or the user, see
section~\ref{sec:jxta_services}.}. When a peer joins a peer group, all services
needed should be loaded according to the peer group's specification.  Thus, a
peer is always a member of at least one group as there would not be any running
services at all otherwise.

A peer may belong to more than one peer group though. The super peer group which
is loaded first is \emph{usually} the world peer group (WPG)\@. All other groups
are direct or indirect children of the WPG\@. This is because only one peer
group can actually handle the network traffic. The specification does not
explicitly mention a parent-child relationship among peer groups but it is
handled that way in JXTA JSE and also in \vampeer\@.

\paragraph{World Peer Group} The WPG is defined as the peer group in which all
JXTA peers reside, even if they are not communicating with each other. The WPG
is somewhat a special peer group which is automatically loaded and may not
support all services. In the C implementation, there is actually no explicit
WPG whereas \vampeer{} uses one for configuration purposes only.

The WPG's ID\footnote{See section~\ref{sec:jxta_ids} for an introduction to
JXTA IDs.} reads \texttt{urn:jxta:jxta-WorldGroup}.

\paragraph{Net Peer Group} The WPG's only direct child is the net peer group
(NPG)\@ which is now a true normally running peer group configured with all JXTA
services.

When talking about the NPG, we usually mean the \emph{public} net peer group
with the ID \texttt{urn:jxta:jxta-NetGroup}. It should only be used for
development and testing purposes (as long as no other group has been created).
Sun provides a public infrastructure for this peer group---unfortunately, the
servers have been unavailable or under heavy load for the last couple
months\footnote{This downtime being cumbersome, it is no problem as one can
easily setup an own rendezvous/relay server, see
subsection~\ref{sec:jxta_rendezvous}.}.

When using JXTA as a framework for a custom \ptp{} application, one should
create a new \emph{private} NPG so that the application will not get in contact
with peers from other applications\footnote{This is not a guarantee however;
every JXTA peer may connect as long as it knows the address and has a
purpose.}. It has its own network.

\paragraph{} Other peer groups are children of the NPG. Although services can be
shared among peer groups, one should have good reasons to split the
application into several groups as peer communication is only possible within
the same group.

\section{IDs}
\label{sec:jxta_ids}

We already mentioned IDs for peer groups. Also other entities in JXTA have an
ID: peers, modules, advertisements and other resources. A JXTA ID must be a
complete identifier referring to a unique resource.

JXTA IDs are in URN format (see \cite{rfc2141}) with the namespace
\texttt{jxta}.  Additionally, the URN namespace specific string is prefixed with
a format ID announcing how the ID is formatted. The general form looks like
this: \texttt{urn:jxta:format-specificid}

Although the format is written explicitly, one should never make any assumptions
about the ID format. Of course, it is allowed to optionally gather some
information from an ID when the format is recognized.

\paragraph{UUID format} Most IDs are in the JXTA \texttt{uuid} format. They are
in hexadecimal form representing 1 up to maximal 64 bytes. The last byte (the
last two hex digits) always specifies the type. Each type stands for its own
bytes schema. Usually, the number contains one or two UUIDs\footnote{A
``Universally Unique Identifier'' (UUID) is a random number (containing also
time information) meant to be universally unique (the probability to create two
same UUIDs in the same context is very small). They are fully defined in
\cite{isoiec9834-8}. You may more like the corresponding RFC \cite{rfc4122}
which describes also the UUID format but focuses more on UUID as a URN
namespace.}, each 16 bytes long.

Table~\ref{tab:uuid_types} shows the defined types for the \texttt{uuid} format
and all the contained information.

Sample IDs look like this:

\begin{itemize}
\item Peer group ID:\\
{\small \texttt{urn:jxta:uuid-822A7C9E6B804759870B81B10070E9C9$\backslash$\\
59616261646162614A7874615032503302}}
\item Module class ID:\\
{\small \texttt{urn:jxta:uuid-261F502615134AA99FDC99E3751E6B8505}}
\end{itemize}

\begin{table}[h]
\begin{center}
\begin{tabular}{|l|l|l|}
\hline
\emph{ID byte} & \emph{Type name} & \emph{Information contained} \\
\hline
01 & Codat ID & Group UUID, Codat UUID, Codat Hash \\
02 & Peer group ID & Group UUID, Parent group UUID \\
03 & Peer ID & Group UUID, Peer UUID \\
04 & Pipe ID & Group UUID, Pipe UUID \\
05 & Module class ID & Module UUID \\
06 & Module specification ID & Module class UUID, Specification UUID \\
\hline
\end{tabular}
\end{center}
\caption{UUID ID types in JXTA IDs}
\label{tab:uuid_types}
\end{table}

\section{Advertisements}
\label{sec:jxta_advertisements}

Another important and widely used term in JXTA is \emph{advertisement}. It is an
XML document advertising any kind of resource. It does not actually contain the
real data, but only metadata. The JXTA protocols are then used to transport
and share advertisements with other peers.

For peers, peer groups and all the other JXTA entities, there are advertisement
schemes defined. It is the JXTA way of creating new types of advertisements for
each used entity. In a file sharing application for example, one would possibly
create a file advertisement containing a codat ID\footnote{A codat is just a
container for any kind of data, for example file content.}, the owner's name,
the creation date and the peer's ID that hosts the file.

An advertisement has an expiration time, so no old advertisements may be passed
around. As advertisements cannot be withdrawn or deleted on remote peers, one
should not set the expiration time too high; the default expiration is two hours,
but implementations may vary. When resources are valid for a longer time than
the expiration time, the advertisement has to be recreated and published again.

We would like to show two advertisement types so that you may better understand
its purpose:

\paragraph{Peer Advertisement} One of the most important advertisements is the
one describing a peer. This is used for example to discover peers. It contains
the peer ID, the peer group ID and an optional peer name as well as a
description. Each service may additionally add a service parameter with some
special configurations.

Listing~\ref{lst:jxta_peeradv} shows a sample peer advertisement. The service
parameter hosts the configuration for the endpoint service: a route
advertisement that advertises the physical endpoint address (IP host and TCP
port).

\begin{lstlisting}[language=XML, caption={A sample peer advertisement for a peer
in the public NPG}, label=lst:jxta_peeradv]
<?xml version="1.0" encoding="UTF-8"?>
<jxta:PA xmlns:jxta="http://jxta.org">
  <PID>urn:jxta:uuid-59616261646162614E50472050325033\
    1A227980E5924E80A3FD8ECD73D4C31803</PID>
  <GID>urn:jxta:jxta-NetGroup</GID>
  <Name>My sample peer node</Name>
  <Desc>Development test peer</Desc>
  <Svc>
    <MCID>urn:jxta:uuid-DEADBEEFDEAFBABAFEEDBABE0000000805</MCID>
    <Parm><jxta:RA><Dst>
      <jxta:APA><EA>tcp://129.132.105.170:32725</EA></jxta:APA>
    </Dst></jxta:RA></Parm>
  </Svc>
</jxta:PA>
\end{lstlisting}

\paragraph{Peer Group Advertisement} The peer group advertisement announces the
existence of a group. It contains the group ID, the module specification
ID\footnote{See section~\ref{sec:jxta_services} for an introduction to modules
and their specifications.} and optionally again a name, description and service
parameters.


\section{Services}
\label{sec:jxta_services}

Until now, we have just introduced \emph{peer groups} and the general terms
\emph{ID} and \emph{advertisement}. In this section, we will look at
\emph{services} and what JXTA protocols they are providing.

JXTA is very modular. Each feature or protocol is available as a service module.
Thus, a peer group can disable unneeded modules.

\paragraph{Module definition}

JXTA modules can be loaded dynamically. This means that a JXTA peer could
theoretically load module code from another peer. For specifying and identifying
modules, there exist several module advertisements:

A class of modules providing the same local behavior and API is identified by a
unique \emph{module class ID} (MCID) and announced with a \emph{module class
advertisement}. 

Specifications for a module class are identified with a \emph{module
specification ID} (MSID). Its \emph{module specification advertisement}
includes a version number and a URI where a human-readable description can be
found.  The specification focuses on the remote behavior and the protocol.

All modules implementing a module specification can be advertised with a
\emph{module implementation advertisement}. This specifies the targeted
environment and it may provide the entire code or only a package name and a
description where the code can be fetched.

The three module definition layers allow to have various specification versions
for a single module class and also any number of module implementations for
each environment.

\paragraph{}

We will now present the main services we have focused on in \vampeer\@:

\subsection{Endpoint service}
\label{sec:jxta_endpoint}

The endpoint service is actually the core service where messages are redirected
to when coming from the network (through the transport modules) and where
messages can be sent to other peers. For the received messages, its job is
therefore to route the messages to the services that are interested.

\paragraph{Endpoint messages}

The endpoint service is dealing with \emph{endpoint messages}. Each transport
module must be able to send them over or read them from the wire.

A message is basically just an ordered list of key/value pairs (elements). The
key is restricted to a namespace whereas the empty namespace and the
\texttt{jxta} namespace are predefined for user respectively JXTA internal
purposes. You may have any number of namespaces and keys\footnote{There are
however some restrictions by the transport modules but beyond of what you will
ever need\ldots}.

An element may additionally define a MIME type\footnote{See \cite{rfc2046} for a
general introduction to MIME types and \cite{rfc3023} for the XML type, which is
used very often in JXTA message elements.} and a signature element which is
rarely used though.

The endpoint service adds some elements to outgoing messages, for loopback
detection and for addressing.

\paragraph{Endpoint addresses}

The message's source and particularly the destination address must somehow be
defined: an \emph{endpoint address} can be used for various forms of addresses.
The string format looks like this:

\begin{center}
\texttt{protocol://address/service/param}
\end{center}

The \emph{protocol} part specifies the transport module to use: \texttt{tcp},
\texttt{jxta}, \texttt{http}, \ldots{} The \emph{address} is protocol specific:
For TCP and HTTP for example, it is of the form: \texttt{ip:port}, for JXTA it
is simply a peer ID in URN format.

For destination addresses, \emph{service} defines the final service;
\emph{param} is an optional parameter for the given service.

Thus to send messages to a peer, one may either create a destination endpoint
address directly with the destination's IP and port number or alternatively just
set the peer ID, which is the preferred way (because dealing with IP addresses
is discouraged in higher level services). It is the \emph{endpoint router}'s
task to resolve the peer ID to the real endpoint address as described in the
next section.

\subsection{Transport modules}

The \emph{transport modules} are responsible for sending endpoint messages to
another peer and for reading incoming messages from the network. Therefore, they
register themselves as available transport module in the \emph{endpoint
service}, each for its own \emph{protocol}. They provide support for sending a
single message to a peer, to ping a peer (looking if the remote peer is online)
or to propagate a message\footnote{Propagating is only available with UDP
multicast.}.

Each transport specifies its own wire representation.

There is no required transport module and protocol but the low level transport
TCP is usually enabled together with the HTTP transport. It would also be
possible to send messages via SMTP or anything else. Whereas the TCP transport
is simple and fast, HTTP has the great advantage to break firewalls since many
firewalls allow HTTP traffic.

The transport modules do not guarantee message delivery even when TCP is used.
This is very important. The original message sender cannot be sure that his
message has arrived at the destination.

The message transport is usually not secured except for the \texttt{jxtatls}
transport which uses TLS to encrypt data. It is based on top of the
\emph{endpoint router} to provide a secure path from the source to the
destination peer.

\paragraph{Endpoint Router}

A special transport is the \emph{endpoint router}. It is not used to transport
messages over the wire but to route messages with a peer ID as destination
address to the correct gateway. For this, it rewrites the destination address
and passes the messages again to the endpoint service, which is then able to
send the message using a real transport.

A message cannot be sent directly to the peer because there is no direct
connection to it. So called \emph{router peers} may then forward messages to
other networks. This module's task is to query for routes and to send the
message to the first route gateway. It will try to connect using the fastest
transport module around (whenever a connection to that peer is already opened,
the related transport is considered fast).

Whenever the module, respectively the peer, is configured as \emph{router}, it
will accept and forward messages from other peers. It would also maintain a
route cache to be able to do its job faster (without always having to first seek
for routes).

\subsection{Rendezvous service}
\label{sec:jxta_rendezvous}

The \emph{rendezvous service} is used for propagating messages through the peer
group and/or the local network. This is a very fundamental service and many
other services rely on it (e.g.\ for sending queries to all peers).

The module has basically two modes: \texttt{server} and \texttt{client}. An edge
peer would only implement and run as rendezvous client.

\paragraph{Rendezvous lease protocol}

A rendezvous client has to subscribe to a rendezvous server to be able to send
messages for propagation and also to receive propagated messages. Therefore, a
lease protocol is defined which manages this.

When the client peer joins a peer group, it tries to connect to a rendezvous
server. There are several ways to find one:

\begin{itemize}
\item The peer's platform configuration contains some endpoint addresses
(addressing the peer directly with TCP or HTTP). Such addresses may also be hard
coded in the user's peer application.

\item The peer's platform configuration specifies a seeding URL where a
rendezvous server list is published. This is actually the most common and simple
way. It is also very useful for maintenance because only one list has to be
updated to point all the new peers to other servers.

\item As soon as a peer got contact with other peers, it may send them
discovery queries for rendezvous peers. It may also cache rendezvous
advertisements\footnote{A \emph{rendezvous advertisement} promotes a peer's
rendezvous server capability.}, so it does not have to find new servers when
starting up every time.

\item Maybe the chance is high that other peers reside in the same local
network. Therefore, a peer may send a discovery query via multicast to the local
net.
\end{itemize}

Once a rendezvous\footnote{Speaking about a \emph{rendezvous}, we always mean a
rendezvous server or the service depending on the context.} gets a lease
request, it may send back a lease granted mes\-sage---a lease which is always
restricted for a certain amount of time, usually 30 minutes. During this time,
messages are also propagated to the subscribed peer which is allowed to send a
propagation message to the rendezvous.

As leases are not eternally valid, a client has to send lease renewal requests
until it gets another lease. Renewal and initial lease requests actually do not
differ. JXTA JSE asks for lease renewal when the lease's first half has passed.

When a peer leaves a peer group, it should send a lease cancel message so the
rendezvous does not try to propagate messages to that peer anymore.

A rendezvous client should only be registered with at most one rendezvous and
should always send a lease cancel message to peers which send propagation
messages without being the used rendezvous.

For further details on lease messages, please consult the specification
\cite{jxta_spec}.

\paragraph{Message propagation protocol}

The \emph{message propagation protocol} is used to propagate messages.  It adds
a message element\footnote{The element name is the peer group ID prefixed with
\texttt{RendezVousPropagate}.} which has an XML document containing a unique
message ID, a TTL, a path and the final destination service's name and
parameter.

The protocol makes sure that duplicated messages get discarded as well as
messages that are too many hops away from the source peer. This can be done by
means of the message ID and the TTL respectively. Every hop decrements the TTL
value, so the message can be filtered out when the value reaches zero.

The protocol is also responsible for loop detection. Every hop adds its peer ID
to the message's path and detects when a message already passed by earlier.

Based on the given service name, the rendezvous service is able to pass the
message to the correct service, for example by using the endpoint service.

Unlike older JXTA versions, a rendezvous service should not repropagate every
incoming message and thus flood the network. Each service decides individually
whether to repropagate a message or not---of course only as long as the peer
is a rendezvous server.

Hence, a rendezvous server should have some knowledge about the network and
which peer may have which information. It is able to direct messages only to
those peers that may have use for the propagated message. Of course, this
heavily depends on the actual service and the message's type.

\paragraph{Peerview protocol}

Rendezvous servers need as mentioned earlier a good knowledge of the peer
network infrastructure. They also need to stay in contact with other rendezvous
servers to share propagated messages because there may be any number of
rendezvous (not just one) and propagated messages are expected to reach
eventually every node in the peer group (not just the subscribed peers of one's
own rendezvous).

To manage and share this knowledge, the \emph{peer view protocol} is used.
Nowadays, only rendezvous servers should actually run it.

\subsection{Resolver service}
\label{sec:jxta_resolver}

The \emph{resolver service}\footnote{The service name is a little confusing: it
means actually to resolve queries to responses\ldots} is our first user of the
rendezvous service as it has to propagate queries. Its task is to provide a
query-response system.

It is able to recognize received responses to a sent query by adding meta
information to queries like a handler name and a query ID. It attaches also the
peer's route information, so remote peers have the possibility to respond even
if they do not know the querying peer. The actual query can be any string.

The resolver service creates an endpoint message and combines the query together
with all the meta information to a XML document as shown in
listing~\ref{lst:jxta_resolverquery}. Unlike in the example, it is common to use
the service module class ID as handler name.

\begin{lstlisting}[language=XML, caption={A sample resolver query XML document},
label=lst:jxta_resolverquery]
<?xml version="1.0" encoding="UTF-8"?>
<jxta:ResolverQuery xmlns:jxta="http://jxta.org">
  <SrcPeerID>urn:jxta:uuid-59616261646162614E50472050325033\
    1A227980E5924E80A3FD8ECD73D4C31803</SrcPeerID>
  <HandlerName>BeerFinder</HandlerName>
  <QueryID>1</QueryID>
  <HC>0</HC>
  <Query>Got a beer?</Query>
  <SrcPeerRoute><jxta:RA><Dst>
    <jxta:APA><EA>tcp://129.132.105.170:32725</EA></jxta:APA>
    </Dst></jxta:RA></SrcPeerRoute>
</jxta:ResolverQuery>
\end{lstlisting}

The hop count, which is incremented on each hop, ensures that the query is not
sent to far away. Although the rendezvous service already does such a check, we
have to do it here again because the resolver does not always need to use the
rendezvous propagation mechanism.

Queries and responses may be propagated or sent directly to a specified peer.
The resolver service is not necessarily dependent on the rendezvous service but
would of course be limited to local network propagation and single message
dispatching in a situation without rendezvous.

A resolver response looks basically the same as a query. The actual response is
also a string. There is no hop count as it does not make any sense here. Also the
source peer ID and route are dropped but there is a response peer ID, so the
recipient knows from which peer the response originates.

A client service sending a query should be able to register a listener for
related responses, so it does not have to check itself if the response matches
the query. That is one of the main tasks of the resolver service.

\subsection{Discovery service}
\label{sec:jxta_discovery}

Whereas the resolver does not maintain much data and only serves as an
intermediate message layer for other services, the \emph{discovery service} is
a fundamental part in JXTA and much information passes it.

The discovery service is dealing with all sorts of advertisements, so it knows
about all peer resources (as advertisements are promoting resources). On one
hand, it serves as an advertisement storage and on the other hand, it is
responsible for finding remote advertisements and for letting other peers know
about the locally stored ones.

\paragraph{Discovery queries and responses}

Whenever a service needs an advertisement, it does a local discovery query
which is equivalent to a storage lookup.

Generally, we distinguish between peer, peer group and other advertisements, so
we specify the advertisement type in a query. A query may restrict the search
additionally with a key and a value name. The key is a XML tag name.  It is
allowed for the value to contain the wild char \texttt{*} in the beginning
and/or at the end. The number of answers may also be limited by setting a
threshold.

Querying remote peers is actually the same but one may choose to send a query
to a single peer or to propagate the message in the group. In both cases, the
resolver service is actually used for sending the messages.

The discovery query is an XML document specifying query type, key, value,
threshold and optionally also the source peer advertisement. Looking at the
final endpoint message, we see a resolver element containing a resolver query
XML document which contains the quoted discovery query XML data\footnote{The
discovery query is quoted because the resolver service currently expects a
simple query string.}.

A peer is not obliged to respond to any remote discovery query. Peers which have
sent a query should expect no, one or multiple responses. They cannot expect
that the threshold is respected, neither as minimum nor maximum.

A discovery response message may contain several matching advertisements. It may
additionally also contain the responding peer advertisement.

Discovery responses are not only used to respond to queries. It is also allowed
to publish advertisements to other peers, especially the rendezvous server,
using a ``response'' message.

To feed the local storage with advertisements, one just publishes them locally.

\paragraph{Shared Resource Distributed Index}

As we have already seen, a rendezvous server will not propagate every message.
In the case of discovery queries, the rendezvous makes use of a \emph{shared
resource distributed index}, henceforth called SRDI\@ (see \cite{jxta_srdi}).

The SRDI is an advertisement index containing certain keys and values together
with peer IDs enabling the rendezvous to lead queries to peers which actually
should have matching advertisements. This monumentally reduces network traffic
as peers that do not have the needed information are not queried.

Note that the SRDI does not contain the entire advertisements but only has some
important keys and their values for every advertisement a peer has.

But how does the rendezvous maintain its SRDI\@? Every edge peer sends its SRDI
to its rendezvous. When newly joining the group, it sends the full index.
Later, it sends regularly (for example every minute) a SRDI delta, that means
only the key/values for newly discovered, created or updated advertisements.
When a rendezvous lease is canceled, the peer's SRDI entries are removed
automatically by the rendezvous.

There exists a generic SRDI XML document used for pushing SRDI entries to the
rendezvous. It contains also a TTL, so an SRDI entry is not valid for ever
(like the advertisements themselves).

The current JXTA implementations do not index each advertisement's XML tag.
When creating a new advertisement type, one should specify which elements are
important and should therefore be indexed. When speaking in database terms, one
should at least index the primary key attributes.

\section{JXTA's P2P infrastructure and peer roles}
\label{sec:jxta_peerroles}

To bring some clarification into the partly insufficiently introduced peer
roles, we would now like to show now a short overview of the entire JXTA \ptp{}
infrastructure:

Although JXTA may use central rendezvous server lists when starting up, we
can definitely see JXTA's structure as a \emph{true} \ptp{} system. It does not
rely on central servers for any core task and uses the \ptp{} structure for all
purposes.

However, there are various peer roles in the network although there may also
exist combinations:

\paragraph{Edge peer} Whenever traffic or CPU power is expensive, an \emph{edge
peer} is surely the right role for a peer. Such a peer heavily relies on other
peers and consumes parts of their attention. Most peers would actually choose
this kind of role.

\paragraph{Rendezvous peer} A \emph{rendezvous peer} is providing a rendezvous
server and enables edge peers to make contact with other peers. In a JXTA
network, we need at least one rendezvous server because we usually want to
discover other peers and do not have the physical locations hard coded of other
peers that we want to communicate with.

\paragraph{Router peer} A \emph{router peer} enables peers to communicate with
others to which they cannot connect directly. This is used for peers behind a
NAT gateway or a firewall. Therefore, router peers may have to manage all their
clients message traffic in one or both directions.

\paragraph{}

Note that peer roles may dynamically change. For example, a peer which cannot
find any rendezvous server could automatically become a rendezvous. This is of
course adjustable in the platform configuration.
