/** \mainpage libPPI : The Packet Processing Infrastructure
- The PPI provides an infrastructure to create packet oriented network processin
- applications. A PPI application is built by combining processing modules in a very flexible
- manner.
+ The PPI provides an infrastructure to create packet oriented network processing applications. A
+ PPI application is built by combining processing modules in a very flexible manner.
\image html scenario.png Target Scenario
\li The PPI is built around reusable \ref modules. Each module is completely independent.
\li Each module has an arbitrary number of \ref connectors, inputs and outputs.
\li The modules are connected to each other using flexible \ref connections.
- \li Data flow throughout the network is governed via flexible automatic or manual \ref throttling.
+ \li Data flow throughout the network is governed via flexible automatic or manual \ref
+ throttling.
\li Modules may register additional external \ref events (file descriptor events or timers).
The PPI thereby builds on the facilities provided by the other components of the SENF
performance optimizations for TCP traffic (PEP). This router is built by combining several
modules.
+ \see \ref overview \n
+ <a href="../../../Examples/RateStuffer/doc/html/index.html">PPI Example Application:
+ RateStuffer</a> \n
+ \ref senf::ppi::module "Modules" \n
+ \ref senf::ppi::connector "Connectors" \n
+ \ref event_group
+ */
+
+/** \page overview PPI Overview and Concepts
+
\section design Design considerations
The PPI interface is designed to be as simple as possible. It provides sane defaults for all
configurable parameters to simplify getting started. It also automates all resource
- management. Especially to simplify resource management, the PPI will take many configuration
- objects by value. Even though this is not as efficient, it frees the user from most resource
- management chores. This decision does not affect the runtime performance since it only affects
- the configuration step.
+ management. The throttling infrastructure handles blocking conditions (like input exhaustion)
+ automatically
\section packets Packets
(network connection, writing packets to disk, generating new packets) whereas processing modules
process packets internally. In the target scenario, <em>TAP</em>, <em>ASI Out</em>, <em>Raw
Socket</em> and in a limited way <em>Generator</em> are I/O modules whereas <em>PEP</em>,
- <em>DiffServ</em>, <em>DVB Enc</em>, <em>GRE/UDLR</em>, <em>TCP Filter</em> and
- <em>Stuffer</em>are processing modules. <em>ASI/MPEG</em> and <em>Net</em> are external I/O
- ports which are integrated via the <em>TAP</em>, <em>ASI Out</em> and <em>Raw Sock</em> modules
- using external events.
+ <em>DiffServ</em>, <em>DVB Enc</em>, <em>GRE/UDLR</em>, <em>TCP Filter</em> and <em>Stuffer</em>
+ are processing modules. <em>ASI/MPEG</em> and <em>Net</em> are external I/O ports which are
+ integrated via the <em>TAP</em>, <em>ASI Out</em> and <em>Raw Sock</em> modules using external
+ events.
The following example module declares three I/O connectors (see below): <tt>payload</tt>,
<tt>stuffing</tt> and <tt>output</tt>. These connectors are defined as <em>public</em> data
members so they can be accessed from the outside. This is important as we will see below.
\code
- class RateStuffer
- : public senf::ppi::Module
+ class RateStuffer
+ : public senf::ppi::module::Module
{
+ senf::ppi::IntervalTimer timer_;
+
public:
- ActiveInput payload;
- ActiveInput stuffing;
- ActiveOutput output;
+ senf::ppi::connector::ActiveInput payload;
+ senf::ppi::connector::ActiveInput stuffing;
+ senf::ppi::connector::ActiveOutput output;
RateStuffer(unsigned packetsPerSecond)
+ : timer_(1000u, packetsPerSecond)
{
route(payload, output);
route(stuffing, output);
- registerEvent(&RateStuffer::tick,
- senf::ppi::IntervalTimer(1000u, packetsPerSecond));
+ registerEvent(&RateStuffer::tick, timer_);
}
private:
};
\endcode
- On module instantiation, it will declare it's flow information with <tt>route</tt> (which
- is inherited from <tt>senf::ppi::Module</tt>). Then the module registers an interval timer which
- will fire <tt>packetsPerSecond</tt> times every <tt>1000</tt> milliseconds.
+ On module instantiation, it will declare it's flow information with <tt>route</tt> (which is
+ inherited from <tt>senf::ppi::module::Module</tt>). Then the module registers an interval timer
+ which will fire <tt>packetsPerSecond</tt> times every <tt>1000</tt> milliseconds.
The processing of the module is very simple: Whenever a timer tick arrives a packet is sent. If
the <tt>payload</tt> input is ready (see throttling below), a payload packet is sent, otherwise
\code
class CopyPacketGenerator
- : public senf::ppi::Module
+ : public senf::ppi::module::Module
{
public:
- PassiveOutput output;
+ senf::ppi::connector::PassiveOutput output;
CopyPacketGenerator(Packet::ptr template)
: template_ (template)
\section connections Connections
+ \image html ratestuffer.png Simple RateStuffer
+
To make use of the modules, they have to be instantiated and connections have to be created
between the I/O connectors. It is possible to connect any pair of input/output connectors as
long as one of them is active and the other is passive.
CopyPacketGenerator generator (stuffingPacket);
senf::UDPv4ClientSocketHandle inputSocket (1111);
- senf::ppi::ActiveSocketInput udpInput (inputSocket);
+ senf::ppi::module::ActiveSocketReader udpInput (inputSocket);
senf::UDPv4ClientSocketHandle outputSocket ("2.3.4.5:2222");
- senf::ppi::PassiveSocketOutput udpOutput (outputSocket);
+ senf::ppi::module::PassiveSocketWriter udpOutput (outputSocket);
- senf::ppi::connect(udpInput.output, rateStuffer.payload,
- dynamicModule<PassiveQueue>()
- -> qdisc(ThresholdQueueing(10,5)) );
+ senf::ppi::module::PassiveQueue adaptor;
+
+ senf::ppi::connect(udpInput.output, adaptor.input);
+ senf::ppi::connect(adaptor.output, rateStuffer.payload);
+ adaptor.qdisc(ThresholdQueueing(10,5));
senf::ppi::connect(generator.output, rateStuffer.stuffing);
senf::ppi::connect(rateStuffer.output, udpOutput.input);
up. The buffering on the udpInput <-> rateStuffer adaptor is changed so the queue will begin to
throttle only if more than 10 packets are in the queue. The connection will be unthrottled as
soon as there are no more than 5 packets left in the queue. This application will read
- udp-packts coming in on port 1111 and will forward them to port 2222 on host 2.3.4.5 with a
+ udp-packets coming in on port 1111 and will forward them to port 2222 on host 2.3.4.5 with a
fixed rate of 10 packets / second.
\section throttling Throttling
<em>forward throttling</em> since it is forwarded along the \e same direction the data
is. Forward throttling notifications are therefore sent towards the output modules.
- Since throttling a passive input may not disable all further packet delivery immediately, any
- passive input contains an input queue. In it's default configuration, the queue will send out
- throttle notifications when it becomes non-empty and unthrottle notifications when it becomes
- empty again. This automatic behavior may however be disabled. This allows a module to collect
- incoming packets in it's input queue before processing a bunch of them in one go.
+ Since throttling a passive input may not disable all further packet delivery immediately, all
+ inputs contains an input queue. In it's default configuration, the queue will send out throttle
+ notifications when it becomes non-empty and unthrottle notifications when it becomes empty
+ again. This automatic behavior may however be disabled. This allows a module to collect incoming
+ packets in it's input queue before processing a bunch of them in one go.
\section events Events
- Modules may register additional events. These external events are very important since the drive
- the PPI framework. Possible event sources are
+ Modules may register additional events. These external events are very important since they
+ drive the PPI framework. Possible event sources are
\li time based events
\li file descriptors.
+ \li internal events (e.g. IdleEvent)
Here some example code implementing the ActiveSocketInput Module:
\code
- class ActiveSocketInput
- : public senf::ppi::Module
+ class ActiveSocketReader
+ : public senf::ppi::module::Module
{
+ typedef senf::ClientSocketHandle<
+ senf::MakeSocketPolicy< senf::ReadablePolicy,
+ senf::DatagramFramingPolicy > > SocketHandle;
+ SocketHandle socket_;
+ DataParser const & parser_;
+ senf::ppi:IOSignaler event_;
+
static PacketParser<senf::DataPacket> defaultParser_;
public:
- ActiveOutput output;
-
- typedef senf::ClientSocketHandle<
- senf::MakeSocketPolicy< senf::ReadablePolicy,
- senf::DatagramFramingPolicy > > Socket;
+ senf::ppi::connector::ActiveOutput output;
// I hestitate taking parser by const & since a const & can be bound to
// a temporary even though a const & is all we need. The real implementation
// will probably make this a template arg. This simplifies the memory management
// from the users pov.
- ActiveSocketInput(Socket socket, DataParser & parser = SocketInput::defaultParser_)
+ ActiveSocketReader(SocketHandle socket,
+ DataParser & parser = ActiveSocketReader::defaultParser_)
: socket_ (socket),
parser_ (parser)
- event_ (registerEvent( &ActiveSocketInput::data,
- senf::ppi::IOSignaler(socket, senf::ppi::IOSignaler::Read) ))
+ event_ (socket, senf::ppi::IOSignaler::Read)
{
+ registerEvent( &ActiveSocketReader::data, event_ );
route(event_, output);
}
private:
- Socket socket_;
- DataParser const & parser_;
- senf::ppi:IOSignaler::EventBinding event_;
void data()
{
by the template argument.
We register an IOSignaler event. This event will be signaled whenever the socket is
- readable. This event is routet to the output. This routing automates throttling for the socket:
+ readable. This event is routed to the output. This routing automates throttling for the socket:
Whenever the output receives a throttle notifications, the event will be temporarily disabled.
Processing arriving packets happens in the \c data() member: This member simple reads a packet
from the socket. It passes this packet to the \c parser_ and sends the generated packet out.
- \note Open Issues
- \li Exception handling. It would be great to have a sane default exception handling freeing us
- from most manual work. However, I don't think this is feasible.
+ \section flows Information Flow
+
+ The above description conceptually introduces three different flow levels:
+
+ \li The <em>data flow</em> is, where the packets are flowing. This flow always goes from output
+ to input connector.
+ \li The <em>execution flow</em> describes the flow of execution from one module to another. This
+ flow always proceeds from active to passive connector.
+ \li The <em>control flow</em> is the flow of throttling notifications. This flow always proceeds
+ \e opposite to the execution flow, from passive to active connector.
- \see \ref ppi_implementation \n
- <a href="http://openfacts.berlios.de/index-en.phtml?title=SENF:_Packet_Processing_Infrastructure">Implementation plan</a>
+ This is the outside view, from without any module. These flows are set up using
+ senf::ppi::connect() statements.
+
+ Within a module, the different flow levels are defined differently depending on the type of
+ flow:
+
+ \li The <em>data flow</em> is defined by how data is processed. The different event and
+ connector callbacks will pass packets around and thereby define the data flow
+ \li Likewise, the <em>execution flow</em> is defined parallel to the data flow (however possible
+ in opposite direction) by how the handler of one connector calls other connectors.
+ \li The <em>control flow</em> is set up using senf::ppi::Module::route statements (as long as
+ automatic throttling is used. Manual throttling defines the control flow within the
+ respective callbacks).
+
+ In nearly all cases, these flows will be parallel. Therefore it makes sense to define the \c
+ route statement as defining the 'conceptual data flow' since this is also how control messages
+ should flow (sans the direction, which is defined by the connectors active/passive property).
+
+ \see \ref ppi_implementation
*/
-/** \page ppi_implementation Implementation Overview
+/** \page ppi_implementation Implementation Notes
\section processing Data Processing
- The processing in the PPI is driven by external events. Without external events <em>nothing will
- happen</em>. When an external event is generated, the module called will probably either send or
- receive data from an active connector.
+ The processing in the PPI is driven by events. Without events <em>nothing will happen</em>. When
+ an event is generated, the called module will probably call one of it's active connectors.
Calling an active connector will directly call the handler registered at the connected passive
connector. This way the call and data are handed across the connections until an I/O module will
finally handle the request (by not calling any other connectors).
Throttling is handled in the same way: Throttling a passive connector will call a corresponding
- (internal) method of the connector active connector. This method will call registered handlers
+ (internal) method of the connected active connector. This method will call registered handlers
and will analyze the routing information of the module for other (passive) connectors to call
and throttle. This will again create a call chain which terminates at the I/O modules. An event
which is called to be throttled will disable the event temporarily. Unthrottling works in the
handler has finished processing. Forward throttling notifications are not generated
automatically within the connector. However, the Passive-Passive adaptor will generate
Forward-throttling notifications whenever the input queue is empty.
+
+ \section class_diagram Class Diagram
+
+ \image html classes.png
*/
\f
// mode: flyspell
// mode: auto-fill
// End:
+
projects / senf.git / blobdiff