2 // Fraunhofer Institut fuer offene Kommunikationssysteme (FOKUS)
3 // Kompetenzzentrum fuer Satelitenkommunikation (SatCom)
4 // Stefan Bund <g0dil@berlios.de>
6 // This program is free software; you can redistribute it and/or modify
7 // it under the terms of the GNU General Public License as published by
8 // the Free Software Foundation; either version 2 of the License, or
9 // (at your option) any later version.
11 // This program is distributed in the hope that it will be useful,
12 // but WITHOUT ANY WARRANTY; without even the implied warranty of
13 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14 // GNU General Public License for more details.
16 // You should have received a copy of the GNU General Public License
17 // along with this program; if not, write to the
18 // Free Software Foundation, Inc.,
19 // 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
21 /** \mainpage libPPI : The Packet Processing Infrastructure
23 The PPI provides an infrastructure to create packet oriented network processin
24 applications. A PPI application is built by combining processing modules in a very flexible
27 \image html scenario.png Target Scenario
29 The PPI concept is built around some key concepts
31 \li The PPI is based on processing \e packets. It does not handle stream oriented channels.
32 \li The PPI is built around reusable \e modules. Each module is completely independent.
33 \li Each module has an arbitrary number of \e connectors, inputs and outputs.
34 \li The modules are connected to each other using flexible \e connections.
35 \li Data flow throughout the network is governed via flexible automatic or manual \e throttling.
36 \li Modules may register additional external \e events (file descriptor events or timers).
38 The PPI thereby builds on the facilities provided by the other components of the SENF
41 Modules are divided roughly in to two categories: I/O modules provide packet sources and sinks
42 (network connection, writing packets to disk, generating new packets) whereas processing modules
43 process packets internally. The target scenario above depicts a diffserv capable UDLR/ULE
44 router including performance optimizations for TCP traffic (PEP). This router is built by
45 combining several modules. In this scenario, <em>TAP</em>, <em>ASI Out</em>, <em>Raw Socket</em>
46 and in a limited way <em>Generator</em> are I/O modules whereas <em>PEP</em>, <em>DiffServ</em>,
47 <em>DVB Enc</em>, <em>GRE/UDLR</em>, <em>TCP Filter</em> and <em>Stuffer</em>are processing
48 modules. <em>ASI/MPEG</em> and <em>Net</em> are external I/O ports which are integrated via the
49 <em>TAP</em>, <em>ASI Out</em> and <em>Raw Sock</em> modules using external events.
51 \section packets Packets
53 The PPI processes packets and uses the <a href="@TOPDIR@/Packets/doc/html/index.html">Packet
54 library</a> to handle them. All packets are passed around as generic Packet::ptr's, the PPI
55 does not enforce any packet type restrictions.
57 \section modules Modules
59 A module is represented by a class type. Each module has several components:
61 \li It may have any number of connectors (inputs and outputs)
62 \li Each module declares flow information which details the route packets take within the
63 module. This information does not define how the information is processed, it only tells,
64 where data arriving on some input will be directed at.
65 \li The module might take additional parameters.
66 \li The module might also register additional events.
70 : public senf::ppi::Module
77 RateStuffer(unsigned packetsPerSecond)
79 route(payload, output);
80 route(stuffing, output);
82 registerEvent(&RateStuffer::tick, IntervalTimer(1000u, packetsPerSecond));
96 This module declares three I/O connectors (see below): <tt>payload</tt>, <tt>stuffing</tt> and
97 <tt>output</tt>. These connectors are defined as <em>public</em> data members so they can be
98 accessed from the outside. This is important as we will see below.
100 On module instantiation, it will declare it's flow information with <tt>route</tt> (which
101 is inherited from <tt>senf::ppi::Module</tt>). Then the module registers an interval timer which
102 will fire <tt>packetsPerSecond</tt> times every <tt>1000</tt> milliseconds.
104 The processing of the module is very simple: Whenever a timer tick arrives a packet is sent. If
105 the <tt>payload</tt> input is ready (see throttling below), a payload packet is sent, otherwise
106 a stuffing packet is sent. The module will therefore provide a constant stream of packets at a
107 fixed rate on <tt>output</tt>
109 An example module to generate the stuffing packets could be
112 class CopyPacketGenerator
113 : public senf::ppi::Module
116 PassiveOutput output;
118 CopyPacketGenerator(Packet::ptr template)
119 : template_ (template)
122 output.onRequest(&CopyPacketGenerator::makePacket);
126 Packet::ptr template_;
130 output(template_.clone());
135 This module just produces a copy of a given packet whenever output is requested.
137 \subsection connectors Connectors
139 Inputs and Outputs can be active and passive. An \e active I/O is <em>activated by the
140 module</em> to send data or to poll for available packets. A \e passive I/O is <em>signaled by
141 the framework</em> to fetch data from the module or to pass data into the module.
143 To send or receive a packet (either actively or after packet reception has been signaled), the
144 module just calls the connector. This allows to generate or process multiple packets in one
145 iteration. However, reading will only succeed, as long as packets are available from the
148 A module might want to queue incoming packets within a passive input or outgoing packets within
149 an active output. This is possible by either not reading any packet even though a new packet has
150 been scheduled on the input or by writing to the output while it is still throttled. To
151 facilitate this use, the connectors provide accessors to access the attached connection and it's
152 queue. This allows to analyze all packets available in the queue and act accordingly.
154 \section connections Connections
156 To make use of the modules, they have to be instantiated and connections have to be created
157 between the I/O connectors. It is possible to connect any pair of input/output connectors as
158 long as at least one of them is active
160 Every connection contains an internal packet queue. Under normal operating conditions (without
161 throttling) the queue will mostly be empty since packets will be processed directly. If a
162 connection is throttled, it can still receive new packets on it's input which will then be
163 queued. This is necessary even though the throttling will be propagated backwards (so no new
164 packets should arrive) since a module may produce more then one result packet from a single
167 To complete our simplified example: Lets say we have a <tt>UdpInput</tt> module and a
168 <tt>UdpOutput</tt> module. We can then use our <tt>RateStuffer</tt> module to build an
169 application which will create a fixed-rate UDP stream:
172 RateStuffer rateStuffer (10);
173 CopyPacketGenerator generator (some_packet_ptr);
174 senf::UDPv4ClientSocketHandle inputSocket (1111);
175 senf::ppi::SocketInput udpInput (inputSocket);
176 senf::UDPv4ClientSocketHandle outputSocket ("2.3.4.5:2222");
177 senf::ppi::SocketOutput udpOutput (outputSocket);
179 senf::ppi::connect(udpInput.output, rateStuffer.payload)
180 .bufferHighThresh(10)
182 senf::ppi::connect(generator.output, rateStuffer.stuffing);
183 senf::ppi::connect(rateStuffer.output, udpOutput.input);
188 First all necessary modules are created. Then the connections between these modules are set
189 up. The buffering of the udpInput <-> rateStuffer connection is changed so the queue will begin
190 to throttle only if more than 10 packets are in the queue. The connection will be unthrottled as
191 soon as there are no more than 5 packets left in the queue. This application will read
192 udp-packts coming in on port 1111 and will forward them to port 2222 on host 2.3.4.5 with a
193 fixed rate of 10 packets / second.
195 \section throttling Throttling
197 If a connection cannot pass packets in it's current state, the connection is \e throttled. In
198 simple cases, throttling is handled automatically by
199 \li the connection if the queue is exceeds the buffering threshold
200 \li I/O modules whenever the external source or sink of the module is not ready
202 Throttling is handled separately in each direction:
203 \li Forward throttling will throttle in the direction of the data flow. Example: No new packets
204 are available from an input. This will activate forward throttling until new data arrives
205 \li Backward throttling will throttle in the direction to the data source. Example: an output
206 device (e.g. serial interface) has no more room for data. This event will activate backward
207 throttling so no new data will arrive until the device can send data again
209 The throttling state is managed within the Connection. Automatic throttling utilizes the routing
210 information (provided in the modules constructor) to forward throttling events across
211 modules. However, automatic throttling can be disabled for each connector. Modules may also
212 register event handlers whenever a throttling event occurs.
214 Whenever a connection is throttled (in the corresponding direction), passive connectors will \e
215 not be called by the framework. This may lead to packets being accumulated in the connection
216 queue. These packets will be sent as soon as the connection is unthrottled. The unthrottle event
217 will hoewever only be forwarded when the queue is empty (or has reached it's lower buffering
221 passiveConnector.autoForwardThrottling(false);
222 passiveConnector.autoBackwardThrotttling(true);
223 passiveConnector.onForwardThrottle(...);
224 passiveConnector.onBackwardUnthrottle(...);
227 Throttling is <em>not</em> enforced: especially a throttled output may still be called, the
228 excessive packets will be queued in the connection queue.
230 \section events Events
232 Modules may register additional events. These external events are very important since the drive
233 the PPI framework. Possible event sources are
234 \li time based events
235 \li file descriptors.
242 // c-file-style: "senf"
243 // indent-tabs-mode: nil
244 // ispell-local-dictionary: "american"