4 // Fraunhofer Institute for Open Communication Systems (FOKUS)
5 // Competence Center NETwork research (NET), St. Augustin, GERMANY
6 // Stefan Bund <g0dil@berlios.de>
8 // This program is free software; you can redistribute it and/or modify
9 // it under the terms of the GNU General Public License as published by
10 // the Free Software Foundation; either version 2 of the License, or
11 // (at your option) any later version.
13 // This program is distributed in the hope that it will be useful,
14 // but WITHOUT ANY WARRANTY; without even the implied warranty of
15 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
16 // GNU General Public License for more details.
18 // You should have received a copy of the GNU General Public License
19 // along with this program; if not, write to the
20 // Free Software Foundation, Inc.,
21 // 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
23 /** \mainpage libPPI : The Packet Processing Infrastructure
25 The PPI provides an infrastructure to create packet oriented network processing applications. A
26 PPI application is built by combining processing modules in a very flexible manner.
28 \image html scenario.png Target Scenario
30 The PPI concept is built around some key concepts
32 \li The PPI is based on processing \ref ppi_packets. It does not handle stream oriented
34 \li The PPI is built around reusable \ref ppi_modules. Each module is completely independent.
35 \li Each module has an arbitrary number of \ref ppi_connectors, inputs and outputs.
36 \li The modules are connected to each other using flexible \ref ppi_connections.
37 \li Data flow throughout the network is governed via flexible automatic or manual \ref
38 ppi_throttling (throttle notifications).
39 \li Modules may register additional external \ref ppi_events (file descriptor events or timers).
41 The PPI thereby builds on the facilities provided by the other components of the SENF
42 framework. The target scenario above depicts a diffserv capable UDLR/ULE router including
43 performance optimizations for TCP traffic (PEP). This router is built by combining several
46 \see \ref ppi_overview \n
47 <a href="../../../Examples/RateStuffer/doc/html/index.html">PPI Example Application:
49 \ref senf::ppi::module "Modules" \n
50 \ref senf::ppi::connector "Connectors" \n
54 /** \page ppi_overview PPI Overview and Concepts
58 \section ppi_design Design considerations
60 The PPI interface is designed to be as simple as possible. It provides sane defaults for all
61 configurable parameters to simplify getting started. It also automates all resource
62 management. The throttling infrastructure handles blocking conditions (like input exhaustion)
65 \section ppi_packets Packets
67 The PPI processes packets and uses the <a href="@TOPDIR@/Packets/doc/html/index.html">Packet
68 library</a> to handle them. All packets are internally passed around as generic \ref
69 senf::Packet references, however connectors may optionally be defined as sending or receiving
70 packets of a specific type only.
72 \section ppi_modules Modules
74 A module is represented by a class derived from senf::ppi::module::Module. Each module has
77 \li It may have any number of \ref ppi_connectors (inputs and outputs)
78 \li Each module declares flow information which details the route packets take within the
79 module. This information does not define how the information is processed, it only tells,
80 where data arriving on some input will be directed at (\ref
81 senf::ppi::module::Module::route())
82 \li The module might take additional parameters.
83 \li The module might also register additional \ref ppi_events.
85 Generally, modules are divided into several categories:
87 \li \ref io_modules receive external data or forward packets out of the PPI
88 \li \ref routing_modules forward packets within the network
89 \li \ref adapter_modules are used to connect incompatible connectors to each other
90 \li Application modules are modules implemented to perform an applications function
92 Of these modules, normally only the application modules need to be implemented since the library
93 provides an extensive set of reusable modules.
95 The following example module declares three \ref ppi_connectors "Connectors": \c payload,
96 \c stuffing and \c output. These connectors are defined as \e public data members so they
97 can be accessed from the outside. This is important as we will see below.
101 : public senf::ppi::module::Module
103 SENF_PPI_MODULE(RateStuffer);
105 senf::ppi::IntervalTimer timer_;
108 senf::ppi::connector::ActiveInput<> payload;
109 senf::ppi::connector::ActiveInput<> stuffing;
110 senf::ppi::connector::ActiveOutput<> output;
112 RateStuffer(unsigned packetsPerSecond)
113 : timer_(1000u, packetsPerSecond)
115 route(payload, output);
116 route(stuffing, output);
118 registerEvent( timer_, &RateStuffer::tick );
132 The constructor will declare flow information using senf::ppi::module::Module::route(). Then the
133 module registers an interval timer which will fire <tt>packetsPerSecond</tt> times every
134 <tt>1000</tt> milliseconds.
136 The module processing is very simple: Whenever a timer tick arrives a packet is sent. If the \c
137 payload input is ready (see \ref ppi_throttling), a payload packet is sent, otherwise a stuffing
138 packet is sent. The module will therefore provide a constant stream of packets at a fixed rate
139 on \c output (see the
140 <a href="@TOPDIR@/Examples/RateStuffer/doc/html/index.html">RateStuffer</a> example application
141 for a slightly different approach)
143 An example module to generate the stuffing packets could be
146 class CopyPacketGenerator
147 : public senf::ppi::module::Module
149 SENF_PPI_MODULE(CopyPacketGenerator);
151 senf::ppi::connector::PassiveOutput<> output;
153 CopyPacketGenerator(Packet template)
154 : template_ (template)
157 output.onRequest(&CopyPacketGenerator::makePacket);
165 output(template_.clone());
170 This module just produces a copy of a given packet whenever output is requested.
172 \see senf::ppi::module::Module
174 \section ppi_connectors Connectors
176 The input and output attachment points of a module are called connectors. Each connector may be
177 active or passive. This gives us 4 types of connectors:
179 \li senf::ppi::connector::ActiveInput
180 \li senf::ppi::connector::ActiveOutput
181 \li senf::ppi::connector::PassiveInput
182 \li senf::ppi::connector::PassiveOutput
184 An \e active connector (input or output) is <em>activated by the module</em> to send data or to
185 poll for available packets. This means, the modules processing routine will call the connector
186 without being signaled by the framework to read the connector. It just actively fetches a
189 A \e passive connector is <em>signaled by the framework</em> to fetch data from the module or to
190 pass data into the module. The module must register a callback which will be called, whenever a
191 packet is requested from the module or whenever a new packet is made available for the module to
194 To send or receive a packet (either actively or passively) the module just calls the
195 connector. It is permissible to generate or process multiple packets in one iteration. However,
196 you must ensure yourself that enough packets are available to be read if more than one packet
197 shall be read. It is also permissible to not handle a packet at all even if signaled to do
198 so. The packet will automatically be queued.
200 To provide this flexibility, all input connectors incorporate a packet queue. This queue is
201 exposed to the module and allows the module to optionally process packets in batches.
203 Connectors take an optional template argument which allows to specify the type of packet this
204 connector sends or received. This template arguments defaults to \ref senf::Packet.
206 \see \ref senf::ppi::connector
208 \section ppi_connections Connections
210 \image html ratestuffer.png Simple RateStuffer
212 To make use of the modules, they have to be instantiated and connections have to be created
213 between its connectors. It is possible to connect any pair of input/output connectors as long as
214 one of them is active and the other is passive.
216 It is possible to connect two active or passive connectors with each other using a special
217 adaptor module (senf::ppi::module::PassiveQueue or senf::ppi::module::ActiveFeeder
220 To complete our simplified example: Lets connet senf::ppi::module::ActiveSocketReader and
221 senf::ppi::module::PassiveSocketWriter to our example module:
224 RateStuffer rateStuffer (10);
226 senf::Packet stuffingPacket = senf::DataPacket::create(...);
227 CopyPacketGenerator generator (stuffingPacket);
229 senf::UDPv4ClientSocketHandle inputSocket (1111);
230 senf::ppi::module::ActiveSocketSource<> udpInput (inputSocket);
232 senf::UDPv4ClientSocketHandle outputSocket ("2.3.4.5:2222");
233 senf::ppi::module::PassiveSocketSink<> udpOutput (outputSocket);
235 senf::ppi::module::PassiveQueue adaptor;
237 senf::ppi::connect(udpInput, adaptor);
238 senf::ppi::connect(adaptor, rateStuffer.payload);
239 adaptor.qdisc(ThresholdQueueing(10,5));
240 senf::ppi::connect(generator, rateStuffer.stuffing);
241 senf::ppi::connect(rateStuffer, udpOutput);
246 This application will read udp-packets coming in on port 1111 and will forward
247 them to port 2222 on host 2.3.4.5 with a fixed rate of 10 packets / second.
249 We start out by instantiating the necessary modules. Then the connections between these modules
250 are set up by successively connecting each output connector to an input connector. As can be
251 seen, the name of the connector can be left of if it is named \c output or \c input
254 The buffering on the udpInput <-> rateStuffer adaptor is changed so the queue will begin to
255 throttle only if more than 10 packets are in the queue. The connection will be unthrottled as
256 soon as there are no more than 5 packets left in the queue (see \ref ppi_throttling).
258 \section ppi_throttling Throttling
260 Throttling and throttle notifications at it's base is about handling blocking conditions. The
261 most straight forward blocking condition is that of a file descriptor not being available for
262 reading resp. writing. Other blocking conditions can arise for example when a queue fills up or
263 if a module for some application specific reason does not want to handle packets for a period of
266 All this is handled using throttle notifications. We need throttle notifications so a passive
267 connector can tell it's connected peer that it cannot service further requests until an
268 unthrottle notification is sent. This tells us, that from the view of someone implementing a
269 module, throttle notifications will always be received on active connectors and be sent on
272 This tells us, that the direction of control flow (the throttle notifications) is from passive
273 to active connectors and does \e not depend on the direction of data flow (which flows from
274 output to input connector). Thinking about this, this makes sense: The module with the active
275 connector is the one initiating the data processing (after all, it is the \e active part) and
276 needs to be told not to request or send packets on it's connector since the connected passive
277 peer cannot handle the request.
279 So if a passive connector cannot handle requests, the connector must be \e throttled. Throttling
280 the connector will forward a throttle notification to its peer. The peer then handles the
281 throttling notification.
283 There are two ways, throttle notifications can be handled: By automatic throttling or by
284 registering callbacks. The default is <em>automatic throttling</em>.
286 <em>Automatic throttling</em> is based on the routing information available to the module. Every
287 notification received is forwarded within the module along all known routes from active to
288 passive connectors (routes which connect to active or passive connectors are absolutely valid,
289 they just are not \e forwarding routes, they are ignored by the throttle
290 notifications). Together with automatic event throttling (see \ref ppi_events), this is all that
291 is normally needed to handle throttle notifications: By forwarding the notifications we ensure,
292 that a module's passive connectors will only be signaled when it's corresponding active
293 connectors are not throttled (as defined by the routing information). The module is therefore
294 not called until the connector(s) are untrhottled.
296 <em>Throttle callbacks</em> can optionaly be registerd (with automatic throttling enabled or
297 disabled, see \ref senf::ppi::connector::ActiveConnector) to be called when a throttle
298 notification is received. The callback may then handle the notification however it sees fit, for
299 example by manually throttling some passive connector (see \ref
300 senf::ppi::connector::PassiveConnector).
302 To enable/disable automatic throttling, the \ref senf::ppi::module::Module::route() command
303 returns a reference to a \ref senf::ppi::Route instance. If this route is \e forwarding route,
304 (that is, of the connectors is passive and the other is active), the return value will be
305 derived from \ref senf::ppi::ForwardingRoute which provides members to control the throttle
306 notification forwarding.
309 senf::ppi::module::Module \n
312 \section ppi_events Events
314 Modules may register additional events. These external events are very important since they
315 drive the PPI framework. Events are like external calls into the module network which are sent
316 whenever some event happens. Some possible events are
317 \li timer events (senf::ppi::IntervalTimer)
318 \li read or write events on some file descriptor (senf::ppi::IOEvent)
319 \li internal events (senf::ppi::IdleEvent)
321 The PPI really is not concerned, how the events are called and what information is needed to
322 perform the call. This is handled by the <a
323 href="@TOPDIR@/Scheduler/doc/html/index.html">Scheduler</a>, which is wrapped by the event
326 All events are derived from senf::ppi::EventDescriptor. The base class allows to enable and
327 disable the event. Each type of event will take descriptor specific constructor arguments to
328 describe the event to be generated. Events are declared as (private) data members of the
329 module and are then registered using senf::ppi::module::Module::registerEvent().
331 Each event when signaled is described by an instance of the descriptor specific \e
332 descriptorType \c ::Event class. This instance will hold the event specific information (like
333 scheduled time of the event, file handle state and so on). This information is passed to the
336 Additionaly, events are valid routing targets. This feature allows events to be disabled and
337 enabled by throtling notifications. For the sake of routing, an event may be used like an active
338 input or output. Iit is \e active from the PPI's point of view since it is signaled from the
339 outside and not by some module. It may be either input or output depending on the operation the
342 If we take into account event routing, we can extend the \c RateStuffer constructor accordingly:
345 RateStuffer(unsigned packetsPerSecond)
346 : timer_(1000u, packetsPerSecond)
348 route(payload, output);
349 route(stuffing, output);
350 route(timer_, output); // (*)
352 registerEvent( timer_, &RateStuffer::tick );
356 We have added the marked route call. This way, the \c timer_ will receive throttling
357 notifications from the output: Whenever the output is throttled, the event will be disabled
358 until the output is unthrottled again.
360 \see senf::ppi::EventDescriptor
362 \section ppi_run Running the network
364 After the network has been set up, senf::ppi::run() is called to execute it. This call will only
365 return after all data has been processed. The PPI knows this, when no events are enabled any
366 more. Without events, nothing will happen any more since it is the events which drive the
367 PPI. Therefore the PPI surmises, that all data has been processed and returns from
370 This works very well with automatic throttling. When no data is available to be processed any
371 more and no more data can be expected to arrive (for Example since data has been read from a
372 file which is now exhausted) all events will be disabled automatically via trhottle
373 notifications and so signal that any processing should stop.
375 \section ppi_flows Information Flow
377 The above description conceptually introduces three different flow levels:
379 \li The <em>data flow</em> is, where the packets are flowing. This flow always goes from output
381 \li The <em>execution flow</em> describes the flow of execution from one module to another. This
382 flow always proceeds from active to passive connector.
383 \li The <em>control flow</em> is the flow of throttling notifications. This flow always proceeds
384 \e opposite to the execution flow, from passive to active connector.
386 This is the outside view, from without any module. These flows are set up using
387 senf::ppi::connect() statements.
389 Within a module, the different flow levels are defined differently depending on the type of
392 \li The <em>data flow</em> is defined by how data is processed. The different event and
393 connector callbacks will pass packets around and thereby define the data flow
394 \li Likewise, the <em>execution flow</em> is defined parallel to the data flow (however possible
395 in opposite direction) by how the handler of one connector calls other connectors.
396 \li The <em>control flow</em> is set up using senf::ppi::Module::route statements (as long as
397 automatic throttling is used. Manual throttling defines the control flow within the
398 respective callbacks).
400 In nearly all cases, these flows will be parallel. Therefore it makes sense to define the \c
401 route statement as defining the 'conceptual data flow' since this is also how control messages
402 should flow (sans the direction, which is defined by the connectors active/passive property).
404 \see \ref ppi_implementation
407 /** \page ppi_implementation Implementation Notes
409 \section processing Data Processing
411 The processing in the PPI is driven by events. Without events <em>nothing will happen</em>. When
412 an event is generated, the called module will probably call one of it's active connectors.
414 Calling an active connector will directly call the handler registered at the connected passive
415 connector. This way the call and data are handed across the connections until an I/O module will
416 finally handle the request (by not calling any other connectors).
418 Throttling is handled in the same way: Throttling a passive connector will call a corresponding
419 (internal) method of the connected active connector. This method will call registered handlers
420 and will analyze the routing information of the module for other (passive) connectors to call
421 and throttle. This will again create a call chain which terminates at the I/O modules. An event
422 which is called to be throttled will disable the event temporarily. Unthrottling works in the
425 This simple structure is complicated by the existence of the input queues. This affects both
426 data forwarding and throttling:
427 \li A data request will only be forwarded, if no data is available in the queue
428 \li The connection will only be throttled when the queue is empty
429 \li Handlers of passive input connectors must be called repeatedly until either the queue is
430 empty or the handler does not take any packets from the queue
433 \section ppi_logistics Managing the Data Structures
435 The PPI itself is a singleton. This simplifies many of the interfaces (We do not need to pass
436 the PPI instance). Should it be necessary to have several PPI systems working in parallel
437 (either by registering all events with the same event handler or by utilizing multiple threads),
438 we can still extend the API by adding an optional PPI instance argument.
440 Every module manages a collection of all it's connectors and every connector has a reference to
441 it's containing module. In addition, every connector maintains a collection of all it's routing
444 All this data is initialized via the routing statements. This is, why \e every connector must
445 appear in at least one routing statement: These statements will as a side effect initialize the
446 connector with it's containing module.
448 Since all access to the PPI via the module is via it's base class, unbound member function
449 pointers can be provided as handler arguments: They will automatically be bound to the current
450 instance. This simplifies the PPI usage considerably. The same is true for the connectors: Since
451 they know the containing module, they can explicitly bind unbound member function pointers to
454 \section ppi_random_notes Random implementation notes
456 Generation of throttle notifications: Backward throttling notifications are automatically
457 generated (if this is not disabled) whenever the input queue is non-empty \e after the event
458 handler has finished processing. Forward throttling notifications are not generated
459 automatically within the connector. However, the Passive-Passive adaptor will generate
460 Forward-throttling notifications whenever the input queue is empty.
462 \section ppi_classdiagram Class Diagram
464 <div class="diamap" name="classes">
465 <span coords="652,428,796,455">\ref senf::ppi::connector::PassiveConnector</span>
466 <span coords="198,381,316,408">\ref senf::ppi::EventManager</span>
467 <span coords="462,543,571,570">\ref senf::ppi::connector::ActiveOutput</span>
468 <span coords="468,494,564,521">\ref senf::ppi::connector::ActiveInput</span>
469 <span coords="414,36,505,63">\ref senf::ppi::RouteBase</span>
470 <span coords="432,325,529,379">\ref senf::ppi::Route</span>
471 <span coords="194,154,319,181">\ref (some module)</span>
472 <span coords="19,293,252,333">\ref senf::ppi::EventImplementation</span>
473 <span coords="225,36,289,63">\ref senf::ppi::module::Module</span>
474 <span coords="309,331,397,358">\ref senf::ppi::connector::Connector</span>
475 <span coords="597,543,717,570">\ref senf::ppi::connector::PassiveOutput</span>
476 <span coords="66,432,210,459">\ref senf::ppi::detail::EventBindingBase</span>
477 <span coords="378,428,505,455">\ref senf::ppi::connector::InputConnector</span>
478 <span coords="491,124,694,210">\ref senf::ppi::detail::RouteImplementation</span>
479 <span coords="283,464,423,491">\ref senf::ppi::connector::OutputConnector</span>
480 <span coords="512,428,645,455">\ref senf::ppi::connector::ActiveConnector</span>
481 <span coords="85,487,259,527">\ref senf::ppi::detail::EventBinding</span>
482 <span coords="39,216,170,243">\ref senf::ppi::EventDescriptor</span>
483 <span coords="604,494,710,521">\ref senf::ppi::connector::PassiveInput</span>
485 \htmlonly <img src="classes.png" border="0" alt="classes" usemap="#classes"> \endhtmlonly
492 // c-file-style: "senf"
493 // indent-tabs-mode: nil
494 // ispell-local-dictionary: "american"