4 // Fraunhofer Institute for Open Communication Systems (FOKUS)
5 // Competence Center NETwork research (NET), St. Augustin, GERMANY
6 // Stefan Bund <g0dil@berlios.de>
8 // This program is free software; you can redistribute it and/or modify
9 // it under the terms of the GNU General Public License as published by
10 // the Free Software Foundation; either version 2 of the License, or
11 // (at your option) any later version.
13 // This program is distributed in the hope that it will be useful,
14 // but WITHOUT ANY WARRANTY; without even the implied warranty of
15 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
16 // GNU General Public License for more details.
18 // You should have received a copy of the GNU General Public License
19 // along with this program; if not, write to the
20 // Free Software Foundation, Inc.,
21 // 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
23 /** \mainpage SENF: The Simple and Extensible Network Framework
25 The SENF Simple and Extensible Network Framework aims to be a complete set of libraries to
26 facilitate the development of network applications focusing on network protocols on the layers
27 below the application layer. However, the framework includes many general purpose utilities and
28 will be expedient to use well beyond its primary objective.
32 The main goals of this library are (in no particular order):
34 \li modular framework design
35 \li utilizing the power of modern C++
36 \li very low overhead for frequently called members
40 \section start Getting started
42 To get started using this library, begin by checking out the code from the <a
43 href="http://developer.berlios.de/svn/?group_id=7489">BerliOS SVN repository</a>. You may find
44 help on using the library at '\ref senf_usage'. If you are interested in SENF, feel free to
45 subscribe to the <a href="http://developer.berlios.de/mail/?group_id=7489">SENF mailing
46 lists</a>. If you want to contribute, read the docs and \e please adhere to the \ref
49 \see \ref senf_usage\n
50 <a href="../../Examples/doc/html/index.html">Examples</a>
52 \section senfutil_overview Building Projects using SENF
54 When building projects using %senf, SENFSCons has a very simple helper module
55 \ref senf_senfutil "senfutil.py" to make the building of libraries utilizing %senf simpler.
57 \see \ref senf_senfutil
60 /** \page senf_senfutil SENF SCons build utility (senfutil.py)
64 \c senfutil helps setting up projects which utilize SENF. It will configure all necessary
65 compiler and linker options and additionally sets up some useful defaults and utilities.
67 \c senfutil really serves three roles
69 \li detect SENF and configure the build accordingly
70 \li make some SCons extensions used within SENF available to other projects
71 \li set default compilation options in the same way, they are set when compiling SENF proper.
73 The last two points are of course optional.
75 \section senfutil_tutorial Tutorial
77 To utilize \c senfutil you need to do two things:
78 \li Update your \c SConstruct file
79 \li add a bootstrap \c senfutil.py to \c site_scons
81 Lets start with the \c SConstruct file
86 senfutil.SetupForSENF(env)
87 senfutil.DefaultOptions(env)
90 PROJECTNAME = 'Example project',
91 PROJECTEMAIL = 'developer@domain.com',
92 DOCLINKS = [ ('Homepage', 'http://www.domain.com') ]
95 sources, tests = senfutil.Glob(env, exclude=['main.cc'])
97 objects = env.Object(sources)
98 example = env.Program('example', objects + ['main.cc'])
99 test = env.BoostUnitTest('test', objects + tests)
103 senfutil.Doxygen(env)
105 senfutil.CleanGlob('all', [ '*~', '#*#' ])
108 This simple sample already enables a lot of functionality:
109 \li support for different \e SENF flavors (debug/normal/final)
110 \li support for different \e build flavors (debug/normal/final)
111 \li sensible default compile options for the different flavors
112 \li support for extended command-line variables
113 \li building documentation with an auto-generated Doxyfile
114 \li running unit-tests
115 \li cleaning backup and temporary files
117 Here a very quick rundown of the important scons commands:
118 \li Build default target:
122 \li Build documentation and unit-tests:
126 \li clean up everything
130 \li Pass custom options on the command-line
132 $ scons CXXFLAGS+=-Wextra
135 Since \c senfutil.py is not on the standard \c python or \c SCons path, some extra steps are
137 \li Either add the possible directories to <tt>sys.path</tt> before importing \c senfutil:
140 sys.path.extend(('/usr/local/lib/senf/site_scons', '/usr/lib/senf/site_scons'))
143 \li Alternatively, install the following utility script as <tt>site_scons/senfutil.py</tt> into
144 your project. This script will search for <tt>site_scons/senfutil.py</tt> in a list of
145 directories and then load the real \c senfutil.py on top of itself. The directories searched
146 include: the current directory and all parents, subdirectories named <tt>senf/</tt>,
147 <tt>Senf/</tt> or <tt>SENF/</tt> thereof, and <tt>/usr/local/lib/senf/</tt> and
148 <tt>/usr/lib/senf/</tt>
151 \section senfutil_features
153 The \c senfutil utility for SCons helps setting up a project to compile against SENF:
155 \li \c senfutil adds all necessary libraries to link against
156 \li \c senfutil will set necessary compile options.
157 \li \c senfutil supports normal, debug and final project build options
158 \li \c senfutil adds support for Boost unit tests
159 \li \c senfutil implements a very simple to use enhanced doxygen build with SENF symbol
161 \li \c senfutil allows specifying variables on the scons command line
162 \li \c senfutil supports more readable compile-time SENF loglevel configuration
164 Using the utility is quite simple
168 sys.path.extend(('senf/site_scons','/usr/lib/senf/site_scons'))
169 import glob, senfutil
172 senfutil.SetupForSENF(env)
173 # senfutil.DefaultOptions(env)
175 # Set or change SCons environment variables with env.Append, env.Replace or env.SetDefault
177 CXXFLAGS = [ '-Wall', '-Woverloaded-virtual' ],
178 CXXFLAGS_final = [ '-O2' ],
179 CXXFLAGS_normal = [ '-O0', '-g' ],
180 CXXFLAGS_debug = [ '$CXXFLAGS_normal' ],
182 LINKFLAGS_normal = [ '-Wl,-S' ],
184 LOGLEVELS_debug = [ 'senf::log::Debug||VERBOSE' ],
186 PROJECTNAME = 'Example project',
187 PROJECTEMAIL = 'developer@domain.com',
188 DOCLINKS = [ ('Homepage', 'http://www.domain.com') ]
191 # Create a list of sources and tests. Sources are all *.cc files, test are *.test.cc
192 sources, tests = senfutil.Glob(env, exclude=['main.cc'] )
194 # Build objects from sources
195 objects = env.Object(sources)
198 env.Default( env.Program( target='example', source=objects + ['main.cc'] ) )
200 # Build a boost unit-test from additional test sources
201 env.BoostUnitTest( 'test', source=objects + tests)
203 # Build a documentation, autogenerates a Doxyfile
204 senfutil.Doxygen(env)
207 This example builds a simple binary from a number of source files (all '.cc' files). It links
208 against the SENF library and automatically sets all the correct compiler options using
209 <tt>senfutil.SetupForSENF( env )</tt>.
211 This script automatically uses a SENF installation either symlinked or imported into the current
212 project in directory 'senf' or, if this directory does not exist, a globally installed SENF.
214 \section senf_senfutil_options Build options
216 \c senfutil supports the <tt>debug=1</tt> or <tt>final=1</tt> build options. These parameters
217 select one of the build configurations 'debug', 'normal' or 'final'. The following variables are
218 supported each with separate values for all three configurations:
225 \c senfutil will detect the type of SENF library used (final or not) and set the correct compile
228 \section senf_senfutil_loglevels Specifying compile-time loglevels
230 To simplify specifying the compile-time loglevel configuration, the build variable \c LOGLEVELS
231 (and it's build configuration specific variants) may be set. This variable will be parsed and
232 converted into the correct \c SENF_LOG_CONF definition. The \c LOGLEVELS Syntax is
234 \par "" \e optional_stream \c | \e optional_area | \e level
236 where \e optional_stream and \e optional_area are optional fully scoped C++ names (e.g. \c
237 senf::log::Debug) and \e level is the loglevel. There must be \e no whitespace in a single
238 specification, multiple specifications are either specified using an array or separated with
241 \section senf_senfutil_default Default options
243 In the example above, all compile options are set manually. To specify the default customary
244 compile options for SENF programs, \c senfutil.DefaultOptions(env) is provided. This function is
248 senfutil.DefaultOptions(env) =>
250 CXXFLAGS = [ '-Wall', '-Woverloaded-virtual' ],
251 CXXFLAGS_final = [ '-O2' ],
252 CXXFLAGS_normal = [ '-O0', '-g' ],
253 CXXFLAGS_debug = [ '$CXXFLAGS_normal' ],
255 LINKFLAGS_normal = [ '-Wl,-S' ],
259 Thus above example can be simplified to
262 sys.path.extend(('senf/site_scons','/usr/lib/senf/site_scons'))
263 import glob, senfutil
266 senfutil.SetupForSENF(env)
267 senfutil.DefaultOptions(env)
269 env.Append( LOGLEVELS_debug = [ 'senf::log::Debug||VERBOSE' ],
270 PROJECTNAME = 'Example project',
271 PROJECTEMAIL = 'developer@domain.com',
272 DOCLINKS = [ ('Homepage', 'http://www.domain.com') ] )
274 sources, tests = senfutil.Glob(env, exclude=['main.cc'] )
276 objects = env.Object(sources)
277 env.Default( env.Program( target='example', source=objects + ['main.cc'] ) )
278 env.BoostUnitTest( 'test', source=objects + tests)
279 senfutil.Doxygen(env)
282 \section senf_senfutil_tests Building unit tests
284 Building unit tests mostly follows a standard pattern
287 # Generate list of sources and tests (sources=*.cc, tests=*.test.cc)
288 extra_sources = ['main.cc']
289 sources, tests = senfutil.Glob(env, exclude=extra_sources)
291 # Build object files needed for both main target and unit tests
292 objects = env.Object(sources)
294 # Build main target, e.g. a Binary with additional sources which are not part of the unit test
295 env.Program('example', objects+extra_sources)
297 # Build unit tests including additional test sources
298 env.BoostUnitTest('test', objects+tests)
301 It is important to exclude the \c main function from the unit-test build since the boost unit
302 test library provides it's own.
304 \section senf_senfutil_doxygen Building documentation
306 Documentation is built using the \c senfutil.Doxygen utility
309 env.Append( PROJECTNAME = "Example project",
310 PROJECTEMAIL = "coder@example.com",
311 COPYRIGHT = "The Master Coders",
312 DOCLINKS = [ ('Homeage', 'http://www.example.com') ],
313 REVISION = 'r'+os.popen('svnversion').read().strip().lower() )
315 senfutil.Doxygen(env)
318 The \c senfutil.Doxygen utility autogenerates a \c Doxyfile.
320 The utility will search for a SENF documentation in the \c senfdoc and \c %senf subdirectories
321 as well as via the senfutil module directory and some other standard locations. If SENF
322 documentation is found, the SENF tagfiles will automatically be added. Links will be resolved
323 to the documentation found.
325 \c senfutil.Doxygen takes some additional optional keyword arguments:
326 \li \c doxyheader: Path of an alternative HTML header
327 \li \c doxyfooter: Path of an alternative HTML footer
328 \li \c doxycss: Path on an alternative CSS file
329 \li \c mydoxyfile: If set to \c True, don't generate or clean the \c Doxyfile\
330 \li \c senfdoc_path: List of additional directories to search for SENF documentation
332 \section senf_senfutil_arguments 'scons' Command line arguments
334 \c senfutil automatically parses SCons command line arguments into the SCons build
335 environment. This allows specifying any parameter on the command line:
337 $ scons CXX=myg++ CXXFLAGS+=-mtune=geode
339 You may either set variables unconditionally using '=' or append values to the end of a list
343 /** \page senf_usage Using the SENF framework
345 The SENF Framework is a collection of loosely coupled modules. The libraries are heavily object
346 oriented and template based. For compatibility reasons, the libraries are therefore built
347 together with every project making use of the framework.
349 When starting a new project based on the SENF framework, it is advisable, to make use of the
350 SENFSCons build environment and use SVN to manage the code repository. This is the
351 configuration, described in this documentation.
353 \see \ref senf_build \n
355 \ref senf_components \n
358 \section senf_preliminaries Preliminaries
360 Before starting the development, make sure to fulfill the following requirements:
362 \li GNU g++, version at least 3.4
363 \li The Boost libraries (http://www.boost.org)
364 \li The SCons build tool (http://www.scons.org)
366 If you want to build the documentation, you additionally need
368 \li Doxygen (http://www.doxygen.org)
369 \li The \c dia diagram editor (http://www.gnome.org/projects/dia/)
370 \li HTML \c tidy (http://tidy.sourceforge.net/)
371 \li The \c xsltproc XSLT processor (http://xmlsoft.org/XSLT/xsltproc2.html)
372 \li The \c graphviz library (http://www.graphviz.org)
374 The library is only tested with gcc-3.4 and 4.0 on Linux. On other POSIX platforms with a BSD
375 Socket API, the library should be usable, possibly with some tweaking (except for the Scheduler,
376 which relies on \c epoll)
378 \section senf_compiler_options Compiler and Linker Options
380 If SENF is compiled in debug mode (SENF_DEBUG is defined), exception messages will automatically
381 include a stack backtrace. For this to work, you need to add the -rdynamic option to all link
382 commands. This feature depends on gcc and the GNU-libc.
384 It is <B>very important</B> that both the SENF library and the application using it are compiled
385 \e both either with or without this compiler switch (-DSENF_DEBUG). Otherwise, the compiler will
386 emit error messages which might be hard to debug.
389 /** \page senf_build Building the SENF framework
391 This procedure will test building the complete framework including the unit tests and the
392 Sniffer test application. This build is \e not needed to use the framework since every project
393 will include the full SENF source code itself (via Subversion).
395 After you have successfully built the library tests, you can continue to setup your own project
398 \see \ref senf_setup \n
399 \ref senf_components \n
402 \section senf_checkout Getting the code
404 To access the code, check out the code from the BerliOS repository. Change to your development
405 directory and use the following subversion command
408 $ svn checkout http://svn.berlios.de/svnroot/repos/senf/trunk senf
411 This will create a new directory \c senf within the current directory. For further documentation
412 on the use of Subversion, see the \c svn man-page or the subversion homepage at
413 http://subversion.tigris.org. A very good introduction and reference to subversion is available
414 at http://svnbook.red-bean.com.
416 \section senf_compile Building
418 To build the library, execute all unit tests and build the Sniffer test application, use
425 in the \c %senf directory. This assumes, that you want to build the library with your default
426 gcc and requires the boost libraries to be available in the system include paths. If this is
427 not the case, you can take a look at <tt>SConfig.template</tt> file. Copy this file to
428 <tt>SConfig</tt> and comment out all the variables you don't want to change (The \e values in
429 the template file are just arbitrary examples).
432 /** \page senf_setup Setting up a new project using SENF
434 The most simple way to use SENF for now is to checkout the svn repository and build SENF
435 yourselves. After you have built SENF, reference your SENF build directory from your build
436 environment. The most flexible way to do this, is to use a symbolic link to your SENF build.
438 Here an example \c SConstruct file for a project using SENF. This script expects SENF to be
439 found in the <tt>%senf</tt> sub-directory of the directory, where the \c SConstruct file is
440 found. This may either be a SENF checkout (if managing your project via subversion, you can use
441 <tt>svn:externals</tt> for this) or a symbolic link to your SENF checkout.
448 LIBS = [ 'senf', 'boost_regex', 'boost_iostreams' ],
449 CXXFLAGS = [ '-Wall', '-Woverloaded-virtual', '-Wno-long-long' ],
455 source = glob.glob('*.cc'),
459 When building against a self-built SENF which will probably be in debug mode, the '-DSENF_DEBUG'
460 option must be added to the compile command.
462 \see \ref senf_components \n
466 /** \page senf_components The SENF modules
468 The framework is made up of several modular components. When using the library, it is possible
469 to selectively choose to use only a subset of the implemented modules.
471 \see \ref senf_overview
473 \section libPPI libPPI: Packet Processing Infrastructure
475 The Packet Processing Infrastructure implements a modular framework for implementing packet
476 oriented network applications. The library provides a large set of pre-defined modules as well
477 as the necessary helpers to implement application specific processing modules.
479 \see <a href="../../senf/PPI/doc/html/index.html">libPPI API reference</a>
481 \section libSocket libSocket: C++ abstraction of the BSD socket API
483 This library provides a high performance and object oriented abstraction of the standard socket
484 API. It utilizes a flexible and extensible policy based design. The library provides predefined
485 types for the important socket types (UDP and TCP sockets etc) including raw and packet sockets.
487 \see <a href="../../senf/Socket/doc/html/index.html">libSocket API reference</a>
489 \section libPackets libPackets: Network packet manipulation
491 This library provides a very flexible infrastructure to parse, create and otherwise manipulate
492 packetized network data. Included is a library of several protocol parsers covering the basic
493 IPv4 and IPv6 network protocols down to the Ethernet layer.
495 \see <a href="../../senf/Packets/doc/html/index.html">libPackets API reference</a>
497 \section libScheduler libScheduler: Asynchronous event handling
499 The scheduler library provides an object oriented interface to the standard UNIX \c select type
500 event dispatcher. It is based on the high performance \c epoll system call. It provides support
501 for read/write events as well as simple timer based events.
503 \see <a href="../../senf/Scheduler/doc/html/index.html">libScheduler API reference</a>
505 \section libUtils libUtils: Collection of arbitrary utilities
507 This library is used be most all of the other modules for miscellaneous tools and utilities. We
510 \li Simple functions to manage daemon processes
511 \li Standard exception classes
512 \li senf::intrusive_refcount to simplify the implementation of classes usable with
514 \li boost::bind extensions
515 \li An interface to the \c g++ de-mangler integrated with type_info
516 \li Typedefs and rudimentary methods to simplify handling high-resolution time values
518 \see <a href="../../senf/Utils/doc/html/index.html">libUtils API reference</a>
520 \section senfscons SENFSCons, the SENF build environment
522 SENF relies on SCons (http://www.scons.org) to build. To further simplify the common tasks, SENF
523 includes a library of custom routines and builders comprising a very concise build
524 environment. Included are a number of templates to help bootstrapping a new project or
527 \see <a href="../../senfscons/doc/html/index.html">SENFSCons reference</a>
530 /** \page senf_overview Introduction to the framework
532 The SENF framework is relatively complex and makes use of advanced features of the C++
533 language. To make the most efficient use of the framework, you should have at least a basic
534 understanding of C++ templates and the standard library concepts.
536 The library implementation at places makes heavy use of advanced template techniques and relies
537 on some very advanced template libraries from Boost. The aim was however for the \e external
538 interface of the library to be as simple as possible without sacrificing important functionality
539 or adversely impacting the runtime performance.
541 As already mentioned several times, the library relies on Boost (http://www.boost.org) as a
542 generic library of high quality reusable C++ components. It also makes frequent use of the
543 standard library. It is designed, to integrate well into both libraries and to use the same
546 \section senf_startup Getting starting developing with SENF
548 To introduce the framework and it's general structure, a simple example application is provided
549 in the SENF repository in the \c %Sniffer module. Peruse this example to get a first look at how
552 When building a network Application with SENF, you will use several modules:
554 \li Use the <a href="../../senf/Socket/doc/html/index.html">Socket library</a> for network
555 communication needs. This library includes support for raw and packet sockets to allow low
556 level network access.
557 \li Use the <a href="../../senf/Scheduler/doc/html/index.html">Scheduler library</a> to
558 coordinate the asynchronous event processing. This drastically reduces the number of threads
559 needed in your application and will greatly enhance the overall responsiveness.
560 \li To interpret low level network packets, use the <a
561 href="../../senf/Packets/doc/html/index.html">Packets library</a>. This library will provide
562 efficient and convenient access to all protocol fields. It supports parsing as well as
563 modifying and creating packets. It has default support for the most important Internet
564 protocols and is highly extensible with new protocols.
565 \li Go over the <a href="../../senf/Utils/doc/html/index.html">Utils library</a>. It contains
567 helpers to simplify tasks like daemonization, exception handling, debugging and so on.
569 The simplest way to get started is: copy the Sniffer application and start to modify it.
571 \see <a href="../../Examples/doc/html/index.html">Examples</a> \n
572 \ref senf_components \n
575 \section senf_conventions Coding Conventions
577 Here we have laid down the coding conventions used throughout the SENF framework. Please ad here
578 to these conventions when changing or adding code. If you use emacs, you can use the C++ IDE for
579 emacs from http://g0dil.de which greatly simplifies following these conventions.
581 \subsection senf_conventions_file_naming File Naming
583 Files should be named according to the main class they define. A single header file should
584 define only one main class. Exceptions to this rule are OK.
587 This simplifies finding the implementation/header for a given class and also reduces the
588 size of each single file.
590 The implementation is divided into a number of different files:
592 <table class="glossary"> <tr><td>\c .h</td><td>C public header</td></tr>
594 <tr><td>\c .hh</td><td>C++ public header</td></tr>
596 <tr><td>\c .ih</td><td>C++ internal header used only by the implementation. This header will
597 probably be included indirectly by the public header but is not meant to be perused by the
598 library user</td></tr>
600 <tr><td>\c .c</td><td>C implementation</td></tr>
602 <tr><td>\c .cc</td><td>C++ implementation of non-inline non-template functions and
605 <tr><td>\c .ct</td><td>C++ implementation of non-inline template functions and members</td></tr>
607 <tr><td>\c .cci</td><td>C++ implementation of inline non-template functions and
610 <tr><td>\c .cti</td><td>C++ implementation of inline template functions and members</td></tr>
612 <tr><td>\c .mpp</td><td>Special include file used for external iteration by the
613 Boost.Preprocessor library</td></tr> </table>
616 There are two part's to this: First, separating the implementation of inlines and templates
617 out of the header file makes the header file much easier to read. This is important, since
618 the header file will be used as a reference by the developers.
620 Separating inline from non-inline members is used together with the \c prefix_ convention
621 below to ensure the correct placement of inline vs non-inline members in the source
622 code. The C++ language requires, that inline members must be included into \e every
623 compilation unit, non-inline members however must be included \e only in one compilation
624 unit. Placing the inline members into a separate file allows to automate this: Simply moving
625 an implementation from one of the inline files into one of the non-inline files will change
626 the type of implementation accordingly.
628 \subsection senf_conventions_type_naming Type Naming
630 SENF prefers the use of the CapitalziedLettersToSeparateWords convention for class names. In
631 this case, class names must start with a capital letter. There are some exceptions to this rule:
632 Types which define new basic data types to be used like other built-in types may be named using
633 lowercase letters plus underscores. Also, if a type or class is directly related to some other
634 library (STL or Boost) which uses the underscore convention, it might be more sensible to follow
635 this convention. This is open to debate.
638 Naming types with capital letters nicely gives a visual clue, that a symbol is a type
639 name. This can also be used by the editor to highlight type names correctly. Additionally,
640 this convention is compact and does not add additional or repeated overhead.
642 \subsection senf_conventions_impl Implementation
644 Only in very few places, SENF allows the use of inline implementations (not to be confused with
645 inline functions). An \e implementation is inline, if it is written directly into the class
646 definition in the header file. Again there are exceptions to this rule but they are very few:
647 \li When defining simple exception classes, the 'what()' member may be defined inline if it
648 returns a string constant.
649 \li It may be OK to use inline implementations for one-line implementations in internal
651 \li The Packet library allows inline implementations for the definition of parsers since doing
652 so outside the declaration just gets to verbose and parsers definitions are quite length but
653 very simple and straight forward.
656 Implementing members inline inside the class declaration makes the declaration much harder
657 to read. Since the declaration in the header file will be used as a reference by the
658 developer, the header files should be as readable as possible.
660 Every function or method implementation in one of the implementation files must \e always be
661 prefixed with \c prefix_. This symbol is defined at the beginning of the file and undefined at
662 the end. The symbol must be defined to be \c inline in the \c .cti and \c .cci files and must be
663 defined empty in the \c .cc and \c .ct files.
666 Together with splitting inlines and non-inlines into separate files, this allows to
667 automatically include the inline definitions at the right places. See above.
669 Private data members are named with a trailing underscore character.
672 This helps distinguishing local variables from parameter names. The trailing underscore
673 does not interfere with other naming conventions and is allowed by the standard (underscore
674 at the beginning of the name are problematic since some classes of names beginning with an
675 underscore are reserved for the standard library implementation)
679 // :vim:textwidth=100
683 // c-file-style: "senf"
684 // indent-tabs-mode: nil
685 // ispell-local-dictionary: "american"
686 // compile-command: "scons doc"