4 // Fraunhofer Institute for Open Communication Systems (FOKUS)
5 // Competence Center NETwork research (NET), St. Augustin, GERMANY
6 // Stefan Bund <g0dil@berlios.de>
8 // This program is free software; you can redistribute it and/or modify
9 // it under the terms of the GNU General Public License as published by
10 // the Free Software Foundation; either version 2 of the License, or
11 // (at your option) any later version.
13 // This program is distributed in the hope that it will be useful,
14 // but WITHOUT ANY WARRANTY; without even the implied warranty of
15 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
16 // GNU General Public License for more details.
18 // You should have received a copy of the GNU General Public License
19 // along with this program; if not, write to the
20 // Free Software Foundation, Inc.,
21 // 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
23 /** \mainpage HowTo: Defining and using a new 'libPacket' Packet Type
25 This howto will introduce the facilities needed to define a new packet type. As example, the
26 \c GREPacket type is defined.
31 \section howto_newpacket_start Getting started
33 Before starting with the implementation, we look at the specification of the GRE packet. This is
34 found in <a href="http://tools.ietf.org/html/rfc2784">RFC 2784</a> in Section 2.1:
37 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
38 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
39 |C| Reserved0 | Ver | Protocol Type |
40 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
41 | Checksum (optional) | Reserved1 (Optional) |
42 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
45 Using this protocol definition, we can decide the first important question: Whether the packet
46 header is fixed size or dynamically sized. As we see above, the header incorporates optional
47 fields. Therefore it must be dynamically sized. The RFC further details, that if the \a Checksum
48 \a Present bit \a C is set, both \a Checksum and \a Reserved1 are present, otherwise they must
51 Another information we take from the RFC is, that the \a Protocol \a Type is used to define the
52 type of payload which directly follows the GRE header. This value is an <a
53 href="http://www.iana.org/assignments/ethernet-numbers">ETHERTYPE</a> value. To allow the packet
54 library to automatically parse the GRE payload data, we need to tell the packet library which
55 ETHERTYPE represents which packet type. This association already exists in form of the
56 senf::EtherTypes registry. Our GRE packet will therefore utilize this registry.
58 To summarize, we have gathered the following information:
60 \li The GRE packet header is a dynamically sized header.
61 \li The GRE packet header utilizes the senf::EtherTypes registry for next-header selection
64 \section howto_newpacket_parser Implementing the GRE Parser
66 The next step in creating a new packet type is to implement the parser. The parser is
67 responsible for turning a bunch of bytes into an interpreted header with specific fields. The
68 parser will later be constructed with an iterator (pointer) to the first byte to be interpreted
69 as a GRE header and will provide member functions to access the header fields. You can implement
70 these members manually but the SENF library provides a large set of helper macros which simplify
71 this task considerably.
74 \subsection howto_newpacket_parser_skeleton The PacketParser skeleton
77 struct GREPacketParser : public senf::PacketParser
79 # include SENF_PARSER()
83 SENF_PARSER_FINALIZE(GREPacketParser);
87 This is the standard skeleton of any parser class: We need to inherit senf::PacketParser and
88 start out by including either \ref SENF_PARSER() or \ref SENF_FIXED_PARSER(). Which, depends on
89 whether we define a fixed size or a dynamically sized parser. As \c GREPacketParser is dynamically
90 sized, we include \ref SENF_PARSER().
92 After the fields are defined, we need to call the \ref SENF_PARSER_FINALIZE() macro to close of
93 the parser definition. This call takes the name of the parser being defined as it's sole
96 This is already a valid parser, albeit not a very usable one since it defines no fields. We now
97 go back to define the parser fields and begin with the simple part: Those fields which are
101 \subsection howto_newpacket_parser_simple Simple field definitions
104 SENF_PARSER_BITFIELD ( checksumPresent, 1, bool );
105 SENF_PARSER_SKIP_BITS ( 12 );
106 SENF_PARSER_BITFIELD ( version, 3, unsigned );
107 SENF_PARSER_BITFIELD ( protocolType, 16, unsigned );
110 This is a direct transcript of the field definition above. There are quite a number of macros
111 which may be used to define fields. All these macros are documented in '\ref
114 This is a correct \c GREPacket header definition but we can optimize a little bit: Since the \a
115 protocolType field is aligned on a byte boundary, instead of defining it as a bitfield, we can
116 define it as a UInt16 field:
119 SENF_PARSER_BITFIELD ( checksumPresent, 1, bool );
120 SENF_PARSER_SKIP_BITS ( 12 );
121 SENF_PARSER_BITFIELD ( version, 3, unsigned );
123 SENF_PARSER_FIELD ( protocolType, senf::UInt16Parser );
126 Whereas \ref SENF_PARSER_BITFIELD can define only bit-fields, \ref SENF_PARSER_FIELD can define
127 almost arbitrary field types. The type is specified by passing the name of another parser to
128 \ref SENF_PARSER_FIELD.
130 It is important to understand, that the accessors do \e not return the parsed field value. They
131 return another \e parser which is used to further interpret the value. This is the inherent
132 recursive nature of the SENF packet parsers. This allows to define wildly complex header formats
133 if needed. Of course, at some point we need the real value. This is, what the so called
134 <em>value parsers</em> do: They interpret some bytes or bits and return the value of that field
135 (not a parser). Examples are the bitfield parsers returned by the accessors generated by
136 SENF_PARSER_BITFIELD (like senf::UIntFieldParser) or the senf::UInt16Parser.
138 What happens in the above macros? Most of the macros define an accessor for a specific field: \a
139 checksumPresent() or \a protocolType(). They also manage a <em>current Offset</em>. This value
140 is advanced according to the field size whenever a new field is defined (and since this parser
141 is defined as a dynamically sized parser, this offset is not a constant, it is an expression
142 which calculates the offset of a field depending on the preceding data).
145 \subsection howto_newpacket_parser_variant Defining optional fields: The 'variant' parser
147 We now come to the optional fields. Since there are two fields which need to be disabled/enabled
148 together, we first need to define an additional sub-parser which combines those two
149 fields. After this parser is defined, we can use \ref SENF_PARSER_VARIANT() to add this parser
150 as an optional parser to the GRE header.
153 struct GREPacketParser_OptFields : public senf::PacketParser
155 # include SENF_FIXED_PARSER()
157 SENF_PARSER_FIELD ( checksum, senf::UInt16Parser );
158 SENF_PARSER_SKIP ( 2 );
160 SENF_PARSER_FINALIZE(GREPacketParser_OptFields);
164 This parser only parses the two optional fields of which the reserved field is just skipped. The
165 parser this time is a fixed size parser. We can now use this parser to continue the \c GREPacketParser
169 SENF_PARSER_BITFIELD ( checksumPresent, 1, bool );
170 SENF_PARSER_SKIP_BITS ( 12 );
171 SENF_PARSER_BITFIELD ( version, 3, unsigned );
173 SENF_PARSER_FIELD ( protocolType, senf::UInt16Parser );
175 SENF_PARSER_VARIANT ( optionalFields, checksumPresent,
176 (senf::VoidPacketParser)
177 (GREPacketParser_OptFields) );
180 For a variant parser, two things need to be specified: A selector and a list of variant
181 parsers. The selector is another parser field which is used to decide, which variant to
182 choose. In this simple case, the field must be an unsigned integer (more precisely a value
183 parser which returns a value which is implicitly convertible to \c unsigned). This value is used
184 as index into the list of variant types. So in our case, 0 is associated with
185 senf::VoidPacketParser whereas 1 is associated with \c
186 GREPacketParser_OptFields. (senf::VoidPacketParser is a special empty parser which is used in a
187 Variant to denote cases in which the variant parser should not parse anything)
189 This parser will work, it is however not very safe and not very usable. If \a p is a GREPacketParser
190 instance, than we access the fields via:
192 p.checksumPresent() = true;
194 p.protocolType() = 0x86dd;
195 p.optionalFields().get<1>().checksum() = 12345u;
198 There are two problems here:
199 \li accessing the checksum field is quite unwieldy
200 \li changing the checksumPresent() value will break the parser
202 The reason for the second problem lies in the fact, that the variant parser needs to be informed
203 whenever the selector (here \a checksumPresent) is changed since the variant parser must ensure,
204 that the header data stays consistent. In this example, whenever the checksumPresent field is
205 enabled, the variant parser needs to insert additional 4 bytes of data and remove those bytes,
206 when the checksumPresent field is disabled.
209 \subsection howto_newpacket_parser_fixvariant Fixing access by providing custom accessor members
211 Since we don't want to allow the \a checksumPresent() field to be changed directly, we mark this
215 SENF_PARSER_BITFIELD_RO ( checksumPresent, 1, bool );
218 To change the \a checksumPresent() value, the variant parser provides special members to change
219 the currently selected variant:
222 p.optionalFields().init<0>();
223 p.optionalFields().init<1>();
226 These statements also change the selector field (in this case \a checksumPresent()) to the
227 correct value: The first statements switches to the first variant and therefore in this case
228 disables the checksum field. The second statement will switch to the second variant and enable
231 Again, these statements are relatively complex. So we complete the parser by providing simple
232 additional members which wrap these complicated calls. While doing this, we also mark the
233 variant as a private field so it is not directly accessible any more (since we now have the
234 additional helpers which are used to access the variant, we don't want anyone to mess around
238 SENF_PARSER_PRIVATE_VARIANT ( optionalFields_, checksumPresent,
239 (senf::VoidPacketParser)
240 (GREPacketParser_OptFields) );
242 typedef GREPacketParser_OptFields::checksum_t checksum_t;
243 checksum_t checksum() const
244 { return optionalFields_().get<1>().checksum(); }
246 void enableChecksum() const { optionalFields_().init<1>(); }
247 void disableChecksum() const { optionalFields_().init<0>(); }
250 Above code has one other twist we need to discuss: the \a checksum_t typedef. This is added as a
251 convenience to the user of this parser. The \c SENF_PARSER_* macros which define a field all
252 define some additional symbols providing further information about the field. Of these
253 additional symbols, the most important is <em>field</em><code>_t</code>, which is the (parser)
254 type returned by the field. This helps to work with a parser in more complex situations
255 (e.g. when using \ref parsecollection) since it allows to access the parser type without exact
256 knowledge of this type (which may become quite complex if templates are involved) as long as the
257 field name is known. Since we provide an accessor for the \a checksum field, we also provide the
258 \a checksum_t typedef for this accessor.
260 The \c GREPacketParser is now simple and safe to use. The only responsibility of the user now is to
261 only access \a checksum() if the \a checksumPresent() field is set. Otherwise, the behavior is
262 undefined (in debug builds, the parser will terminate the application with an assert).
265 \subsection howto_newpacket_parser_add Providing additional functionality
267 The \c GREPacketParser is now complete. But we can do better: A packet parser is not restricted
268 to simply parsing data. Depending on the packet type, additional members can be arbitrarily
269 defined. In the case of \c GREPacket, we provide one additional member, \a calculateChecksum()
270 which does just that: It calculates the checksum of the GRE packet.
273 checksum_t::value_type calculateChecksum() const
275 if (!checksumEnabled())
279 cs.feed( i(), i()+4 );
280 // Skip even number of 0 bytes (the 2 bytes checksum field)
281 // cs.feed(0); cs.feed(0);
282 cs.feed( i()+6, data().end() );
288 This code just implements what is defined in the RFC: The checksum covers the complete GRE
289 packet including it's header with the checksum field temporarily set to 0. Instead of really
290 changing the checksum field we manually pass the correct data to \a cs.
292 In this code we utilize some additional information provided by senf::PacketParserBase. The \a
293 i() member returns an iterator to the first byte the parser is interpreting whereas \a data()
294 returns a reference to the packet data container for the packet being parsed. Access to \a
295 data() should be restricted as much as possible. It is safe when defining new packet parsers
296 (like GREPacketParser). It's usage from sub parsers (like GREPacketParser_OptFields or even
297 senf::UInt16Parser) would be much more arcane and should be avoided.
300 \subsection howto_newpacket_parser_final The complete GREPacketParser implementation
302 So this is now the complete implementation of the \c GREPacketParser:
305 #include <senf/Packets.hh>
306 #include <senf/Utils/IpChecksum.hh>
308 struct GREPacketParser_OptFields : public senf::PacketParser
310 # include SENF_FIXED_PARSER()
312 SENF_PARSER_FIELD ( checksum, senf::UInt16Parser );
313 SENF_PARSER_SKIP ( 2 );
315 SENF_PARSER_FINALIZE(GREPacketParser_OptFields);
318 struct GREPacketParser : public senf::PacketParser
320 # include SENF_PARSER()
322 SENF_PARSER_BITFIELD_RO ( checksumPresent, 1, bool );
323 SENF_PARSER_SKIP_BITS ( 12 );
324 SENF_PARSER_BITFIELD ( version, 3, unsigned );
326 SENF_PARSER_FIELD ( protocolType, senf::UInt16Parser );
328 SENF_PARSER_PRIVATE_VARIANT ( optionalFields_, checksumPresent,
329 (senf::VoidPacketParser)
330 (GREPacketParser_OptFields) );
332 typedef GREPacketParser_OptFields::checksum_t checksum_t;
333 checksum_t checksum() const
334 { return optionalFields_().get<1>().checksum(); }
336 void enableChecksum() const { optionalFields_().init<1>(); }
337 void disableChecksum() const { optionalFields_().init<0>(); }
339 SENF_PARSER_FINALIZE(GREPacketParser);
341 checksum_t::value_type calculateChecksum() const;
344 GREPacketParser::checksum_t::value_type GREPacketParser::calculateChecksum() const
346 if (!checksumEnabled())
350 cs.feed( i(), i()+4 );
351 // Skip even number of 0 bytes (the 2 bytes checksum field)
352 // cs.feed(0); cs.feed(0);
353 cs.feed( i()+6, data().end() );
360 \section howto_newpacket_type Defining the packet type
362 After we have implemented the \c GREPacketParser we need now to build the packet type. This is
363 done by providing a special policy class called the 'packet type'. This class encapsulates all
364 the information the packet library needs to know about a packet:
367 \subsection howto_newpacket_type_skeleton The packet type skeleton
369 For every type of packet, the 'packet type' class will look roughly the same. If the packet
370 utilizes a registry and is not hopelessly complex, the packet type will almost always look like
374 : public senf::PacketTypeBase,
375 public senf::PacketTypeMixin<GREPacketType, EtherTypes>
377 typedef senf::PacketTypeMixin<GREPacketType, EtherTypes> mixin;
378 typedef senf::ConcretePacket<GREPacketType> packet;
379 typedef senf::GREPacketParser parser;
381 using mixin::nextPacketRange;
382 using mixin::nextPacketType;
384 using mixin::initSize;
386 // Define members here
389 We note, that \c GREPacketType derives from two classes: senf::PacketTypeBase and
390 senf::PacketTypeMixin. senf::PacketTypeBase must be inherited by every packet type class. the
391 senf::PacketTypeMixin provides default implementations for some members which are useful for
392 most kinds of packets. If a packet type is very complex and these defaults don't work, the mixin
393 class can and should be left out.
395 Of the typedefs, only \a parser is mandatory. It defines the packet parser to use to interpret
396 this type of packet. \a mixin and \a packet are defined to simplify the following
397 definitions (More on \a packet and senf::ConcretePacket later).
399 The next block of statements imports all the default implementations provided by the mixin
402 \li \a nextPacketRange provides information about where the next packet lives within the GRE
404 \li \a nextPacketType provides the type of the next packet from information in the GRE packet.
405 \li \a init is called to initialize a new GRE packet. This call is forwarded to \c
406 GREPacketParser::init.
407 \li \a initSize is called to find the size of an empty (newly create) GRE packet. This is also
408 provided by GREPacketParser.
410 With these default implementations provided by the mixin, only a few additional members are
411 needed to complete the \c GREPacketType: \a nextPacketKey, \a finalize, and \a dump
414 \subsection howto_newpacket_type_registry Utilizing the packet registry
416 We want the GRE packet to utilize the senf::EtherTypes registry to find the type of packet
417 contained in the GRE payload. A registry maps an arbitrary key value to a packet type
418 represented by a packet factory instance. The details have already been taken care of by the
419 senf::PacketTypeMixin (it provides the \a nextPacketType member). However, to lookup the packet
420 in the registry, the mixin needs to know the key value. To this end, we implement \a
421 nextPacketKey(), which is very simple:
424 static key_t nextPacketKey(packet p) { return p->protocolType(); }
427 All \c GREPacketType members are static. They are passed the packet in question as an
428 argument. \a nextPacketKey() just needs to return the value of the correct packet field. And
429 since the \c parser type (as defined as a typedef) allows direct access to the packet parser
430 using the <tt>-></tt> operator, we can simply access that value.
432 The \c key_t return type is a typedef provided by the mixin class it is taken from the type of
433 registry, in this case it is senf::EtherTypes::key_t (which is defined as a 16 bit unsigned
436 With this information, the packet library can now find out the type of packet needed to parse
437 the GRE payload -- as long as the protocolType() is registered with the senf::EtherTypes
438 registry. If this is not the case, the packet library will not try to interpret the payload, it
439 will return a senf::DataPacket.
441 One special case of GRE encapsulation occurs when layer 2 frames and especially ethernet frames
442 are carried in the GRE payload. The ETHERTYPE registry normally only contains layer 3 protocols
443 (like IP or IPX) however for this special case, the value 0x6558 has been added to the ETHERTYPE
444 registry. So we need to add this value to inform the packet library to parse the payload as an
445 ethernet packet if the \a protocolType() is 0x6558. This happens in the implementation file (the
449 SENF_PACKET_REGISTRY_REGISTER( senf::EtherTypes, 0x6558, senf::EthernetPacket );
452 This macro registers the value 0x6558 in the senf::EtherTypes registry and associates it with
453 the packet type senf::EthernetPacket. This macro declares an anonymous static variable, it
454 therefore must always be placed in the implementation file and \e never in an include file.
456 Additionally, we want the GRE packet to be parsed when present as an IP payload. Therefore we
457 additionally need to register GRE in the senf::IpTypes registry. Looking at the <a
458 href="http://www.iana.org/assignments/protocol-numbers">IP protocol numbers</a>, we find that
459 GRE has been assigned the value 47:
462 SENF_PACKET_REGISTRY_REGISTER( senf::IpTypes, 47, GREPacket );
465 But wait -- what is \c GREPacket ? This question is answered a few section further down.
468 \subsection howto_newpacket_type_invariants Providing packet invariants
470 Many packets have some invariants that must hold: The payload size must be equal to some field,
471 a checksum must match and so on. When packets are newly created or changed, these invariants
472 have to be updated to be correct. This is the responsibility of the \a finalize() member. This
473 is also the place, where the \a protocolType() field is assigned.
476 static void finalize(packet p)
478 if (p->checksumPresent())
479 p->checksum() << p->calculateChecksum();
480 p->protocolType() << key(p->next(senf::nothrow));
484 \a finalize() first updates the \a checksum() field if present. It then sets the \a
485 protocolType() field depending on the \e next packet. The \c key() function is provided by the
486 mixin class: It will lookup the \e type of a packet in the registry and return that packets key
489 Here we are using the more generic parser assignment expressed using the \c << operator. This
490 operator in the most cases works like an ordinary assignment, however it can also be used to
491 assign parsers to each other efficiently and it supports 'optional values' (as provided by <a
492 href="http://www.boost.org/libs/optional/doc/optional.html">Boost.Optional</a> and as returned
495 \fixme Document the needed \c \#include files
496 \fixme Provide an advanced section with additional info: How to ensure, that the first 5 bits in
497 reserver0 are not set. How to enforce version == 0 (that is, make version() read-only and
498 return no_factory for the next packet type if any of the conditions is violated)
505 // comment-column: 40
506 // c-file-style: "senf"
507 // indent-tabs-mode: nil
508 // ispell-local-dictionary: "american"
509 // compile-command: "scons -u doc"