4 // Fraunhofer Institute for Open Communication Systems (FOKUS)
6 // The contents of this file are subject to the Fraunhofer FOKUS Public License
7 // Version 1.0 (the "License"); you may not use this file except in compliance
8 // with the License. You may obtain a copy of the License at
9 // http://senf.berlios.de/license.html
11 // The Fraunhofer FOKUS Public License Version 1.0 is based on,
12 // but modifies the Mozilla Public License Version 1.1.
13 // See the full license text for the amendments.
15 // Software distributed under the License is distributed on an "AS IS" basis,
16 // WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
17 // for the specific language governing rights and limitations under the License.
19 // The Original Code is Fraunhofer FOKUS code.
21 // The Initial Developer of the Original Code is Fraunhofer-Gesellschaft e.V.
22 // (registered association), Hansastraße 27 c, 80686 Munich, Germany.
25 // Stefan Bund <g0dil@berlios.de>
28 /** \mainpage Defining and using a new 'libPacket' Packet Type
30 This howto will introduce the facilities needed to define a new packet type. As example, the
31 \c GREPacket type is defined.
36 \section howto_newpacket_overview Overview
38 The packet library supports two basic packet representations, the more generic one being
39 senf::Packet. This representation does not know anything about the type of packet, its fields or
40 properties. It really only is a bunch of bytes. Possibly there is a preceding packet (header) or
41 a following one, but that is all, a senf::Packet knows.
43 The second representation is implemented by senf::ConcretePacket. This representation derives
44 from senf::Packet and adds information about the packet type, its fields, eventually some
45 invariants or packet specific operations etc. In what follows, we will concentrate on this
46 latter representation.
48 A concrete packet type in senf provides a lot of detailed information about a specific type of
51 \li It provides access to the packets fields
52 \li It may provide additional packet specific functions (e.g. calculating or validating a
54 \li It provides information on the nesting of packets
55 \li It implements packet invariants
57 To define a new packet type, we need to implement two classes which together provide all this
60 \li a \e parser (a class derived from senf::PacketParserBase). This class defines the data
61 fields of the packet header and may also provide additional packet specific functionality.
62 \li a \e packet \e type (a class derived from senf::PacketTypeBase). This class defines, how
63 packets are nested and how to initialize and maintain invariants.
65 The following sections describe how to define these classes. Where appropriate, we will use GRE
66 (Generic Routing Encapsulation) as an example.
68 \section howto_newpacket_gre Introducing the GRE example packet type
70 When defining a new packet type, we start out by answering two important questions:
72 \li What kind of parser is needed for this packet type (fixed size or variable sized).
73 \li Whether the packet has another packet as payload (a nested packet) and how the type of this
74 payload is found (whether a registry is used and if yes, which).
76 In the case of GRE, these questions can be answered by just looking at the GRE specification in
77 <a href="http://tools.ietf.org/html/rfc2784">RFC 2784</a>. In Section 2.1 we find the header
81 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
82 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
83 |C| Reserved0 | Ver | Protocol Type |
84 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
85 | Checksum (optional) | Reserved1 (Optional) |
86 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
89 This header is followed by the payload data.
91 Using this protocol definition, we see that the header incorporates optional fields. Therefore
92 it must be dynamically sized: if the \a Checksum \a Present bit \a C is set, both \a Checksum
93 and \a Reserved1 are present, otherwise both must be omitted.
95 Further inspection of the RFC reveals that the \a Protocol \a Type is used to define the type of
96 payload which directly follows the GRE header. This value is an <a
97 href="http://www.iana.org/assignments/ethernet-numbers">ETHERTYPE</a> value. To allow the packet
98 library to automatically parse the GRE payload data, we need to tell the packet library which
99 ETHERTYPE is implemented by which packet type. This kind of association already exists in the
100 form of the senf::EtherTypes registry. Our GRE packet will therefore use this registry.
104 \li The GRE packet header is a dynamically sized header.
105 \li The GRE packet header uses the senf::EtherTypes registry for next-header selection.
108 \section howto_newpacket_parser Implementing the packet parser
110 Each parser is responsible for turning a bunch of bytes into an interpreted header with specific
111 fields. A parser instance is initialized with an iterator (pointer) to the first byte to be
112 interpreted (the first byte of the packet data) and provides member functions to access the
113 header fields. You could implement these members manually, but the SENF library provides a large
114 set of helper macros which simplify this task considerably.
116 \subsection howto_newpacket_parser_skeleton The PacketParser skeleton
119 #include <senf/Packets.hh>
121 struct GREPacketParser : public senf::PacketParserBase
123 # include SENF_PARSER()
128 SENF_PARSER_FINALIZE(GREPacketParser);
132 This is the standard skeleton of any parser class: We need to inherit senf::PacketParserBase and
133 start out by including either \ref SENF_PARSER() or \ref SENF_FIXED_PARSER(), depending on
134 whether we define a fixed size or a dynamically sized parser. As \c GREPacketParser is
135 dynamically sized, we include \ref SENF_PARSER().
137 The definition of fields will be described in the next subsection.
139 After the fields have been defined, we need to call the \ref SENF_PARSER_FINALIZE() macro to
140 close of the parser definition. This call takes the name of the parser being defined as it's
143 This is already a valid parser, albeit not a very usable one, since it does not define any
144 fields. We now go back to define the parser fields and begin with the simple part: fields which
147 \subsection howto_newpacket_parser_simple Simple field definitions
149 Packet parser fields are defined using special \ref packetparsermacros. We take the fields
150 directly from the packet definition (the GRE RFC in this case). This will give us to the
151 following code fragment:
154 SENF_PARSER_BITFIELD ( checksumPresent, 1, bool );
155 SENF_PARSER_SKIP_BITS ( 12 );
156 SENF_PARSER_BITFIELD ( version, 3, unsigned );
157 SENF_PARSER_BITFIELD ( protocolType, 16, unsigned );
160 This is a correct \c GREPacket header definition, but there is room for a small optimization:
161 Since the \a protocolType field is exactly 2 bytes wide and is aligned on a byte boundary, we
162 can define it as a UInt16 field (instead of a bitfield):
165 SENF_PARSER_BITFIELD ( checksumPresent, 1, bool );
166 SENF_PARSER_SKIP_BITS ( 12 );
167 SENF_PARSER_BITFIELD ( version, 3, unsigned );
168 SENF_PARSER_FIELD ( protocolType, senf::UInt16Parser );
171 Whereas \ref SENF_PARSER_BITFIELD can only define bit-fields, \ref SENF_PARSER_FIELD can define
172 almost arbitrary field types. The type is specified by passing the name of another parser to
173 \ref SENF_PARSER_FIELD.
175 It is important to understand, that the accessors do \e not return the parsed field value.
176 Instead, they return another \e parser which is used to further interpret the value. This is due
177 to the inherently recursive nature of the SENF packet parsers, that allows us to define rather
178 complex header formats if needed. Of course, at some point we will hit bottom and need real
179 values. This is, what <em>value parsers</em> do: they interpret some bytes or bits and return
180 the value of that field (not a parser). Examples are the bitfield parsers returned by the
181 accessors generated by SENF_PARSER_BITFIELD (like senf::UIntFieldParser) or the
184 What is going on inside the macros above? Basically, they define accessor functions for a
185 specific field, like \a checksumPresent() or \a protocolType(). They also manage a <em>current
186 Offset</em>. This value is advanced according to the field size whenever a new field is defined
187 (and since this parser is defined as a dynamically sized parser, this offset is not constant but
188 an expression which calculates the offset of a field depending on the preceding data).
190 \subsection howto_newpacket_parser_variant Defining optional fields: The 'variant' parser
192 The parser is currently very simple, and it could have been defined as a fixed size parser. Now
193 for the tricky part: defining parsers the optional fields. The mechanism described here is
194 suitable for a single optional field as well as for an optional contiguous sequence of fields.
196 In our GRE example, there are two fields which need to be enabled/disabled en bloc. We first
197 define an auxiliary sub-parser which combines the two fields.
200 struct GREPacketParser_OptFields : public senf::PacketParserBase
202 # include SENF_FIXED_PARSER()
204 // typedef checksum_t uint16_t; XXX defined automatically???
206 SENF_PARSER_FIELD ( checksum, senf::UInt16Parser );
207 SENF_PARSER_SKIP ( 2 );
209 SENF_PARSER_FINALIZE(GREPacketParser_OptFields);
213 This parser only parses the two optional fields, the second ("Reserved1") field just being
214 skipped. It is a fixed size parser, as indicated by the SENF_FIXED_PARSER() macro. We can
215 now use \ref SENF_PARSER_VARIANT() to add it as an optional parser to the GRE header in our \c
216 GREPacketParser implementation (the typedef'ed checksum_t will be used later on):
219 SENF_PARSER_BITFIELD ( checksumPresent, 1, bool );
220 SENF_PARSER_SKIP_BITS ( 12 );
221 SENF_PARSER_BITFIELD ( version, 3, unsigned );
223 SENF_PARSER_FIELD ( protocolType, senf::UInt16Parser );
225 SENF_PARSER_VARIANT ( optionalFields, checksumPresent,
226 (senf::VoidPacketParser)
227 (GREPacketParser_OptFields) );
230 For a variant parser, two things need to be specified: a selector and a list of variant parsers.
231 The selector is a distinct parser field that is used to decide which variant to choose. In this
232 simple case, the field must be an unsigned integer (more precisely: a value parser returning a
233 value which is implicitly convertible to \c unsigned). This value is used as an index into the
234 list of variant types. So in our case, the value 0 (zero) is associated with
235 senf::VoidPacketParser, whereas the value 1 (one) is associated with \c
236 GREPacketParser_OptFields. senf::VoidPacketParser is a special (empty or no-op) parser which is
237 used in a variant to denote a case in which the variant parser should not parse anything.
239 This parser will work, it is however not very safe and not very usable. If \a p is a
240 GREPacketParser instance, than we would access the fields via:
242 p.checksumPresent() = true;
244 p.protocolType() = 0x86dd;
245 p.optionalFields().get<1>().checksum() = 12345u;
248 This code has two problems:
249 \li accessing the checksum field is quite unwieldy
250 \li changing the checksumPresent() value will break the parser
252 The second problem is caused by the fact that the variant parser needs to be informed whenever
253 the selector (here \a checksumPresent) is changed, since the variant parser must ensure that the
254 header data stays consistent. Whenever the checksumPresent field is enabled, the variant parser
255 needs to insert additional 4 bytes of data. And it must remove those bytes whenever the
256 checksumPresent field is disabled.
258 \subsection howto_newpacket_parser_fixvariant Fixing access by providing custom accessor members
260 The problems outlined above will happen whenever we use variant parsers, and they will often
261 occur with other complex parsers too (most XXX \ref parsercollection reference some field
262 external to themselves, and they will break if that value is changed without them knowing about
263 it). There might be other reasons to restrict access to a field: the field may be set
264 automatically or it may be calculated from other values (we'll see later how to do this).
266 In all these cases we will want to disallow the user to directly change the value, while still
267 allowing to read the value. To do this, we can mark \e value \e fields as read-only:
270 SENF_PARSER_BITFIELD_RO ( checksumPresent, 1, bool );
273 \e Value \e fields are fields implemented by parsers returning a simple value (i.e. bit-field,
274 integer and some additional parsers like those parsing network addresses) as apposed to complex
277 In this case however, we still want to allow the user to change the field value, albeit not
278 directly. We will need to go through the collection parser, in this case the variant.
280 The syntax for accessing a variant is quite cumbersome. Therefore we adjust the variant
281 definition to generate a more usable interface:
284 SENF_PARSER_VARIANT ( optionalFields_, checksumPresent,
285 (novalue(disable_checksum, senf::VoidPacketParser))
286 ( id(checksum, GREPacketParser_OptFields)) );
289 Here, we added some optional information to the variants type list.
291 With this information, \ref SENF_PARSER_VARIANT() will create some additional \e public accessor
292 members and will automatically make the variant itself private. The members generated work like:
294 void disable_checksum() const { optionalFields_().init<0>; }
296 typedef GREPacketParser_OptFields checksum_t;
297 checksum_t checksum() const { return optionalFields_().get<1>(); }
298 void init_checksum() const { optionalFields_().init<1>; }
299 bool has_checksum() const { return optionalFields_().variant() == 1; }
301 (Again: We don't implement these fields ourselves, this is done by \ref SENF_PARSER_VARIANT())
303 \c disable_checksum() and \c init_checksum() change the selected variant. This will
304 automatically change the \c checksumPresent() field accordingly.
306 The \c GREPacketParser is now simple and safe to use. The only responsibility of the user now is to
307 only access \a checksum() if the \a checksumPresent() field is set. Otherwise, the behavior is
308 undefined (in debug builds, the parser will terminate the application with an assert).
311 \subsection howto_newpacket_parser_add Providing additional functionality
313 We have now implemented parsing all the header fields. However, often packets would benefit from
314 additional functionality. In the case of GRE, this could be a function to calculate the checksum
315 value if it is enabled. Defining this member will also show, how to safely access the raw packet
316 data from a parser member.
319 #include <senf/Utils/IpChecksum.hh>
321 checksum_t::checksum_t::value_type calculateChecksum() const
323 if (!checksumEnabled())
327 cs.feed( i(), i(4) );
328 // Skip even number of 0 bytes (the 2 bytes checksum field)
329 // cs.feed(0); cs.feed(0);
330 cs.feed( i(6), data().end() );
336 This code just implements what is defined in the RFC: The checksum covers the complete GRE
337 packet including it's header with the checksum field temporarily set to 0. Instead of really
338 changing the checksum field we manually pass the correct data to \a cs.
340 We use the special <tt>i(</tt><i>offset</i><tt>)</tt> helper to get iterators \a offset number
341 of bytes into the data. This helper has the additional benefit of range-checking the returned
342 iterator and is thereby safe from errors due to truncated packets: If the offset is out of
343 range, a TruncatedPacketException will be thrown.
345 The \a data() function on the other hand returns a reference to the complete data container of
346 the packet under inspection (the GRE packet in this case). Access to \a data() should be
347 restricted as much as possible. It is safe when defining new packet parsers (parsers, which
348 parser a complete packet like GREPacketParser). It's usage from sub parsers (like
349 GREPacketParser_OptFields or even senf::UInt16Parser) would be much more arcane and should be
353 \subsection howto_newpacket_parser_final The complete GREPacketParser implementation
355 So this is now the complete implementation of the \c GREPacketParser:
358 #include <senf/Packets.hh>
360 struct GREPacketParser_OptFields : public senf::PacketParserBase
362 # include SENF_FIXED_PARSER()
364 SENF_PARSER_FIELD ( checksum, senf::UInt16Parser );
365 SENF_PARSER_SKIP ( 2 );
367 SENF_PARSER_FINALIZE(GREPacketParser_OptFields);
370 struct GREPacketParser : public senf::PacketParserBase
372 # include SENF_PARSER()
374 SENF_PARSER_BITFIELD_RO ( checksumPresent, 1, bool );
375 SENF_PARSER_SKIP_BITS ( 12 );
376 SENF_PARSER_BITFIELD ( version, 3, unsigned );
378 SENF_PARSER_FIELD ( protocolType, senf::UInt16Parser );
380 SENF_PARSER_PRIVATE_VARIANT ( optionalFields_, checksumPresent,
381 (novalue(disable_checksum, senf::VoidPacketParser))
382 ( id(checksum, GREPacketParser_OptFields)) );
384 SENF_PARSER_FINALIZE(GREPacketParser);
386 checksum_t::checksum_t::value_type calculateChecksum() const;
389 // In the implementation (.cc) file:
391 #include <senf/Utils/IpChecksum.hh>
393 GREPacketParser::checksum_t::value_type GREPacketParser::calculateChecksum() const
395 if (!checksumEnabled())
400 cs.feed( i(), i()+4 );
401 // Skip even number of 0 bytes (the 2 bytes checksum field)
402 // cs.feed(0); cs.feed(0);
403 cs.feed( i()+6, data().end() );
410 \section howto_newpacket_type Defining the packet type
412 After defining the packet parser, the <em>packet type</em> must be defined. This class is used
413 as a policy and collects all the information necessary to be known about the packet type.
415 The <em>packet type</em> class is \e never instantiated. It has only typedef, constants or
418 \subsection howto_newpacket_type_skeleton The packet type skeleton
420 For every type of packet, the <em>packet type</em> class will look roughly the same. If the
421 packet uses a registry and is not hopelessly complex, the packet type will almost always
425 #include <senf/Packets.hh>
428 : public senf::PacketTypeBase,
429 public senf::PacketTypeMixin<GREPacketType, EtherTypes>
431 typedef senf::PacketTypeMixin<GREPacketType, EtherTypes> mixin;
432 typedef senf::ConcretePacket<GREPacketType> packet;
433 typedef senf::GREPacketParser parser;
435 using mixin::nextPacketRange;
436 using mixin::nextPacketType;
438 using mixin::initSize;
440 // Define members here
444 We note, that it derives from two classes: senf::PacketTypeBase and
445 senf::PacketTypeMixin. senf::PacketTypeBase must be inherited by every packet type class. the
446 senf::PacketTypeMixin provides default implementations for some members which are useful for
447 most kinds of packets. If a packet type is very complex and these defaults don't work, the mixin
448 class can and should be left out. More on this (what the default members do exactly and when the
449 mixin can be used) can be found in the senf::PacketTypeMixin documentation.
451 Of the typedefs, only \a parser is mandatory. It defines the packet parser to use to interpret
452 this type of packet. \a mixin and \a packet are defined to simplify the following
453 definitions (More on \a packet and senf::ConcretePacket later).
455 The next block of statements imports all the default implementations provided by the mixin
458 \li \a nextPacketRange provides information about where the next packet lives within the GRE
460 \li \a nextPacketType provides the type of the next packet from information in the GRE packet.
461 \li \a init is called to initialize a new GRE packet. This call is forwarded to \c
462 GREPacketParser::init.
463 \li \a initSize is called to find the size of an empty (newly create) GRE packet. This is also
464 provided by GREPacketParser.
466 With these default implementations provided by the mixin, only a few additional members are
467 needed to complete the \c GREPacketType: \a nextPacketKey, \a finalize, and \a dump.
470 \subsection howto_newpacket_type_registry Utilizing the packet registry
472 A packet registry maps an arbitrary key value to a type of packet represented by a packet
473 factory instance. There may be any number of packet registries. When working with packet
474 registries, there are three separate steps:
475 \li Using the registry to tell the packet library, what type of packet to instantiate for the
477 \li Given a payload packet of some type, set the appropriate payload type field in the packet
478 header to the correct value (inverse of above).
479 \li Adding packets to the registry.
481 We want the GRE packet to utilize the senf::EtherTypes registry to find the type of packet
482 contained in the GRE payload. The details have already been taken care of by the
483 senf::PacketTypeMixin (it provides the \a nextPacketType member). However, to lookup the packet
484 in the registry, the mixin needs to know the key value. To this end, we implement \a
485 nextPacketKey(), which is very simple:
488 static key_t nextPacketKey(packet p) { return p->protocolType(); }
491 Since all \c GREPacketType members are static, they are passed the packet in question as an
492 argument. \a nextPacketKey() just needs to return the value of the correct packet field. And
493 since the \c packet type (as defined as a typedef) allows direct access to the packet parser
494 using the <tt>-></tt> operator, we can simply access that value.
496 The \c key_t return type is a typedef provided by the mixin class. It is taken from the type of
497 registry, in this case it is senf::EtherTypes::key_t (which is defined as a 16 bit unsigned
500 With this information, the packet library can now find out the type of packet needed to parse
501 the GRE payload -- as long as the \a protocolType() is registered with the senf::EtherTypes
502 registry. If this is not the case, the packet library will not try to interpret the payload, it
503 will return a senf::DataPacket.
505 One special case of GRE encapsulation occurs when layer 2 frames and especially ethernet frames
506 are carried in the GRE payload. The ETHERTYPE registry normally only contains layer 3 protocols
507 (like IP or IPX) however for this special case, the value 0x6558 has been added to the ETHERTYPE
508 registry. So we need to add this value to inform the packet library to parse the payload as an
509 ethernet packet if the \a protocolType() is 0x6558. This happens in the implementation file (the
513 #include <senf/Packets/DefaultBundle/EthernetPacket.hh>
515 SENF_PACKET_REGISTRY_REGISTER( senf::EtherTypes, 0x6558, senf::EthernetPacket );
518 This macro registers the value 0x6558 in the senf::EtherTypes registry and associates it with
519 the packet type senf::EthernetPacket. This macro declares an anonymous static variable, it
520 therefore must always be placed in the implementation file and \e never in an include file.
522 Additionally, we want the GRE packet to be parsed when present as an IP payload. Therefore we
523 additionally need to register GRE in the senf::IpTypes registry. Looking at the <a
524 href="http://www.iana.org/assignments/protocol-numbers">IP protocol numbers</a>, we find that
525 GRE has been assigned the value 47:
528 #include <senf/Packets/DefaultBundle/IPv4Packet.hh>
530 SENF_PACKET_REGISTRY_REGISTER( senf::IpTypes, 47, GREPacket );
533 But wait -- what is \c GREPacket ? This question is answered a few section further down.
535 The last thing we need to do is, we need to set the \a protocolType() field to the correct value
536 when packets are newly created or changed. This is done within \a finalize:
539 static void finalize(packet p) { p->protocolType() << key(p.next(senf::nothrow)); }
542 The \c key() function is provided by the mixin class: It will lookup the \e type of a packet in
543 the registry and return that packets key in the registry. If the key cannot be found, the return
544 value is such that the assignment is effectively skipped.
547 \subsection howto_newpacket_type_invariants Providing packet invariants
549 Many packets have some invariants that must hold: The payload size must be equal to some field,
550 a checksum must match and so on. When packets are newly created or changed, these invariants
551 have to be updated to be correct. This is the responsibility of the \a finalize() member.
554 static void finalize(packet p)
556 p->protocolType() << key(p.next(senf::nothrow));
557 if (p->checksumPresent())
558 p->checksum() << p->calculateChecksum();
562 We already used finalize above to set the \a protocolType() field. Now we add code to update the
563 \a checksum() field if present (this always needs to be done last since the checksum depends on
564 the other field values).
566 Here we are using the more generic parser assignment expressed using the \c << operator. This
567 operator in the most cases works like an ordinary assignment, however it can also be used to
568 assign parsers to each other efficiently and it supports 'optional values' (as provided by <a
569 href="http://www.boost.org/doc/libs/release/libs/optional/index.html">Boost.Optional</a> and
570 as returned by \c key()).
573 \subsection howto_newpacket_type_dump Writing out a complete packet: The 'dump()' member
575 For diagnostic purposes, every packet should provide a meaningful \a dump() member which writes
576 out the complete packet. This member is simple to implement and is often very helpful when
577 tracking down problems.
580 #include <boost/io/ios_state.hpp>
582 static void dump(packet p, std::ostream & os)
584 boost::io::ios_all_saver ias(os);
585 os << "General Routing Encapsulation:\n"
586 << " checksum present : " << p->checksumPresent() ? "true" : "false" << "\n"
587 << " version : " << p->version() << "\n"
588 << " protocol type : 0x" << std::hex << std::setw(4) << std::setfill('0')
589 << p->protocolType() << "\n";
590 if (p->checksumPresent())
591 os << " checksum : 0x" << std::hex << std::setw(4)
592 << std::setfill('0') << p->checksum() << "\n";
596 This member is quite straight forward. We should try to adhere to the formating standard shown
597 above: The first line should be the type of packet/header being dumped followed by one line for
598 each protocol field. The colon's should be aligned at column 33 with the field name indented by
601 The \c boost::ios_all_saver is just used to ensure, that the stream formatting state is restored
602 correctly at the end of the method. An instance of this type will save the stream state when
603 constructed and will restore that state when destructed.
605 \subsection howto_newpacket_type_final Final touches
607 The \c GREPacket implementation is now almost complete. The only thing missing is the \c
608 GREPacket itself. \c GREPacket is just a typedef for a specific senf::ConcretePacket template
609 instantiation. Here the complete GREPacket definition:
612 #include <senf/Packets.hh>
615 : public senf::PacketTypeBase,
616 public senf::PacketTypeMixin<GREPacketType, EtherTypes>
618 typedef senf::PacketTypeMixin<GREPacketType, EtherTypes> mixin;
619 typedef senf::ConcretePacket<GREPacketType> packet;
620 typedef senf::GREPacketParser parser;
622 using mixin::nextPacketRange;
623 using mixin::nextPacketType;
625 using mixin::initSize;
627 static key_t nextPacketKey(packet p) { return p->protocolType(); }
629 static void finalize(packet p) {
630 p->protocolType() << key(p.next(senf::nothrow));
631 if (p->checksumPresent()) p->checksum() << p->calculateChecksum();
634 static void dump(packet p, std::ostream & os);
637 typedef GREPacketType::packet GREPacket;
639 // In the implementation (.cc) file:
641 #include <senf/Packets/DefaultBundle/EthernetPacket.hh>
642 #include <senf/Packets/DefaultBundle/IPv4Packet.hh>
644 SENF_PACKET_REGISTRY_REGISTER( senf::EtherTypes, 0x6558, senf::EthernetPacket );
645 SENF_PACKET_REGISTRY_REGISTER( senf::IpTypes, 47, GREPacket );
647 void GREPacketType::dump(packet p, std::ostream & os)
649 boost::io::ios_all_saver ias(os);
650 os << "General Routing Encapsulation:\n"
651 << " checksum present : " << p->checksumPresent() ? "true" : "false" << "\n"
652 << " version : " << p->version() << "\n"
653 << " protocol type : 0x" << std::hex << std::setw(4) << std::setfill('0')
654 << p->protocolType() << "\n";
655 if (p->checksumPresent())
656 os << " checksum : 0x" << std::hex << std::setw(4)
657 << std::setfill('0') << p->checksum() << "\n";
662 \section howto_newpacket_advanced Going further
664 \subsection howto_newpacket_advanced_valid Checking the GRE packet for validity
666 We now know how to define packets, but there is more. In this section we will explore the
667 features available to make the packet chaining more flexible. We will show, how to implement
668 more complex logic than simple registry lookup to find the nested packet (the payload) type.
670 In our concrete example, reading the RFC we find there are some restrictions which a GRE packet
671 needs to obey to be considered valid. If the packet is not valid it cannot be parsed and should
672 be dropped. We can't drop it here but if the packet is invalid, we certainly must refrain from
673 trying to parser any payload since we cannot assume the packet to have the format we assume our
676 There are two conditions defined in the RFC which render a GRE packet invalid: If one of the \a
677 reserved0() fields first 5 bits is set or if the version is not 0. We will add a \a valid()
678 check to the parser and utilize this check in the packet type.
680 So first lets update the parser. We will need to change the fields a little bit so we have
681 access to the first 5 bits of \a reserved0. We therefore replace the first three field
685 SENF_PARSER_BITFIELD_RO ( checksumPresent, 1, bool );
686 SENF_PARSER_PRIVATE_BITFIELD ( reserved0_5bits_, 5, unsigned );
687 SENF_PARSER_SKIP_BITS ( 7 );
688 SENF_PARSER_BITFIELD_RO ( version, 3, unsigned );
691 We have added an additional private bitfield \a reserved0_5bits_() and we made the \a version()
694 We will now add a simple additional member to the parser:
697 bool valid() const { return version() == 0 && reserved0_5bits_() == 0; }
700 I think, this is quite straight forward: \a valid() will just check the restrictions as defined
703 Now to the packet type. We want to refrain from parsing the payload if the packet is
704 invalid. This is important: If the packet is not valid, we have no idea, whether the payload is
705 what we surmise it to be (if any of the \a reserved0_5bits_() are set, the packet is from an
706 older GRE RFC and the header is quite a bit longer so the payload will be incorrect).
708 So we need to change the logic which is used by the packet library to find the type of the next
709 packet. We have two ways to do this: We keep using the default \c nextPacketType()
710 implementation as provided by the senf::PacketTypeMixin and have our \a nextPacketKey()
711 implementation return a key value which is guaranteed never to be registered in the registry.
713 The more flexible possibility is implementing \c nextPacketType() ourselves. In this case, the
714 first method would suffice, but we will choose to go the second route to show how to write the
715 \c nextPacketType() member. We therefore remove the \c using declaration of \c nextPacketType()
716 and also remove the \a nextPacketKey() implementation. Instead we add the following to the
720 // disabled: using mixin::nextPacketType;
722 factory_t nextPacketType(packet p) { return p->valid() ? lookup(p->protocolType()) : no_factory(); }
725 As we see, this is still quite simple. \c factory_t is provided by senf::PacketTypeBase. For our
726 purpose it is an opaque type which somehow enables the packet library to create a new packet of
727 a specified packet type. The \c factory_t has a special value, \c no_factory() which stands for
728 the absence of any concrete factory. In a boolean context this (and only this) \c factory_t
729 value tests \c false.
731 The \c lookup() member is provided by the senf::PacketTypeMixin. It looks up the key passed as
732 argument in the registry and returns the factory or \c no_factory(), if the key was not found in
735 In this case this is all. But let's elaborate on this example. What if we need to return some
736 specific factory from \a nextPacketType(), e.g. what, if we want to handle the case of
737 transparent ethernet bridging explicitly instead of registering the value in the
738 senf::EtherTypes registry ? Here one way to do this:
741 factory_t nextPacketType(packet p) {
743 if (p->protocolType() == 0x6558) return senf::EthernetPacket::factory();
744 else return lookup(p->protocolType());
746 else return no_factory();
750 As can be seen above, every packet type has a (static) \a factory() member which returns the
751 factory for this type of packet.
754 \subsection howto_newpacket_advanced_init Non-trivial packet initialization
756 Every packet when created is automatically initialized with 0 bytes (all data bytes will be
757 0). In the case of GRE this is enough. But other packets will need other more complex
758 initialization to be performed.
760 Lets just for the sake of experiment assume, the GRE packet would have to set \a version() to 1
761 not 0. In this case, the default initialization would not suffice. It is however very simple to
762 explicitly initialize the packet. The initialization happens within the parser. We just add
765 SENF_PARSER_INIT() { version_() << 1u; }
768 to \c GREPacketParser. For every read-only defined field, the macros automatically define a \e
769 private read-write accessor which may be used internally. This read-write accessor is used here
770 to initialize the value.
773 \section howto_newpacket_final The ultimate GRE packet implementation completed
775 So here we now have \c GREPacket finally complete in all it's glory. First the header file \c
779 #ifndef HH_GREPacket_
780 #define HH_GREPacket_
782 #include <senf/Packets.hh>
784 struct GREPacketParser_OptFields : public senf::PacketParserBase
786 # include SENF_FIXED_PARSER()
788 SENF_PARSER_FIELD ( checksum, senf::UInt16Parser );
789 SENF_PARSER_SKIP ( 2 );
791 SENF_PARSER_FINALIZE(GREPacketParser_OptFields);
794 struct GREPacketParser : public senf::PacketParserBase
796 # include SENF_PARSER()
798 SENF_PARSER_BITFIELD_RO ( checksumPresent, 1, bool );
799 SENF_PARSER_PRIVATE_BITFIELD ( reserved0_5bits_, 5, unsigned );
800 SENF_PARSER_SKIP_BITS ( 7 );
801 SENF_PARSER_BITFIELD_RO ( version, 3, unsigned );
803 SENF_PARSER_FIELD ( protocolType, senf::UInt16Parser );
805 SENF_PARSER_PRIVATE_VARIANT ( optionalFields_, checksumPresent,
806 (novalue(disable_checksum, senf::VoidPacketParser))
807 ( id(checksum, GREPacketParser_OptFields)) );
809 bool valid() const { return version() == 0 && reserved0_5bits_() == 0; }
811 SENF_PARSER_FINALIZE(GREPacketParser);
813 checksum_t::checksum_t::value_type calculateChecksum() const;
817 : public senf::PacketTypeBase,
818 public senf::PacketTypeMixin<GREPacketType, EtherTypes>
820 typedef senf::PacketTypeMixin<GREPacketType, EtherTypes> mixin;
821 typedef senf::ConcretePacket<GREPacketType> packet;
822 typedef senf::GREPacketParser parser;
824 using mixin::nextPacketRange;
826 using mixin::initSize;
828 factory_t nextPacketType(packet p)
829 { return p->valid() ? lookup(p->protocolType()) : no_factory(); }
831 static void finalize(packet p) {
832 p->protocolType() << key(p.next(senf::nothrow));
833 if (p->checksumPresent()) p->checksum() << p->calculateChecksum();
836 static void dump(packet p, std::ostream & os);
839 typedef GREPacketType::packet GREPacket;
844 And the implementation file \c GREPacket.cc:
847 #include "GREPacket.hh"
848 #include <senf/Utils/IpChecksum.hh>
849 #include <senf/Packets/DefaultBundle/EthernetPacket.hh>
850 #include <senf/Packets/DefaultBundle/IPv4Packet.hh>
852 SENF_PACKET_REGISTRY_REGISTER( senf::EtherTypes, 0x6558, senf::EthernetPacket );
853 SENF_PACKET_REGISTRY_REGISTER( senf::IpTypes, 47, GREPacket );
855 GREPacketParser::checksum_t::checksum_t::value_type GREPacketParser::calculateChecksum() const
857 if (!checksumEnabled())
862 cs.feed( i(), i()+4 );
863 // Skip even number of 0 bytes (the 2 bytes checksum field)
864 // cs.feed(0); cs.feed(0);
865 cs.feed( i()+6, data().end() );
870 void GREPacketType::dump(packet p, std::ostream & os)
872 boost::io::ios_all_saver ias(os);
873 os << "General Routing Encapsulation:\n"
874 << " checksum present : " << p->checksumPresent() ? "true" : "false" << "\n"
875 << " version : " << p->version() << "\n"
876 << " protocol type : 0x" << std::hex << std::setw(4) << std::setfill('0')
877 << p->protocolType() << "\n";
878 if (p->checksumPresent())
879 os << " checksum : 0x" << std::hex << std::setw(4)
880 << std::setfill('0') << p->checksum() << "\n";
885 \section howto_newpacket_using Using the newly created GRE packet type
887 The GRE packet is now fully integrated into the packet library framework. For example, we can
888 read GRE packets from a raw INet socket and forward decapsulated Ethernet frames to a packet
893 #include <senf/Packets.hh>
894 #include <senf/Packets/DefaultBundle/EthernetPacket.hh>
895 #include <senf/Socket/Protocols/INet/RawINetProtocol.hh>
896 #include <senf/Socket/Protocols/Raw/PacketSocketHandle.hh>
897 #include "GREPacket.hh"
899 int main(int, char const **)
901 senf::RawV6ClientSocketHandle isock (47u); // 47 = Read GRE packets
902 senf::PacketSocketHandle osock;
906 GREPacket gre (GREPacket::create(senf::noinit));
907 isock.read(gre.data(),0u);
908 if (gre->checksumPresent() && gre->checksum() != gre->calculateChecksum())
909 throw InvalidPacketChainException();
910 osock.write(gre.next<EthernetPacket>().data())
912 catch (senf::TruncatedPacketException & ex) {
913 std::cerr << "Malformed packet dropped\n";
915 catch (senf::InvalidPacketChainException & ex) {
916 std::cerr << "Invalid GRE packet dropped\n";
922 Or we can do the opposite: Read ethernet packets from a \c tap device and send them out GRE
927 #include <senf/Packets.hh>
928 #include <senf/Packets/DefaultBundle/EthernetPacket.hh>
929 #include <senf/Socket/Protocols/INet/RawINetProtocol.hh>
930 #include <senf/Socket/Protocols/Raw/TunTapSocketHandle.hh>
931 #include "GREPacket.hh"
933 int main(int argc, char const ** argv)
936 std::cerr << "Usage: " << argv[0] << " <tunnel endpoint>\n";
940 senf::TapSocketHandle tap ("tap0");
941 senf::ConnectedRawV6ClientSocketHandle osock (47u, senf::INet6SocketAddress(argv[1]));
944 senf::EthernetPacket eth (senf::EthernetPacket::create(senf::noinit));
945 isock.read(eth.data(),0u);
946 GREPacket gre (senf::GREPacket::createBefore(eth));
948 osock.write(gre.data());
954 \section howto_newpacket_further Further reading
956 Lets start with references to the important API's (Use the <i>List of all members</i> link to
957 get the complete API of one of the classes and templates):
959 <table class="senf fixedcolumn">
961 <tr><td>senf::ConcretePacket</td> <td>this is the API provided by the packet handles.</td></tr>
963 <tr><td>senf::PacketData</td> <td>this API provides raw data access accessible via the handles
964 'data' member.</td></tr>
966 <tr><td>senf::PacketParserBase</td> <td>this is the generic parser API. This API is accessible
967 via the packets \c -> operator or via the sub-parsers returned by the field accessors.</td></tr>
971 When implementing new packet's, the following information will be helpful:
973 <table class="senf fixedcolumn">
975 <tr><td>senf::PacketTypeBase</td> <td>here you find a description of the members which need to
976 be implemented to provide a 'packet type'. Most of these members will normally be provided by
977 the mixin helper.</td></tr>
979 <tr><td>senf::PacketTypeMixin</td> <td>here you find all about the packet type mixin and how to
982 <tr><td>\ref packetparser</td> <td>This section describes the packet parser facility.</td></tr>
984 <tr><td>\link packetparsermacros Packet parser macros\endlink</td> <td>A complete list and
985 documentation of all the packet parser macros.</td></tr>
987 <tr><td>\ref parseint, \n \ref parsecollection</td> <td>There are several lists of available
988 reusable packet parsers. However, these lists are not complete as there are other protocol
989 specific reusable parsers (without claiming to be exhaustive: senf::INet4AddressParser,
990 senf::INet6AddressParser, senf::MACAddressParser)</td></tr>
1000 // comment-column: 40
1001 // c-file-style: "senf"
1002 // indent-tabs-mode: nil
1003 // ispell-local-dictionary: "american"
1004 // compile-command: "scons -u doc"
1007 // vim:filetype=doxygen:textwidth=100: