X-Git-Url: http://g0dil.de/git?a=blobdiff_plain;f=HowTos%2FNewPacket%2FMainpage.dox;h=054754398b4185c3a93067243e39cbf9853c4bd8;hb=d270abd20469df1cac466fe24d8dfa3dc119d987;hp=8769831a388e02a250f774a5b5fb10ebca61ef68;hpb=6d612eda73e43f9304bd178aff02020a6e7be3fe;p=senf.git diff --git a/HowTos/NewPacket/Mainpage.dox b/HowTos/NewPacket/Mainpage.dox index 8769831..0547543 100644 --- a/HowTos/NewPacket/Mainpage.dox +++ b/HowTos/NewPacket/Mainpage.dox @@ -28,10 +28,49 @@ \autotoc - \section howto_newpacket_start Getting started + \section howto_newpacket_overview Overview - Before starting with the implementation, we look at the specification of the GRE packet. This is - found in RFC 2784 in Section 2.1: + The packet library supports two basic packet representations, the more generic one being + senf::Packet. This representation does not know anything about the type of packet, its fields or + properties. It really only is a bunch of bytes. Possibly there is a preceding packet (header) or + a following one, but that is all, a senf::Packet knows. + + The second representation is implemented by senf::ConcretePacket. This representation derives + from senf::Packet and adds information about the packet type, its fields, eventually some + invariants or packet specific operations etc. In what follows, we will concentrate on this + latter representation. + + A concrete packet type in senf provides a lot of detailed information about a specific type of + packet: + + \li It provides access to the packets fields + \li It may provide additional packet specific functions (e.g. calculating or validating a + checksum) + \li It provides information on the nesting of packets + \li It implements packet invariants + + To define a new packet type, we need to implement two classes which together provide all this + information: + + \li a \e parser (a class derived from senf::PacketParserBase). This class defines the data + fields of the packet header and may also provide additional packet specific functionality. + \li a \e packet \e type (a class derived from senf::PacketTypeBase). This class defines, how + packets are nested and how to initialize and maintain invariants. + + The following sections describe how to define these classes. Where appropriate, we will use GRE + (Generic Routing Encapsulation) as an example. + + \section howto_newpacket_gre Introducing the GRE example packet type + + When defining a new packet type, we start out by answering two important questions: + + \li What kind of parser is needed for this packet type (fixed size or variable sized). + \li Whether the packet has another packet as payload (a nested packet) and how the type of this + payload is found (whether a registry is used and if yes, which). + + In the case of GRE, these questions can be answered by just looking at the GRE specification in + RFC 2784. In Section 2.1 we find the header + layout:
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 @@ -42,34 +81,32 @@ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- Using this protocol definition, we can decide the first important question: Whether the packet - header is fixed size or dynamically sized. As we see above, the header incorporates optional - fields. Therefore it must be dynamically sized. The RFC further details, that if the \a Checksum - \a Present bit \a C is set, both \a Checksum and \a Reserved1 are present, otherwise they must - both be omitted. + This header is followed by the payload data. + + Using this protocol definition, we see that the header incorporates optional fields. Therefore + it must be dynamically sized: if the \a Checksum \a Present bit \a C is set, both \a Checksum + and \a Reserved1 are present, otherwise both must be omitted. - Another information we take from the RFC is, that the \a Protocol \a Type is used to define the - type of payload which directly follows the GRE header. This value is an ETHERTYPE value. To allow the packet library to automatically parse the GRE payload data, we need to tell the packet library which - ETHERTYPE represents which packet type. This association already exists in form of the - senf::EtherTypes registry. Our GRE packet will therefore utilize this registry. + ETHERTYPE is implemented by which packet type. This kind of association already exists in the + form of the senf::EtherTypes registry. Our GRE packet will therefore use this registry. - To summarize, we have gathered the following information: + To summarize: \li The GRE packet header is a dynamically sized header. - \li The GRE packet header utilizes the senf::EtherTypes registry for next-header selection + \li The GRE packet header uses the senf::EtherTypes registry for next-header selection. - \section howto_newpacket_parser Implementing the GRE Parser - - The next step in creating a new packet type is to implement the parser. The parser is - responsible for turning a bunch of bytes into an interpreted header with specific fields. The - parser will later be constructed with an iterator (pointer) to the first byte to be interpreted - as a GRE header and will provide member functions to access the header fields. You can implement - these members manually but the SENF library provides a large set of helper macros which simplify - this task considerably. + \section howto_newpacket_parser Implementing the packet parser + Each parser is responsible for turning a bunch of bytes into an interpreted header with specific + fields. A parser instance is initialized with an iterator (pointer) to the first byte to be + interpreted (the first byte of the packet data) and provides member functions to access the + header fields. You could implement these members manually, but the SENF library provides a large + set of helper macros which simplify this task considerably. \subsection howto_newpacket_parser_skeleton The PacketParser skeleton @@ -81,27 +118,33 @@ # include SENF_PARSER() // Define fields + // (see below) SENF_PARSER_FINALIZE(GREPacketParser); }; \endcode This is the standard skeleton of any parser class: We need to inherit senf::PacketParserBase and - start out by including either \ref SENF_PARSER() or \ref SENF_FIXED_PARSER(). Which, depends on + start out by including either \ref SENF_PARSER() or \ref SENF_FIXED_PARSER(), depending on whether we define a fixed size or a dynamically sized parser. As \c GREPacketParser is dynamically sized, we include \ref SENF_PARSER(). - After the fields are defined, we need to call the \ref SENF_PARSER_FINALIZE() macro to close of - the parser definition. This call takes the name of the parser being defined as it's sole - argument. + The definition of fields will be described in the next subsection. - This is already a valid parser, albeit not a very usable one since it defines no fields. We now - go back to define the parser fields and begin with the simple part: Those fields which are - always present. + After the fields have been defined, we need to call the \ref SENF_PARSER_FINALIZE() macro to + close of the parser definition. This call takes the name of the parser being defined as it's + sole argument. + This is already a valid parser, albeit not a very usable one, since it does not define any + fields. We now go back to define the parser fields and begin with the simple part: fields which + are always present. \subsection howto_newpacket_parser_simple Simple field definitions + Packet parser fields are defined using special \ref packetparsermacros. We take the fields + directly from the packet definition (the GRE RFC in this case). This will give us to the + following code fragment: + \code SENF_PARSER_BITFIELD ( checksumPresent, 1, bool ); SENF_PARSER_SKIP_BITS ( 12 ); @@ -109,52 +152,51 @@ SENF_PARSER_BITFIELD ( protocolType, 16, unsigned ); \endcode - This is a direct transcript of the field definition above. There are quite a number of macros - which may be used to define fields. All these macros are documented in '\ref - packetparsermacros'. - - This is a correct \c GREPacket header definition but we can optimize a little bit: Since the \a - protocolType field is aligned on a byte boundary, instead of defining it as a bitfield, we can - define it as a UInt16 field: + This is a correct \c GREPacket header definition, but there is room for a small optimization: + Since the \a protocolType field is exactly 2 bytes wide and is aligned on a byte boundary, we + can define it as a UInt16 field (instead of a bitfield): \code SENF_PARSER_BITFIELD ( checksumPresent, 1, bool ); SENF_PARSER_SKIP_BITS ( 12 ); SENF_PARSER_BITFIELD ( version, 3, unsigned ); - SENF_PARSER_FIELD ( protocolType, senf::UInt16Parser ); \endcode - Whereas \ref SENF_PARSER_BITFIELD can define only bit-fields, \ref SENF_PARSER_FIELD can define + Whereas \ref SENF_PARSER_BITFIELD can only define bit-fields, \ref SENF_PARSER_FIELD can define almost arbitrary field types. The type is specified by passing the name of another parser to \ref SENF_PARSER_FIELD. - It is important to understand, that the accessors do \e not return the parsed field value. They - return another \e parser which is used to further interpret the value. This is the inherent - recursive nature of the SENF packet parsers. This allows to define wildly complex header formats - if needed. Of course, at some point we need the real value. This is, what the so called - value parsers do: They interpret some bytes or bits and return the value of that field - (not a parser). Examples are the bitfield parsers returned by the accessors generated by - SENF_PARSER_BITFIELD (like senf::UIntFieldParser) or the senf::UInt16Parser. + It is important to understand, that the accessors do \e not return the parsed field value. + Instead, they return another \e parser which is used to further interpret the value. This is due + to the inherently recursive nature of the SENF packet parsers, that allows us to define rather + complex header formats if needed. Of course, at some point we will hit bottom and need real + values. This is, what value parsers do: they interpret some bytes or bits and return + the value of that field (not a parser). Examples are the bitfield parsers returned by the + accessors generated by SENF_PARSER_BITFIELD (like senf::UIntFieldParser) or the + senf::UInt16Parser. + + What is going on inside the macros above? Basically, they define accessor functions for a + specific field, like \a checksumPresent() or \a protocolType(). They also manage a current + Offset. This value is advanced according to the field size whenever a new field is defined + (and since this parser is defined as a dynamically sized parser, this offset is not constant but + an expression which calculates the offset of a field depending on the preceding data). - What happens in the above macros? Most of the macros define an accessor for a specific field: \a - checksumPresent() or \a protocolType(). They also manage a current Offset. This value - is advanced according to the field size whenever a new field is defined (and since this parser - is defined as a dynamically sized parser, this offset is not a constant, it is an expression - which calculates the offset of a field depending on the preceding data). + \subsection howto_newpacket_parser_variant Defining optional fields: The 'variant' parser + The parser is currently very simple, and it could have been defined as a fixed size parser. Now + for the tricky part: defining parsers the optional fields. The mechanism described here is + suitable for a single optional field as well as for an optional contiguous sequence of fields. - \subsection howto_newpacket_parser_variant Defining optional fields: The 'variant' parser + In our GRE example, there are two fields which need to be enabled/disabled en bloc. We first + define an auxiliary sub-parser which combines the two fields. - We now come to the optional fields. Since there are two fields which need to be disabled/enabled - together, we first need to define an additional sub-parser which combines those two - fields. After this parser is defined, we can use \ref SENF_PARSER_VARIANT() to add this parser - as an optional parser to the GRE header. - \code struct GREPacketParser_OptFields : public senf::PacketParserBase { # include SENF_FIXED_PARSER() + + // typedef checksum_t uint16_t; XXX defined automatically??? SENF_PARSER_FIELD ( checksum, senf::UInt16Parser ); SENF_PARSER_SKIP ( 2 ); @@ -163,9 +205,10 @@ }; \endcode - This parser only parses the two optional fields of which the reserved field is just skipped. The - parser this time is a fixed size parser. We can now use this parser to continue the \c GREPacketParser - implementation: + This parser only parses the two optional fields, the second ("Reserved1") field just being + skipped. It is a fixed size parser, as indicated by the SENF_FIXED_PARSER() macro. We can + now use \ref SENF_PARSER_VARIANT() to add it as an optional parser to the GRE header in our \c + GREPacketParser implementation (the typedef'ed checksum_t will be used later on): \code SENF_PARSER_BITFIELD ( checksumPresent, 1, bool ); @@ -179,17 +222,17 @@ (GREPacketParser_OptFields) ); \endcode - For a variant parser, two things need to be specified: A selector and a list of variant - parsers. The selector is another parser field which is used to decide, which variant to - choose. In this simple case, the field must be an unsigned integer (more precisely a value - parser which returns a value which is implicitly convertible to \c unsigned). This value is used - as index into the list of variant types. So in our case, 0 is associated with - senf::VoidPacketParser whereas 1 is associated with \c - GREPacketParser_OptFields. (senf::VoidPacketParser is a special empty parser which is used in a - Variant to denote cases in which the variant parser should not parse anything) - - This parser will work, it is however not very safe and not very usable. If \a p is a GREPacketParser - instance, than we access the fields via: + For a variant parser, two things need to be specified: a selector and a list of variant parsers. + The selector is a distinct parser field that is used to decide which variant to choose. In this + simple case, the field must be an unsigned integer (more precisely: a value parser returning a + value which is implicitly convertible to \c unsigned). This value is used as an index into the + list of variant types. So in our case, the value 0 (zero) is associated with + senf::VoidPacketParser, whereas the value 1 (one) is associated with \c + GREPacketParser_OptFields. senf::VoidPacketParser is a special (empty or no-op) parser which is + used in a variant to denote a case in which the variant parser should not parse anything. + + This parser will work, it is however not very safe and not very usable. If \a p is a + GREPacketParser instance, than we would access the fields via: \code p.checksumPresent() = true; p.version() = 4u; @@ -197,67 +240,63 @@ p.optionalFields().get<1>().checksum() = 12345u; \endcode - There are two problems here: + This code has two problems: \li accessing the checksum field is quite unwieldy \li changing the checksumPresent() value will break the parser - The reason for the second problem lies in the fact, that the variant parser needs to be informed - whenever the selector (here \a checksumPresent) is changed since the variant parser must ensure, - that the header data stays consistent. In this example, whenever the checksumPresent field is - enabled, the variant parser needs to insert additional 4 bytes of data and remove those bytes, - when the checksumPresent field is disabled. - + The second problem is caused by the fact that the variant parser needs to be informed whenever + the selector (here \a checksumPresent) is changed, since the variant parser must ensure that the + header data stays consistent. Whenever the checksumPresent field is enabled, the variant parser + needs to insert additional 4 bytes of data. And it must remove those bytes whenever the + checksumPresent field is disabled. \subsection howto_newpacket_parser_fixvariant Fixing access by providing custom accessor members - Since we don't want to allow the \a checksumPresent() field to be changed directly, we mark this - field as read-only: + The problems outlined above will happen whenever we use variant parsers, and they will often + occur with other complex parsers too (most XXX \ref parsercollection reference some field + external to themselves, and they will break if that value is changed without them knowing about + it). There might be other reasons to restrict access to a field: the field may be set + automatically or it may be calculated from other values (we'll see later how to do this). + + In all these cases we will want to disallow the user to directly change the value, while still + allowing to read the value. To do this, we can mark \e value \e fields as read-only: \code SENF_PARSER_BITFIELD_RO ( checksumPresent, 1, bool ); \endcode - To change the \a checksumPresent() value, the variant parser provides special members to change - the currently selected variant: + \e Value \e fields are fields implemented by parsers returning a simple value (i.e. bit-field, + integer and some additional parsers like those parsing network addresses) as apposed to complex + sub-parsers. + + In this case however, we still want to allow the user to change the field value, albeit not + directly. We will need to go through the collection parser, in this case the variant. + + The syntax for accessing a variant is quite cumbersome. Therefore we adjust the variant + definition to generate a more usable interface: \code - p.optionalFields().init<0>(); - p.optionalFields().init<1>(); + SENF_PARSER_VARIANT ( optionalFields_, checksumPresent, + (novalue(disable_checksum, senf::VoidPacketParser)) + ( id(checksum, GREPacketParser_OptFields)) ); \endcode - - These statements also change the selector field (in this case \a checksumPresent()) to the - correct value: The first statements switches to the first variant and therefore in this case - disables the checksum field. The second statement will switch to the second variant and enable - the checksum field. - Again, these statements are relatively complex. So we complete the parser by providing simple - additional members which wrap these complicated calls. While doing this, we also mark the - variant as a private field so it is not directly accessible any more (since we now have the - additional helpers which are used to access the variant, we don't want anyone to mess around - with it directly). + Here, we added some optional information to the variants type list. + With this information, \ref SENF_PARSER_VARIANT() will create some additional \e public accessor + members and will automatically make the variant itself private. The members generated work like: \code - SENF_PARSER_PRIVATE_VARIANT ( optionalFields_, checksumPresent, - (senf::VoidPacketParser) - (GREPacketParser_OptFields) ); - - typedef GREPacketParser_OptFields::checksum_t checksum_t; - checksum_t checksum() const - { return optionalFields_().get<1>().checksum(); } + void disable_checksum() const { optionalFields_().init<0>; } - void enableChecksum() const { optionalFields_().init<1>(); } - void disableChecksum() const { optionalFields_().init<0>(); } + typedef GREPacketParser_OptFields checksum_t; + checksum_t checksum() const { return optionalFields_().get<1>(); } + void init_checksum() const { optionalFields_().init<1>; } + bool has_checksum() const { return optionalFields_().variant() == 1; } \endcode + (Again: We don't implement these fields ourselves, this is done by \ref SENF_PARSER_VARIANT()) - Above code has one other twist we need to discuss: the \a checksum_t typedef. This is added as a - convenience to the user of this parser. The \c SENF_PARSER_* macros which define a field all - define some additional symbols providing further information about the field. Of these - additional symbols, the most important is field
_t
, which is the (parser)
- type returned by the field. This helps to work with a parser in more complex situations
- (e.g. when using \ref parsecollection) since it allows to access the parser type without exact
- knowledge of this type (which may become quite complex if templates are involved) as long as the
- field name is known. Since we provide an accessor for the \a checksum field, we also provide the
- \a checksum_t typedef for this accessor.
+ \c disable_checksum() and \c init_checksum() change the selected variant. This will
+ automatically change the \c checksumPresent() field accordingly.
The \c GREPacketParser is now simple and safe to use. The only responsibility of the user now is to
only access \a checksum() if the \a checksumPresent() field is set. Otherwise, the behavior is
@@ -266,15 +305,15 @@
\subsection howto_newpacket_parser_add Providing additional functionality
- The \c GREPacketParser is now complete. But we can do better: A packet parser is not restricted
- to simply parsing data. Depending on the packet type, additional members can be arbitrarily
- defined. In the case of \c GREPacket, we provide one additional member, \a calculateChecksum()
- which does just that: It calculates the checksum of the GRE packet.
+ We have now implemented parsing all the header fields. However, often packets would benefit from
+ additional functionality. In the case of GRE, this could be a function to calculate the checksum
+ value if it is enabled. Defining this member will also show, how to safely access the raw packet
+ data from a parser member.
\code
#include