Serialization

Contents

This document acts as a manual. For further implementation details see Serialization Framework (Implementation Details).

For information on serialization format changes, see Serialization format changes.

What is the serialization framework ?

Serialization is the process of converting a data structure or object into a sequence of bits so that it can be stored in a file or memory buffer, or transmitted across a network connection link to be "resurrected" (deserialized) later in the same or another computer environment. The serialization framework that is implemented by MIRA achieves this in a very generic way and additionally extends the C++ language by a very basic "Reflection" concept.

"Reflection" is also known from higher level programming languages of the 3rd generation like Java and C#. It allows to retrieve information on the structures of the program at runtime, e.g. to query the names and types of variables and methods of classes at runtime.

The provided serialization framework supports:

These capabilities are making the serialization framework an important technique that is used for several concepts within the MIRA framework:

What types are supported ?

The serialization framework can serialize and deserialize built-in fundamental C/C++ types and essential STL types like strings natively, i.e. without including any additional header file:

Type

Remarks

fundamental types:
e.g. char, int, uint16, float, bool, ...

native support

arrays:
e.g. int array[10]

native support

enums

native support

std::string

native support

user-defined classes

need to implement a reflect method

Other user-defined types and classes need to implement a "reflect()" method (see Serialization of user-defined types). Most MIRA classes already provide such a method. For classes of external libraries such as STL, boost, etc. several adapters are provided. Those classes can be serialized by simply including the corresponding header:

Type

MIRA header to include

std::list<>

include <serialization/adapters/std/list>

std::vector<>

include <serialization/adapters/std/vector>

std::map<>

include <serialization/adapters/std/map>

std::multimap<>

include <serialization/adapters/std/map>

std::deque<>

include <serialization/adapters/std/deque>

std::set<>

include <serialization/adapters/std/set>

std::multiset<>

include <serialization/adapters/std/set>

std::pair<>

include <serialization/adapters/std/pair>

std::shared_ptr<>

include <serialization/adapters/std/shared_ptr.hpp>

boost::shared_ptr<>

include <serialization/adapters/boost/shared_ptr.hpp>

boost::array<>

include <serialization/adapters/boost/array.hpp>

boost::multi_array<>

include <serialization/adapters/boost/multi_array.hpp>

boost::optional<>

include <serialization/adapters/boost/optional.hpp>

boost::tuple<>

include <serialization/adapters/boost/tuple.hpp>

Eigen::Matrix

include <serialization/adapters/Eigen/Eigen>

cv::Size

include <serialization/adapters/opencv2/core/core.hpp>

cv::Rect

include <serialization/adapters/opencv2/core/core.hpp>

cv::Mat

include <serialization/adapters/opencv2/core/core.hpp>

attention.png

Please be careful with the serialization of platform dependent types, like: size_t, std::size_t and derived types std::streamoff, std::streampos, etc. These types express sizes of memory blocks or positions in those buffers and therefore are different on 32bit and 64bit systems. The problem becomes apparent when the type is serialized as binary content, since data serialized on 32bit system will not be compatible to those generated on 64bit systems. Use uint64 or uint32 instead, which specify the size explicitly.

Usage of Serializers/Deserializers

The actual serialization and deserialization of a value or object is performed by Serializers and Deserializers, respectively. There are different Serializers and Deserializers each of which is able to serialize the data in different formats:

Serializers provide the serialize() method to serialize the value or object into their provided format:

serializer.serialize("myValue", myValue, "A comment describing the value");

Beside the object that is serialized, a name of that value and a description have to be specified. The name is used to identify the value in the serialized data. The description should give the meaning of the value in detail, and is used by serializers in different ways, e.g. it can be stored as comment with the respective data in an XML document (by the XMLSerializer), or for properties it can be shown to the user in a property editor.

The following example shows how to serialize an STL vector into an XML file:

#include <serialization/adapters/std/vector>

// the data to be serialized
std::vector<int> myValue;
...

// create a XMLSerializer that writes into an XML document
XMLDom myXmlDocument;
XMLSerializer serializer(myXmlDocument);

// serialize the data
serializer.serialize("myValue", myValue, "A comment describing the value");

// write the XML document to a file
myXmlDocument.saveToFile("myfile.xml");

The example will generate the following xml output:

<?xml version="1.0" encoding="UTF-8"?>
<root>
<!--A comment describing the value-->
<myValue>
    <item>1</item>
    <item>2</item>
</myValue>
</root>

For deserialization the Deserializers provide the deserialize() method:

deserializer.deserialize("myValue", myValue);

This method again takes the name of the value that should be deserialized and a reference to the object where the content should be deserialized.

The following example deserializes an STL vector from an XML file:

// load the XML document and create the XMLDeserializer:
XMLDom myXmlDocument;
myXmlDocument.loadFromFile("myfile.xml");
XMLDeserializer deserializer(myXmlDocument);

// the object that will be filled with the content
std::vector<int> myValue;

// deserialize the value
deserializer.deserialize("myValue", myValue);

// myValue is now filled with the content that was stored in the XML file
// within the "myValue" node.

The usage of the other serializers is similar. For examples of their usage please refer to the documentation of XMLSerializer, JSONSerializer, BinarySerializer and the corresponding deserializers.

Serialization of user-defined types

In order to use all above mentioned features for your own classes, all you need to do is to add a special reflect() method, that exposes all important members and methods of your class to the serialization and reflection framework. There are two ways of making a class "reflectable": either via an intrusive (modifying the class) or a non-intrusive (not modifying the class) reflect method.

Intrusive way - Modify the class

To make your class "reflectable", the reflect method must be a member of your class:

class MyClass
{
public:
    template<typename Reflector>
    void reflect(Reflector& r)
    {
        r.member("i", mI,      "my integral member");
        r.member("v", mVector, "my vector member");
        r.member("myPtr", mPtr,  "my pointer member");
    }
private:

    int  mI;
    std::vector<float> mVector;
    boost::shared_ptr<Pose2> mPtr;
};

Please note that the reflect() method must exist for each type of reflector as parameter. The easiest and most common way to achieve that is to provide a template method with the following declaration:

template<typename Reflector>
void reflect(Reflector& r)
{
}

If you use Eclipse for software development, you can also use the reflect-method template that is provided within the MIRA code templates by typing reflect and pressing Ctrl+Space.

(In some very special cases, reflect() should behave differently for a specific reflector, this can be achieved by overloading reflect() methods for particular reflector parameter types.)

The reflect() method will be invoked each time an object of that class is serialized/deserialized. The serializer/deserializer object (the "reflector") is passed as parameter to the method. For each member you want to serialize, you must call the member method of the reflector to specify the name of the member, the member variable and a comment that describes the member. The member can be of every type that is serializable itself, i.e fundamental types like float, etc., STL containers, or instances of complex classes that contain a reflect method themselves. In the latter case this process will continue recursively, calling the reflect method of that class until all the data contained in the class is serialized/deserialized.

attention.png

When subclassing, the reflect() method is inherited from a base class automatically. However, if you reimplement the reflect() method in a derived class, please make sure that the reflect() method of your base class is executed, in order to maintain the inheritance of reflected parameters and properties. However, instead of calling that method directly, this should be done through the serializer using its reflectBase() template method, or the macro MIRA_REFLECT_BASE, as shown in the following example:

class MyUnit : public MicroUnit
{
...
    template<typename Reflector>
    void reflect(Reflector& r)
    {
        // call base class reflect
        MIRA_REFLECT_BASE(r, MicroUnit);
        
        r.member("Member", mA, "Comment on A");
        r.property("Property", mB, setter(&MyUnit::setB, this), "Comment on B", 1);
    }
...
};

Non-intrusive version

Sometimes you can not alter the code of a class, e.g. types provided by an external library. Therefore MIRA provides a way to make the class "reflectable" in a non-intrusive way. You just have to add the reflect method as a global function:

class TheirClass : public TheirBase
{
public:

    int  i;
    std::vector<float> c;
    boost::shared_ptr<Pose2> ptr;
};

template<typename Reflector>
void reflect(Reflector& r, TheirBase& ioTheirBase)
{
    ...
}

template<typename Reflector>
void reflect(Reflector& r, TheirClass& ioTheirClass)
{
    // call base class reflect
    MIRA_REFLECT_BASE_NONINTRUSIVE(r, TheirBase, ioTheirClass);

    r.member("i", ioTheirClass.i,      "integral member");
    r.member("v", ioTheirClass.v, "vector member");
    r.member("myPtr", ioTheirClass.ptr,  "pointer member");
}

Again, the reflect() method must have overloads for each type of reflector or be a template method with the following declaration:

template<typename Reflector>
void reflect(Reflector& r, TypeOfClass& ioTypeOfClass)
{
}

Note also that the members must be public accessible for the above example to work. However, if the members are protected and the class provides getter and setter methods, you can use these to reflect the members. See Getters and Setters.

Pointer Serialization and Object Tracking

The serialization framework also supports the serialization of pointers and smart pointers (boost::shared_ptr, std::shared_ptr). When serializing a pointer, it is not sufficient to store the value of the pointer, rather the object it points to must be saved. When the member is loaded later, a new object is created and a new pointer to the object is loaded into the class member.

If the same pointer (pointing at the same object address) is serialized more than once within one object, only one instance is added to the serialized data. When deserialized, data is read back in only for the first pointer, the second (and further) pointer is set to point to the same address as the first one. To do so, all stored objects are tracked by the serialization framework. If you try to serialize a pointer to a previously serialized object, the framework will store a reference to the previously stored object instead of storing the content of the object again. In order to reference other objects, each object has a unique id, that is formed using the object's name and the names of its parent objects separated by a ".".

The following class:

class MyData
{
public:

    template<typename Reflector>
    void reflect(Reflector& r)
    {
        r.member("i", i, "");
        r.member("f", f, "");
        r.member("s", s, "");
    }
    int i;
    float f;
    std::string s;
};

class MyClass
{
public:

    template<typename Reflector>
    void reflect(Reflector& r)
    {
        r.member("v", mVector, "");
        r.member("ptr1", mPtr1, "");
        r.member("ptr2", mPtr2, ""); // points to the same object as mPtr1
        r.member("ptr3", mPtr3, ""); // points to the second element of mVector
    }

private:

    std::vector<MyData> mVector;
    MyData* mPtr1;
    MyData* mPtr2;
    MyData* mPtr3;
};

will be serialized using the XMLSerializer as:

<myObject>
    <v>
        <item>
            <i>1</i>
            <f>3.141</f>
            <s>a string</s>
        </item>
        <item>
            <i>2</i>
            <f>3.282</f>
            <s>another string</s>
        </item>
        <item>
            <i>3</i>
            <f>9.423</f>
            <s>a third string</s>
        </item>
    </v>
    <ptr1>
        <i>1234</i>
        <f>1234.5678</f>
        <s>test test test</s>
    </ptr1>
    <ptr2 ref="myObject.ptr1"/>
    <ptr3 ref="myObject.v[1]"/>
</myObject>

Note, that the pointers "ptr2" and "ptr3", pointing to values already stored before, use references instead of storing the values twice.

attention.png

If you deserialize a normal pointer, the object the pointer points to will be created by the serialization framework using the new operator. You have to make sure that this object is destroyed correctly in order to avoid memory leaks:

MyObject* obj = NULL;

// deserialize the "pointer": a new object will be created and a pointer
// to that object is stored in "obj":
deserializer.deserialize("myObject", obj);

...

// make sure to delete obj, if you do not need it any longer
delete obj; 

Like in many other cases, it is safer to use smart pointers instead:

std::shared_ptr<MyObject> obj;

// deserialize the "pointer"
deserializer.deserialize("myObject", obj);

// object will be freed automatically by the smart pointer

Polymorphic Classes

Special care must be taken when serializing pointers to base classes of polymorphic types, since the pointer may point to one of several possible concrete derived classes. So when the pointer is saved, the class name must be saved, too.

When the pointer is deserialized, the class name is read and an instance of the corresponding class is constructed using the class factory. Finally, the data can be loaded to the newly created instance of the correct type.

Since the serialization framework works closely together with the class factory, when deserializing polymorphic classes, your polymorphic classes must be instantiable by the class factory. Hence, if you want to serialize and deserialize polymorphic classes, these classes must be derived from Object and must contain a MIRA_OBJECT macro. Moreover, these classes must be registered in the class factory and the serialization framework using the MIRA_CLASS_SERIALIZATION as shown in these examples:

class MyBaseClass : public Object
{
MIRA_OBJECT( MyBaseClass  )
public:

    virtual ~MyBaseClass() {}

    template<typename Reflector>
    void reflect(Reflector& r)
    {
    }
};

MIRA_CLASS_SERIALIZATION( MyBaseClass, Object );

class MyClass1 : public MyBaseClass
{
MIRA_OBJECT( MyClass1 )
public:

    template<typename Reflector>
    void reflect(Reflector& r)
    {
        ...
    }
};

MIRA_CLASS_SERIALIZATION( MyClass1, MyBaseClass );


class MyClass2 : public MyBaseClass
{
MIRA_OBJECT( MyClass2 )
public:

    template<typename Reflector>
    void reflect(Reflector& r)
    {
        ...
    }
};

MIRA_CLASS_SERIALIZATION( MyClass2, MyBaseClass );

In the following XML file the class name of a polymorphic object instance is specified:

<myObject class="MyClass2">
...
</myObject>

When the object is deserialized from the above XML file, an object of the class "MyClass2" will be created automatically and the pointer to that class is stored in the pointer "object" which is of the type MyBaseClass*:

MyBaseClass* object;
deserializer.deserialize("myObject", object);
attention.png

Please note that the MIRA_CLASS_SERIALIZATION macro usually needs to be placed within the source file (instead of the class header), to make sure the registration code is instantiated only once.

Class Versioning

Versioning of classes is optional, but can be used to maintain backward compatibility when changes in the serialized members are necessary (adding additional members, removing members, changing the name or order of the members, etc).

If multiple versions have existed in the past, but only a certain version is supported now, you can add a call to requireVersion() to specify a certain version in the reflect() method:

template<typename Reflector>
void reflect(Reflector& r)
{
    r.requireVersion(3, this);
    ...
}

This specifies the current version is 3, and only this version can be used.

When serialized by an XMLSerializer, the output will look like this:

<myObject>
    <version type="MyClass">3</version>
    ...
</myObject>

On deserialization, requireVersion() will throw an exception if the available version differs from the required version when deserializing the class.

If you want to support different versions, you can use version() instead of requireVersion().

template<typename Reflector>
void reflect(Reflector& r)
{
    serialization::VersionType version = r.version(3, this);

    // class history:
    // had only member a in version 1
    //
    // added member b in version 2,
    // version 2 can be created from version 1 by initializing b to 0
    //
    // replaced a and b by c and d in version 3 (current version)
    // version 3 can be created from version 2 by directly reading values of c/d from a/b

    if(version==1) {              // version 1 only had member a,
        r.member("a", c, "");     // corresponding to c in current version
        d = 0;                    // we must not even try reading a second value here, as e.g. for binary
                                  // serialized data there is no structural information in the data itself
                                  // -> the BinaryDeserializer would read the serialized data wrong!
    } else if(version==2) {
        r.member("a", c, "");     // version 2 had members a and b
        r.member("b", d, "");     // corresponding to c and d in current version
    } else if(version==3) {
        r.member("c", c, "");
        r.member("d", d, "");
    }
}

When deserializing the object, version() will return the available class version that is stored in the XML file, etc. Afterwards you can deserialize the specific members depending on the version as in the example above. When serializing an object, version() will always return the version that was specified as parameter since this is the version that is written. That also means older versions are only considered for the case of deserialization, never for serialization.

Do not specify your class to have version 0, always start with version 1 (0 is used as a dummy version value by various reflectors for objects not providing version information).

Versioning in Class Hierarchies

When a class is declared inheriting from a base class, it may happen that both the base and the subclass independently undergo changes over time and different versions exist for both. In that case, it is possible to independently declare a version in each of the reflect() methods.

class Base
{
public:
    template<typename Reflector>
    void reflect(Reflector& r)
    {
        serialization::VersionType v = r.version(2, this);
        if (v == 1) {
            r.member("a", a, "");
            r.member("b", b, "");
        }
        if (v == 2) {
            r.member("a2", a, "");
            r.member("b2", b, "");
        }
    }

    int a;
    int b;
};

class Derived : public Base
{
public:
    template<typename Reflector>
    void reflect(Reflector& r)
    {
        MIRA_REFLECT_BASE(r, Base);
        r.version(1, this);
        r.member("c", c, "");
    }

    int c;
};

This will serialize e.g. to XML like this:

<myObject>
    <version type="Base">2</version>
    <version type="Derived">1</version>
    ...
</myObject>

Here, different versions are assigned to different parts of the same object (which are reflected in separate parts of code), distinguished by type (type name). In order to tell the reflector which type the version refers to, version<Type>() is a template method that is called either using an explicit type template parameter, or with a pointer to the object as additional parameter (employing automatic type deduction by the compiler). In intrusive reflect(), a this pointer can just be used as additional parameter, as seen in the examples above. In non-intrusive reflection, the first form is more common:

template<typename Reflector>
void reflect(Reflector& r, Class& ioClass)
{
    r.template version<Class>(2);
}

On the other hand, not all serializers store the type name in serialized data to distinguish between versions (e.g. the BinarySerializer does not store any meta data). For these, it is very important to not just call the base class' reflect() directly, but use reflectBase() or MIRA_REFLECT_BASE/MIRA_REFLECT_BASE_NONINTRUSIVE to make sure the serializer can separate these portions of reflection and understand they (at least potentially) use own version numbers. This is the case even if version() is not used in one or both parts. (Not yet! Someone might want to add it in later versions of those classes!)

Default Values

Instead of using versioning, using default values often is sufficient to maintain backward compatibility when new members are added to classes. Default values can be specified as optional parameter of the member() method:

template<typename Reflector>
void reflect(Reflector& r)
{
    r.member("i", mI, "my integral member", 123);
}

In the above example mI will be set to the default value 123 if the XML file does not contain the member "i". Additionally, a warning will be printed via the error logging framework. If no default value was specified instead, deserialzing the above object would result in an exception if the member "i" is missing.

Default values that are specified within a class' reflect() method, can also be used to initialize the corresponding members within the constructor. Therefore, a special "DefaultInitializer" reflector is provided which visits the reflect method and initializes all members with the specified default values. To simplify this process even more, you can use the MIRA_INITIALIZE_THIS macro as shown in the following example:

 #include <serialization/DefaultInitializer.h>
 class MyClass
 {
 public:
   MyClass() {
     // initialize our members using their default values
     MIRA_INITIALIZE_THIS;
   }

   template <typename Reflector>
   void reflect(Reflector& r)
   {
     // default value of mMember is 123.45f
     r.member("MyMember", mMember, "", 123.45f);
   }
};
attention.png

If your reflection contains setters or notifiers, MIRA_INITIALIZE_THIS executes them. As with any call from within the constructor, be careful if you end up calling virtual functions (in particular not to try calling a pure virtual function).

Ignoring Missing Parameters

Instead of using a default value, you can also specify serialization::IgnoreMissing as last parameter:

template <typename Reflector>
void reflect(Reflector& r)
{
    r.member("Value", mValue, "will not be set if 'Value' is missing", serialization::IgnoreMissing());
}

This will neither produce an exception, nor set a default value if the parameter "Value" is missing. Instead, the parameter is ignored and its value is not changed at all. This behavior is useful, if the value was set correctly before (e.g. in the constructor) and should not be altered if it is not specified in the configuration file.

Getters and Setters

Instead of using the variable of a member in the reflect method you can specify getter and setter methods the serializers and deserializers should use to access the member. This is useful when additional values or look-up-tables need to be computed after a certain member is deserialized or for converting the values of members before they are serialized and deserialized (e.g. for converting the angle from rad to deg in getAngle() before storing it and for converting it back in setAngle() after restoring it in the example below).

class MyClass
{
public:

    template<typename Reflector>
    void reflect(Reflector& r)
    {
        // A member where the setter is called when new data was
        // deserialized and should be set into the member.
        // When reading, the data is read from the member directly.
        r.member("value", mValue,
                setter(&MyClass::setValue, this), "");

        // A member where the setter and the getter is called
        // for serializing and deserializing the data.
        // Here the member does not need to be specified anymore,
        // it is accessed through the getter and setter only
        r.member("angle",
                getter(&MyClass::getAngle, this),
                setter(&MyClass::setAngle, this), "");
    }

    void setValue(const int &val);

    float getAngle()
    {
        // convert from rad to deg
        return mAngle * 180.0f / M_PI;
    }

    void setAngle(const float &val)
    {
        // convert from deg to rad and set the value
        mAngle = val * M_PI / 180.0f;
    }

private:

    int   mValue;
    float mAngle;
};

Properties

Properties are parameters that can be changed at runtime via a property editor. There are two kinds of properties - read/write properties and read-only properties. They support the same syntax as members, but additionally they provide mechanisms to specify hints like limits or enumerations. Read-only properties also can not have setters. Let's start with a simple example.

template<typename Reflector>
void reflect(Reflector& r)
{
    // a property with default value 1
    r.property("prop1", mValue1, "comment", 1);

    // property with getter and setter
    r.property("prop2", getter(&MyClass::getValue2, this),
            setter(&MyClass::getValue2, this), "");

    // a read only property
    r.roproperty("ROProp", mInt4, "comment");
}

For a graphical property editor it can be useful to specify limits for a property in order to limit input ranges for used editors like spinboxes or sliders. Therefore property hints are used.

template<typename Reflector>
void reflect(Reflector& r)
{
    // a property with default value 1 and a limited range from 0 to 10
    r.property("IntProp1", mInt1, "comment", 1, PropertyHints::limits(0, 10));

    // a property with only a minimum value given
    r.property("IntProp2", mInt2, "comment", PropertyHints::minimum(5));

    // a property with only a maximum value given
    r.property("IntProp3", mInt3, "comment", PropertyHints::maximum(2000));
}

For some editors like sliders or spinboxes it can be useful to specify steps for changing the value. e.g. for a property that should be incremented/decremented in steps of 10 one could write

template<typename Reflector>
void reflect(Reflector& r)
{
    // a property with a step of 10
    r.property("IntProp", mInt, "comment", PropertyHints::step(10));
}

It is even possible to combine these hints in order to allow specifying limits and steps at once:

template<typename Reflector>
void reflect(Reflector& r)
{
    // a property with a step of 10 and a limit of 0 to 1000
    r.property("IntProp", mInt, "comment", PropertyHints::limits(0, 1000) | PropertyHints::step(10));
}

To be able to choose the right editor widget for the property one can specify the type of the property.

template<typename Reflector>
void reflect(Reflector& r)
{
    // a property with type slider
    r.property("IntProp", mInt, "comment", PropertyHints::type("slider"));
}

For convenience there are already two hints for sliders and spin boxes defined:

template<typename Reflector>
void reflect(Reflector& r)
{
    // a property with type slider and a range from 0 to 100 in steps of 2
    r.property("IntProp1", mInt1, "comment", PropertyHints::slider(0,100,2));

    // a property with type spin and a range from 0 to 100 in steps of 2
    r.property("IntProp2", mInt2, "comment", PropertyHints::spin(0,100,2));
}

Some properties allow the user to select from a given set of values. This is called an enumeration and the graphical editor will display a combobox for these properties.

template<typename Reflector>
void reflect(Reflector& r)
{
    // an enumeration property that allows selection of 3 values 0=Value1, 1=Value2 and 2=Value3
    r.property("IntProp1", mInt1, "comment", PropertyHints::enumeration("Value1;Value2;Value3"));
    // an enumeration property that allows selection of index=value pairs where the index is explicitely given
    r.property("IntProp2", mInt2, "comment", PropertyHints::enumeration("-1000=Value1;333=Value2;2000=Value3"));
    r.property("IntProp2", mString, "comment", PropertyHints::enumeration("A=Value1;B=Value2;C=Value3"));

}

Getting notifications when a property is set

Setters offer a powerful mechanism to handle a changed value of a member or property in different ways. However, in some cases you just want to get notified whenever the value of one or more properties is changed. This usually is the case when writing visualization classes. These classes usually have a large number of properties that control the appearance. When such a property is set, usually no special setter shall be called, but the visualisation should be notified to redraw itself in order to visualize the changes immediately. For this purpose, the setterNotify() method is provided. It can be used to create a predefined setter that takes the member whose value should be set and a user defined callback function that is called, whenever the value changes:

#include <serialization/SetterNotify.h>
    
    ...
    template <typename Reflector>
    void reflect(Reflector& r)
    {
        r.property("Foo", mFoo, setterNotify(mFoo, &MyVisualization::redraw, this), "foo");
        r.property("Bar", mBar, setterNotify(mBar, boost::bind(&MyVisualization::redrawEx, this, 123)), "");
    }
    ...
    
    void redraw() {...}
    void redrawEx(int param) {...}

As you can see in the above example, the setterNotify() method takes the member, whose value should be set, as first parameter and the notification function as second parameter. The latter one, can be a member function (as in the first line) or a function binded using boost::bind (the second line),

Format for Serialized Collections

Support for STL containers: vector, list, deque, set, multiset:

Serialized content in XML format:

<myContainer>;
    <item>1</item>
    <item>2</item>
    <item>3</item>
</myContainer>

Serialized content in JSON format:

[1,2,3]

Support for map, multimap:

Serialized content in XML format:

<myContainer>
    <key>a</key>
    <item>1</item>
    <key>b</key>
    <item>2</item>
    <key>c</key>
    <item>3</item>
</myContainer>

Serialized content in JSON format:

["a",1,
 "b",2,
 "c",3]

Advanced Techniques

This section is for advanced users that are familar with the usage of the serialization framework.

Delegation and 'Transparent Members'

Imagine you have the following class:

class Foo
{
public:
    template <typename Reflector>
    void reflect(Reflector& r)
    {
        r.member("Value", mValue, "My Value");
    }
    std::string mValue;
};

In XML an instance of Foo will be serialized as:

<foo>
    <Value>abc</Value>
</foo>

In most cases this will be satisfactory. However, sometimes a more convenient form of storage is desired, which avoids the occurence of the additional "Value" tag. Instead the object should be stored as:

<foo>abc</foo>

In other words, the "Value" should be transparent to the user and the "Foo" class should be serialized as if it was from the underlying type of "Value" (in this example 'stdstring').

To achieve this, you need to modify the above example as follows:

#include <serialization/IsTransparentSerializable.h>

namespace myns {

class Foo
{
public:
    template <typename Reflector>
    void reflect(Reflector& r)
    {
        r.delegate(mValue);
    }
    std::string mValue;
};

}

namespace mira {

template <typename SerializerTag>
class IsTransparentSerializable<myns::Foo,SerializerTag> : public std::true_type {};

}

Note, that the "member" call in the reflect method was replaced by "delegate" and that a specialization of the IsTransparentSerializable type trait was added.

attention.png

The specialization of the template class IsTransparentSerializable must be done in the mira namespace.

attention.png

You only can make classes "transparent serializable" that contain a SINGLE member only which is serialized. Multiple calls of "delegate" or the combined usage of "delegate", "member" or "property" from the same reflect method is not allowed and results in undefined behavior.

Delegation can also be used with getters and setters:

#include <serialization/GetterSetter.h>
#include <serialization/IsTransparentSerializable.h>

namespace myns {

class Foo
{
public:
    template <typename Reflector>
    void reflect(Reflector& r)
    {
        r.delegate(mValue,
                   setter(&Foo::setValue, this));
// alternatively:                   
//        r.delegate(getter(&Foo::getValue, this),
//                   setter(&Foo::setValue, this));
    }
    std::string getValue();
    void setValue(std::string value);
    std::string mValue;
};

}

namespace mira {

template <typename SerializerTag>
class IsTransparentSerializable<myns::Foo,SerializerTag> : public std::true_type {};

}

Splitting reflect in read and write parts

Normally a single reflect for serialization and deserialization is used as members are serialized and deserialized in the same way. But sometimes you want to transform a member into something else or serialize it in a different format. In that case different code must be used for reading and writing data from your class. The serialization framework supports this by allowing to split the reflect method in two parts - reflectRead and reflectWrite.

First a macro must be used inside or outside your class depending if you want to define your reflect methods intrusive or non-intrusive (either MIRA_SPLIT_REFLECT_MEMBER or MIRA_SPLIT_REFLECT). After that you need to implement the two methods - reflectRead for serializing your class members and relfectWrite to deserialize your class members.

In the example a uint8 bitfield is used as member but should be reflected bitwise.

class MyClass
{
public:

    MIRA_SPLIT_REFLECT_MEMBER

    template <typename Reflector>

    void reflectRead(Reflector& r)
    {
        bool b1 = flags & 0x01 > 0;
        r.member("Bit0", b1, "");
        bool b2 = flags & 0x02 > 0;
        r.member("Bit1", b2, "");
        ....
    }

    template<typename Reflector>
    void reflectWrite(Reflector& r)
    {
        flags = 0x00;
        bool b1;
        r.member("Bit0", b1, "");
        if (b1)
            flags |= 0x01;
        bool b2;
        r.member("Bit1", b2, "");
        if (b2)
            flags |= 0x02;
        ....
    }

    uint8 flags;
};
attention.png

If your class is to be serialized via BinarySerializer it is crucial to have the same number, types and order of your members in both reflect methods.

Limitations

Pointers on pointers

Pointers that point to pointers can not be serialized. If you try to serialize pointers on pointers you will get the following compiler error:

error: static assertion failed "Pointers on pointers cannot be serialized"

Instead of serializing the pointer to a pointer you should serialize the pointer that is pointed to. There should never be a need to serialize a pointer to a pointer, if it is, you really should think about your code.

Pointers on fundamental types

Pointers that point to fundamental types (int, float, etc.) can not be serialized. This restriction is made for performance reasons. If you try to serialize pointers to fundamental types you will get the following compiler error:

error: static assertion failed "Pointers on fundamental types cannot be serialized"

If you really need to serialize a pointer to a fundamental type, you must wrap the fundamental type into a class or struct.

Pointer conflicts

When pointers are serialized improperly a so-called pointer conflict may arise as shown in the following example:

struct MyClass
{
    MyClass() 
    {
        // ptr points to obj
        ptr = &obj;
    }    
    
    template<typename Reflector>
    void reflect(Reflector& r)
    {
        r.member("ptr", ptr, "");
        r.member("obj", obj, "");
    }
    
    Foo* ptr;
    Foo obj;
};

In this example, the pointer "ptr" points to the object "obj". Moreover, the pointer is reflected BEFORE the object. Here, the problem occurs. When the pointer "ptr" is serialized the underlying object "obj" was not serialized yet, hence the serialization framework will serialize the whole content of the object. Afterwards, the object "obj" will be serialized. However, the object was already serialized before using the pointer "ptr" and should not be serialized twice. In this case, an XIO exception will be thrown to indicate the problem.

To resolve this conflict one only has to switch the serialization order of the pointer and the object:

    template<typename Reflector>
    void reflect(Reflector& r)
    {
        r.member("obj", obj, "");
        r.member("ptr", ptr, "");
    }
};

Now, the object "obj" will be serialized first. When the pointer "ptr" is serialized afterwards, the underlying object will not be serialized a second time, instead a reference to the previously serialized object will be stored for the pointer and hence there is no conflict here.

Abstract types

Abstract classes can be serialized only, if they are subclassed from mira::Object. Otherwise you will get the following compiler error:

error: static assertion failed "You tried to serialize an abstract class that is not a mira::Object"

The reason for this restriction is, that objects of abstract classes cannot be created during the process of deserialization. This can be achieved using the class factory only, which will create an object of the derived (non-abstract) class. Hence, if you want to serialize abstract types, they need to be inherited from the mira::Object in order to use the class factory. Note that abstract classes are just a special case of polymorphic classes (identifiable at compile time), and that ALL polymorphic classes need to be derived from mira::Object to work with serialization properly. See Polymorphic Classes for details.

 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines

Generated on 5 Apr 2020 for MIRA by  doxygen 1.6.1