Implementing Macros operating on sequences

Sequences give you a convenient array-like data-structure wich can be passed to macros:

#define HEAD(sequence) \
    BOOST_PP_SEQ_ELEM(0, sequence)

std::cout << HEAD((0)(1)(2)); // prints "0"

Now how can that be implemented? First there are two characters that have special status for the preprocessor: commas and parentheses.
Parentheses can be used to

• wrap expressions to be left alone by macro expansion, e.g. because they contain commas
• can be bound to macro names as parameter lists

The binding of parameter lists doesn't have to happen immediately, so the following works fine:

#define IDENTITY(x) x
#define HEAD(sequence) IDENTITY sequence
std::cout << HEAD((42)); // prints "42"

The macro-expansion here is the following:

HEAD((42))
IDENTITY (42)
42

That already works fine for a one-element sequence, but what if it had more elements:

std::cout << HEAD((42)(23));
// results in:
std::cout << 42(23);

... which is obviously wrong. We need to get rid of the rest of the sequence by going through another macro:

#define FIRST(x) x, DOESNT_EXIST
#define HEAD(sequence) FIRST sequence
std::cout << HEAD((42)(23));

This now results in:

HEAD((42)(23))
FIRST (42)(23) // FIRST binds (42)
42, DOESNT_EXIST(23)

Passing this result through another macro that takes two arguments, we can ignore the second part:

#define HEAD(sequence) IGNORE_SECOND(FIRST sequence)
#define FIRST(x) x, FOO
#define IGNORE_SECOND(x) IGNORE_SECOND_I(x) // needed for expansion of x
#define IGNORE_SECOND_I(x, _) x

Which now expands to:

HEAD((42)(23))
IGNORE_SECOND(FIRST (42)(23)) // FIRST binds (42)
IGNORE_SECOND_I(42, DOESNT_EXIST(23))
42

Getting elements by index

Now the macros we pass the comma-separated expansion to don't have to throw away the second part:

#define SECOND(sequence) IGNORE_SECOND(TAIL sequence)
#define TAIL(x) FIRST // throw away first element

... with which we get something like the following expansion:

SECOND((1)(2)(3))
IGNORE_SECOND(TAIL (1)(2)(3))
IGNORE_SECOND_I(FIRST (2)(3))
IGNORE_SECOND_I(2, DOESNT_EXIST(3))
2

Now combining that with the concatenation operator ## we can define macros that operate index-based:

#define GET_BY_INDEX(index, sequence) IGNORE_SECOND( GET_##index sequence )
#define GET_0(x) x, DOESNT_EXIST
#define GET_1(_) GET_0
#define GET_2(_) GET_1
// ...

... which could lead to the following expansion:

GET_BY_INDEX(2, (1)(2)(3)(4))
IGNORE_SECOND(GET_2 (1)(2)(3)(4))
IGNORE_SECOND_I(GET_1 (2)(3)(4))
IGNORE_SECOND_I(GET_0 (3)(4))
IGNORE_SECOND_I(3, DOESNT_EXIST(4))
3

Counting elements

To get the size of a sequence the elements in it need to be counted. The trick here is that substitution for macros that take arguments stops if there are no parameter-lists left that could be bound, so the following:

#define SIZE(sequence) SIZE_0 sequence
#define SIZE_0(_) SIZE_1
#define SIZE_1(_) SIZE_2
#define SIZE_2(_) SIZE_3
// ...

would expand SIZE((1)(2)) to SIZE_2. All that is left then is to map that name to a number:

#define SIZE(sequence) \
    CONCAT(SIZE_, SIZE_0 sequence) // concatenate SIZE_ with SIZE_N
#define CONCAT(a, b) CONCAT_I(a, b) // allow for macro expansion
#define CONCAT_I(a, b) a ## b
#define SIZE_SIZE_0 0
#define SIZE_SIZE_1 1
#define SIZE_SIZE_2 2
// ...

... which would lead to expansions like:

SIZE((1)(2))
CONCAT(SIZE_, SIZE_0 (1)(2))
CONCAT_I(SIZE_, SIZE_0 (1)(2))
CONCAT_I(SIZE_, SIZE_1 (2))
CONCAT_I(SIZE_, SIZE_2)
SIZE_ ## SIZE_2
SIZE_SIZE_2
2

- Georg

In a pure C++ world we can generate interfacing or glue code between different components or interfaces at compile time, using a combination of template-based compile-time and runtime-techniques (to e.g. mostly automatically marshall to/from calls using legacy types).

When having to interface C++ applications with Objective-C/Cocoa for GUI, system integration or IPC though, things become harder due to the less strict typing – yet often not more then a flat repetitive interface layer is needed: thin bridging delegates have to be defined or conversion code to language bridging calls has to be written.

If you have to deal with interfaces of non-trivial size and want to avoid script-based code generation this quickly becomes cumbersome and is just a pain every time refactorings have to take place. Using a combination of (template) metaprogramming and the Objective-C runtime library, it should be possible to reduce the amount of code considerably…

As i didn’t get any answers that were satisfactory for me, i started on a prototype.

The following is an overview on the idea behind the prototype.

Motivation

What annoyed me most is that every time i just need to get some events of an Objective-C instance to some C++ instance, i need to create a separate Objective-C class and manually forward from there to the target (C++) object.

Lets say we have the following informal delegate protocol:

- (NSString*)concatString:(NSString*)s1 withString:(NSString*)s2;

Currently i have to first create an Objective-C instance:

@interface Glue : NSObject {
    CppClass* instance;
}
-(id)initWithCppInstance:(CppClass*)ins;
-(NSString*)concatString:(NSString*)s1 withString:(NSString*)s2;
@end

… with the appropriate implementation:

@implementation Glue
-(id)initWithCppInstance:(CppClass*)ins {
    if(self = [super init])
        instance = ins;
    return self;
}
-(NSString*)concatString:(NSString*)s1 withString:(NSString*)s2 {
    const std::string s =
        instance->concatStrings(toStdString(s1), toStdString(s2));
    return toNsString(s);
}
@end

Then i still have to take care of holding and initializing that Objective-C instance in the C++ instance, with possibly additional work for opaque pointers because i don’t want to bring Objective-C into the C++ header.

Objective-Glue

The idea was that there should be a way to at least cut down on the amount of code and to mostly automatically generate the bridging code.

I now found some time and came up with a basic prototype. Given the following class definition:

struct CppClass {
    std::string concatStrings(const std::string& s1, const std::string& s2) const
    {
        return s1+s2;
    }
};

… a suitable delegate can be build as follows:

CppClass cpp;
og::ObjcClass objc("MyGlueClass");
objc.add_handler<NSString* (NSString*, NSString*)>
       ("concatString:withString:", &cpp, &CppClass::concatStrings);

Here the explicit template parameter to add_handler() specifies the Objective-C signature, the first parameter specifies the method name and the last two pass the C++ instance and the method to call on it.

Free functions can also be used:

objc.add_handler<NSString* (NSString*, NSString*)>
       ("concatString:withString:", &concatStrings);

Boost.Function objects can also be passed.

The Objective-C instance can then be called like the following:

id ins = objc.get_instance();
NSString* result = [ins concatString:@"abcd" withString:@"efgh"];
assert([result compare:@"abcdefgh"] == NSOrderedSame);

How?

While the Objective-C signature has to be specified explicitly even for formal protocols (as in the type encodings the type information for Objective-C classes is lost), the C++ signature can be looked up at compile time. Knowing those two signatures we can use template functions to get a suitable IMP function, that “stores” the knowledge of the signatures and does the necessary conversions.

Note: Behind the scenes, Objective-C methods are simple C functions that take two additional parameters, of type id for the instance and SEL for the method being called. These functions are called IMP or implementation functions.

Combining that with the fact that the Objective-C runtime allows to add and replace method implementations via its API, we can generate classes that call suitable IMP functions. To allow for member functions, free functions or Boost.Bind generated expressions to be invoked, functors (Boost.Function) are used to store the callees.

The functors are stored in a map in the indexed ivar section of the instance, look-up is done via the selector that IMP functions are passed.
The IMP function can then cast the retrieved pointer to the correct signature type and invoke the functor.

Generating the instance

Using the Objective-C runtime, we can generate a custom class. We first need the super-class (if any):

std::string superClassName = "NSObject";
std::string className      = "MyObjcClass";
id superClass = objc_getClass(superClassName.c_str());

Then we need to allocate a new class pair and register it:

Class myClass = objc_allocateClassPair
                    (superClass, className.c_str(), sizeof(void*));
objc_registerClassPair(myClass);

The last parameter to objc_allocateClassPair() reserves additional space per instance for a pointer, so we can store the callee map later.

Now that the class is registered, we can create an instance:

id myInstance = class_createInstance(myClass, 0);

To add methods, we need to use class_addMethod() if the class doesn’t have such a method yet, or method_setImplementation() to override an existing one:

std::string methodName = "concatString:withString:";
SEL mySelector = sel_registerName(methodName.c_str());
BOOL added = class_addMethod(myClass, mySelector, myImp, "");
if(!added) {
    // class already has such a method, override it
    Method m = class_getInstanceMethod(myClass, mySelector);
    method_setImplementation(m, myImp);
}

Now for expressions like [myObj concatString:s1 withString:s2], the custom implementation myImp would get called. But how do we generate a suitable IMP function?

Generating IMPs

As we know the ObjcC and the C++ signatures, we can use template specialization to get a suitable function. To avoid too much repitition, partial specialization of a template class with a static member function is used:

template<size_t Arity, bool returnsVoid, class ObjcSig, class CppSig>
struct imp;

For e.g. CppClass::concatStrings, we have an arity of 2 and it doesn’t return void. This would require the following specialization:

template<class ObjcSig, class CppSig>
struct imp<2, false, ObjcSig, CppSig> {
    static ??? f(id self, SEL sel, ??? a1, ??? a2)
    {
        // ... invoke Callee
    }
};

Looking up the needed forms of the ObjC and C++ arguments and return types can be done via helper meta functions using Boosts FunctionTypes and TypeTraits libraries. With those types known, f() becomes something like this:

static ObjcRetType f(id self, SEL sel, ObjcArg0 a0, ObjcArg1 a1)
{
    boost::function<CppSig> callee = lookup_callee<CppSig>(self, sel);
    return convert<PlainObjcRet, PlainCppRet> // convert<ToType,FromType>
        (callee
          (convert<PlainCppArg0, PlainObjcArg0>(a0),
           convert<PlainCppArg1, PlainObjcArg1>(a1)));
}

Note: The Plain* types denote argument or return types stripped of references and constness.

With such an approach, we can choose a suitable IMP function at compile-time:

typedef imp<function_arity, is_same<ReturnType, void>::value,
            ObjcSig, CppSig> ImpType;
IMP myImp = (IMP)&ImpType::f;

Conversions

Conversions are done via a static member function of a templated struct og::converter, which allows for partial specialization in user code. It is declared as following:

template<class To, class From> struct converter {
    static To convert(typename boost::call_traits::param_type<From>::type from);
};

Note: param_type is a Boost utility that maps the type to the appropriate form (pass-by-value for fundamental types, pass-by-const-reference for complex types, …).

To add additional conversions, specializations of converter are added anywhere before the add_handler() call that would use them. E.g. to enable conversions from NSString to std::string the following (simplified) specialization can be added:

template<> struct converter<std::string, NSString*> {
    static std::string convert(NSString* from) {
        return [from UTF8String];
    }
};

Conclusion

While the prototype shows that the basic idea works, there are still some issues that need to be worked out:

• support for formal protocols
• support for properties
• support for type-encodings (in a non-painful way)
• allowing full customization of class and instance (currently dealloc is needed and no class methods supported)
• convenience for wrapping complete C++ instance as Objective-C instances?
• … probably more

For those interested, here is some background reading material:

• Objective-C Runtime Programming Guide
• Understanding the Objective-C runtime
• Objective-C Runtime Reference

As previously mentioned, this is a first prototype… so constructive input and ideas are welcome. The current source can be viewed online via a bitbucket repository.
The prototype relies on some parts of Boost and was tested with Boost 1.42.

– Georg