Daily news, dev blogs, and stories from Game Developer straight to your inbox
Sponsored By
Featured Blog | This community-written post highlights the best of what the game industry has to offer. Read more like it on the Game Developer Blogs.
Modern C++ lightweight binary RPC framework without code generation
In this article, I go over the foundations of a new C++ RPC framework that requires no code generation. Using Modern C++, with type traits, variadic templates, and some other tricks, the final API is surprisingly simple.
49 Min Read
<p>This was initially posted on my blog at:&nbsp;&nbsp;<a href="http://www.crazygaze.com/blog/2016/06/06/modern-c-lightweight-binary-rpc-framework-without-code-generation/">http://www.crazygaze.com/blog/2016/06/06/modern-c-lightweight-binary-rpc-framework-without-code-generation/</a></p> <h1>Table of Contents</h1> <ol> <li><a href="#id-introduction">Introduction</a> <ol> <li><a href="#id-why-i-needed-this">Why I needed this</a></li> </ol> </li> <li><a href="#id-rpc-parameters">RPC parameters</a> <ol> <li><a href="#id-parameter-traits">Parameter Traits</a></li> <li><a href="#id-serialization">Serialization</a></li> <li><a href="#id-deserialization">Deserialization</a></li> <li><a href="#id-from-tuple-to-function-parameters">From tuple to function parameters</a></li> </ol> </li> <li><a href="#id-the-rpc-api">The RPC API</a> <ol> <li><a href="#id-header">Header</a></li> <li><a href="#id-table">Table</a></li> <li><a href="#id-transport">Transport</a></li> <li><a href="#id-result">Result</a></li> <li><a href="#id-out-processor">OutProcessor</a></li> <li><a href="#id-in-processor">InProcessor</a></li> <li><a href="#id-connection">Connection</a></li> </ol> </li> <li><a href="#id-improvements">Improvements</a></li> </ol> <p><a name="id-introduction"></a></p> <h2>Introduction</h2> <p><img alt="" class="alignnone size-full wp-image-652" src="http://www.crazygaze.com/blog/wp-content/uploads/2016/06/img_57561179204f5.png" /></p> <p>This article explores a C++ RPC framework I&#39;ve been working on which requires no code generation step for glue code. Before I start rambling on implementation details, and so you know what to expect, here is a feature list:</p> <ul> <li>Source available at <a href="https://bitbucket.org/ruifig/czrpc">https://bitbucket.org/ruifig/czrpc</a> <ul> <li>The source code shown in this article is by no means complete. It&#39;s meant to show the foundations upon which the framework was built. Also, to shorten things a bit, it&#39;s a mix of code from the repository at the time of writing and custom sample code, so it might have errors.</li> <li>Some of the source code which is not directly related to the problem at hand is left intentionally simple with disregard for performance. Any improvements will later be added to source code repository.</li> </ul> </li> <li>Modern C++ (C++11/14) <ul> <li>Requires at least <strong>Visual Studio 2015</strong>. Clang/GCC is fine too, but might not work as-is, since VS is less strict.</li> </ul> </li> <li>Type-safe <ul> <li>The framework detects at <strong>compile time</strong> invalid RPC calls, such as unknown RPC names, wrong number of parameters, or wrong parameter types.</li> </ul> </li> <li>Relatively small API and not too verbose (considering it requires no code generation)</li> <li>Multiple ways to handle RPC replies <ul> <li>Asynchronous handler</li> <li>Futures</li> <li>A client can detect if an RPC caused an exception server side</li> </ul> </li> <li>Allows the use of potentially any type in RPC parameters <ul> <li>Provided the user implements the required functions to deal with that type.</li> </ul> </li> <li>Bidirectional RPCs (A server can call RPCs on a client) <ul> <li>Typically, client code cannot be trusted, but since the framework is to be used between trusted parties, this is not a problem.</li> </ul> </li> <li>Non intrusive <ul> <li>An object being used for RPC calls doesn&#39;t need to know anything about RPCs or network.</li> <li>This makes it possible to wrap third party classes for RPC calls.</li> </ul> </li> <li>Minimal bandwidth overhead per RPC call</li> <li>No external dependencies <ul> <li>Although the supplied transport (in the source code repository) uses Asio/Boost Asio, the framework itself does not depend on it. You can plug in your own transport.</li> </ul> </li> <li>No security features provided <ul> <li>Because the framework is intended to be used between trusted parties (e.g: between servers).</li> <li>The application can specify its own transport, therefore having a chance to encrypt anything if required.</li> </ul> </li> </ul> <p>Even though the source code shown is not complete, the article is still very heavy on code. Code is presented in small portions and every section builds on the previous, but is still an overwhelming amount of code. So that you have an idea of how it will look like in the end, here is a fully functional sample using the source code repository at the time of writing:</p> <pre> <code class="language-cpp">////////////////////////////////////////////////////////////////////////// // Useless RPC-agnostic class that performs calculations. ////////////////////////////////////////////////////////////////////////// class Calculator { public: double add(double a, double b) { return a + b; } }; ////////////////////////////////////////////////////////////////////////// // Define the RPC table for the Calculator class // This needs to be seen by both the server and client code ////////////////////////////////////////////////////////////////////////// #define RPCTABLE_CLASS Calculator #define RPCTABLE_CONTENTS \ REGISTERRPC(add) #include "crazygaze/rpc/RPCGenerate.h" ////////////////////////////////////////////////////////////////////////// // A Server that only accepts 1 client, then shuts down // when the client disconnects ////////////////////////////////////////////////////////////////////////// void RunServer() { asio::io_service io; // Start thread to run Asio's the io_service // we will be using for the server std::thread th = std::thread([&amp;io] { asio::io_service::work w(io); io.run(); }); // Instance we will be using to serve RPC calls. // Note that it's an object that knows nothing about RPCs Calculator calc; // start listening for a client connection. // We specify what Calculator instance clients will use, auto acceptor = AsioTransportAcceptor&lt;Calculator, void&gt;::create(io, calc); // Start listening on port 9000. // For simplicity, we are only expecting 1 client using ConType = Connection&lt;Calculator, void&gt;; std::shared_ptr&lt;ConType&gt; con; acceptor-&gt;start(9000, [&amp;io, &amp;con](std::shared_ptr&lt;ConType&gt; con_) { con = con_; // Since this is just a sample, close the server once the first client // disconnects reinterpret_cast&lt;BaseAsioTransport*&gt;(con-&gt;transport.get()) -&gt;setOnClosed([&amp;io] { io.stop(); }); }); th.join(); } ////////////////////////////////////////////////////////////////////////// // A client that connects to the server, calls 1 RPC // then disconnects, causing everything to shut down ////////////////////////////////////////////////////////////////////////// void RunClient() { // Start a thread to run our Asio io_service asio::io_service io; std::thread th = std::thread([&amp;io] { asio::io_service::work w(io); io.run(); }); // Connect to the server (localhost, port 9000) auto con = AsioTransport&lt;void, Calculator&gt;::create(io, "127.0.0.1", 9000).get(); // Call one RPC (the add method), specifying an asynchronous handler for // when the result arrives CZRPC_CALL(*con, add, 1, 2) .async([&amp;io](Result&lt;double&gt; res) { printf("Result=%f\n", res.get()); // Prints 3.0 // Since this is a just a sample, stop the io_service after we get // the result, // so everything shuts down io.stop(); }); th.join(); } // For testing simplicity, run both the server and client on the same machine, void RunServerAndClient() { auto a = std::thread([] { RunServer(); }); auto b = std::thread([] { RunClient(); }); a.join(); b.join(); }</code></pre> <p>This code is mostly setup code, since the provided transport uses Asio. The RPC calls itself can be as simple as:</p> <pre> <code class="language-cpp">// RPC call using asynchronous handler to handle the result CZRPC_CALL(*con, add, 1, 2).async([](Result&lt;double&gt; res) { printf("Result=%f\n", res.get()); // Prints 3.0 }); // RPC call using std::future to handle the result Result&lt;double&gt; res = CZRPC_CALL(*con, add, 1, 2).ft().get(); printf("Result=%f\n", res.get()); // Prints 3.0</code></pre> <p><a name="id-why-i-needed-this"></a></p> <h3>Why I needed this</h3> <p>The game I&#39;ve been working on for a couple of years now (<a href="https://bitbucket.org/ruifig/g4devkit">code named G4</a>), gives players fully simulated little in-game computers they can code for whatever they want. That requires me to have a couple of server types running:</p> <ul> <li>Gameplay Server(s)</li> <li>VM Server(s) (Simulates the in-game computers) <ul> <li>So that in-game computers can be simulated even if the player is not currently online</li> </ul> </li> <li>VM Disk Server(s) <ul> <li>Deals with in-game computer&#39;s storage, like floppies or hard drives.</li> </ul> </li> <li>Database server(s)</li> <li>Login server(s)</li> </ul> <p>All these servers need to exchange data, therefore the need for a flexible RPC framework.</p> <p>Initially I had a custom solution where I would tag methods of a class with certain attributes, then have a Clang based parser (<a href="https://github.com/Celtoys/clReflect">clReflect</a>) generate any required serialization and glue code.</p> <p>Although it worked fine for the most part, for the past year or so I kept wondering how could I use the new C++11/14 features to create a minimal type safe C++ RPC framework. Something that would not need a code generation step for glue code, while still keeping an acceptable API.</p> <p>For serialization of non-fundamental types, code generation is still useful, so I don&#39;t need to manually define how to serialize all the fields of a given struct/class. Although defining those manually is not a big deal, I believe.</p> <p><a name="id-rpc-parameters"></a></p> <h2>RPC Parameters</h2> <p>Given a function, in order to have type safe RPC calls, there are a few things we need to be able to do:</p> <ul> <li>Identify at compile time if this function is a valid RPC function (Right number of parameters, right type of parameters, etc)</li> <li>Check if the supplied parameters match (or can be converted) to what the function signature specifies.</li> <li>Serialize all parameters</li> <li>Deserialize all parameters</li> <li>Call the desired function</li> </ul> <p><a name="id-parameter-traits"></a></p> <h3>Parameter Traits</h3> <p>The first problem you&#39;ll face is in deciding what type of parameters are accepted. Some RPC frameworks only accept a limited number of types, such as Thrift. Let&#39;s check the problem.</p> <p>Given these function signatures:</p> <pre> <code class="language-cpp">void func1(int a, float b, char c); void func2(const char* a, std::string b, const std::string&amp; c); void func3(Foo a, Bar* b); </code></pre> <p>How can we make compile time checks regarding the parameters? Fundamental types are easy enough and should definitely be supported by the framework. A dumb memory copy will do the trick in those cases unless you want to trade a bit of performance for bandwidth usage by cutting down the number of bits needed. But how about complex types such std::string, std::vector, or your own classes? How about pointers, references, const references, rvalues?</p> <p>We can get some inspiration from what the C++ Standard Library does in the <a href="http://en.cppreference.com/w/cpp/types">type_traits</a> header. We need to be able to query a given type regarding its RPC properties. Let&#39;s put that concept in a template class <code>ParamTraits&lt;T&gt;</code>, with the following layout .</p> <table> <thead> <tr> <th><strong>Member constants</strong></th> <th>&nbsp;</th> </tr> </thead> <tbody> <tr> <td><code>valid</code></td> <td><code>true</code> if T is valid for RPC parameters, <code>false</code> otherwise</td> </tr> <tr> <td><strong>Member types</strong></td> <td>&nbsp;</td> </tr> <tr> <td><code>store_type</code></td> <td>Type used to hold the temporary copy needed when deserializing</td> </tr> <tr> <td><strong>Member functions</strong></td> <td>&nbsp;</td> </tr> <tr> <td><code>write</code></td> <td>Writes the parameter to a stream</td> </tr> <tr> <td><code>read</code></td> <td>Reads a parameter into a <code>store_type</code></td> </tr> <tr> <td><code>get</code></td> <td>Given a <code>store_type</code> parameter, it returns what can be passed to the RPC function as a parameter</td> </tr> </tbody> </table> <p>As an example, let&#39;s implement <code>ParamTraits&lt;T&gt;</code> for arithmetic types, considering we have a stream class with a <code>write</code> and <code>read</code> methods:</p> <pre> <code class="language-cpp">namespace cz { namespace rpc { // By default, all types for which ParamTraits is not specialized are invalid template &lt;typename T, typename ENABLED = void&gt; struct ParamTraits { using store_type = int; static constexpr bool valid = false; }; // Specialization for arithmetic types template &lt;typename T&gt; struct ParamTraits&lt; T, typename std::enable_if&lt;std::is_arithmetic&lt;T&gt;::value&gt;::type&gt; { using store_type = typename std::decay&lt;T&gt;::type; static constexpr bool valid = true; template &lt;typename S&gt; static void write(S&amp; s, typename std::decay&lt;T&gt;::type v) { s.write(&amp;v, sizeof(v)); } template &lt;typename S&gt; static void read(S&amp; s, store_type&amp; v) { s.read(&amp;v, sizeof(v)); } static store_type get(store_type v) { return v; } }; } // namespace rpc } // namespace cz </code></pre> <p>And a simple test:</p> <pre> <code class="language-cpp">#define TEST(exp) printf("%s = %s\n", #exp, exp ? "true" : "false"); void testArithmetic() { TEST(ParamTraits&lt;int&gt;::valid); // true TEST(ParamTraits&lt;const int&gt;::valid); // true TEST(ParamTraits&lt;int&amp;&gt;::valid); // false TEST(ParamTraits&lt;const int&amp;&gt;::valid); // false } </code></pre> <p><code>ParamTraits&lt;T&gt;</code> is also used to check if return types are valid, and since a void function is valid, we need to specialize <code>ParamTraits</code> for void too.</p> <pre> <code class="language-cpp">namespace cz { namespace rpc { // void type is valid template &lt;&gt; struct ParamTraits&lt;void&gt; { static constexpr bool valid = true; using store_type = void; }; } // namespace rpc } // namespace cz </code></pre> <p>The apparently strange thing with the specialization for <code>void</code> is that it also specifies a <code>store_type</code>. We can&#39;t use it to store anything, but will make some of the later template code easier.</p> <p>With these <code>ParamTraits</code> examples, references are not valid RPC parameters. In practice you do want to allow const references at least, especially for fundamental types. A tweak can be added to enable support for <code>const T&amp;</code> for any valid T if your application needs it.</p> <pre> <code class="language-cpp">// Make "const T&amp;" valid for any valid T #define CZRPC_ALLOW_CONST_LVALUE_REFS \ namespace cz { \ namespace rpc { \ template &lt;typename T&gt; \ struct ParamTraits&lt;const T&amp;&gt; : ParamTraits&lt;T&gt; { \ static_assert(ParamTraits&lt;T&gt;::valid, \ "Invalid RPC parameter type. Specialize ParamTraits if " \ "required."); \ }; \ } \ } </code></pre> <p>Similar tweaks can be made to enable support for <code>T&amp;</code> or <code>T&amp;&amp;</code> if required, although if the function makes changes to those parameters, those changes will be lost.</p> <p>Let&#39;s try adding support for a complex type, such as <code>std::vector&lt;T&gt;</code>. For <code>std::vector&lt;T&gt;</code> to be supported, <code>T</code> needs to be supported too.</p> <pre> <code class="language-cpp">namespace cz { namespace rpc { template &lt;typename T&gt; struct ParamTraits&lt;std::vector&lt;T&gt;&gt; { using store_type = std::vector&lt;T&gt;; static constexpr bool valid = ParamTraits&lt;T&gt;::valid; static_assert(ParamTraits&lt;T&gt;::valid == true, "T is not valid RPC parameter type."); // std::vector serialization is done by writing the vector size, followed by // each element template &lt;typename S&gt; static void write(S&amp; s, const std::vector&lt;T&gt;&amp; v) { int len = static_cast&lt;int&gt;(v.size()); s.write(&amp;len, sizeof(len)); for (auto&amp;&amp; i : v) ParamTraits&lt;T&gt;::write(s, i); } template &lt;typename S&gt; static void read(S&amp; s, std::vector&lt;T&gt;&amp; v) { int len; s.read(&amp;len, sizeof(len)); v.clear(); while (len--) { T i; ParamTraits&lt;T&gt;::read(s, i); v.push_back(std::move(i)); } } static std::vector&lt;T&gt;&amp;&amp; get(std::vector&lt;T&gt;&amp;&amp; v) { return std::move(v); } }; } // namespace rpc } // namespace cz // A simple test void testVector() { TEST(ParamTraits&lt;std::vector&lt;int&gt;&gt;::valid); // true // true if support for const refs was enabled TEST(ParamTraits&lt;const std::vector&lt;int&gt;&amp;&gt;::valid); }</code></pre> <p>For convenience, we can use the <code>&lt;&lt;</code> and <code>&gt;&gt;</code> operators with <code>Stream</code> class (not shown here). Those operators simply call the respective <code>ParamTraits&lt;T&gt;</code> <code>read</code> and <code>write</code> functions.</p> <p>Now that we can check if a specific type is allowed for RPC parameters, we can build on that and check if a function can be used for RPCs. This is implemented with variadic templates.</p> <p>First let&#39;s create a template to tells us if a bunch of parameters are valid.</p> <pre> <code class="language-cpp">namespace cz { namespace rpc { // // Validate if all parameter types in a parameter pack can be used for RPC // calls // template &lt;typename... T&gt; struct ParamPack { static constexpr bool valid = true; }; template &lt;typename First&gt; struct ParamPack&lt;First&gt; { static constexpr bool valid = ParamTraits&lt;First&gt;::valid; }; template &lt;typename First, typename... Rest&gt; struct ParamPack&lt;First, Rest...&gt; { static constexpr bool valid = ParamTraits&lt;First&gt;::valid &amp;&amp; ParamPack&lt;Rest...&gt;::valid; } } // namespace rpc } // namespace cz // Usage example: void testParamPack() { TEST(ParamPack&lt;&gt;::valid); // true (No parameters is a valid too) TEST((ParamPack&lt;int, double&gt;::valid)); // true TEST((ParamPack&lt;int, int*&gt;::valid)); // false }</code></pre> <p>Using <code>ParamPack</code>, we can now create a <code>FunctionTraits</code> template to query a function&#39;s properties.</p> <pre> <code class="language-cpp">namespace cz { namespace rpc { template &lt;class F&gt; struct FunctionTraits {}; // For free function pointers template &lt;class R, class... Args&gt; struct FunctionTraits&lt;R (*)(Args...)&gt; : public FunctionTraits&lt;R(Args...)&gt; {}; // For method pointers template &lt;class R, class C, class... Args&gt; struct FunctionTraits&lt;R (C::*)(Args...)&gt; : public FunctionTraits&lt;R(Args...)&gt; { using class_type = C; }; // For const method pointers template &lt;class R, class C, class... Args&gt; struct FunctionTraits&lt;R (C::*)(Args...) const&gt; : public FunctionTraits&lt;R(Args...)&gt; { using class_type = C; }; template &lt;class R, class... Args&gt; struct FunctionTraits&lt;R(Args...)&gt; { // Tells if both the return type and parameters are valid for RPC calls static constexpr bool valid = ParamTraits&lt;R&gt;::valid &amp;&amp; ParamPack&lt;Args...&gt;::valid; using return_type = R; // Number of parameters static constexpr std::size_t arity = sizeof...(Args); // A tuple that can store all parameters using param_tuple = std::tuple&lt;typename ParamTraits&lt;Args&gt;::store_type...&gt;; // Allows us to get the type of each parameter, given an index template &lt;std::size_t N&gt; struct argument { static_assert(N &lt; arity, "error: invalid parameter index."); using type = typename std::tuple_element&lt;N, std::tuple&lt;Args...&gt;&gt;::type; }; }; } // namespace rpc } // namespace cz // A simple test... struct FuncTraitsTest { void func1() const {} void func2(int) {} int func3(const std::vector&lt;int&gt;&amp;) { return 0; } int* func4() { return 0; } }; void testFunctionTraits() { TEST(FunctionTraits&lt;decltype(&amp;FuncTraitsTest::func1)&gt;::valid); // true TEST(FunctionTraits&lt;decltype(&amp;FuncTraitsTest::func2)&gt;::valid); // true TEST(FunctionTraits&lt;decltype(&amp;FuncTraitsTest::func3)&gt;::valid); // true TEST(FunctionTraits&lt;decltype(&amp;FuncTraitsTest::func4)&gt;::valid); // false }</code></pre> <p><code>FunctionTraits</code> gives us a couple of properties that will be used later. Note for example that <code>FunctionTraits::param_tuple</code> builds on <code>ParamTraits&lt;T&gt;::store_type</code> . This is needed, since at some point we need a way to deserialize all parameters into a tuple before calling the function.</p> <p><a name="id-serialization"></a></p> <h3>Serialization</h3> <p>Since we now have the required code for querying parameters, return types and validating functions, we can put together the code to serialize a function call. Also, it is type safe. It will not compile if given the wrong number or type of parameters, or if the function itself is not valid for RPCs (e.g: unsupported return/parameter types).</p> <pre> <code class="language-cpp">namespace cz { namespace rpc { namespace details { template &lt;typename F, int N&gt; struct Parameters { template &lt;typename S&gt; static void serialize(S&amp;) {} template &lt;typename S, typename First, typename... Rest&gt; static void serialize(S&amp; s, First&amp;&amp; first, Rest&amp;&amp;... rest) { using Traits = ParamTraits&lt;typename FunctionTraits&lt;F&gt;::template argument&lt;N&gt;::type&gt;; Traits::write(s, std::forward&lt;First&gt;(first)); Parameters&lt;F, N + 1&gt;::serialize(s, std::forward&lt;Rest&gt;(rest)...); } }; } // namespace details template &lt;typename F, typename... Args&gt; void serializeMethod(Stream&amp; s, Args&amp;&amp;... args) { using Traits = FunctionTraits&lt;F&gt;; static_assert(Traits::valid, "Function signature not valid for RPC calls. Check if " "parameter types are valid"); static_assert(Traits::arity == sizeof...(Args), "Invalid number of parameters for RPC call."); details::Parameters&lt;F, 0&gt;::serialize(s, std::forward&lt;Args&gt;(args)...); } } // namespace rpc } // namespace cz // // A simple test // struct SerializeTest { void func1(int, const std::vector&lt;int&gt;) {} void func2(int*) {} }; void testSerializeCall() { Stream s; serializeMethod&lt;decltype(&amp;SerializeTest::func1)&gt;(s, 1, std::vector&lt;int&gt;{1, 2, 3}); // These fail to compile because of the wrong number of parameters // serializeMethod&lt;decltype(&amp;SerializeTest::func1)&gt;(s); // serializeMethod&lt;decltype(&amp;SerializeTest::func1)&gt;(s, 1); // Doesn't compile because of wrong type of parameters // serializeMethod&lt;decltype(&amp;SerializeTest::func1)&gt;(s, 1, 2); // Doesn't compile because the function can't be used for RPCs. // int a; // serializeMethod&lt;decltype(&amp;SerializeTest::func2)&gt;(s, &amp;a); } </code></pre> <p><a name="id-deserialization"></a></p> <h3>Deserialization</h3> <p>As mentioned above, <code>FunctionTraits&lt;F&gt;::param_tuple</code> is the <code>std::tuple</code> type we can use to hold all the function&#39;s parameters. In order to be able to use this tuple to deserialize parameters, we need to specialize <code>ParamTraits</code> for tuples. This has the nice side effect of also making it possible to use <code>std::tuple</code> for RPC parameters.</p> <pre> <code class="language-cpp">namespace cz { namespace rpc { namespace details { template &lt;typename T, bool Done, int N&gt; struct Tuple { template &lt;typename S&gt; static void deserialize(S&amp; s, T&amp; v) { s &gt;&gt; std::get&lt;N&gt;(v); Tuple&lt;T, N == std::tuple_size&lt;T&gt;::value - 1, N + 1&gt;::deserialize(s, v); } template &lt;typename S&gt; static void serialize(S&amp; s, const T&amp; v) { s &lt;&lt; std::get&lt;N&gt;(v); Tuple&lt;T, N == std::tuple_size&lt;T&gt;::value - 1, N + 1&gt;::serialize(s, v); } }; template &lt;typename T, int N&gt; struct Tuple&lt;T, true, N&gt; { template &lt;typename S&gt; static void deserialize(S&amp;, T&amp;) {} template &lt;typename S&gt; static void serialize(S&amp;, const T&amp;) {} }; } // namespace details template &lt;typename... T&gt; struct ParamTraits&lt;std::tuple&lt;T...&gt;&gt; { using tuple_type = std::tuple&lt;T...&gt;; // for internal use using store_type = tuple_type; static constexpr bool valid = ParamPack&lt;T...&gt;::valid; static_assert( ParamPack&lt;T...&gt;::valid == true, "One or more tuple elements is not a valid RPC parameter type."); template &lt;typename S&gt; static void write(S&amp; s, const tuple_type&amp; v) { details::Tuple&lt;tuple_type, std::tuple_size&lt;tuple_type&gt;::value == 0, 0&gt;::serialize(s, v); } template &lt;typename S&gt; static void read(S&amp; s, tuple_type&amp; v) { details::Tuple&lt;tuple_type, std::tuple_size&lt;tuple_type&gt;::value == 0, 0&gt;::deserialize(s, v); } static tuple_type&amp;&amp; get(tuple_type&amp;&amp; v) { return std::move(v); } }; } // namespace rpc } // namespace cz // A simple test void testDeserialization() { Stream s; serializeMethod&lt;decltype(&amp;SerializeTest::func1)&gt;(s, 1, std::vector&lt;int&gt;{1, 2}); // deserialize the parameters into a tuple. // the tuple is of type std::tuple&lt;int,std::vector&lt;int&gt;&gt; FunctionTraits&lt;decltype(&amp;SerializeTest::func1)&gt;::param_tuple params; s &gt;&gt; params; }; </code></pre> <p><a name="id-from-tuple-to-function-parameters"></a></p> <h3>From tuple to function parameters</h3> <p>After we deserialize all the parameters into a tuple, we now need to figure out how to unpack the tuple to call a matching function. This is once again done with variadic templates.</p> <pre> <code class="language-cpp">namespace cz { namespace rpc { namespace detail { template &lt;typename F, typename Tuple, bool Done, int Total, int... N&gt; struct callmethod_impl { static decltype(auto) call(typename FunctionTraits&lt;F&gt;::class_type&amp; obj, F f, Tuple&amp;&amp; t) { return callmethod_impl&lt;F, Tuple, Total == 1 + sizeof...(N), Total, N..., sizeof...(N)&gt;::call(obj, f, std::forward&lt;Tuple&gt;(t)); } }; template &lt;typename F, typename Tuple, int Total, int... N&gt; struct callmethod_impl&lt;F, Tuple, true, Total, N...&gt; { static decltype(auto) call(typename FunctionTraits&lt;F&gt;::class_type&amp; obj, F f, Tuple&amp;&amp; t) { using Traits = FunctionTraits&lt;F&gt;; return (obj.*f)( ParamTraits&lt;typename Traits::template argument&lt;N&gt;::type&gt;::get( std::get&lt;N&gt;(std::forward&lt;Tuple&gt;(t)))...); } }; } // namespace details template &lt;typename F, typename Tuple&gt; decltype(auto) callMethod(typename FunctionTraits&lt;F&gt;::class_type&amp; obj, F f, Tuple&amp;&amp; t) { static_assert(FunctionTraits&lt;F&gt;::valid, "Function not usable as RPC"); typedef typename std::decay&lt;Tuple&gt;::type ttype; return detail::callmethod_impl&lt; F, Tuple, 0 == std::tuple_size&lt;ttype&gt;::value, std::tuple_size&lt;ttype&gt;::value&gt;::call(obj, f, std::forward&lt;Tuple&gt;(t)); } } // namespace rpc } // namespace cz // A simple test void testCall() { Stream s; // serialize serializeMethod&lt;decltype(&amp;SerializeTest::func1)&gt;(s, 1, std::vector&lt;int&gt;{1, 2}); // deserialize FunctionTraits&lt;decltype(&amp;SerializeTest::func1)&gt;::param_tuple params; s &gt;&gt; params; // Call func1 on an object, unpacking the tuple into parameters SerializeTest obj; callMethod(obj, &amp;SerializeTest::func1, std::move(params)); }; </code></pre> <p>So, we now know how to validate a function, serialize, deserialize, and call it. That&#39;s the low level code done. The layer we&#39;ll now build on top of this code will be the actual RPC API.</p> <p><a name="id-the-rpc-api"></a></p> <h2>The RPC API</h2> <p><a name="id-header"></a></p> <h3>Header</h3> <p>The header contains the following information:</p> <p><img alt="" class="alignnone size-full wp-image-575" src="http://www.crazygaze.com/blog/wp-content/uploads/2016/05/img_574cbb33e8851.png" /></p> <table> <thead> <tr> <th>Field</th> <th>&nbsp;</th> </tr> </thead> <tbody> <tr> <td>size</td> <td>Total size in bytes of the RPC. Having the size as part of the header greatly simplifies things, since we can check if we received all the data before trying to process the RPC.</td> </tr> <tr> <td>counter</td> <td>Call number. Every time we call an RPC, a counter is incremented and assigned to that RPC call.</td> </tr> <tr> <td>rpcid</td> <td>The function to call</td> </tr> <tr> <td>isReply</td> <td>If true, it&#39;s a reply to an RPC. If false, it&#39;s an RPC call.</td> </tr> <tr> <td>success</td> <td>This only applies to replies (isReply==true). If true, the call was successful and the data is the reply. If false, the data is the exception information</td> </tr> </tbody> </table> <p>The counter and rpcid form a key that identifies an RPC call instance. This is needed to match an incoming RPC reply to the RPC call that caused it.</p> <pre> <code class="language-cpp">namespace cz { namespace rpc { // Small utility struct to make it easier to work with the RPC headers struct Header { enum { kSizeBits = 32, kRPCIdBits = 8, kCounterBits = 22, }; explicit Header() { static_assert(sizeof(*this) == sizeof(uint64_t), "Invalid size. Check the bitfields"); all_ = 0; } struct Bits { uint32_t size : kSizeBits; unsigned counter : kCounterBits; unsigned rpcid : kRPCIdBits; unsigned isReply : 1; // Is it a reply to a RPC call ? unsigned success : 1; // Was the RPC call a success ? }; uint32_t key() const { return (bits.counter &lt;&lt; kRPCIdBits) | bits.rpcid; } union { Bits bits; uint64_t all_; }; }; inline Stream&amp; operator&lt;&lt;(Stream&amp; s, const Header&amp; v) { s &lt;&lt; v.all_; return s; } inline Stream&amp; operator&gt;&gt;(Stream&amp; s, Header&amp; v) { s &gt;&gt; v.all_; return s; } } // namespace rpc } // namespace cz </code></pre> <p><a name="id-table"></a></p> <h3>Table</h3> <p>We already managed to serialize and deserialize an RPC, but not a way to map a serialized RPC to the right function on the server side. To solve this, we need to assign an ID to each function. The client knows what function it wants to call and fills the header with the right ID. The server checks the header, and knowing the ID, it dispatches to the right handler. Let&#39;s create the basics to define such dispatching tables.</p> <pre> <code class="language-cpp">namespace cz { namespace rpc { // // Helper code to dispatch a call. namespace details { // Handle RPCs with return values template &lt;typename R&gt; struct CallHelper { template &lt;typename OBJ, typename F, typename P&gt; static void impl(OBJ&amp; obj, F f, P&amp;&amp; params, Stream&amp; out) { out &lt;&lt; callMethod(obj, f, std::move(params)); } }; // Handle void RPCs template &lt;&gt; struct CallHelper&lt;void&gt; { template &lt;typename OBJ, typename F, typename P&gt; static void impl(OBJ&amp; obj, F f, P&amp;&amp; params, Stream&amp; out) { callMethod(obj, f, std::move(params)); } }; } struct BaseRPCInfo { BaseRPCInfo() {} virtual ~BaseRPCInfo(){}; std::string name; }; class BaseTable { public: BaseTable() {} virtual ~BaseTable() {} bool isValid(uint32_t rpcid) const { return rpcid &lt; m_rpcs.size(); } protected: std::vector&lt;std::unique_ptr&lt;BaseRPCInfo&gt;&gt; m_rpcs; }; template &lt;typename T&gt; class TableImpl : public BaseTable { public: using Type = T; struct RPCInfo : public BaseRPCInfo { std::function&lt;void(Type&amp;, Stream&amp; in, Stream&amp; out)&gt; dispatcher; }; template &lt;typename F&gt; void registerRPC(uint32_t rpcid, const char* name, F f) { assert(rpcid == m_rpcs.size()); auto info = std::make_unique&lt;RPCInfo&gt;(); info-&gt;name = name; info-&gt;dispatcher = [f](Type&amp; obj, Stream&amp; in, Stream&amp; out) { using Traits = FunctionTraits&lt;F&gt;; typename Traits::param_tuple params; in &gt;&gt; params; using R = typename Traits::return_type; details::CallHelper&lt;R&gt;::impl(obj, f, std::move(params), out); }; m_rpcs.push_back(std::move(info)); } }; template &lt;typename T&gt; class Table : public TableImpl&lt;T&gt; { static_assert(sizeof(T) == 0, "RPC Table not specified for the type."); }; } // namespace rpc } // namespace cz </code></pre> <p>The <code>Table</code> template needs to be specialized for the class we want to use for RPC calls. Say we have a <code>Calculator</code> class we want to be able to use for RPC calls:</p> <pre> <code class="language-cpp">class Calculator { public: double add(double a, double b) { m_ans = a + b; return m_ans; } double sub(double a, double b) { m_ans = a - b; return m_ans; } double ans() { return m_ans; } private: double m_ans = 0; }; </code></pre> <p>We can specialize the <code>Table</code> template for <code>Calculator</code>, so both the client and server have something to work with :</p> <pre> <code class="language-cpp">// Table specialization for Calculator template &lt;&gt; class cz::rpc::Table&lt;Calculator&gt; : cz::rpc::TableImpl&lt;Calculator&gt; { public: enum class RPCId { add, sub, ans }; Table() { registerRPC((int)RPCId::add, "add", &amp;Calculator::add); registerRPC((int)RPCId::sub, "sub", &amp;Calculator::sub); registerRPC((int)RPCId::ans, "ans", &amp;Calculator::ans); } static const RPCInfo* get(uint32_t rpcid) { static Table&lt;Calculator&gt; tbl; assert(tbl.isValid(rpcid)); return static_cast&lt;RPCInfo*&gt;(tbl.m_rpcs[rpcid].get()); } }; </code></pre> <p>Given an ID, the <code>get</code> function returns the dispatcher for the right <code>Calculator</code> method. We can then pass our <code>Calculator</code> instance, input and output streams to the dispatcher, which takes care of the rest.</p> <p>These specializations are quite verbose and error-prone, since the enums and the <code>registerRPC</code> calls need to match. But we can greatly shorten that with some macros. But first, let&#39;s see a verbose example on how to use this table :</p> <pre> <code class="language-cpp">void testCalculatorTable() { // Both the client and server need to have access to the necessary table using CalcT = Table&lt;Calculator&gt;; // // Client sends RPC Stream toServer; RPCHeader hdr; hdr.bits.rpcid = (int)CalcT::RPCId::add; toServer &lt;&lt; hdr; serializeMethod&lt;decltype(&amp;Calculator::add)&gt;(toServer, 1.0, 9.0); // // Server receives RPC, and sends back a reply Calculator calc; // object used to receive the RPCs toServer &gt;&gt; hdr; auto&amp;&amp; info = CalcT::get(hdr.bits.rpcid); Stream toClient; // Stream for the reply // Call the desired Calculator function. info-&gt;dispatcher(calc, toServer, toClient); // // Client receives a reply double r; toClient &gt;&gt; r; printf("%2.0f\n", r); // Will print "10" } </code></pre> <p>Again, this is quite verbose, but its just to show the code flow. That will be improved later.</p> <p>Now, how can we simplify the table specializations? If we put the gist of tables specialization in an unguarded header, all you need is a couple of defines followed by an include of that unguarded header to generate what the equivalent of what we did manually.</p> <p>Example:</p> <pre> <code class="language-cpp">#define RPCTABLE_CLASS Calculator #define RPCTABLE_CONTENTS \ REGISTERRPC(add) \ REGISTERRPC(sub) \ REGISTERRPC(ans) #include "RPCGenerate.h" </code></pre> <p>That&#39;s surprisingly simple, no?</p> <p>The <code>RPCGenerate.h</code> is an unguarded header that looks like this:</p> <pre> <code class="language-cpp">#ifndef RPCTABLE_CLASS #error "Macro RPCTABLE_CLASS needs to be defined" #endif #ifndef RPCTABLE_CONTENTS #error "Macro RPCTABLE_CONTENTS needs to be defined" #endif #define RPCTABLE_TOOMANYRPCS_STRINGIFY(arg) #arg #define RPCTABLE_TOOMANYRPCS(arg) RPCTABLE_TOOMANYRPCS_STRINGIFY(arg) template&lt;&gt; class cz::rpc::Table&lt;RPCTABLE_CLASS&gt; : cz::rpc::TableImpl&lt;RPCTABLE_CLASS&gt; { public: using Type = RPCTABLE_CLASS; #define REGISTERRPC(rpc) rpc, enum class RPCId { RPCTABLE_CONTENTS NUMRPCS }; Table() { static_assert((unsigned)((int)RPCId::NUMRPCS-1)&lt;(1&lt;&lt;Header::kRPCIdBits), RPCTABLE_TOOMANYRPCS(Too many RPCs registered for class RPCTABLE_CLASS)); #undef REGISTERRPC #define REGISTERRPC(func) registerRPC((uint32_t)RPCId::func, #func, &amp;Type::func); RPCTABLE_CONTENTS } static const RPCInfo* get(uint32_t rpcid) { static Table&lt;RPCTABLE_CLASS&gt; tbl; assert(tbl.isValid(rpcid)); return static_cast&lt;RPCInfo*&gt;(tbl.m_rpcs[rpcid].get()); } }; #undef REGISTERRPC #undef RPCTABLE_START #undef RPCTABLE_END #undef RPCTABLE_CLASS #undef RPCTABLE_CONTENTS #undef RPCTABLE_TOOMANYRPCS_STRINGIFY #undef RPCTABLE_TOOMANYRPCS </code></pre> <p>As a bonus, specializing tables like this makes it easy to support inheritance. Imagine we have a <code>ScientificCalculator</code> that inherits from Calculator:</p> <pre> <code class="language-cpp">class ScientificCalculator : public Calculator { public: double sqrt(double a) { return std::sqrt(a); } }; </code></pre> <p>By separately defining the contents of <code>Calculator</code>, we can reuse that define:</p> <pre> <code class="language-cpp">// Separately define the Calculator contents so it can be reused #define RPCTABLE_CALCULATOR_CONTENTS \ REGISTERRPC(add) \ REGISTERRPC(sub) \ REGISTERRPC(ans) // Calculator table #define RPCTABLE_CLASS Calculator #define RPCTABLE_CONTENTS \ RPCTABLE_CALCULATOR_CONTENTS #include "RPCGenerate.h" // ScientificCalculator table #define RPCTABLE_CLASS ScientificCalculator #define RPCTABLE_CONTENTS \ RPCTABLE_CALCULATOR_CONTENTS \ REGISTERRPC(sqrt) #include "RPCGenerate.h" </code></pre> <p><a name="id-transport"></a></p> <h3>Transport</h3> <p>We need to define how data will be transported between client and server. Let&#39;s put that in a <code>Transport</code> interface class. The interface is intentionally left very simple so the application can specify a custom transport . All we need is a method to send, receive, and to close.</p> <pre> <code class="language-cpp">namespace cz { namespace rpc { class Transport { public: virtual ~Transport() {} // Send one single RPC virtual void send(std::vector&lt;char&gt; data) = 0; // Receive one single RPC // dst : Will contain the data for one single RPC, or empty if no RPC // available // return: true if the transport is still alive, false if the transport // closed virtual bool receive(std::vector&lt;char&gt;&amp; dst) = 0; // Close connection to the peer virtual void close() = 0; }; } // namespace rpc } // namespace cz </code></pre> <p><a name="id-result"></a></p> <h3>Result</h3> <p>How should a RPC result look like? Whenever we make an RPC call, the result can come in 3 forms.</p> <table> <thead> <tr> <th>Form</th> <th>Meaning</th> </tr> </thead> <tbody> <tr> <td>Valid</td> <td>We got a reply from the server with the result value of the RPC call</td> </tr> <tr> <td>Aborted</td> <td>The connection failed or was closed, and therefore we don&#39;t have result value</td> </tr> <tr> <td>Exception</td> <td>We got a reply from the server with an exception string (The RPC call caused an exception server side)</td> </tr> </tbody> </table> <pre> <code class="language-cpp">namespace cz { namespace rpc { class Exception : public std::exception { public: Exception(const std::string&amp; msg) : std::exception(msg.c_str()) {} }; template &lt;typename T&gt; class Result { public: using Type = T; Result() : m_state(State::Aborted) {} explicit Result(Type&amp;&amp; val) : m_state(State::Valid), m_val(std::move(val)) {} Result(Result&amp;&amp; other) { moveFrom(std::move(other)); } Result(const Result&amp; other) { copyFrom(other); } ~Result() { destroy(); } Result&amp; operator=(Result&amp;&amp; other) { if (this == &amp;other) return *this; destroy(); moveFrom(std::move(other)); return *this; } // Construction from an exception needs to be separate. so // RPCReply&lt;std::string&gt; works. // Otherwise we would have no way to tell if constructing from a value, or // from an exception static Result fromException(std::string ex) { Result r; r.m_state = State::Exception; new (&amp;r.m_ex) std::string(std::move(ex)); return r; } template &lt;typename S&gt; static Result fromStream(S&amp; s) { Type v; s &gt;&gt; v; return Result(std::move(v)); }; bool isValid() const { return m_state == State::Valid; } bool isException() const { return m_state == State::Exception; }; bool isAborted() const { return m_state == State::Aborted; } T&amp; get() { if (!isValid()) throw Exception(isException() ? m_ex : "RPC reply was aborted"); return m_val; } const T&amp; get() const { if (!isValid()) throw Exception(isException() ? m_ex : "RPC reply was aborted"); return m_val; } const std::string&amp; getException() { assert(isException()); return m_ex; }; private: void destroy() { if (m_state == State::Valid) m_val.~Type(); else if (m_state == State::Exception) { using String = std::string; m_ex.~String(); } m_state = State::Aborted; } void moveFrom(Result&amp;&amp; other) { m_state = other.m_state; if (m_state == State::Valid) new (&amp;m_val) Type(std::move(other.m_val)); else if (m_state == State::Exception) new (&amp;m_ex) std::string(std::move(other.m_ex)); } void copyFrom(const Result&amp; other) { m_state = other.m_state; if (m_state == State::Valid) new (&amp;m_val) Type(other.m_val); else if (m_state == State::Exception) new (&amp;m_ex) std::string(other.m_ex); } enum class State { Valid, Aborted, Exception }; State m_state; union { Type m_val; std::string m_ex; }; }; // void specialization template &lt;&gt; class Result&lt;void&gt; { public: Result() : m_state(State::Aborted) {} Result(Result&amp;&amp; other) { moveFrom(std::move(other)); } Result(const Result&amp; other) { copyFrom(other); } ~Result() { destroy(); } Result&amp; operator=(Result&amp;&amp; other) { if (this == &amp;other) return *this; destroy(); moveFrom(std::move(other)); return *this; } static Result fromException(std::string ex) { Result r; r.m_state = State::Exception; new (&amp;r.m_ex) std::string(std::move(ex)); return r; } template &lt;typename S&gt; static Result fromStream(S&amp; s) { Result r; r.m_state = State::Valid; return r; } bool isValid() const { return m_state == State::Valid; } bool isException() const { return m_state == State::Exception; }; bool isAborted() const { return m_state == State::Aborted; } const std::string&amp; getException() { assert(isException()); return m_ex; }; void get() const { if (!isValid()) throw Exception(isException() ? m_ex : "RPC reply was aborted"); } private: void destroy() { if (m_state == State::Exception) { using String = std::string; m_ex.~String(); } m_state = State::Aborted; } void moveFrom(Result&amp;&amp; other) { m_state = other.m_state; if (m_state == State::Exception) new (&amp;m_ex) std::string(std::move(other.m_ex)); } void copyFrom(const Result&amp; other) { m_state = other.m_state; if (m_state == State::Exception) new (&amp;m_ex) std::string(other.m_ex); } enum class State { Valid, Aborted, Exception }; State m_state; union { bool m_dummy; std::string m_ex; }; }; } // namespace rpc } // namespace cz </code></pre> <p>The <code>Result&lt;void&gt;</code> specialization is needed, since for that case there isn&#39;t a result, but the caller still wants to know if the RPC call was properly processed. Initially, I considered using <a href="https://en.wikipedia.org/wiki/Andrei_Alexandrescu"><code>Expected&lt;T&gt;</code></a> for RPC replies. But <code>Expected&lt;T&gt;</code> has basically 2 states (Value or Exception), and we need 3 (Value, Exception, and Aborted). One can think that Aborted could be considered an exception, but from a client point of view, it&#39;s not always the case. In some cases you want to know an RPC failed because the connection was closed, and not because the server replied with an exception.</p> <p><a name="id-out-processor"></a></p> <h3>OutProcessor</h3> <p>We need to track ongoing RPC calls so the user code can get the results when they arrive. Handling a result can be done in two ways. Throught an asynchronous handler (similar to <a href="https://en.wikipedia.org/wiki/Asio_C%2B%2B_library">Asio</a> ), or with a future.</p> <p>Two classes are needed for this. An outgoing processor and a wrapper for a single RPC call. Another class is needed (<code>Connection</code>) that ties together outgoing and incoming processors. It will be introduced later.</p> <pre> <code class="language-cpp">namespace cz { namespace rpc { class BaseOutProcessor { public: virtual ~BaseOutProcessor() {} protected: template &lt;typename R&gt; friend class Call; template &lt;typename L, typename R&gt; friend struct Connection; template &lt;typename F, typename H&gt; void commit(Transport&amp; transport, uint32_t rpcid, Stream&amp; data, H&amp;&amp; handler) { std::unique_lock&lt;std::mutex&gt; lk(m_mtx); Header hdr; hdr.bits.size = data.writeSize(); hdr.bits.counter = ++m_replyIdCounter; hdr.bits.rpcid = rpcid; *reinterpret_cast&lt;Header*&gt;(data.ptr(0)) = hdr; m_replies[hdr.key()] = [handler = std::move(handler)](Stream * in, Header hdr) { using R = typename ParamTraits&lt; typename FunctionTraits&lt;F&gt;::return_type&gt;::store_type; if (in) { if (hdr.bits.success) { handler(Result&lt;R&gt;::fromStream((*in))); } else { std::string str; (*in) &gt;&gt; str; handler(Result&lt;R&gt;::fromException(std::move(str))); } } else { // if the stream is nullptr, it means the result is being aborted handler(Result&lt;R&gt;()); } }; lk.unlock(); transport.send(data.extract()); } void processReply(Stream&amp; in, Header hdr) { std::function&lt;void(Stream*, Header)&gt; h; { std::unique_lock&lt;std::mutex&gt; lk(m_mtx); auto it = m_replies.find(hdr.key()); assert(it != m_replies.end()); RPCs on a "Calculator" server OutProcessor<Calculator> outPrc; // Handle with an asynchronous handler outPrc.call<decltype(&Calculator::add)>( trp, (int)Table<Calculator>::RPCId::add, 1.0, 2.0) .async([](Result<double> res) { printf("%2.0f\n", res.get()); // prints '3' }); // Handle with a future Result<double> res = outPrc.call<decltype(&Calculator::add)>( trp, (int)Table<Calculator>::RPCId::add, 1.0, 3.0) .ft().get(); printf("%2.0f\n", res.get()); // prints '4' }
Again, a bit verbose, since I haven't introduced all the code that wraps things up. But shows how the OutProcessor<T> and Call interfaces work. The std::future implementation simply builds on the asynchronous implementation.
InProcessor
Now that we can send an RPC and wait for the result, let's look at what we need to do on the other side. What to do when an RPC call is received on the server.
Let's create a InProcessor<T> class. Contrary to OutProcessor<T>, InProcessor<T> needs to keep a reference to an object of type T. This is so when an RPC is received, it can call the requested method on that object, and send the result back to the client.
namespace cz { namespace rpc { class BaseInProcessor { public: virtual ~BaseInProcessor() {} }; template <typename T> class InProcessor : public BaseInProcessor { public: using Type = T; InProcessor(Type* obj, bool doVoidReplies = true) : m_obj(*obj), m_voidReplies(doVoidReplies) {} void processCall(Transport& transport, Stream& in, Header hdr) { Stream out; // Reuse the header as the header for the reply, so we keep the counter // and rpcid hdr.bits.size = 0; hdr.bits.isReply = true; hdr.bits.success = true; auto&& info = Table<Type>::get(hdr.bits.rpcid); #if CZRPC_CATCH_EXCEPTIONS try { #endif out << hdr; // Reserve space for the header info->dispatcher(m_obj, in, out); #if CZRPC_CATCH_EXCEPTIONS } catch (std::exception& e) { out.clear(); out << hdr; // Reserve space for the header hdr.bits.success = false; out << e.what(); } #endif if (m_voidReplies || (out.writeSize() > sizeof(hdr))) { hdr.bits.size = out.writeSize(); *reinterpret_cast<Header*>(out.ptr(0)) = hdr; transport.send(out.extract()); } } protected: Type& m_obj; bool m_voidReplies = false; }; template <> class InProcessor<void> { public: InProcessor(void*) {} void processCall(Transport&, Stream&, Header) { assert(0 && "Incoming RPC not allowed for void local type"); } }; } // namespace rpc } // namespace cz
The CZRPC_CATCH_EXCEPTIONS define allows us to tweak if we want server side exceptions to be passed to the clients.
It's the use of InProcessor<T> (and Table<T>) that allows calling RPCs on objects that don't know anything about RPCs or network. For example, consider this dummy example:
void calculatorServer() { // The object we want to use for RPC calls Calculator calc; // The server processor. It will call the appropriate methods on 'calc' when // an RPC is received InProcessor<Calculator> serverProcessor(&calc); while (true) { // calls to serverProcessor::processCall whenever there is data } }
The Calculator object used for RPCs doesn't know anything about RPCs. The InProcessor<Calculator> does all the required work. This makes it possible use third party classes for RPCs. In some situations, we do want have the class used for RPCs to know about RPCs and/or network. For example, if you're creating a chat system, you have the clients sending messages (RPC calls) to the server. The server needs to know what clients are connected, so it can broadcast messages.
Connection
We can now send and receive RPCs, although with a bit of a verbose API. The OutProcessor<T> and InProcessor<T> template classes deal with what happens to data at both ends of the connection. So, what we need now is exactly that. A Connection to tie in one place everything needed to send and receive data, and simplify the API.
namespace cz { namespace rpc { struct BaseConnection { virtual ~BaseConnection() {} //! Process any incoming RPCs or replies // Return true if the connection is still alive, false otherwise virtual bool process() = 0; }; template <typename LOCAL, typename REMOTE> struct Connection : public BaseConnection { using Local = LOCAL; using Remote = REMOTE; using ThisType = Connection<Local, Remote>; Connection(Local* localObj, std::shared_ptr<Transport> transport) : localPrc(localObj), transport(std::move(transport)) {} template <typename F, typename... Args> auto call(Transport& transport, uint32_t rpcid, Args&&... args) { return remotePrc.template call<F>(transport, rpcid, std::forward<Args>(args)...); } static ThisType* getCurrent() { auto it = Callstack<ThisType>::begin(); return (*it) == nullptr ? nullptr : (*it)->getKey(); } virtual bool process() override { // Place a callstack marker, so other code can detect we are serving an // RPC typename Callstack<ThisType>::Context ctx(this); std::vector<char> data; while (true) { if (!transport->receive(data)) { // Transport is closed remotePrc.abortReplies(); return false; } if (data.size() == 0) return true; // No more pending data to process Header hdr; Stream in(std::move(data)); in >> hdr; if (hdr.bits.isReply) { remotePrc.processReply(in, hdr); } else { localPrc.processCall(*transport, in, hdr); } } } InProcessor<Local> localPrc; OutProcessor<Remote> remotePrc; std::shared_ptr<Transport> transport; }; } // namespace rpc } // namespace cz
This puts together the output processor, the input processor, and the transport. To make it possible for user code to detect if it's in the middle of serving an RPC, it uses a class I introduced in an earlier post. The Callstack class. This allows the creation of RPC/network aware code if necessary, like server classes.
So, how this simplifies the API ? Since the Connection<T> has everything we need, one macro taking as parameters the connection object, a function name and the parameters, does everything, including type checks so it doesn't compile if it's an invalid call.
#define CZRPC_CALL(con, func, ...) \ (con).call<decltype(&std::decay<decltype(con)>::type::Remote::func)>( \ *(con).transport, \ (uint32_t)cz::rpc::Table< \ std::decay<decltype(con)>::type::Remote>::RPCId::func, \ ##__VA_ARGS__)
Using this macro, RPC calls syntax is surprisingly simple. For example, consider this client code:
// Some class to use for RPC calls class MagicSauce { public: int func1(int a, int b) { return a + b; } int func2(int a, int b) { return a + b; } }; // Define RPC table for MagicSauce #define RPCTABLE_CLASS MagicSauce #define RPCTABLE_CONTENTS REGISTERRPC(func1) #include "RPCGenerate.h" // 'trp' is a fully functional transport void test_Connection(std::shared_ptr<Transport> trp) { Connection<void, MagicSauce> con(nullptr, trp); // Doesn't compile : Invalid number of parameters // CZRPC_CALL(con, func1, 1); // Doesn't compile : Wrong type of parameters // CZRPC_CALL(con, func1, 1, "hello"); // Doesn't compile: func3 is not a MagicSauce method // CZRPC_CALL(con, func3, 1, 2); // Doesn't compile: func2 is a method of MagicSauce, but not registered as // RPC // CZRPC_CALL(con, func2, 1, 2); // Compiles fine, since everything is valid CZRPC_CALL(con, func1, 1, 2).async([](Result<int> res) { printf("%d\n", res.get()); // print '3' }); }
Notice the void and nullptr used when creating the connection with Connection<void, MagicSauce> con(nullptr, trp); ? This accommodates for bidirectional RPCs (the server can also call RPCs on a client). In this case, we don't expect client side RPCs, so the client side Connection object doesn't have a local object to call RPCs on.
A simplified example (not functional) of bidirectional RPCs can be something like this:
class ChatClient; class ChatServer { public: // Called by clients to post new messages void msg(const char* msg); void addNewClient(std::shared_ptr<Transport> trp); private: // Connection specifies both a LOCAL, and REMOTE object types std::vector<std::unique_ptr<Connection<ChatServer, ChatClient>>> m_clients; }; #define RPCTABLE_CLASS ChatServer #define RPCTABLE_CONTENTS REGISTERRPC(msg) #include "RPCGenerate.h" class ChatClient { public: void onMsg(const char* msg); }; #define RPCTABLE_CLASS ChatClient #define RPCTABLE_CONTENTS REGISTERRPC(onMsg) #include "RPCGenerate.h" void ChatServer::msg(const char* msg) { // Simply broadcast the message to all clients for (auto&& c : m_clients) { CZRPC_CALL(*c, onMsg, msg); } } void ChatServer::addNewClient(std::shared_ptr<Transport> trp) { auto con = std::make_unique<Connection<ChatServer, ChatClient>>(this, trp); m_clients.push_back(std::move(con)); } void ChatClient::onMsg(const char* msg) { printf("%s\n", msg); } void test_ChatServer() { ChatServer server; while (true) { // Wait for connections, and call ChatServer::addClient } } // 'trp' is some fully functional transport connected to the ChatServer void test_ChatClient(std::shared_ptr<Transport> trp) { ChatClient client; // In this case, we have a client side object to answer RPCs Connection<ChatClient, ChatServer> con(&client, trp); while (true) { // call the 'msg' RPC whenever the user types something, like this: CZRPC_CALL(con, msg, "some message"); // The server will call our client 'onMsg' when other clients send a // message } }
The Connection template parameters in the server and client are reversed. Whenever data is received, the Connection object forwards processing to the InProcessor if it's an incoming RPC call (to call on our side), or to the OutProcessor if it's a reply to a previous outgoing RPC call. Data flow for bidirectional RPCs looks like this:
Improvements
A couple of things were left out of the framework intentionally, so the application can decide what's best. For example:
Transport initialization
The transport interface is very simple, so it doesn't impose any specific way of initialization or detecting incoming data. It's up to the application to provide a fully functional and connected transport to the Connection class. This is also the reason why I avoided showing any transport initialization, since I would have to present a fully functional transport implementation for that.
At the time of writing, the source code repository has one transport implementation using Boost Asio (or standalone Asio)
Disconnection detection
As with initialization, there is minimal code for dealing with or detecting shutdowns. It is up to the application to decide how to do this with whatever custom transports it provides.
At some point I had support for server side functions that return std::futures. But as I was writing this article I removed that to simplify things. That will be added back to the repository soon, since I need it for my own projects.
This is needed to support wrapping classes whose API returns std::future instances. Say you have a Login server class with a login method. That login method is in itself probably asynchronous (returns a std::future) since you'll be checking a database for the login details.
It's not particularly hard to implement, but it has one niggle I couldn't get rid of without introducing a lot more code. The bulk of the changes would be in the InProcessor<T> class. It would have to keep track of ongoing calls whose std::future<T> instances are not ready yet. The problem here is how would InProcessor<T> detect when those futures are ready so it can send the result back to the client? I can think of two approaches:
Regular polling all pending std::futures to see if they are ready.
Use continuations (std::future::then), which are not available in the standard yet. That would be the easiest solution. InProcessor<T> would set a continuation to send the result back to the client whenever it is ready. No need for polling anything.
This would affect no client side code whatsoever. Say a server side function returns std::future<bool> . All the client sees and needs is still just a Result<bool>.
Read more about:
Featured BlogsAbout the Author(s)
You May Also Like
