Ground Up Functional API Design in C++

There are plenty of reasons why functional APIs look and feel different than more common object-oriented APIs. A tell-tale sign of a functional APIs is existence of a core abstraction and a set of methods with algebraic properties. The abstraction part is of course about being able to talk about just the necessary details and nothing more. The algebraic part is about being able to take one or more instances of the same abstraction and being able to create a new one following some laws. I.e., Lawfully composing smaller parts into a bigger thing that is an instance of the same abstraction the original instances were.

Composition by itself is nothing new to OO programmers. Composite, Decorator design patterns are all about putting together smaller pieces into something bigger. However, they miss out on the lawful part. Functional paradigm gives you extra guard rails so that you can bake in some well-understood mathematical properties into the abstraction.

I think most people with even basic OO experience "get" the abstraction part. But the algebraic part, however, needs some thinking, some training, and most importantly some well-made, simple to understand examples.

Speaking of examples, C++ has already adopted the ranges library as a technical specification (TS). At its core there are lawful algebraic properties. Understanding the functional principles from an industrial implementation is possible but harder. RxCpp is another example. Both libraries use operator overloading, which I think is a convenience and not a core part of the design. My focus today is understandability. So, here's my attempt to demystify the art of lawful algebraic api design with a simple example of data generators. See cpp-generators on github.


This talk presents two classic techniques from the functional domain — composable data generators and property-based testing — implemented in C++14 for testing a generic serialization and deserialization library (RefleX). It presents a systematic technique of constructing data generators from a mere random number generator and random type generation using compile-time meta-programming. The talk describes the laws of monoids, functors, applicative, and monads and how they are implemented in a simple to understand abstraction of data generators.

Check out Slides. Transcripts below for a quick read.

2:30
This is an attempt to demystify algebraic api design and buzz words like functor, monoid, and monads.

4:05
I wish the book Functional Programming in C++ by Manning had a chapter on algebraic api design like the material covered in this talk.

7:30
Property-based testing---the motivation behind data generators. Testing a serialization and deserialization library (Reflex)

12:00
The RefleX library---Compile-time reflection in C++ for DDS-XTypes

14:30
How to test RefleX?  What types and data should be used to test RefleX? We'll use property-based testing.

17:50
A code and type generator machinery that hammers the code with unthinkable tests. It is developer's nightmare but also the best friend.

Progression of the data generator abstraction...

20:00
A simple random number generator---a built-in library function

22:00 
A bool generator, an uppercase character generator---a custom function on top of the library function.

23:00
A generator of random numbers within a range---a function with some state (a closure)

26:00
A oneof generator from an std::initializer_list<t>.

28:00
A Gen<t> template with a generate function. A named type for the function object. It's structurally same as a closure. So it's not more powerful.

29:00
Is a function object (think closures) a sufficient abstraction for all kinds of generators we might need? No.

31:30
Now a much farther jump into abstraction level---a composable generator---an algebraic generator with methods such as map, concat, reduce, zip, where.

33:00
Composition is not going to be arbitrary. It will be governed by some well-known rules (laws). These are guard rails.

34:45
Mapping over a generator. Composition of a primitive generator and a mapping function gives another generator.

36:48
An excellent question from the audience. Why is the while loop infinite? The reason is that we've not seen termination as a requirement for the generators. It later become important to make sure the generator abstraction is a well-behaved monoid. To satisfy the monoid laws, especially the identity law, we've to have an empty generator and a way to convey that a generator ended. Hence we introduce termination in the core abstraction. Guard rails and all...

37:30
Introducing Functor---you can map over it. Functor laws. Composition law and the identity law.

40:30
Implementing the map function for generator

42:30
Overloading lvalue and rvalue reference qualified function (e.g., map) for fluent api. Algebraic APIs commonly implemented using the Fluent Interface technique generates a number of temporary objects. The overhead of temporary objects can be eliminated using move semantics. You can overload functions specifically for temporary objects (rvalues) using rvalue reference qualification.

44:00
Introducing Monoid---The fundamental ability to combine smaller generators to create a bigger generator. Identity law---appending an empty generator gives back the original generator. Without an empty generator it's not a monoid because identity law is not satisfied yet. A notion of termination must be introduced. We do that by just throwing an exception std::out_of_range.

50:00
Associative law of Monoid. Grouping does not matter.

53:00
Introducing Applicatives: Zipping generators. Implementation of zip_with. Generator is an applicative.

56:30
Introducing Monad to handle dependent generators. Generator of generators come up often. concat_map flattens nested generators. Monad laws.

1:03:00
Recapping algebraic api design based on mathematical abstractions. Algebra = a first-class lawful abstraction + compositional api.

1:06:00
Catagory theoretic intuition is much a deeper thing. I'm not sure I get it. For example, intuitively understanding "Monad is just a monoid in the category of endofunctors".

1:08:00
We can generate random types at compile-time if we can generate random numbers at compile-time.

1:10:00
Generating random numbers at compile-time. A 16-bit Linear Feedback Shift Register (LFSR) constexpr function.

constexpr int LFSR_bit(int prev) { 
  return (((prev >> 0) ^ (prev >> 2) ^ 
           (prev >> 3) ^ (prev >> 5)) & 1); 
} 

constexpr int LFSR(int prev) { 
  return ((prev >> 1) | (LFSR_bit(prev) << 15)); 
}

1:13:30
Random type generation at compile-time using a recursive TypeMap and an LFSR. See type_generator.

1:20:00
Generating a random 51 levels deep nested tuple or worse a tuple of static size 2.7 MB. Test your generic library with monster types.