Skip to main content

Bootstrapping a vcpkg-based project in Linux and Windows with idiomatic cmake

This blog is part #2 in the series of trying out different package managers to bootstrap a cmake project. Checkout part #1 about Bootstrapping a vcpkg-based cmake project in Visual Studio. Part #3 is about bootstrapping Hunter-based cmake project in Linux and Visual Studio. The cmake code in the previous post works well on Linux too. After all, both cmake and vcpkg are designed for cross-platform build management. So what's new here?

This time around we'll get the same project off the ground in both Linux and Windows with cmake proper. Last time, the cmake script CMakeLists.txt felt like a poorly written bash script. Since that blogpost, I received a lot of feedback.

Feedback from Carlos ORyan (Google) forms the basis of this blog post. It would be more accurate to say that I'm downright stealing the cmake-vcpkg integration scripts he shared with me. They are open-source and available at google-cloud-cpp/super. I've copied them nearly verbatim to my vcpkg_cmake_blog branch for ease of use and long term stability of the hyperlinks. Thanks Carlos!

The objective is the same: bootstrap a vcpkg-based cmake project. The mechanics are much more sophisticated and feel idiomatic cmake. Let's get started.

Cmake Project Structure

vcpkg_test
├── cmake
│   ├── AutoVcpkg.cmake
│   └── VcpkgBootstrap.cmake
├── CMakeLists.txt
├── include
│   └── driver.h
├── src
│   └── driver.cpp
└── test
    └── driver_test.cpp
There're two more files under the cmake directory. These are cmake scripts designed to download, install, configure vcpkg instances in both Linux and Windows. They also expose suitable cmake function for use to use in CMakeLists.txt. This integration is much nicer (but also complex).

The CMakeLists.txt looks as follows. 
cmake_minimum_required (VERSION 3.12)
set(MY_PROJECT_DEPENDENCIES boost-core boost-optional boost-filesystem) 

# This section of cmake is using AutoVcpkg to download, install, and configure vcpkg.
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_LIST_DIR}/cmake")
include(AutoVcpkg)
vcpkg_install(${MY_PROJECT_DEPENDENCIES})
message(STATUS "CMAKE_TOOLCHAIN_FILE=${CMAKE_TOOLCHAIN_FILE}")

# The CMakeLists from this point on is the same as that of part 1. 
project (vcpkg_test CXX)
set(CMAKE_CXX_STANDARD 17)
find_package(Boost 1.67 REQUIRED COMPONENTS filesystem)

add_executable(driver src/driver.cpp)
target_include_directories(driver PUBLIC ${Boost_INCLUDE_DIRS} ${PROJECT_SOURCE_DIR}/include )
target_link_libraries(driver ${Boost_LIBRARIES})

enable_testing()
include(CTest)
add_executable(driver_test ${PROJECT_SOURCE_DIR}/test/driver_test.cpp)
add_test(NAME driver COMMAND driver_test)
find_package finds and loads settings from an external project (package). Boost_FOUND will be set to indicate whether the Boost package was found. add_executable simply adds a target named driver to be built from the sources (src/driver.cpp). The Boost library dependencies are specified next for the driver target. First, a set of include directories are specified. Next, a set of libraries are specified. Note that boost-filesystem must be linked to driver program. Hence, target_link_libraries is essential. The variables Boost_INCLUDE_DIRS, Boost_LIBRARIES are set by find_package (only upon success).

vcpkg_install

Here's the full code of AutoVcpkg.cmake. Here's the github branch vcpkg_cmake_blog_idiomatic.

We're including the files under cmake directory as "modules" and simply invoking them using vcpkg_install. The code is mostly self-explanatory. If you are new to cmake, you might have to stare at it for a while though.

The vcpkg-download is a separate cmake project. The CMakeLists.txt for this project is created while generating the build files for the driver project. I.e., It allows every project to bootstrap it's own vcpkg repository. This may or may not be desirable. For smaller project it might be an overkill. For large projects where controlling the exact library version separate from vcpkg repository HEAD is desirable, one might want a dedicated vcpkg instance. Here's the ExternalProject vcpkg_download. 
cmake_minimum_required(VERSION 3.12)
project(vcpkg-download)

include(ExternalProject)
ExternalProject_Add(vcpkg
            GIT_REPOSITORY @AUTO_VCPKG_GIT_REPOSITORY@
            # GIT_TAG 52870c7595a63ade069ae51d5f4ee3a85fe4123f # TODO: Debug this 
            GIT_SHALLOW ON
            SOURCE_DIR @AUTO_VCPKG_ROOT@
            PATCH_COMMAND ""
            CONFIGURE_COMMAND  ""
            BUILD_COMMAND ""
            INSTALL_COMMAND ""
            LOG_DOWNLOAD ON
            LOG_CONFIGURE ON
            LOG_INSTALL ON)
So instead of simply forking off and launching git clone directly from cmake, this external project allows a plethora of options and configure the download step.

The vcpkg_download function spits out and runs this project (with another invocation of cmake) only if needed. I ended up passing additional flags to cmake on Windows. Having to pass additional flags like CMAKE_EXE_LINKER_FLAGS, CMAKE_C_COMPILER, and CMAKE_CXX_COMPILER (from parent to the nested invocation of cmake) indicates that cmake integration with Visual Studio is still rough on the edges. Here's a snippet. 
function (vcpkg_download)
    if (DEFINED AUTO_VCPKG_ROOT)
        return()
    endif ()
    set(AUTO_VCPKG_ROOT "${CMAKE_BINARY_DIR}/vcpkg")
    # Generate the vcpkg_download project if necessary.
    file(WRITE "${CMAKE_BINARY_DIR}/vcpkg-download/CMakeLists.txt" "${vcpkg_download_contents}")
    if(WIN32)
      get_filename_component(VC_COMPILER_PATH ${CMAKE_C_COMPILER} DIRECTORY)
      set(VCRT_LIB_PATH "${VC_COMPILER_PATH}/../../../lib/x86")
      execute_process(COMMAND "${CMAKE_COMMAND}"
              "-H${CMAKE_BINARY_DIR}/vcpkg-download"
              "-B${CMAKE_BINARY_DIR}/vcpkg-download"
              "-DCMAKE_C_COMPILER:FILEPATH=${CMAKE_C_COMPILER}"
              "-DCMAKE_CXX_COMPILER:FILEPATH=${CMAKE_CXX_COMPILER}"
              "-DCMAKE_EXE_LINKER_FLAGS=/LIBPATH:\"${VCRT_LIB_PATH}\"")
      execute_process(COMMAND "${CMAKE_COMMAND}"
              "--build" "${CMAKE_BINARY_DIR}/vcpkg-download")
    else()
      # Linux here.
    endif()
If the previous step does not succeed in building vcpkg successfully (i.e., if AUTO_VCPKG_EXECUTABLE is undefined), there's plan B. The plan B is to do pretty much fork off a child cmake process and run vcpkg bootstrap.sh or bootstrap.bat directly. We saw a very simple version of it in part #1. 
function (vcpkg_bootstrap)
    find_program(AUTO_VCPKG_EXECUTABLE vcpkg PATHS ${AUTO_VCPKG_ROOT})
    if (NOT AUTO_VCPKG_EXECUTABLE)
        execute_process(COMMAND ${CMAKE_COMMAND} -E copy "${CMAKE_CURRENT_LIST_DIR}/cmake/VcpkgBootstrap.cmake" "${AUTO_VCPKG_ROOT}")
        execute_process(COMMAND ${CMAKE_COMMAND} -P "${AUTO_VCPKG_ROOT}/VcpkgBootstrap.cmake"
                WORKING_DIRECTORY ${AUTO_VCPKG_ROOT})
    endif ()
endfunction ()

###### VcpkgBootstrap.cmake file
find_program(VCPKG_EXECUTABLE
        vcpkg PATHS "${CMAKE_CURRENT_LIST_DIR}")
if (NOT VCPKG_EXECUTABLE)
    if (WIN32)
        execute_process(COMMAND "${CMAKE_CURRENT_LIST_DIR}/bootstrap-vcpkg.bat"
                WORKING_DIRECTORY "${CMAKE_CURRENT_LIST_DIR}")
    else ()
        execute_process(COMMAND  "${CMAKE_CURRENT_LIST_DIR}/bootstrap-vcpkg.sh"
                WORKING_DIRECTORY "${CMAKE_CURRENT_LIST_DIR}")
    endif ()
endif ()
At this point we've covered the gist. There are a lot of new things I learned about cmake.

The main differences between part #1 and this cmake project are the following.
  1. vcpkg is cloned from the github repository, compiled, and bootstrapped in the cmake binary directory. The directory you use for out-of-source builds (e.g., build). Previously, vcpkg is cloned, compiled, and bootstrapped in $ENV{HOMEDRIVE}$ENV{HOMEPATH}/vcpkg_cpptruths
  2. The vcpkg-download project is a real cmake project that generates a Makefile for bootstrapping vcpkg. On Windows, it generates a solution file under $ENV{HOMEDRIVE}$ENV{HOMEPATH}\CMakeBuilds\...\build\x86-Debug\vcpkg-download. Things are really meta at this point. cmake ExternalProject is used for that. Some tweaks in execute_process were necessary to pass the right ${CMAKE_EXE_LINKER_FLAGS}to build vcpkg with Visual Studio.
The projects seems to contain some meta targets that are unrelated to the main "driver" project. Here's how it looks like.

Observations

There are a couple of things vcpkg.cmake could make the experience better.
  1. GIT_TAG ... simply did not work for me in ExternalProject_Add. Cloning a specific tag/branch/commit hash of vcpkg is important for reproducible builds. Btw, Why aren't there any official releases of vcpkg? There's not a single tag as of this writing.
  2. The technique is this post is lower level but feels much more well-integrated. However, the end effect is the same. Not sure if it's worth the increased complexity. Especially because I had to overcome vcpkg build error "LINK : fatal error LNK1104: cannot open file 'MSVCRTD.lib'" that did not happen in part #1. The resulting Visual Studio project has some cruft too.

Comments

Post a Comment

Popular Content

Unit Testing C++ Templates and Mock Injection Using Traits

Unit testing your template code comes up from time to time. (You test your templates, right?) Some templates are easy to test. No others. Sometimes it's not clear how to about injecting mock code into the template code that's under test. I've seen several reasons why code injection becomes challenging. Here I've outlined some examples below with roughly increasing code injection difficulty. Template accepts a type argument and an object of the same type by reference in constructor Template accepts a type argument. Makes a copy of the constructor argument or simply does not take one Template accepts a type argument and instantiates multiple interrelated templates without virtual functions Lets start with the easy ones. Template accepts a type argument and an object of the same type by reference in constructor This one appears straight-forward because the unit test simply instantiates the template under test with a mock type. Some assertion might be tested in
Post a Comment
Read more

Covariance and Contravariance in C++ Standard Library

Covariance and Contravariance are concepts that come up often as you go deeper into generic programming. While designing a language that supports parametric polymorphism (e.g., templates in C++, generics in Java, C#), the language designer has a choice between Invariance, Covariance, and Contravariance when dealing with generic types. C++'s choice is "invariance". Let's look at an example. struct Vehicle {}; struct Car : Vehicle {}; std::vector<Vehicle *> vehicles; std::vector<Car *> cars; vehicles = cars; // Does not compile The above program does not compile because C++ templates are invariant. Of course, each time a C++ template is instantiated, the compiler creates a brand new type that uniquely represents that instantiation. Any other type to the same template creates another unique type that has nothing to do with the earlier one. Any two unrelated user-defined types in C++ can't be assigned to each-other by default. You have to provide a
14 comments
Read more

Multi-dimensional arrays in C++11

What new can be said about multi-dimensional arrays in C++? As it turns out, quite a bit! With the advent of C++11, we get new standard library class std::array. We also get new language features, such as template aliases and variadic templates. So I'll talk about interesting ways in which they come together. It all started with a simple question of how to define a multi-dimensional std::array. It is a great example of deceptively simple things. Are the following the two arrays identical except that one is native and the other one is std::array? int native[3][4]; std::array<std::array<int, 3>, 4> arr; No! They are not. In fact, arr is more like an int[4][3]. Note the difference in the array subscripts. The native array is an array of 3 elements where every element is itself an array of 4 integers. 3 rows and 4 columns. If you want a std::array with the same layout, what you really need is: std::array<std::array<int, 4>, 3> arr; That's quite annoying for
20 comments
Read more