Skip to main content

Bootstrapping a vcpkg-based project in Linux and Windows with idiomatic cmake

This blog is part #2 in the series of trying out different package managers to bootstrap a cmake project. Checkout part #1 about Bootstrapping a vcpkg-based cmake project in Visual Studio. The cmake code in the previous post works well on Linux too. After all, both cmake and vcpkg are designed for cross-platform build management. So what's new here?

This time around we'll get the same project off the ground in both Linux and Windows with cmake proper. Last time, the cmake script CMakeLists.txt felt like a poorly written bash script. Since that blogpost, I received a lot of feedback.

Feedback from Carlos ORyan (Google) forms the basis of this blog post. It would be more accurate to say that I'm downright stealing the cmake-vcpkg integration scripts he shared with me. They are open-source and available at google-cloud-cpp/super. I've copied them nearly verbatim to my vcpkg_cmake_blog branch for ease of use and long term stability of the hyperlinks. Thanks Carlos!

The objective is the same: bootstrap a vcpkg-based cmake project. The mechanics are much more sophisticated and feel idiomatic cmake. Let's get started.

Cmake Project Structure

vcpkg_test
├── cmake
│   ├── AutoVcpkg.cmake
│   └── VcpkgBootstrap.cmake
├── CMakeLists.txt
├── include
│   └── driver.h
├── src
│   └── driver.cpp
└── test
    └── driver_test.cpp
There're two more files under the cmake directory. These are cmake scripts designed to download, install, configure vcpkg instances in both Linux and Windows. They also expose suitable cmake function for use to use in CMakeLists.txt. This integration is much nicer (but also complex).

The CMakeLists.txt looks as follows.
cmake_minimum_required (VERSION 3.12)
set(MY_PROJECT_DEPENDENCIES boost-core boost-optional)

# This section of cmake is using AutoVcpkg to download, install, and configure vcpkg.
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_LIST_DIR}/cmake")
include(AutoVcpkg)
vcpkg_install(${MY_PROJECT_DEPENDENCIES})
message(STATUS "CMAKE_TOOLCHAIN_FILE=${CMAKE_TOOLCHAIN_FILE}")

# The CMakeLists from this point on is the same as that of part 1. 
project (vcpkg_test CXX)
set(CMAKE_CXX_STANDARD 17)
find_package(Boost 1.67 REQUIRED)

add_executable(driver src/driver.cpp)
target_include_directories(driver PUBLIC ${Boost_INCLUDE_DIR} ${PROJECT_SOURCE_DIR}/include )

enable_testing()
include(CTest)
add_executable(driver_test ${PROJECT_SOURCE_DIR}/test/driver_test.cpp)
add_test(NAME driver COMMAND driver_test)

vcpkg_install

Here's the full code of AutoVcpkg.cmake. Here's the github branch vcpkg_cmake_blog_idiomatic.

We're including the files under cmake directory as "modules" and simply invoking them using vcpkg_install. The code is mostly self-explanatory. If you are new to cmake, you might have to stare at it for a while though.

The vcpkg-download is a separate cmake project. The CMakeLists.txt for this project is created while generating the build files for the driver project. I.e., It allows every project to bootstrap it's own vcpkg repository. This may or may not be desirable. For smaller project it might be an overkill. For large projects where controlling the exact library version separate from vcpkg repository HEAD is desirable, one might want a dedicated vcpkg instance. Here's the ExternalProject vcpkg_download.
cmake_minimum_required(VERSION 3.12)
project(vcpkg-download)

include(ExternalProject)
ExternalProject_Add(vcpkg
            GIT_REPOSITORY @AUTO_VCPKG_GIT_REPOSITORY@
            # GIT_TAG 52870c7595a63ade069ae51d5f4ee3a85fe4123f # TODO: Debug this 
            GIT_SHALLOW ON
            SOURCE_DIR @AUTO_VCPKG_ROOT@
            PATCH_COMMAND ""
            CONFIGURE_COMMAND  ""
            BUILD_COMMAND ""
            INSTALL_COMMAND ""
            LOG_DOWNLOAD ON
            LOG_CONFIGURE ON
            LOG_INSTALL ON)
So instead of simply forking off and launching git clone directly from cmake, this external project allows a plethora of options and configure the download step.

The vcpkg_download function spits out and runs this project (with another invocation of cmake) only if needed. I ended up passing additional flags to cmake on Windows. Having to pass additional flags like CMAKE_EXE_LINKER_FLAGS, CMAKE_C_COMPILER, and CMAKE_CXX_COMPILER (from parent to the nested invocation of cmake) indicates that cmake integration with Visual Studio is still rough on the edges. Here's a snippet.
function (vcpkg_download)
    if (DEFINED AUTO_VCPKG_ROOT)
        return()
    endif ()
    set(AUTO_VCPKG_ROOT "${CMAKE_BINARY_DIR}/vcpkg")
    # Generate the vcpkg_download project if necessary.
    file(WRITE "${CMAKE_BINARY_DIR}/vcpkg-download/CMakeLists.txt" "${vcpkg_download_contents}")
    if(WIN32)
      get_filename_component(VC_COMPILER_PATH ${CMAKE_C_COMPILER} DIRECTORY)
      set(VCRT_LIB_PATH "${VC_COMPILER_PATH}/../../../lib/x86")
      execute_process(COMMAND "${CMAKE_COMMAND}"
              "-H${CMAKE_BINARY_DIR}/vcpkg-download"
              "-B${CMAKE_BINARY_DIR}/vcpkg-download"
              "-DCMAKE_C_COMPILER:FILEPATH=${CMAKE_C_COMPILER}"
              "-DCMAKE_CXX_COMPILER:FILEPATH=${CMAKE_CXX_COMPILER}"
              "-DCMAKE_EXE_LINKER_FLAGS=/LIBPATH:\"${VCRT_LIB_PATH}\"")
      execute_process(COMMAND "${CMAKE_COMMAND}"
              "--build" "${CMAKE_BINARY_DIR}/vcpkg-download")
    else()
      # Linux here.
    endif()
If the previous step does not succeed in building vcpkg successfully (i.e., if AUTO_VCPKG_EXECUTABLE is undefined), there's plan B. The plan B is to do pretty much fork off a child cmake process and run vcpkg bootstrap.sh or bootstrap.bat directly. We saw a very simple version of it in part #1.
function (vcpkg_bootstrap)
    find_program(AUTO_VCPKG_EXECUTABLE vcpkg PATHS ${AUTO_VCPKG_ROOT})
    if (NOT AUTO_VCPKG_EXECUTABLE)
        execute_process(COMMAND ${CMAKE_COMMAND} -E copy "${CMAKE_CURRENT_LIST_DIR}/cmake/VcpkgBootstrap.cmake" "${AUTO_VCPKG_ROOT}")
        execute_process(COMMAND ${CMAKE_COMMAND} -P "${AUTO_VCPKG_ROOT}/VcpkgBootstrap.cmake"
                WORKING_DIRECTORY ${AUTO_VCPKG_ROOT})
    endif ()
endfunction ()

###### VcpkgBootstrap.cmake file
find_program(VCPKG_EXECUTABLE
        vcpkg PATHS "${CMAKE_CURRENT_LIST_DIR}")
if (NOT VCPKG_EXECUTABLE)
    if (WIN32)
        execute_process(COMMAND "${CMAKE_CURRENT_LIST_DIR}/bootstrap-vcpkg.bat"
                WORKING_DIRECTORY "${CMAKE_CURRENT_LIST_DIR}")
    else ()
        execute_process(COMMAND  "${CMAKE_CURRENT_LIST_DIR}/bootstrap-vcpkg.sh"
                WORKING_DIRECTORY "${CMAKE_CURRENT_LIST_DIR}")
    endif ()
endif ()
At this point we've covered the gist. There are a lot of new things I learned about cmake.

The main differences between part #1 and this cmake project are the following.
  1. vcpkg is cloned from the github repository, compiled, and bootstrapped in the cmake binary directory. The directory you use for out-of-source builds (e.g., build). Previously, vcpkg is cloned, compiled, and bootstrapped in $ENV{HOMEDRIVE}$ENV{HOMEPATH}/vcpkg_cpptruths
  2. The vcpkg-download project is a real cmake project that generates a Makefile for bootstrapping vcpkg. On Windows, it generates a solution file under $ENV{HOMEDRIVE}$ENV{HOMEPATH}\CMakeBuilds\...\build\x86-Debug\vcpkg-download. Things are really meta at this point. cmake ExternalProject is used for that. Some tweaks in execute_process were necessary to pass the right ${CMAKE_EXE_LINKER_FLAGS}to build vcpkg with Visual Studio.
The projects seems to contain some meta targets that are unrelated to the main "driver" project. Here's how it looks like.

Observations

There are a couple of things vcpkg.cmake could make the experience better.
  1. GIT_TAG ... simply did not work for me in ExternalProject_Add. Cloning a specific tag/branch/commit hash of vcpkg is important for reproducible builds. Btw, Why aren't there any official releases of vcpkg? There's not a single tag as of this writing.
  2. The technique is this post is lower level but feels much more well-integrated. However, the end effect is the same. Not sure if it's worth the increased complexity. Especially because I had to overcome vcpkg build error "LINK : fatal error LNK1104: cannot open file 'MSVCRTD.lib'" that did not happen in part #1. The resulting Visual Studio project has some cruft too.

Comments

Popular Content

Multi-dimensional arrays in C++11

What new can be said about multi-dimensional arrays in C++? As it turns out, quite a bit! With the advent of C++11, we get new standard library class std::array. We also get new language features, such as template aliases and variadic templates. So I'll talk about interesting ways in which they come together.

It all started with a simple question of how to define a multi-dimensional std::array. It is a great example of deceptively simple things. Are the following the two arrays identical except that one is native and the other one is std::array?

int native[3][4];
std::array<std::array<int, 3>, 4> arr;

No! They are not. In fact, arr is more like an int[4][3]. Note the difference in the array subscripts. The native array is an array of 3 elements where every element is itself an array of 4 integers. 3 rows and 4 columns. If you want a std::array with the same layout, what you really need is:

std::array<std::array<int, 4>, 3> arr;

That's quite annoying for two r…

Covariance and Contravariance in C++ Standard Library

Covariance and Contravariance are concepts that come up often as you go deeper into generic programming. While designing a language that supports parametric polymorphism (e.g., templates in C++, generics in Java, C#), the language designer has a choice between Invariance, Covariance, and Contravariance when dealing with generic types. C++'s choice is "invariance". Let's look at an example.
struct Vehicle {}; struct Car : Vehicle {}; std::vector<Vehicle *> vehicles; std::vector<Car *> cars; vehicles = cars; // Does not compile The above program does not compile because C++ templates are invariant. Of course, each time a C++ template is instantiated, the compiler creates a brand new type that uniquely represents that instantiation. Any other type to the same template creates another unique type that has nothing to do with the earlier one. Any two unrelated user-defined types in C++ can't be assigned to each-other by default. You have to provide a c…

Inheritance vs std::variant

C++17 added std::variant and std::visit in its repertoire. They are worth a close examination. I've been wondering about whether they are always better than inheritance for modeling sum-types (fancy name for discriminated unions) and if not, under what circumstances they are not. We'll compare the two approaches in this blog post. So here it goes.

Inheritancestd::variantNeed not know all the derived types upfront (open-world assumption)Must know all the cases upfront (closed-world assumption)Dynamic Allocation (usually)No dynamic allocationIntrusive (must inherit from the base class)Non-intrusive (third-party classes can participate)Reference semantics (think how you copy a vector of pointers to base class?)Value semantics (copying is trivial)Algorithm scattered into classesAlgorithm in one placeLanguage supported (Clear errors if pure-virtual is not implemented)Library supported (poor error messages)Creates a first-class abstractionIt’s just a containerKeeps fluent interfaces…