Autocmake

About Autocmake

Building libraries and executables from sources can be a complex task. Several solutions exist to this problem: GNU Makefiles is the traditional approach. Today, CMake is one of the trendier alternatives which can generate Makefiles starting from a file called CMakeLists.txt.

Autocmake composes CMake building blocks into a CMake project and generates CMakeLists.txt as well as a setup script, which serves as a front-end to CMakeLists.txt. All this is done based on a lightweight autocmake.yml file:

python update.py --self
     |                                   |
     | fetches Autocmake                 |
     | infrastructure                    |
     | and updates the update.py script  |
     |                                   |
     v                           Developer maintaining
autocmake.yml                        Autocmake
     |                                   |
     | python update.py ..               |
     |                                   |
     v                                   v
CMakeLists.txt (and setup front-end)
     |                                   |
     | python setup or ./setup           |
     | which invokes CMake               |
     v                             User of the code
Makefile (or something else)             |
     |                                   |
     | make                              |
     |                                   |
     v                                   v
Build/install/test targets

Our main motivation to create Autocmake as a CMake framework library and CMake module composer is to simplify CMake code transfer between programs. We got tired of manually diffing and copy-pasting boiler-plate CMake code and watching it diverge while maintaining the CMake infrastructure in a growing number of scientific projects which typically have very similar requirements:

  • Fortran and/or C and/or C++ support
  • Tuning of compiler flags
  • Front-end script with good defaults
  • Support for parallelization: MPI, OMP, CUDA
  • Math libraries: BLAS, LAPACK

Our other motivation for Autocmake was to make it easier for developers who do not know CMake to provide a higher-level entry point to CMake.

Autocmake is a chance to provide a well documented and tested set of CMake plug-ins. With this we wish to give also users of codes the opportunity to introduce the occasional tweak without the need to dive deep into CMake documentation.

Requirements and dependencies

Autocmake update and test scripts require Python 2.7 or higher. We also support Python 3 (we automatically test with 2.7 and 3.5).

The generated setup script runs with Python >= 2.6 (also tested with Python 3.5).

To generate CMakeLists.txt and the setup script, Autocmake requires the pyyaml package.

Getting help

Autocmake discussion forum: http://groups.google.com/group/autocmake

FAQ for developers

Which files do I need to edit?

Let us start with files which you normally never edit: CMakeLists.txt and setup - these are generated based on autocmake.yml. Have a look in autocmake.yml and you will see all CMake files which are assembled into CMakeLists.txt. You can edit those files. If you change the order of files listed in autocmake.yml or if you add or remove CMake modules, then you need to rerun the update.py script to refresh CMakeLists.txt and setup.

Autocmake does not do feature X - I really need feature X and a setup flag –X

The Autocmake developers have to be very conservative and only a very limited set of portable features of absolutely general interest become part of the Autocmake core or an Autocmake module. Autocmake developers are also busy.

Our recommendation is to not wait for the feature to be implemented: Implement it yourself. Here we show you how. Code your feature in a module (i.e. my_feature.cmake) and place the module under cmake/custom/ (the directory name is just a suggestion, Autocmake does not enforce a directory naming):

cmake/custom/my_feature.cmake

And include this feature to the main CMakeLists.txt in autocmake.yml under the modules section:

modules:
- my_feature:
  - source:
    - custom/my_feature.cmake

Now your code is included in the main CMakeLists.txt. Perhaps you also want a setup script flag to toggle the feature:

- my_feature:
  - docopt: "--enable-my-feature Enable my feature [default: False]."
  - define: "'-DENABLE_MY_FEATURE={0}'.format(arguments['--enable-my-feature'])"
  - source:
    - custom/my_feature.cmake

Implement your ideas, test them, and share them. If your module is portable, good code quality, and of general interest, you can suggest it to be part of the standard set of modules or even a core feature.

How can I get a setup flag –X that toggles a CMake variable?

The following will add a --something flag which toggles the CMake variable ENABLE_SOMETHING:

- my_section:
  - docopt: "--something Enable something [default: False]."
  - define: "'-DENABLE_SOMETHING={0}'.format(arguments['--enable-something'])"

Can I change the name of the setup script?

Yes you can do that in autocmake.yml. Here we for instance change the name to “configure”:

name: myproject
min_cmake_version: 2.8
default_build_type: release
setup_script: configure
language:
  - Fortran
  - C
  - CXX

Can I prevent Autocmake from creating any front-end setup script?

Yes, with setup_script: None.

In CMake I can do feature X - can I do that also with Autocmake?

Yes. Autocmake is really just a simplistic script which helps to organize CMake code across projects. Everything that can be done in CMake can be realized in Autocmake.

Should I include and track also files generated by Autocmake in my repository?

Yes, you probably want to do that. Autocmake downloads and generates a number of files which in principle could be generated at configure- or build-time. However, you probably do not want the users of your code to run any Autocmake scripts like update.py to generate the files they need to build the project. The users of your code will run setup directly and typically expect everything to just work (TM). Note also that the users of your code will not need to install the pyyaml package.

The update.py script is overwriting my CMakeLists.txt and setup, isn’t this bad?

No, it is not as bad as it first looks. It is a feature. Normally CMakeLists.txt and setup should not contain any explicit customization and therefore should not contain anything that could not be regenerated. In any case you should use version control so that you can inspect and compare changes introduced to CMakeLists.txt and setup and possibly revert them. See also the next remark.

But I need to manually edit and customize CMakeLists.txt and setup every time I run update.py!?

You typically never need to manually edit and customize CMakeLists.txt and setup directly. You can introduce customizations in autocmake.yml which get assembled into the front-end scripts.

Where is a good place to list my sources and targets?

As mentioned above CMakeLists.txt is not a good place because this file is generated from autocmake.yml and your modifications would become overwritten at some point. A good standard is to organize your sources under src/ and to list your sources and targets in src/CMakeLists.txt. You can include the latter in autocmake.yml using:

- my_sources:
  - source:
    - https://github.com/dev-cafe/autocmake/raw/master/modules/src.cmake

If you really do not like to do it this way, you can describe your sources and targets in a custom module in a local file and include it like this:

- my_sources:
  - source:
    - custom/my_sources.cmake

How can I do some more sophisticated validation of setup flags?

Sometimes you need to do more sophisticated validation and post-processing of setup flags. This can be done by placing a module called extensions.py under cmake/ (or wherever you have autocmake.yml). This file should implement a function with the following signature:

def postprocess_args(sys_argv, arguments):
    # sys_argv is the sys.argv from the setup script
    # arguments is the dictionary of arguments returned by docopt

    # do something here ...

    return arguments

In this function you can do any validation and post-processing you like. This function is run after the flags are parsed and before the CMake command is run.

Example for a validation of MPI flags to the setup script:

import sys

def contains_flag(sys_argv, flag):
    return (any(x for x in sys_argv if x.startswith('--{0}='.format(flag))))

def postprocess_args(sys_argv, arguments):

    # if --mpi is selected and compilers are not selected
    # then compilers default to mpif90, mpicc, and mpicxx
    if arguments['--mpi']:
        if not contains_flag(sys_argv, 'fc') and not contains_flag(sys_argv, 'cc') and not contains_flag(sys_argv, 'cxx'):
           arguments['--fc'] = 'mpif90'
           arguments['--cc'] = 'mpicc'
           arguments['--cxx'] = 'mpicxx'

    # if one of the compilers contains "mpi" and --mpi is not selected, it is probably a user error
    # in this case stop the configuration
    asking_for_mpi_compiler = False
    for flag in ['fc', 'cc', 'cxx']:
        if contains_flag(sys_argv, 'fc'):
            if 'mpi' in arguments['--fc']:
                asking_for_mpi_compiler = True
    if asking_for_mpi_compiler and not arguments['--mpi']:
        sys.stderr.write('ERROR: you ask for an MPI compiler but have not specified --mpi\n')
        sys.exit(1)

    return arguments

Bootstrapping a new project

Bootstrapping Autocmake

Download the update.py and execute it with --self to fetch other infrastructure files which will be needed to build the project:

$ mkdir cmake  # does not have to be called "cmake" - take the name you prefer
$ cd cmake
$ wget https://github.com/dev-cafe/autocmake/raw/master/update.py
$ virtualenv venv
$ source venv/bin/activate
$ pip install pyyaml
$ python update.py --self

On the MS Windows system, you can use the PowerShell wget-replacement:

$ Invoke-WebRequest https://github.com/dev-cafe/autocmake/raw/master/update.py -OutFile update.py

This creates (or updates) the following files (an existing autocmake.yml is not overwritten by the script):

cmake/
    autocmake.yml      # edit this file
    update.py          # no need to edit
    autocmake/         # no need to edit
        ...            # no need to edit

Note that update.py and files under autocmake/ are overwritten (use version control).

Generating the CMake infrastructure

Now customize autocmake.yml to your needs (see Configuring autocmake.yml) and then run the update.py script which creates CMakeLists.txt and a setup script in the target path:

$ python update.py ..

The script also downloads external CMake modules specified in autocmake.yml to a directory called downloaded/:

cmake/
    autocmake.yml      # edit this file
    update.py          # no need to edit
    autocmake/         # no need to edit
        ...            # no need to edit
    downloaded/        # contains CMake modules fetched from the web

Building the project

Now you have CMakeLists.txt and setup script in the project root and the project can be built:

$ cd ..
$ python setup [-h]
$ cd build
$ make

Configuring autocmake.yml

The script autocmake.yml is the high level place where you configure your project. Here is an example. We will discuss it in detail further below:

name: numgrid

min_cmake_version: 2.8

default_build_type: release

language:
  - Fortran
  - C
  - CXX

url_root: https://github.com/dev-cafe/autocmake/raw/master/

modules:
- compilers:
  - source:
    - '%(url_root)modules/fc.cmake'
    - '%(url_root)modules/cc.cmake'
    - '%(url_root)modules/cxx.cmake'
- flags:
  - source:
    - '%(url_root)compilers/GNU.CXX.cmake'
    - '%(url_root)compilers/Intel.CXX.cmake'
    - 'compilers/Clang.CXX.cmake'
- plugins:
  - source:
    - '%(url_root)modules/ccache.cmake'
    - 'custom/rpath.cmake'
    - '%(url_root)modules/definitions.cmake'
    - '%(url_root)modules/code_coverage.cmake'
    - '%(url_root)modules/safeguards.cmake'
    - '%(url_root)modules/default_build_paths.cmake'
    - '%(url_root)modules/src.cmake'
    - '%(url_root)modules/googletest.cmake'
    - 'custom/api.cmake'
    - 'custom/test.cmake'

Name and order of sections

First we define the project name (here “numgrid”). This section has to be there and it has to be called “project” (but it does not have to be on top).

We also have to define min_cmake_version as well as set the supported language(s).

The definition url_root is an interpolation (see Interpolation) and we use it to avoid retyping the same line over and over and to be able to change it in one place. The explicit name “url_root” has no special meaning to Autocmake and we could have chosen a different name.

The section modules is a list of CMake plugins. The names of the list elements (here “compilers”, “flags”, and “plugins”) does not matter to Autocmake. We could have called them “one”, “two”, and “whatever”, but it would not make much sense. It is better to choose names that are meaningful to you and readers of your code.

The order of the elements under modules does matter and the list will be processed in the exact order as you specify them in autocmake.yml.

Minimal example

As a minimal example we take an autocmake.yml which only contains:

name: minime
min_cmake_version: 2.8
language: C

If you don’t have the update.py script yet, you need to fetch it from the web:

$ wget https://github.com/dev-cafe/autocmake/raw/master/update.py

First we make sure that the update.py script is up-to-date and that it has access to all libraries it needs:

$ python update.py --self

- creating .gitignore
- fetching autocmake/configure.py
- fetching autocmake/__init__.py
- fetching autocmake/external/docopt.py
- fetching autocmake/external/__init__.py
- fetching autocmake/generate.py
- fetching autocmake/extract.py
- fetching autocmake/interpolate.py
- fetching autocmake/parse_rst.py
- fetching autocmake/parse_yaml.py
- fetching update.py

Good. Now we can generate CMakeLists.txt and the setup script:

$ python update.py ..

- parsing autocmake.yml
- generating CMakeLists.txt
- generating setup script

Excellent. Here is the generated CMakeLists.txt:

# set minimum cmake version
cmake_minimum_required(VERSION 2.8 FATAL_ERROR)

# project name
project(minime C)

# do not rebuild if rules (compiler flags) change
set(CMAKE_SKIP_RULE_DEPENDENCY TRUE)

# if CMAKE_BUILD_TYPE undefined, we set it to Debug
if(NOT CMAKE_BUILD_TYPE)
    set(CMAKE_BUILD_TYPE "Debug")
endif()

set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${PROJECT_SOURCE_DIR}/cmake/downloaded)

This is the very bare minimum. Every Autocmake project will have at least these settings.

And we also got a setup script (front-end to CMakeLists.txt) with the following default options:

Usage:
  ./setup [options] [<builddir>]
  ./setup (-h | --help)

Options:
  --type=<TYPE>                          Set the CMake build type (debug, release, or relwithdeb) [default: release].
  --generator=<STRING>                   Set the CMake build system generator [default: Unix Makefiles].
  --show                                 Show CMake command and exit.
  --cmake-executable=<CMAKE_EXECUTABLE>  Set the CMake executable [default: cmake].
  --cmake-options=<STRING>               Define options to CMake [default: ''].
  --prefix=<PATH>                        Set the install path for make install.
  <builddir>                             Build directory.
  -h --help                              Show this screen.

That’s not too bad although currently we cannot do much with this since there are no sources listed, no targets, hence nothing to build. We need to flesh out CMakeLists.txt by extending autocmake.yml and this is what we will do in the next section.

Assembling CMake plugins

The preferred way to extend CMakeLists.txt is by editing autocmake.yml and using the source option:

- compilers:
  - source:
    - '%(url_root)modules/fc.cmake'
    - '%(url_root)modules/cc.cmake'
    - '%(url_root)modules/cxx.cmake'

This will download fc.cmake, cc.cmake, and cxx.cmake, and include them in CMakeLists.txt, in this order.

You can also include local CMake modules, e.g.:

- source:
  - 'custom/rpath.cmake'

It is also OK to include several modules at once as we have seen above. The modules will be included in the same order as they appear in autocmake.yml.

Fetching files without including them in CMakeLists.txt

Sometimes you want to fetch a file without including it in CMakeLists.txt. This can be done with the fetch option. This is for instance done by the git_info.cmake module (see https://github.com/dev-cafe/autocmake/blob/master/modules/git_info/git_info.cmake#L10-L13).

If fetch is invoked in autocmake.yml, then the fetched file is placed under downloaded/. If fetch is invoked from within a CMake module documentation (see below), then the fetched file is placed into the same directory as the CMake module file which fetches it.

Generating setup options

Options for the setup script can be generated with the docopt option. As an example, the following autocmake.yml snippet will add a --something flag:

- my_section:
  - docopt: "--something Enable something [default: False]."

Setting CMake options

Configure-time CMake options can be generated with the define option. Consider the following example which toggles the CMake variable ENABLE_SOMETHING:

- my_section:
  - docopt: "--something Enable something [default: False]."
  - define: "'-DENABLE_SOMETHING={0}'.format(arguments['--enable-something'])"

Setting environment variables

You can export environment variables at configure-time using the export option. Consider the following example:

docopt:
  - "--cc=<CC> C compiler [default: gcc]."
  - "--extra-cc-flags=<EXTRA_CFLAGS> Extra C compiler flags [default: '']."
export: "'CC={0}'.format(arguments['--cc'])"
define: "'-DEXTRA_CFLAGS=\"{0}\"'.format(arguments['--extra-cc-flags'])"

Auto-generating configurations from the documentation

To avoid a boring re-typing of boilerplate autocmake.yml code it is possible to auto-generate configurations from the documentation. This is the case for many core modules which come with own options once you have sourced them.

The lines following # autocmake.yml configuration:: are understood by the update.py script to infer autocmake.yml code from the documentation. As an example consider https://github.com/dev-cafe/autocmake/blob/master/modules/cc.cmake#L20-L26. Here, update.py will infer the configurations for docopt, export, and define.

Example hello world project

This is a brief example for the busy and impatient programmer. For a longer tour please see Configuring autocmake.yml.

We start with a mixed Fortran-C project with the following sources:

feature1.F90
feature2.c
main.F90

First thing we do is to create a directory src/ and we move all sources there. This is not necessary for Autocmake but it is a generally good practice:

.
`-- src
    |-- feature1.F90
    |-- feature2.c
    `-- main.F90

Now we create cmake/ and fetch update.py:

$ mkdir cmake  # does not have to be called "cmake" - take the name you prefer
$ cd cmake
$ wget https://github.com/dev-cafe/autocmake/raw/master/update.py
$ python update.py --self

Now from top-level our file tree looks like this:

.
|-- cmake
|   |-- autocmake
|   |   |-- __init__.py
|   |   |-- configure.py
|   |   |-- external
|   |   |   |-- __init__.py
|   |   |   `-- docopt.py
|   |   |-- extract.py
|   |   |-- generate.py
|   |   |-- interpolate.py
|   |   |-- parse_rst.py
|   |   `-- parse_yaml.py
|   |-- autocmake.yml
|   `-- update.py
`-- src
    |-- feature1.F90
    |-- feature2.c
    `-- main.F90

Now we edit cmake/autocmake.yml to look like this:

name: hello

min_cmake_version: 2.8

language:
  - Fortran
  - C

url_root: https://github.com/dev-cafe/autocmake/raw/master/

modules:
- compilers:
  - source:
    - '%(url_root)modules/fc.cmake'
    - '%(url_root)modules/cc.cmake'
- src_support:
  - source:
    - '%(url_root)modules/src.cmake'

What we have specified here is the project name and that we wish Fortran and C support. The src.cmake module tells CMake to include a src/CMakeLists.txt. We need to create src/CMakeLists.txt which can look like this:

add_executable(
    hello.x
    main.F90
    feature1.F90
    feature2.c
    )

We wrote that we want to get an executable “hello.x” built from our sources.

Now we have everything to generate CMakeLists.txt and a setup script:

$ cd cmake
$ python update ..

- parsing autocmake.yml
- assembling modules: [##############################] (3/3)
- generating CMakeLists.txt
- generating setup script

And this is what we got:

.
|-- CMakeLists.txt
|-- cmake
|   |-- autocmake
|   |   |-- __init__.py
|   |   |-- configure.py
|   |   |-- external
|   |   |   |-- __init__.py
|   |   |   `-- docopt.py
|   |   |-- extract.py
|   |   |-- generate.py
|   |   |-- interpolate.py
|   |   |-- parse_rst.py
|   |   `-- parse_yaml.py
|   |-- autocmake.yml
|   |-- downloaded
|   |   |-- autocmake_cc.cmake
|   |   |-- autocmake_fc.cmake
|   |   `-- autocmake_src.cmake
|   `-- update.py
|-- setup
`-- src
    |-- CMakeLists.txt
    |-- feature1.F90
    |-- feature2.c
    `-- main.F90

Now we are ready to build:

$ ./setup --fc=gfortran --cc=gcc

FC=gfortran CC=gcc cmake -DEXTRA_FCFLAGS="''" -DEXTRA_CFLAGS="''" -DCMAKE_BUILD_TYPE=release -G "Unix Makefiles" /home/user/hello

-- The C compiler identification is GNU 6.1.1
-- The CXX compiler identification is GNU 6.1.1
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- The Fortran compiler identification is GNU 6.1.1
-- Check for working Fortran compiler: /usr/bin/gfortran
-- Check for working Fortran compiler: /usr/bin/gfortran  -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether /usr/bin/gfortran supports Fortran 90
-- Checking whether /usr/bin/gfortran supports Fortran 90 -- yes
-- Configuring done
-- Generating done
-- Build files have been written to: /home/user/hello/build

   configure step is done
   now you need to compile the sources:
   $ cd build
   $ make

$ cd build
$ make

Scanning dependencies of target hello.x
[ 25%] Building Fortran object src/CMakeFiles/hello.x.dir/main.F90.o
[ 50%] Building Fortran object src/CMakeFiles/hello.x.dir/feature1.F90.o
[ 75%] Building C object src/CMakeFiles/hello.x.dir/feature2.c.o
[100%] Linking Fortran executable hello.x
[100%] Built target hello.x

Excellent! But this was a lot of typing and file creating just to get a simple executable compiled!? Of course, all that could have been done with few command lines directly but now we have a cross-platform project and can extend it and customize it and we also got a front-end script and command-line parser for free. Now go out and explore more Autocmake modules and features.

Customizing CMake modules

The update.py script assembles modules listed in autocmake.yml into CMakeLists.txt. Those that are fetched from the web are placed inside downloaded/. You have several options to customize downloaded CMake modules:

Directly inside the generated directory

The CMake modules can be customized directly inside downloaded/ but this is the least elegant solution since the customizations may be overwritten by the update.py script (use version control).

Adapt local copies of CMake modules

A slightly better solution is to download the CMake modules that you wish you customize to a separate directory (e.g. custom/) and source the customized CMake modules in autocmake.yml. Alternatively you can serve your custom modules from your own http server.

Fork and branch the CMake modules

You can fork and branch the mainline Autocmake development and include the branched customized versions. This will make it easier for you to stay up-to-date with upstream development.

Overriding settings

If you source a module which contains directives such as define, docopt, export, or fetch, and you wish to modify those, then you can override these settings in autocmake.yml. Settings in autocmake.yml take precedence over settings imported by a sourced module.

As an example consider the Boost module which defines and uses interpolation variables major, minor, patch, and components, see https://github.com/dev-cafe/autocmake/blob/master/modules/boost/boost.cmake#L52-L55.

The recommended way to customize these is in autocmake.yml, e.g.: https://github.com/dev-cafe/autocmake/blob/master/test/boost_libs/cmake/autocmake.yml#L12-L17.

Create own CMake modules

Of course you can also create own modules and source them in autocmake.yml.

Contribute customizations to the “standard library”

If you think that your customization will be useful for other users as well, you may consider contributing the changes directly to https://github.com/dev-cafe/autocmake/. We very much encourage such contributions. But we also strive for generality and portability.

Updating CMake modules

To update CMake modules fetched from the web you need to run the update.py script:

$ cd cmake
$ python update.py ..

The CMake modules are not fetched or updated at configure time or build time. In other words, if you never re-run update.py script and never modify the CMake module files, then the CMake modules will remain forever frozen.

How to pin CMake modules to a certain version

Sometimes you may want to avoid using the latest version of a CMake module and rather fetch an older version, for example with the hash abcd123. To achieve this, instead of:

- my_feature:
  - source: https://github.com/dev-cafe/autocmake/raw/master/modules/foo.cmake

pin the version to abcd123 (you do not need to specify the full Git hash, a unique beginning will do):

- my_feature:
  - source: https://github.com/dev-cafe/autocmake/raw/abcd123/modules/foo.cmake

Interpolation

In a custom extension to the YAML specification you can define and reuse variables like this (observe how we interpolate url_root, major, minor, patch, and components in this example):

url_root: https://github.com/dev-cafe/autocmake/raw/master/
major: 1
minor: 48
patch: 0
components: ""
fetch:
  - "%(url_root)modules/boost/boost_unpack.cmake"
  - "%(url_root)modules/boost/boost_userconfig.cmake"
  - "%(url_root)modules/boost/boost_configure.cmake"
  - "%(url_root)modules/boost/boost_build.cmake"
  - "%(url_root)modules/boost/boost_install.cmake"
  - "%(url_root)modules/boost/boost_headers.cmake"
  - "%(url_root)modules/boost/boost_cleanup.cmake"
  - "http://sourceforge.net/projects/boost/files/boost/%(major).%(minor).%(patch)/boost_%(major)_%(minor)_%(patch).zip"
docopt:
  - "--boost-headers=<BOOST_INCLUDEDIR> Include directories for Boost [default: '']."
  - "--boost-libraries=<BOOST_LIBRARYDIR> Library directories for Boost [default: '']."
  - "--build-boost=<FORCE_CUSTOM_BOOST> Deactivate Boost detection and build on-the-fly <ON/OFF> [default: OFF]."
define:
  - "'-DBOOST_INCLUDEDIR=\"{0}\"'.format(arguments['--boost-headers'])"
  - "'-DBOOST_LIBRARYDIR=\"{0}\"'.format(arguments['--boost-libraries'])"
  - "'-DFORCE_CUSTOM_BOOST={0}'.format(arguments['--build-boost'])"
  - "'-DBOOST_MINIMUM_REQUIRED=\"%(major).%(minor).%(patch)\"'"
  - "'-DBOOST_COMPONENTS_REQUIRED=\"%(components)\"'"

FAQ for users

TL;DR How do I compile the code?

$ python setup [-h]
$ cd build
$ make

How can I specify the compiler?

By default the setup script will attempt GNU compilers. You can specify compilers manually like this:

$ python setup --fc=ifort --cc=icc --cxx=icpc

How can I add compiler flags?

You can do this with --extra-fc-flags, --extra-cc-flags, or --extra-cxx-flags (depending on the set of enabled languages):

$ python setup --fc=gfortran --extra-fc-flags='-some-exotic-flag'

How can I redefine compiler flags?

If you export compiler flags using the environment variables FCFLAGS, CFLAGS, or CXXFLAGS, respectively, then the configuration will use those flags and neither augment them, nor redefine them. Setting these environment variables you have full control over the flags without editing CMake files.

How can I select CMake options via the setup script?

Like this:

$ python setup --cmake-options='"-DTHIS_OPTION=ON -DTHAT_OPTION=OFF"'

We use two sets of quotes because the shell swallows one set of them before passing the arguments to Python. Yeah that’s not nice, but nothing we can do about it on the Python side. If you do not use two sets of quotes then the setup command may end up incorrectly saved in build/setup_command.

Contributing guidelines

Please follow the excellent guide: http://www.contribution-guide.org.

We do not require any formal copyright assignment or contributor license agreement. Any contributions intentionally sent upstream are presumed to be offered under terms of the OSI-approved BSD 3-clause license.

Testing Autocmake

You will need to install pytest.

Check also the Travis build and test recipe for other requirements.

Your contributions and changes should preserve the test set and be PEP8 conform. You can run locally all tests with:

$ pycodestyle --ignore E501 update.py
$ pycodestyle --ignore E501,E265 autocmake
$ py.test -vv autocmake/*
$ py.test -vv test/test.py

You can also select individual tests, for example those with fc_blas string in the name:

$ py.test -k fc_blas test/test.py

For more options, see the py.test flags.

This test set is run upon each push to the central repository. See also the Travis build and test history.

Contributing to the documentation

Contributions and patches to the documentation are most welcome.

This documentation is refreshed upon each push to the central repository.

The module reference documentation is generated from the module sources using #.rst: tags (compare for instance http://autocmake.readthedocs.io/en/latest/module-reference.html#cc-cmake with https://github.com/dev-cafe/autocmake/blob/master/modules/cc.cmake).

Please note that the lines following # autocmake.yml configuration:: are understood by the update.py script to infer autocmake.yml code from the documentation. As an example consider https://github.com/dev-cafe/autocmake/blob/master/modules/cc.cmake#L20-L26. Here, update.py will infer the configurations for docopt, export, and define.

Module reference

boost.cmake

[Source code]

Detect, build, and link Boost libraries. This modules downloads the .zip archive from SourceForge at Autocmake update time. Note that the build-up commands are not Windows-compatible!

Your autocmake.yml should look like this:

- boost:

Cross-dependencies between required components are not checked for. For example, Boost.Timer depends on Boost.Chrono and Boost.System thus you should ask explicitly for all three. If the self-build of Boost components is triggered the BUILD_CUSTOM_BOOST variable is set to TRUE. The CMake target custom_boost is also added. You should use these two to ensure the right dependencies between your targets and the Boost headers/libraries, in case the self-build is triggered. For example:

if(BUILD_CUSTOM_BOOST)
  add_dependencies(your_target custom_boost)
endif()

will ensure that your_target is built after custom_boost if and only if the self-build of Boost took place. This is an important step to avoid race conditions when building on multiple processes.

Dependencies:

mpi                        - Only if the Boost.MPI library is a needed component
python_libs                - Only if the Boost.Python library is a needed component

Variables used:

BOOST_MINIMUM_REQUIRED     - Minimum required version of Boost
BOOST_COMPONENTS_REQUIRED  - Components (compiled Boost libraries) required
PROJECT_SOURCE_DIR
PROJECT_BINARY_DIR
CMAKE_BUILD_TYPE
MPI_FOUND
BUILD_CUSTOM_BOOST

autocmake.yml configuration:

url_root: https://github.com/dev-cafe/autocmake/raw/master/
major: 1
minor: 48
patch: 0
components: ""
fetch:
  - "%(url_root)modules/boost/boost_unpack.cmake"
  - "%(url_root)modules/boost/boost_userconfig.cmake"
  - "%(url_root)modules/boost/boost_configure.cmake"
  - "%(url_root)modules/boost/boost_build.cmake"
  - "%(url_root)modules/boost/boost_install.cmake"
  - "%(url_root)modules/boost/boost_headers.cmake"
  - "%(url_root)modules/boost/boost_cleanup.cmake"
  - "http://sourceforge.net/projects/boost/files/boost/%(major).%(minor).%(patch)/boost_%(major)_%(minor)_%(patch).zip"
docopt:
  - "--boost-headers=<BOOST_INCLUDEDIR> Include directories for Boost [default: '']."
  - "--boost-libraries=<BOOST_LIBRARYDIR> Library directories for Boost [default: '']."
  - "--build-boost=<FORCE_CUSTOM_BOOST> Deactivate Boost detection and build on-the-fly <ON/OFF> [default: OFF]."
define:
  - "'-DBOOST_INCLUDEDIR=\"{0}\"'.format(arguments['--boost-headers'])"
  - "'-DBOOST_LIBRARYDIR=\"{0}\"'.format(arguments['--boost-libraries'])"
  - "'-DFORCE_CUSTOM_BOOST={0}'.format(arguments['--build-boost'])"
  - "'-DBOOST_MINIMUM_REQUIRED=\"%(major).%(minor).%(patch)\"'"
  - "'-DBOOST_COMPONENTS_REQUIRED=\"%(components)\"'"

cc.cmake

[Source code]

Adds C support. Appends EXTRA_CFLAGS to CMAKE_C_FLAGS. If environment variable CFLAGS is set, then the CFLAGS are used and no other flags are used or appended.

Variables used:

EXTRA_CFLAGS

Variables modified:

CMAKE_C_FLAGS

Environment variables used:

CFLAGS

autocmake.yml configuration:

docopt:
  - "--cc=<CC> C compiler [default: gcc]."
  - "--extra-cc-flags=<EXTRA_CFLAGS> Extra C compiler flags [default: '']."
define: "'-DCMAKE_C_COMPILER={0} -DEXTRA_CFLAGS=\"{1}\"'.format(arguments['--cc'], arguments['--extra-cc-flags'])"

ccache.cmake

[Source code]

Adds ccache support. The user should export the appropriate environment variables to tweak the program’s behaviour, as described in its manpage. Notice that some additional compiler flags might be needed in order to avoid unnecessary warnings.

Variables defined:

CCACHE_FOUND

autocmake.yml configuration:

docopt: "--ccache=<USE_CCACHE> Toggle use of ccache <ON/OFF> [default: ON]."
define: "'-DUSE_CCACHE={0}'.format(arguments['--ccache'])"

code_coverage.cmake

[Source code]

Enables code coverage by appending corresponding compiler flags.

Variables modified (provided the corresponding language is enabled):

CMAKE_Fortran_FLAGS
CMAKE_C_FLAGS
CMAKE_CXX_FLAGS

autocmake.yml configuration:

docopt: "--coverage Enable code coverage [default: OFF]."
define: "'-DENABLE_CODE_COVERAGE={0}'.format(arguments['--coverage'])"

custom_color_messages.cmake

[Source code]

Colorize CMake output. Code was found on StackOverflow: http://stackoverflow.com/a/19578320

Usage within CMake code:
message(“This is normal”) message(“${Red}This is Red${ColourReset}”) message(“${Green}This is Green${ColourReset}”) message(“${Yellow}This is Yellow${ColourReset}”) message(“${Blue}This is Blue${ColourReset}”) message(“${Magenta}This is Magenta${ColourReset}”) message(“${Cyan}This is Cyan${ColourReset}”) message(“${White}This is White${ColourReset}”) message(“${BoldRed}This is BoldRed${ColourReset}”) message(“${BoldGreen}This is BoldGreen${ColourReset}”) message(“${BoldYellow}This is BoldYellow${ColourReset}”) message(“${BoldBlue}This is BoldBlue${ColourReset}”) message(“${BoldMagenta}This is BoldMagenta${ColourReset}”) message(“${BoldCyan}This is BoldCyan${ColourReset}”) message(“${BoldWhite}This is BoldWhitenn${ColourReset}”)

Has no effect on WIN32.

cxx.cmake

[Source code]

Adds C++ support. Appends EXTRA_CXXFLAGS to CMAKE_CXX_FLAGS. If environment variable CXXFLAGS is set, then the CXXFLAGS are used and no other flags are used or appended.

Variables used:

EXTRA_CXXFLAGS

Variables modified:

CMAKE_CXX_FLAGS

Environment variables used:

CXXFLAGS

autocmake.yml configuration:

docopt:
  - "--cxx=<CXX> C++ compiler [default: g++]."
  - "--extra-cxx-flags=<EXTRA_CXXFLAGS> Extra C++ compiler flags [default: '']."
define: "'-DCMAKE_CXX_COMPILER={0} -DEXTRA_CXXFLAGS=\"{1}\"'.format(arguments['--cxx'], arguments['--extra-cxx-flags'])"

default_build_paths.cmake

[Source code]

Sets binary and library output directories to ${CMAKE_BINARY_DIR}/${CMAKE_INSTALL_BINDIR} and ${CMAKE_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR}, respectively.

Variables modified:

CMAKE_ARCHIVE_OUTPUT_DIRECTORY
CMAKE_LIBRARY_OUTPUT_DIRECTORY
CMAKE_RUNTIME_OUTPUT_DIRECTORY

definitions.cmake

[Source code]

Add preprocessor definitions (example: –add-definitions=”-DTHIS -DTHAT=137”). These are passed all the way down to the compiler.

Variables used:

PREPROCESSOR_DEFINITIONS

autocmake.yml configuration:

docopt: "--add-definitions=<STRING> Add preprocesor definitions [default: '']."
define: "'-DPREPROCESSOR_DEFINITIONS=\"{0}\"'.format(arguments['--add-definitions'])"

export_header.cmake

[Source code]

Generates export header for your API using best practices. This module wraps https://cmake.org/cmake/help/v3.0/module/GenerateExportHeader.html and it is needed because of these, Very Good Indeed, reasons: https://gcc.gnu.org/wiki/Visibility. If you are not afraid of jargon, please also have a look at https://www.akkadia.org/drepper/dsohowto.pdf.

Variables used:

PROJECT_NAME (defined by project())

fc.cmake

[Source code]

Adds Fortran support. Appends EXTRA_FCFLAGS to CMAKE_Fortran_FLAGS. If environment variable FCFLAGS is set, then the FCFLAGS are used and no other flags are used or appended.

Variables used:

EXTRA_FCFLAGS

Variables defined:

CMAKE_Fortran_MODULE_DIRECTORY

Variables modified:

CMAKE_Fortran_FLAGS

Environment variables used:

FCFLAGS

autocmake.yml configuration:

docopt:
  - "--fc=<FC> Fortran compiler [default: gfortran]."
  - "--extra-fc-flags=<EXTRA_FCFLAGS> Extra Fortran compiler flags [default: '']."
define: "'-DCMAKE_Fortran_COMPILER={0} -DEXTRA_FCFLAGS=\"{1}\"'.format(arguments['--fc'], arguments['--extra-fc-flags'])"

fc_optional.cmake

[Source code]

Adds optional Fortran support. Appends EXTRA_FCFLAGS to CMAKE_Fortran_FLAGS. If environment variable FCFLAGS is set, then the FCFLAGS are used and no other flags are used or appended.

Variables used:

EXTRA_FCFLAGS
ENABLE_FC_SUPPORT

Variables defined:

CMAKE_Fortran_MODULE_DIRECTORY

Variables modified:

CMAKE_Fortran_FLAGS

Variables defined:

ENABLE_FC_SUPPORT

Environment variables used:

FCFLAGS

autocmake.yml configuration:

docopt:
  - "--fc=<FC> Fortran compiler [default: gfortran]."
  - "--extra-fc-flags=<EXTRA_FCFLAGS> Extra Fortran compiler flags [default: '']."
  - "--fc-support=<FC_SUPPORT> Toggle Fortran language support (ON/OFF) [default: ON]."
define:
  - "'-DCMAKE_Fortran_COMPILER={0} -DEXTRA_FCFLAGS=\"{1}\"'.format(arguments['--fc'], arguments['--extra-fc-flags'])"
  - "'-DENABLE_FC_SUPPORT={0}'.format(arguments['--fc-support'])"

git_info.cmake

[Source code]

Creates git_info.h in the build directory. This file can be included in sources to print Git repository version and status information to the program output.

autocmake.yml configuration:

url_root: https://github.com/dev-cafe/autocmake/raw/master/
fetch:
  - "%(url_root)modules/git_info/git_info.h.in"

googletest.cmake

[Source code]

Includes Google Test sources and adds a library “googletest”.

Variables used:

GOOGLETEST_ROOT

autocmake.yml configuration:

define: "'-DGOOGLETEST_ROOT=external/googletest/googletest'"

int64.cmake

[Source code]

Enables 64-bit integer support for Fortran projects.

Variables modified (provided the corresponding language is enabled):

CMAKE_Fortran_FLAGS

autocmake.yml configuration:

docopt: "--int64 Enable 64bit integers [default: False]."
define: "'-DENABLE_64BIT_INTEGERS={0}'.format(arguments['--int64'])"

accelerate.cmake

[Source code]

Find and link to ACCELERATE.

Variables defined:

ACCELERATE_FOUND - describe me, uncached
ACCELERATE_LIBRARIES - describe me, uncached
ACCELERATE_INCLUDE_DIR - describe me, uncached

autocmake.yml configuration:

url_root: https://github.com/dev-cafe/autocmake/raw/master/
docopt: "--accelerate Find and link to ACCELERATE [default: False]."
define: "'-DENABLE_ACCELERATE={0}'.format(arguments['--accelerate'])"
fetch:
  - "%(url_root)modules/find/find_libraries.cmake"
  - "%(url_root)modules/find/find_include_files.cmake"

acml.cmake

[Source code]

Find and link to ACML.

Variables defined:

ACML_FOUND
ACML_LIBRARIES
ACML_INCLUDE_DIR

autocmake.yml configuration:

docopt: "--acml Find and link to ACML [default: False]."
define: "'-DENABLE_ACML={0}'.format(arguments['--acml'])"

atlas.cmake

[Source code]

Find and link to ATLAS.

Variables defined:

ATLAS_FOUND
ATLAS_LIBRARIES
ATLAS_INCLUDE_DIR

autocmake.yml configuration:

docopt: "--atlas Find and link to ATLAS [default: False]."
define: "'-DENABLE_ATLAS={0}'.format(arguments['--atlas'])"

blas.cmake

[Source code]

Find and link to BLAS.

Variables defined:

BLAS_FOUND
BLAS_LIBRARIES
BLAS_INCLUDE_DIR

autocmake.yml configuration:

docopt: "--blas Find and link to BLAS [default: False]."
define: "'-DENABLE_BLAS={0}'.format(arguments['--blas'])"

cblas.cmake

[Source code]

Find and link to CBLAS.

Variables defined:

CBLAS_FOUND - describe me, uncached
CBLAS_LIBRARIES - describe me, uncached
CBLAS_INCLUDE_DIR - describe me, uncached

autocmake.yml configuration:

url_root: https://github.com/dev-cafe/autocmake/raw/master/
docopt: "--cblas Find and link to CBLAS [default: False]."
define: "'-DENABLE_CBLAS={0}'.format(arguments['--cblas'])"
fetch:
  - "%(url_root)modules/find/find_libraries.cmake"
  - "%(url_root)modules/find/find_include_files.cmake"

goto.cmake

[Source code]

Find and link to Goto BLAS.

Variables defined:

GOTO_FOUND
GOTO_LIBRARIES
GOTO_INCLUDE_DIR

autocmake.yml configuration:

docopt: "--goto Find and link to GOTO [default: False]."
define: "'-DENABLE_GOTO={0}'.format(arguments['--goto'])"

lapack.cmake

[Source code]

Find and link to LAPACK.

Variables defined:

LAPACK_FOUND
LAPACK_LIBRARIES
LAPACK_INCLUDE_DIR

autocmake.yml configuration:

docopt: "--lapack Find and link to LAPACK [default: False]."
define: "'-DENABLE_LAPACK={0}'.format(arguments['--lapack'])"

lapacke.cmake

[Source code]

Find and link to LAPACKE.

Variables defined:

LAPACKE_FOUND - describe me, uncached
LAPACKE_LIBRARIES - describe me, uncached
LAPACKE_INCLUDE_DIR - describe me, uncached

autocmake.yml configuration:

url_root: https://github.com/dev-cafe/autocmake/raw/master/
docopt: "--lapacke Find and link to LAPACKE [default: False]."
define: "'-DENABLE_LAPACKE={0}'.format(arguments['--lapacke'])"
fetch:
  - "%(url_root)modules/find/find_libraries.cmake"
  - "%(url_root)modules/find/find_include_files.cmake"

math_libs.cmake

[Source code]

Detects and links to BLAS and LAPACK libraries.

Variables used:

MATH_LIB_SEARCH_ORDER, example: set(MATH_LIB_SEARCH_ORDER MKL ESSL OPENBLAS ATLAS ACML SYSTEM_NATIVE)
ENABLE_STATIC_LINKING
ENABLE_BLAS
ENABLE_LAPACK
BLAS_FOUND
LAPACK_FOUND
BLAS_LANG
LAPACK_LANG
BLAS_TYPE
LAPACK_TYPE
ENABLE_64BIT_INTEGERS
CMAKE_HOST_SYSTEM_PROCESSOR
BLACS_IMPLEMENTATION
MKL_FLAG

Variables defined:

MATH_LIBS
BLAS_FOUND
LAPACK_FOUND

Variables modified:

CMAKE_EXE_LINKER_FLAGS

Environment variables used:

MATH_ROOT
BLAS_ROOT
LAPACK_ROOT
MKL_ROOT
MKLROOT

autocmake.yml configuration:

docopt:
  - "--blas=<BLAS> Detect and link BLAS library (auto or off) [default: auto]."
  - "--lapack=<LAPACK> Detect and link LAPACK library (auto or off) [default: auto]."
  - "--mkl=<MKL> Pass MKL flag to the Intel compiler and linker and skip BLAS/LAPACK detection (sequential, parallel, cluster, or off) [default: off]."
  - "--scalapack=<SCALAPACK_LIBRARIES> Link line for ScaLAPACK libraries [default: ]"
  - "--blacs=<BLACS_IMPLEMENTATION> Implementation of BLACS for MKL ScaLAPACK (openmpi, intelmpi, sgimpt) [default: openmpi]"
define:
  - "'-DENABLE_BLAS={0}'.format(arguments['--blas'])"
  - "'-DENABLE_LAPACK={0}'.format(arguments['--lapack'])"
  - "'-DMKL_FLAG={0}'.format(arguments['--mkl'])"
  - "'-DMATH_LIB_SEARCH_ORDER=\"MKL;ESSL;OPENBLAS;ATLAS;ACML;SYSTEM_NATIVE\"'"
  - "'-DBLAS_LANG=Fortran'"
  - "'-DLAPACK_LANG=Fortran'"
  - "'-DSCALAPACK_LIBRARIES=\"{0}\"'.format(arguments['--scalapack'])"
  - "'-DBLACS_IMPLEMENTATION=\"{0}\"'.format(arguments['--blacs'])"
warning: "the math_libs.cmake module is deprecated and will be removed in future versions"

mpi.cmake

[Source code]

Enables MPI support.

Variables used:

ENABLE_MPI
MPI_FOUND

Variables modified (provided the corresponding language is enabled):

CMAKE_Fortran_FLAGS
CMAKE_C_FLAGS
CMAKE_CXX_FLAGS

autocmake.yml configuration:

docopt: "--mpi Enable MPI parallelization [default: False]."
define: "'-DENABLE_MPI={0}'.format(arguments['--mpi'])"

omp.cmake

[Source code]

Enables OpenMP support.

Variables used:

ENABLE_OPENMP
OPENMP_FOUND

Variables modified (provided the corresponding language is enabled):

CMAKE_Fortran_FLAGS
CMAKE_C_FLAGS
CMAKE_CXX_FLAGS

autocmake.yml configuration:

docopt: "--omp Enable OpenMP parallelization [default: False]."
define: "'-DENABLE_OPENMP={0}'.format(arguments['--omp'])"

profile.cmake

[Source code]

Enable profiling by appending corresponding compiler flags.

Variables modified (provided the corresponding language is enabled):

CMAKE_Fortran_FLAGS
CMAKE_C_FLAGS
CMAKE_CXX_FLAGS

autocmake.yml configuration:

docopt: "--profile Enable profiling [default: False]"
define: "'-DENABLE_PROFILING={0}'.format(arguments['--profile'])"

python_interpreter.cmake

[Source code]

Detects Python interpreter.

Variables used:

PYTHON_INTERPRETER         - User-set path to the Python interpreter

Variables defined:

PYTHONINTERP_FOUND         - Was the Python executable found
PYTHON_EXECUTABLE          - path to the Python interpreter
PYTHON_VERSION_STRING      - Python version found e.g. 2.5.2
PYTHON_VERSION_MAJOR       - Python major version found e.g. 2
PYTHON_VERSION_MINOR       - Python minor version found e.g. 5
PYTHON_VERSION_PATCH       - Python patch version found e.g. 2

autocmake.yml configuration:

docopt: "--python=<PYTHON_INTERPRETER> The Python interpreter (development version) to use. [default: '']."
define: "'-DPYTHON_INTERPRETER=\"{0}\"'.format(arguments['--python'])"

python_libs.cmake

[Source code]

Detects Python libraries and headers. Detection is done basically by hand as the proper CMake package will not find libraries and headers matching the interpreter version.

Dependencies:

python_interpreter         - Sets the Python interpreter for headers and libraries detection

Variables used:

PYTHONINTERP_FOUND         - Was the Python executable found

Variables defined:

PYTHONLIBS_FOUND           - have the Python libs been found
PYTHON_LIBRARIES           - path to the python library
PYTHON_INCLUDE_DIRS        - path to where Python.h is found
PYTHONLIBS_VERSION_STRING  - version of the Python libs found (since CMake 2.8.8)

safeguards.cmake

[Source code]

Provides safeguards against in-source builds and bad build types.

Variables used:

PROJECT_SOURCE_DIR
PROJECT_BINARY_DIR
CMAKE_BUILD_TYPE

save_flags.cmake

[Source code]

Take care of updating the cache for fresh configurations.

Variables modified (provided the corresponding language is enabled):

DEFAULT_Fortran_FLAGS_SET
DEFAULT_C_FLAGS_SET
DEFAULT_CXX_FLAGS_SET

src.cmake

[Source code]

Adds ${PROJECT_SOURCE_DIR}/src as subdirectory containing CMakeLists.txt.

version.cmake

[Source code]

Determine program version from file “VERSION” (example: “14.1”) The reason why this information is stored in a file and not as CMake variable is that CMake-unaware programs can parse and use it (e.g. Sphinx). Also web-based hosting frontends such as GitLab automatically use the file “VERSION” if present.

Variables defined:

PROGRAM_VERSION