Autocmake¶
About Autocmake¶
Building libraries and executables from sources can be a complex task. Several
solutions exist to this problem: GNU Makefiles is the traditional approach.
Today, CMake is one of the trendier alternatives which can generate Makefiles
starting from a file called CMakeLists.txt
.
Autocmake composes CMake building blocks into a CMake project and generates
CMakeLists.txt
as well as a setup script, which serves as a front-end to
CMakeLists.txt
. All this is done based on a lightweight autocmake.yml
file:
python update.py --self
| |
| fetches Autocmake |
| infrastructure |
| and updates the update.py script |
| |
v Developer maintaining
autocmake.yml Autocmake
| |
| python update.py .. |
| |
v v
CMakeLists.txt (and setup front-end)
| |
| python setup or ./setup |
| which invokes CMake |
v User of the code
Makefile (or something else) |
| |
| make |
| |
v v
Build/install/test targets
Our main motivation to create Autocmake as a CMake framework library and CMake module composer is to simplify CMake code transfer between programs. We got tired of manually diffing and copy-pasting boiler-plate CMake code and watching it diverge while maintaining the CMake infrastructure in a growing number of scientific projects which typically have very similar requirements:
- Fortran and/or C and/or C++ support
- Tuning of compiler flags
- Front-end script with good defaults
- Support for parallelization: MPI, OMP, CUDA
- Math libraries: BLAS, LAPACK
Our other motivation for Autocmake was to make it easier for developers who do not know CMake to provide a higher-level entry point to CMake.
Autocmake is a chance to provide a well documented and tested set of CMake plug-ins. With this we wish to give also users of codes the opportunity to introduce the occasional tweak without the need to dive deep into CMake documentation.
Requirements and dependencies¶
Autocmake update and test scripts require Python 2.7 or higher. We also support Python 3 (we automatically test with 2.7 and 3.5).
The generated setup script runs with Python >= 2.6 (also tested with Python 3.5).
To generate CMakeLists.txt
and the setup
script, Autocmake
requires the pyyaml package.
Getting help¶
Autocmake discussion forum: http://groups.google.com/group/autocmake
FAQ for developers¶
Which files do I need to edit?¶
Let us start with files which you normally never edit: CMakeLists.txt
and
setup
- these are generated based on autocmake.yml
. Have a look in
autocmake.yml
and you will see all CMake files which are assembled into
CMakeLists.txt
. You can edit those files. If you change the order of files
listed in autocmake.yml
or if you add or remove CMake modules, then you
need to rerun the update.py
script to refresh CMakeLists.txt
and
setup
.
Autocmake does not do feature X - I really need feature X and a setup flag –X¶
The Autocmake developers have to be very conservative and only a very limited set of portable features of absolutely general interest become part of the Autocmake core or an Autocmake module. Autocmake developers are also busy.
Our recommendation is to not wait for the feature to be implemented: Implement
it yourself. Here we show you how. Code your feature in a module (i.e.
my_feature.cmake
) and place the module under cmake/custom/
(the
directory name is just a suggestion, Autocmake does not enforce a directory
naming):
cmake/custom/my_feature.cmake
And include this feature to the main CMakeLists.txt
in autocmake.yml
under the modules
section:
modules:
- my_feature:
- source:
- custom/my_feature.cmake
Now your code is included in the main CMakeLists.txt
. Perhaps you also
want a setup script flag to toggle the feature:
- my_feature:
- docopt: "--enable-my-feature Enable my feature [default: False]."
- define: "'-DENABLE_MY_FEATURE={0}'.format(arguments['--enable-my-feature'])"
- source:
- custom/my_feature.cmake
Implement your ideas, test them, and share them. If your module is portable, good code quality, and of general interest, you can suggest it to be part of the standard set of modules or even a core feature.
How can I get a setup flag –X that toggles a CMake variable?¶
The following will add a --something
flag which toggles the CMake variable
ENABLE_SOMETHING
:
- my_section:
- docopt: "--something Enable something [default: False]."
- define: "'-DENABLE_SOMETHING={0}'.format(arguments['--enable-something'])"
Can I change the name of the setup script?¶
Yes you can do that in autocmake.yml
. Here we for instance change the name to “configure”:
name: myproject
min_cmake_version: 2.8
default_build_type: release
setup_script: configure
language:
- Fortran
- C
- CXX
Can I prevent Autocmake from creating any front-end setup script?¶
Yes, with setup_script: None
.
In CMake I can do feature X - can I do that also with Autocmake?¶
Yes. Autocmake is really just a simplistic script which helps to organize CMake code across projects. Everything that can be done in CMake can be realized in Autocmake.
Should I include and track also files generated by Autocmake in my repository?¶
Yes, you probably want to do that. Autocmake downloads and generates a number
of files which in principle could be generated at configure- or build-time.
However, you probably do not want the users of your code to run any Autocmake
scripts like update.py
to generate the files they need to build the
project. The users of your code will run setup
directly and typically expect
everything to just work (TM). Note also that the users of your code will
not need to install the pyyaml package.
The update.py script is overwriting my CMakeLists.txt and setup, isn’t this bad?¶
No, it is not as bad as it first looks. It is a feature. Normally
CMakeLists.txt
and setup
should not contain any explicit
customization and therefore should not contain anything that could not be
regenerated. In any case you should use version control so that you can inspect
and compare changes introduced to CMakeLists.txt
and setup
and
possibly revert them. See also the next remark.
But I need to manually edit and customize CMakeLists.txt and setup every time I run update.py!?¶
You typically never need to manually edit and customize CMakeLists.txt
and
setup
directly. You can introduce customizations in autocmake.yml
which get assembled into the front-end scripts.
Where is a good place to list my sources and targets?¶
As mentioned above CMakeLists.txt
is not a good place because this file is
generated from autocmake.yml
and your modifications would become
overwritten at some point. A good standard is to organize your sources under
src/
and to list your sources and targets in src/CMakeLists.txt
. You
can include the latter in autocmake.yml
using:
- my_sources:
- source:
- https://github.com/dev-cafe/autocmake/raw/master/modules/src.cmake
If you really do not like to do it this way, you can describe your sources and targets in a custom module in a local file and include it like this:
- my_sources:
- source:
- custom/my_sources.cmake
How can I do some more sophisticated validation of setup flags?¶
Sometimes you need to do more sophisticated validation and post-processing
of setup flags. This can be done by placing a module called extensions.py
under cmake/
(or wherever you have autocmake.yml
).
This file should implement a function with the following signature:
def postprocess_args(sys_argv, arguments):
# sys_argv is the sys.argv from the setup script
# arguments is the dictionary of arguments returned by docopt
# do something here ...
return arguments
In this function you can do any validation and post-processing you like. This function is run after the flags are parsed and before the CMake command is run.
Example for a validation of MPI flags to the setup script:
import sys
def contains_flag(sys_argv, flag):
return (any(x for x in sys_argv if x.startswith('--{0}='.format(flag))))
def postprocess_args(sys_argv, arguments):
# if --mpi is selected and compilers are not selected
# then compilers default to mpif90, mpicc, and mpicxx
if arguments['--mpi']:
if not contains_flag(sys_argv, 'fc') and not contains_flag(sys_argv, 'cc') and not contains_flag(sys_argv, 'cxx'):
arguments['--fc'] = 'mpif90'
arguments['--cc'] = 'mpicc'
arguments['--cxx'] = 'mpicxx'
# if one of the compilers contains "mpi" and --mpi is not selected, it is probably a user error
# in this case stop the configuration
asking_for_mpi_compiler = False
for flag in ['fc', 'cc', 'cxx']:
if contains_flag(sys_argv, 'fc'):
if 'mpi' in arguments['--fc']:
asking_for_mpi_compiler = True
if asking_for_mpi_compiler and not arguments['--mpi']:
sys.stderr.write('ERROR: you ask for an MPI compiler but have not specified --mpi\n')
sys.exit(1)
return arguments
Bootstrapping a new project¶
Bootstrapping Autocmake¶
Download the update.py
and execute it with --self
to fetch other
infrastructure files which will be needed to build the project:
$ mkdir cmake # does not have to be called "cmake" - take the name you prefer
$ cd cmake
$ wget https://github.com/dev-cafe/autocmake/raw/master/update.py
$ virtualenv venv
$ source venv/bin/activate
$ pip install pyyaml
$ python update.py --self
On the MS Windows system, you can use the PowerShell wget-replacement:
$ Invoke-WebRequest https://github.com/dev-cafe/autocmake/raw/master/update.py -OutFile update.py
This creates (or updates) the following files (an existing autocmake.yml
is
not overwritten by the script):
cmake/
autocmake.yml # edit this file
update.py # no need to edit
autocmake/ # no need to edit
... # no need to edit
Note that update.py
and files under autocmake/
are overwritten (use version control).
Generating the CMake infrastructure¶
Now customize autocmake.yml
to your needs
(see Configuring autocmake.yml)
and then run the update.py
script which
creates CMakeLists.txt
and a setup script in the target path:
$ python update.py ..
The script also downloads external CMake modules specified in autocmake.yml
to a directory
called downloaded/
:
cmake/
autocmake.yml # edit this file
update.py # no need to edit
autocmake/ # no need to edit
... # no need to edit
downloaded/ # contains CMake modules fetched from the web
Building the project¶
Now you have CMakeLists.txt
and setup script in the project root and the project
can be built:
$ cd ..
$ python setup [-h]
$ cd build
$ make
Configuring autocmake.yml¶
The script autocmake.yml
is the high level place where you configure
your project. Here is an example. We will discuss it in detail further
below:
name: numgrid
min_cmake_version: 2.8
default_build_type: release
language:
- Fortran
- C
- CXX
url_root: https://github.com/dev-cafe/autocmake/raw/master/
modules:
- compilers:
- source:
- '%(url_root)modules/fc.cmake'
- '%(url_root)modules/cc.cmake'
- '%(url_root)modules/cxx.cmake'
- flags:
- source:
- '%(url_root)compilers/GNU.CXX.cmake'
- '%(url_root)compilers/Intel.CXX.cmake'
- 'compilers/Clang.CXX.cmake'
- plugins:
- source:
- '%(url_root)modules/ccache.cmake'
- 'custom/rpath.cmake'
- '%(url_root)modules/definitions.cmake'
- '%(url_root)modules/code_coverage.cmake'
- '%(url_root)modules/safeguards.cmake'
- '%(url_root)modules/default_build_paths.cmake'
- '%(url_root)modules/src.cmake'
- '%(url_root)modules/googletest.cmake'
- 'custom/api.cmake'
- 'custom/test.cmake'
Name and order of sections¶
First we define the project name (here “numgrid”). This section has to be there and it has to be called “project” (but it does not have to be on top).
We also have to define min_cmake_version
as well as set the supported language(s).
The definition url_root
is an interpolation (see Interpolation) and
we use it to avoid retyping the same line over and over and to be able to
change it in one place. The explicit name “url_root” has no special meaning to
Autocmake and we could have chosen a different name.
The section modules
is a list of CMake plugins. The names of the list
elements (here “compilers”, “flags”, and “plugins”) does not matter to
Autocmake. We could have called them “one”, “two”, and “whatever”, but it would
not make much sense. It is better to choose names that are meaningful to you
and readers of your code.
The order of the elements under modules
does matter and the list will be
processed in the exact order as you specify them in autocmake.yml
.
Minimal example¶
As a minimal example we take an autocmake.yml
which only contains:
name: minime
min_cmake_version: 2.8
language: C
If you don’t have the update.py
script yet, you need to fetch it from the web:
$ wget https://github.com/dev-cafe/autocmake/raw/master/update.py
First we make sure that the update.py
script is up-to-date and that it has access
to all libraries it needs:
$ python update.py --self
- creating .gitignore
- fetching autocmake/configure.py
- fetching autocmake/__init__.py
- fetching autocmake/external/docopt.py
- fetching autocmake/external/__init__.py
- fetching autocmake/generate.py
- fetching autocmake/extract.py
- fetching autocmake/interpolate.py
- fetching autocmake/parse_rst.py
- fetching autocmake/parse_yaml.py
- fetching update.py
Good. Now we can generate CMakeLists.txt
and the setup script:
$ python update.py ..
- parsing autocmake.yml
- generating CMakeLists.txt
- generating setup script
Excellent. Here is the generated CMakeLists.txt
:
# set minimum cmake version
cmake_minimum_required(VERSION 2.8 FATAL_ERROR)
# project name
project(minime C)
# do not rebuild if rules (compiler flags) change
set(CMAKE_SKIP_RULE_DEPENDENCY TRUE)
# if CMAKE_BUILD_TYPE undefined, we set it to Debug
if(NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE "Debug")
endif()
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${PROJECT_SOURCE_DIR}/cmake/downloaded)
This is the very bare minimum. Every Autocmake project will have at least these settings.
And we also got a setup script (front-end to CMakeLists.txt
) with
the following default options:
Usage:
./setup [options] [<builddir>]
./setup (-h | --help)
Options:
--type=<TYPE> Set the CMake build type (debug, release, or relwithdeb) [default: release].
--generator=<STRING> Set the CMake build system generator [default: Unix Makefiles].
--show Show CMake command and exit.
--cmake-executable=<CMAKE_EXECUTABLE> Set the CMake executable [default: cmake].
--cmake-options=<STRING> Define options to CMake [default: ''].
--prefix=<PATH> Set the install path for make install.
<builddir> Build directory.
-h --help Show this screen.
That’s not too bad although currently we cannot do much with this since there
are no sources listed, no targets, hence nothing to build. We need to flesh out
CMakeLists.txt
by extending autocmake.yml
and this is what we will do
in the next section.
Assembling CMake plugins¶
The preferred way to extend CMakeLists.txt
is by editing autocmake.yml
and using the source
option:
- compilers:
- source:
- '%(url_root)modules/fc.cmake'
- '%(url_root)modules/cc.cmake'
- '%(url_root)modules/cxx.cmake'
This will download fc.cmake
, cc.cmake
, and cxx.cmake
, and include
them in CMakeLists.txt
, in this order.
You can also include local CMake modules, e.g.:
- source:
- 'custom/rpath.cmake'
It is also OK to include several modules at once as we have seen above. The
modules will be included in the same order as they appear in autocmake.yml
.
Fetching files without including them in CMakeLists.txt¶
Sometimes you want to fetch a file without including it in CMakeLists.txt
.
This can be done with the fetch
option. This is for instance done by the
git_info.cmake
module (see
https://github.com/dev-cafe/autocmake/blob/master/modules/git_info/git_info.cmake#L10-L13).
If fetch
is invoked in autocmake.yml
, then the fetched file is placed
under downloaded/
. If fetch
is invoked from within a CMake module
documentation (see below), then the fetched file is placed into the same
directory as the CMake module file which fetches it.
Generating setup options¶
Options for the setup script can be generated with the docopt
option. As an example, the following autocmake.yml
snippet will add a
--something
flag:
- my_section:
- docopt: "--something Enable something [default: False]."
Setting CMake options¶
Configure-time CMake options can be generated with the define
option.
Consider the following example which toggles the CMake variable
ENABLE_SOMETHING
:
- my_section:
- docopt: "--something Enable something [default: False]."
- define: "'-DENABLE_SOMETHING={0}'.format(arguments['--enable-something'])"
Setting environment variables¶
You can export environment variables at configure-time using the export
option. Consider the following example:
docopt:
- "--cc=<CC> C compiler [default: gcc]."
- "--extra-cc-flags=<EXTRA_CFLAGS> Extra C compiler flags [default: '']."
export: "'CC={0}'.format(arguments['--cc'])"
define: "'-DEXTRA_CFLAGS=\"{0}\"'.format(arguments['--extra-cc-flags'])"
Auto-generating configurations from the documentation¶
To avoid a boring re-typing of boilerplate autocmake.yml
code it is possible
to auto-generate configurations from the documentation. This is the case
for many core modules which come with own options once you have sourced them.
The lines following # autocmake.yml configuration::
are
understood by the update.py
script to infer autocmake.yml
code from the
documentation. As an example consider
https://github.com/dev-cafe/autocmake/blob/master/modules/cc.cmake#L20-L26.
Here, update.py
will infer the configurations for docopt
, export
,
and define
.
Example hello world project¶
This is a brief example for the busy and impatient programmer. For a longer tour please see Configuring autocmake.yml.
We start with a mixed Fortran-C project with the following sources:
feature1.F90
feature2.c
main.F90
First thing we do is to create a directory src/
and we move all sources
there. This is not necessary for Autocmake but it is a generally good practice:
.
`-- src
|-- feature1.F90
|-- feature2.c
`-- main.F90
Now we create cmake/
and fetch update.py
:
$ mkdir cmake # does not have to be called "cmake" - take the name you prefer
$ cd cmake
$ wget https://github.com/dev-cafe/autocmake/raw/master/update.py
$ python update.py --self
Now from top-level our file tree looks like this:
.
|-- cmake
| |-- autocmake
| | |-- __init__.py
| | |-- configure.py
| | |-- external
| | | |-- __init__.py
| | | `-- docopt.py
| | |-- extract.py
| | |-- generate.py
| | |-- interpolate.py
| | |-- parse_rst.py
| | `-- parse_yaml.py
| |-- autocmake.yml
| `-- update.py
`-- src
|-- feature1.F90
|-- feature2.c
`-- main.F90
Now we edit cmake/autocmake.yml
to look like this:
name: hello
min_cmake_version: 2.8
language:
- Fortran
- C
url_root: https://github.com/dev-cafe/autocmake/raw/master/
modules:
- compilers:
- source:
- '%(url_root)modules/fc.cmake'
- '%(url_root)modules/cc.cmake'
- src_support:
- source:
- '%(url_root)modules/src.cmake'
What we have specified here is the project name and that we wish Fortran and C
support. The src.cmake
module tells CMake to include a src/CMakeLists.txt
.
We need to create src/CMakeLists.txt
which can look like this:
add_executable(
hello.x
main.F90
feature1.F90
feature2.c
)
We wrote that we want to get an executable “hello.x” built from our sources.
Now we have everything to generate CMakeLists.txt
and a setup script:
$ cd cmake
$ python update ..
- parsing autocmake.yml
- assembling modules: [##############################] (3/3)
- generating CMakeLists.txt
- generating setup script
And this is what we got:
.
|-- CMakeLists.txt
|-- cmake
| |-- autocmake
| | |-- __init__.py
| | |-- configure.py
| | |-- external
| | | |-- __init__.py
| | | `-- docopt.py
| | |-- extract.py
| | |-- generate.py
| | |-- interpolate.py
| | |-- parse_rst.py
| | `-- parse_yaml.py
| |-- autocmake.yml
| |-- downloaded
| | |-- autocmake_cc.cmake
| | |-- autocmake_fc.cmake
| | `-- autocmake_src.cmake
| `-- update.py
|-- setup
`-- src
|-- CMakeLists.txt
|-- feature1.F90
|-- feature2.c
`-- main.F90
Now we are ready to build:
$ ./setup --fc=gfortran --cc=gcc
FC=gfortran CC=gcc cmake -DEXTRA_FCFLAGS="''" -DEXTRA_CFLAGS="''" -DCMAKE_BUILD_TYPE=release -G "Unix Makefiles" /home/user/hello
-- The C compiler identification is GNU 6.1.1
-- The CXX compiler identification is GNU 6.1.1
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- The Fortran compiler identification is GNU 6.1.1
-- Check for working Fortran compiler: /usr/bin/gfortran
-- Check for working Fortran compiler: /usr/bin/gfortran -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether /usr/bin/gfortran supports Fortran 90
-- Checking whether /usr/bin/gfortran supports Fortran 90 -- yes
-- Configuring done
-- Generating done
-- Build files have been written to: /home/user/hello/build
configure step is done
now you need to compile the sources:
$ cd build
$ make
$ cd build
$ make
Scanning dependencies of target hello.x
[ 25%] Building Fortran object src/CMakeFiles/hello.x.dir/main.F90.o
[ 50%] Building Fortran object src/CMakeFiles/hello.x.dir/feature1.F90.o
[ 75%] Building C object src/CMakeFiles/hello.x.dir/feature2.c.o
[100%] Linking Fortran executable hello.x
[100%] Built target hello.x
Excellent! But this was a lot of typing and file creating just to get a simple executable compiled!? Of course, all that could have been done with few command lines directly but now we have a cross-platform project and can extend it and customize it and we also got a front-end script and command-line parser for free. Now go out and explore more Autocmake modules and features.
Customizing CMake modules¶
The update.py
script assembles modules listed in autocmake.yml
into
CMakeLists.txt
. Those that are fetched from the web are placed inside
downloaded/
. You have several options to customize downloaded CMake
modules:
Directly inside the generated directory¶
The CMake modules can be customized directly inside downloaded/
but this is
the least elegant solution since the customizations may be overwritten by the
update.py
script (use version control).
Adapt local copies of CMake modules¶
A slightly better solution is to download the CMake modules that you wish you customize
to a separate directory (e.g. custom/
) and source the customized CMake
modules in autocmake.yml
. Alternatively you can serve your custom modules
from your own http server.
Fork and branch the CMake modules¶
You can fork and branch the mainline Autocmake development and include the branched customized versions. This will make it easier for you to stay up-to-date with upstream development.
Overriding settings¶
If you source a module which contains directives such as
define
,
docopt
,
export
, or
fetch
, and you wish to modify those,
then you can override these settings in autocmake.yml
.
Settings in autocmake.yml
take precedence over
settings imported by a sourced module.
As an example consider the Boost module which defines and uses
interpolation variables major
, minor
, patch
, and components
, see
https://github.com/dev-cafe/autocmake/blob/master/modules/boost/boost.cmake#L52-L55.
The recommended way to customize these is in autocmake.yml
, e.g.:
https://github.com/dev-cafe/autocmake/blob/master/test/boost_libs/cmake/autocmake.yml#L12-L17.
Create own CMake modules¶
Of course you can also create own modules and source them in autocmake.yml
.
Contribute customizations to the “standard library”¶
If you think that your customization will be useful for other users as well, you may consider contributing the changes directly to https://github.com/dev-cafe/autocmake/. We very much encourage such contributions. But we also strive for generality and portability.
Updating CMake modules¶
To update CMake modules fetched from the web you need to run the update.py
script:
$ cd cmake
$ python update.py ..
The CMake modules are not fetched or updated at configure time or build time.
In other words, if you never re-run update.py
script and never modify the
CMake module files, then the CMake modules will remain forever frozen.
How to pin CMake modules to a certain version¶
Sometimes you may want to avoid using the latest version of a CMake module and
rather fetch an older version, for example with the hash abcd123
. To
achieve this, instead of:
- my_feature:
- source: https://github.com/dev-cafe/autocmake/raw/master/modules/foo.cmake
pin the version to abcd123
(you do not need to specify the full Git hash, a unique
beginning will do):
- my_feature:
- source: https://github.com/dev-cafe/autocmake/raw/abcd123/modules/foo.cmake
Interpolation¶
In a custom extension to the YAML specification you can define and reuse
variables like this (observe how we interpolate url_root
, major
,
minor
, patch
, and components
in this example):
url_root: https://github.com/dev-cafe/autocmake/raw/master/
major: 1
minor: 48
patch: 0
components: ""
fetch:
- "%(url_root)modules/boost/boost_unpack.cmake"
- "%(url_root)modules/boost/boost_userconfig.cmake"
- "%(url_root)modules/boost/boost_configure.cmake"
- "%(url_root)modules/boost/boost_build.cmake"
- "%(url_root)modules/boost/boost_install.cmake"
- "%(url_root)modules/boost/boost_headers.cmake"
- "%(url_root)modules/boost/boost_cleanup.cmake"
- "http://sourceforge.net/projects/boost/files/boost/%(major).%(minor).%(patch)/boost_%(major)_%(minor)_%(patch).zip"
docopt:
- "--boost-headers=<BOOST_INCLUDEDIR> Include directories for Boost [default: '']."
- "--boost-libraries=<BOOST_LIBRARYDIR> Library directories for Boost [default: '']."
- "--build-boost=<FORCE_CUSTOM_BOOST> Deactivate Boost detection and build on-the-fly <ON/OFF> [default: OFF]."
define:
- "'-DBOOST_INCLUDEDIR=\"{0}\"'.format(arguments['--boost-headers'])"
- "'-DBOOST_LIBRARYDIR=\"{0}\"'.format(arguments['--boost-libraries'])"
- "'-DFORCE_CUSTOM_BOOST={0}'.format(arguments['--build-boost'])"
- "'-DBOOST_MINIMUM_REQUIRED=\"%(major).%(minor).%(patch)\"'"
- "'-DBOOST_COMPONENTS_REQUIRED=\"%(components)\"'"
FAQ for users¶
TL;DR How do I compile the code?¶
$ python setup [-h]
$ cd build
$ make
How can I specify the compiler?¶
By default the setup script will attempt GNU compilers. You can specify compilers manually like this:
$ python setup --fc=ifort --cc=icc --cxx=icpc
How can I add compiler flags?¶
You can do this with --extra-fc-flags
, --extra-cc-flags
, or
--extra-cxx-flags
(depending on the set of enabled languages):
$ python setup --fc=gfortran --extra-fc-flags='-some-exotic-flag'
How can I redefine compiler flags?¶
If you export compiler flags using the environment variables FCFLAGS
,
CFLAGS
, or CXXFLAGS
, respectively, then the configuration will use
those flags and neither augment them, nor redefine them. Setting
these environment variables you have full control over the flags
without editing CMake files.
How can I select CMake options via the setup script?¶
Like this:
$ python setup --cmake-options='"-DTHIS_OPTION=ON -DTHAT_OPTION=OFF"'
We use two sets of quotes because the shell swallows one set of them before passing the arguments to Python. Yeah that’s not nice, but nothing we can do about it on the Python side. If you do not use two sets of quotes then the setup command may end up incorrectly saved in build/setup_command.
Contributing guidelines¶
Please follow the excellent guide: http://www.contribution-guide.org.
We do not require any formal copyright assignment or contributor license agreement. Any contributions intentionally sent upstream are presumed to be offered under terms of the OSI-approved BSD 3-clause license.
Testing Autocmake¶
You will need to install pytest.
Check also the Travis build and test recipe for other requirements.
Your contributions and changes should preserve the test set and be PEP8 conform. You can run locally all tests with:
$ pycodestyle --ignore E501 update.py
$ pycodestyle --ignore E501,E265 autocmake
$ py.test -vv autocmake/*
$ py.test -vv test/test.py
You can also select individual tests, for example those with fc_blas
string in the name:
$ py.test -k fc_blas test/test.py
For more options, see the py.test
flags.
This test set is run upon each push to the central repository. See also the Travis build and test history.
Contributing to the documentation¶
Contributions and patches to the documentation are most welcome.
This documentation is refreshed upon each push to the central repository.
The module reference documentation is generated from the module sources using
#.rst:
tags (compare for instance
http://autocmake.readthedocs.io/en/latest/module-reference.html#cc-cmake with
https://github.com/dev-cafe/autocmake/blob/master/modules/cc.cmake).
Please note that the lines following # autocmake.yml configuration::
are
understood by the update.py
script to infer autocmake.yml code from the
documentation. As an example consider
https://github.com/dev-cafe/autocmake/blob/master/modules/cc.cmake#L20-L26.
Here, update.py
will infer the configurations for docopt
, export
,
and define
.
Module reference¶
boost.cmake¶
Detect, build, and link Boost libraries. This modules downloads the .zip archive from SourceForge at Autocmake update time. Note that the build-up commands are not Windows-compatible!
Your autocmake.yml should look like this:
- boost:
- major: 1
- minor: 59
- patch: 0
- components: “chrono;timer;system”
- source: “https://github.com/dev-cafe/autocmake/raw/master/modules/boost/boost.cmake”
Cross-dependencies between required components are not checked for. For example, Boost.Timer depends on Boost.Chrono and Boost.System thus you should ask explicitly for all three. If the self-build of Boost components is triggered the BUILD_CUSTOM_BOOST variable is set to TRUE. The CMake target custom_boost is also added. You should use these two to ensure the right dependencies between your targets and the Boost headers/libraries, in case the self-build is triggered. For example:
if(BUILD_CUSTOM_BOOST)
add_dependencies(your_target custom_boost)
endif()
will ensure that your_target is built after custom_boost if and only if the self-build of Boost took place. This is an important step to avoid race conditions when building on multiple processes.
Dependencies:
mpi - Only if the Boost.MPI library is a needed component
python_libs - Only if the Boost.Python library is a needed component
Variables used:
BOOST_MINIMUM_REQUIRED - Minimum required version of Boost
BOOST_COMPONENTS_REQUIRED - Components (compiled Boost libraries) required
PROJECT_SOURCE_DIR
PROJECT_BINARY_DIR
CMAKE_BUILD_TYPE
MPI_FOUND
BUILD_CUSTOM_BOOST
autocmake.yml configuration:
url_root: https://github.com/dev-cafe/autocmake/raw/master/
major: 1
minor: 48
patch: 0
components: ""
fetch:
- "%(url_root)modules/boost/boost_unpack.cmake"
- "%(url_root)modules/boost/boost_userconfig.cmake"
- "%(url_root)modules/boost/boost_configure.cmake"
- "%(url_root)modules/boost/boost_build.cmake"
- "%(url_root)modules/boost/boost_install.cmake"
- "%(url_root)modules/boost/boost_headers.cmake"
- "%(url_root)modules/boost/boost_cleanup.cmake"
- "http://sourceforge.net/projects/boost/files/boost/%(major).%(minor).%(patch)/boost_%(major)_%(minor)_%(patch).zip"
docopt:
- "--boost-headers=<BOOST_INCLUDEDIR> Include directories for Boost [default: '']."
- "--boost-libraries=<BOOST_LIBRARYDIR> Library directories for Boost [default: '']."
- "--build-boost=<FORCE_CUSTOM_BOOST> Deactivate Boost detection and build on-the-fly <ON/OFF> [default: OFF]."
define:
- "'-DBOOST_INCLUDEDIR=\"{0}\"'.format(arguments['--boost-headers'])"
- "'-DBOOST_LIBRARYDIR=\"{0}\"'.format(arguments['--boost-libraries'])"
- "'-DFORCE_CUSTOM_BOOST={0}'.format(arguments['--build-boost'])"
- "'-DBOOST_MINIMUM_REQUIRED=\"%(major).%(minor).%(patch)\"'"
- "'-DBOOST_COMPONENTS_REQUIRED=\"%(components)\"'"
cc.cmake¶
Adds C support. Appends EXTRA_CFLAGS to CMAKE_C_FLAGS. If environment variable CFLAGS is set, then the CFLAGS are used and no other flags are used or appended.
Variables used:
EXTRA_CFLAGS
Variables modified:
CMAKE_C_FLAGS
Environment variables used:
CFLAGS
autocmake.yml configuration:
docopt:
- "--cc=<CC> C compiler [default: gcc]."
- "--extra-cc-flags=<EXTRA_CFLAGS> Extra C compiler flags [default: '']."
define: "'-DCMAKE_C_COMPILER={0} -DEXTRA_CFLAGS=\"{1}\"'.format(arguments['--cc'], arguments['--extra-cc-flags'])"
ccache.cmake¶
Adds ccache support. The user should export the appropriate environment variables to tweak the program’s behaviour, as described in its manpage. Notice that some additional compiler flags might be needed in order to avoid unnecessary warnings.
Variables defined:
CCACHE_FOUND
autocmake.yml configuration:
docopt: "--ccache=<USE_CCACHE> Toggle use of ccache <ON/OFF> [default: ON]."
define: "'-DUSE_CCACHE={0}'.format(arguments['--ccache'])"
code_coverage.cmake¶
Enables code coverage by appending corresponding compiler flags.
Variables modified (provided the corresponding language is enabled):
CMAKE_Fortran_FLAGS
CMAKE_C_FLAGS
CMAKE_CXX_FLAGS
autocmake.yml configuration:
docopt: "--coverage Enable code coverage [default: OFF]."
define: "'-DENABLE_CODE_COVERAGE={0}'.format(arguments['--coverage'])"
custom_color_messages.cmake¶
Colorize CMake output. Code was found on StackOverflow: http://stackoverflow.com/a/19578320
- Usage within CMake code:
- message(“This is normal”) message(“${Red}This is Red${ColourReset}”) message(“${Green}This is Green${ColourReset}”) message(“${Yellow}This is Yellow${ColourReset}”) message(“${Blue}This is Blue${ColourReset}”) message(“${Magenta}This is Magenta${ColourReset}”) message(“${Cyan}This is Cyan${ColourReset}”) message(“${White}This is White${ColourReset}”) message(“${BoldRed}This is BoldRed${ColourReset}”) message(“${BoldGreen}This is BoldGreen${ColourReset}”) message(“${BoldYellow}This is BoldYellow${ColourReset}”) message(“${BoldBlue}This is BoldBlue${ColourReset}”) message(“${BoldMagenta}This is BoldMagenta${ColourReset}”) message(“${BoldCyan}This is BoldCyan${ColourReset}”) message(“${BoldWhite}This is BoldWhitenn${ColourReset}”)
Has no effect on WIN32.
cxx.cmake¶
Adds C++ support. Appends EXTRA_CXXFLAGS to CMAKE_CXX_FLAGS. If environment variable CXXFLAGS is set, then the CXXFLAGS are used and no other flags are used or appended.
Variables used:
EXTRA_CXXFLAGS
Variables modified:
CMAKE_CXX_FLAGS
Environment variables used:
CXXFLAGS
autocmake.yml configuration:
docopt:
- "--cxx=<CXX> C++ compiler [default: g++]."
- "--extra-cxx-flags=<EXTRA_CXXFLAGS> Extra C++ compiler flags [default: '']."
define: "'-DCMAKE_CXX_COMPILER={0} -DEXTRA_CXXFLAGS=\"{1}\"'.format(arguments['--cxx'], arguments['--extra-cxx-flags'])"
default_build_paths.cmake¶
Sets binary and library output directories to ${CMAKE_BINARY_DIR}/${CMAKE_INSTALL_BINDIR} and ${CMAKE_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR}, respectively.
Variables modified:
CMAKE_ARCHIVE_OUTPUT_DIRECTORY
CMAKE_LIBRARY_OUTPUT_DIRECTORY
CMAKE_RUNTIME_OUTPUT_DIRECTORY
definitions.cmake¶
Add preprocessor definitions (example: –add-definitions=”-DTHIS -DTHAT=137”). These are passed all the way down to the compiler.
Variables used:
PREPROCESSOR_DEFINITIONS
autocmake.yml configuration:
docopt: "--add-definitions=<STRING> Add preprocesor definitions [default: '']."
define: "'-DPREPROCESSOR_DEFINITIONS=\"{0}\"'.format(arguments['--add-definitions'])"
export_header.cmake¶
Generates export header for your API using best practices. This module wraps https://cmake.org/cmake/help/v3.0/module/GenerateExportHeader.html and it is needed because of these, Very Good Indeed, reasons: https://gcc.gnu.org/wiki/Visibility. If you are not afraid of jargon, please also have a look at https://www.akkadia.org/drepper/dsohowto.pdf.
Variables used:
PROJECT_NAME (defined by project())
fc.cmake¶
Adds Fortran support. Appends EXTRA_FCFLAGS to CMAKE_Fortran_FLAGS. If environment variable FCFLAGS is set, then the FCFLAGS are used and no other flags are used or appended.
Variables used:
EXTRA_FCFLAGS
Variables defined:
CMAKE_Fortran_MODULE_DIRECTORY
Variables modified:
CMAKE_Fortran_FLAGS
Environment variables used:
FCFLAGS
autocmake.yml configuration:
docopt:
- "--fc=<FC> Fortran compiler [default: gfortran]."
- "--extra-fc-flags=<EXTRA_FCFLAGS> Extra Fortran compiler flags [default: '']."
define: "'-DCMAKE_Fortran_COMPILER={0} -DEXTRA_FCFLAGS=\"{1}\"'.format(arguments['--fc'], arguments['--extra-fc-flags'])"
fc_optional.cmake¶
Adds optional Fortran support. Appends EXTRA_FCFLAGS to CMAKE_Fortran_FLAGS. If environment variable FCFLAGS is set, then the FCFLAGS are used and no other flags are used or appended.
Variables used:
EXTRA_FCFLAGS
ENABLE_FC_SUPPORT
Variables defined:
CMAKE_Fortran_MODULE_DIRECTORY
Variables modified:
CMAKE_Fortran_FLAGS
Variables defined:
ENABLE_FC_SUPPORT
Environment variables used:
FCFLAGS
autocmake.yml configuration:
docopt:
- "--fc=<FC> Fortran compiler [default: gfortran]."
- "--extra-fc-flags=<EXTRA_FCFLAGS> Extra Fortran compiler flags [default: '']."
- "--fc-support=<FC_SUPPORT> Toggle Fortran language support (ON/OFF) [default: ON]."
define:
- "'-DCMAKE_Fortran_COMPILER={0} -DEXTRA_FCFLAGS=\"{1}\"'.format(arguments['--fc'], arguments['--extra-fc-flags'])"
- "'-DENABLE_FC_SUPPORT={0}'.format(arguments['--fc-support'])"
git_info.cmake¶
Creates git_info.h in the build directory. This file can be included in sources to print Git repository version and status information to the program output.
autocmake.yml configuration:
url_root: https://github.com/dev-cafe/autocmake/raw/master/
fetch:
- "%(url_root)modules/git_info/git_info.h.in"
googletest.cmake¶
Includes Google Test sources and adds a library “googletest”.
Variables used:
GOOGLETEST_ROOT
autocmake.yml configuration:
define: "'-DGOOGLETEST_ROOT=external/googletest/googletest'"
int64.cmake¶
Enables 64-bit integer support for Fortran projects.
Variables modified (provided the corresponding language is enabled):
CMAKE_Fortran_FLAGS
autocmake.yml configuration:
docopt: "--int64 Enable 64bit integers [default: False]."
define: "'-DENABLE_64BIT_INTEGERS={0}'.format(arguments['--int64'])"
accelerate.cmake¶
Find and link to ACCELERATE.
Variables defined:
ACCELERATE_FOUND - describe me, uncached
ACCELERATE_LIBRARIES - describe me, uncached
ACCELERATE_INCLUDE_DIR - describe me, uncached
autocmake.yml configuration:
url_root: https://github.com/dev-cafe/autocmake/raw/master/
docopt: "--accelerate Find and link to ACCELERATE [default: False]."
define: "'-DENABLE_ACCELERATE={0}'.format(arguments['--accelerate'])"
fetch:
- "%(url_root)modules/find/find_libraries.cmake"
- "%(url_root)modules/find/find_include_files.cmake"
acml.cmake¶
Find and link to ACML.
Variables defined:
ACML_FOUND
ACML_LIBRARIES
ACML_INCLUDE_DIR
autocmake.yml configuration:
docopt: "--acml Find and link to ACML [default: False]."
define: "'-DENABLE_ACML={0}'.format(arguments['--acml'])"
atlas.cmake¶
Find and link to ATLAS.
Variables defined:
ATLAS_FOUND
ATLAS_LIBRARIES
ATLAS_INCLUDE_DIR
autocmake.yml configuration:
docopt: "--atlas Find and link to ATLAS [default: False]."
define: "'-DENABLE_ATLAS={0}'.format(arguments['--atlas'])"
blas.cmake¶
Find and link to BLAS.
Variables defined:
BLAS_FOUND
BLAS_LIBRARIES
BLAS_INCLUDE_DIR
autocmake.yml configuration:
docopt: "--blas Find and link to BLAS [default: False]."
define: "'-DENABLE_BLAS={0}'.format(arguments['--blas'])"
cblas.cmake¶
Find and link to CBLAS.
Variables defined:
CBLAS_FOUND - describe me, uncached
CBLAS_LIBRARIES - describe me, uncached
CBLAS_INCLUDE_DIR - describe me, uncached
autocmake.yml configuration:
url_root: https://github.com/dev-cafe/autocmake/raw/master/
docopt: "--cblas Find and link to CBLAS [default: False]."
define: "'-DENABLE_CBLAS={0}'.format(arguments['--cblas'])"
fetch:
- "%(url_root)modules/find/find_libraries.cmake"
- "%(url_root)modules/find/find_include_files.cmake"
goto.cmake¶
Find and link to Goto BLAS.
Variables defined:
GOTO_FOUND
GOTO_LIBRARIES
GOTO_INCLUDE_DIR
autocmake.yml configuration:
docopt: "--goto Find and link to GOTO [default: False]."
define: "'-DENABLE_GOTO={0}'.format(arguments['--goto'])"
lapack.cmake¶
Find and link to LAPACK.
Variables defined:
LAPACK_FOUND
LAPACK_LIBRARIES
LAPACK_INCLUDE_DIR
autocmake.yml configuration:
docopt: "--lapack Find and link to LAPACK [default: False]."
define: "'-DENABLE_LAPACK={0}'.format(arguments['--lapack'])"
lapacke.cmake¶
Find and link to LAPACKE.
Variables defined:
LAPACKE_FOUND - describe me, uncached
LAPACKE_LIBRARIES - describe me, uncached
LAPACKE_INCLUDE_DIR - describe me, uncached
autocmake.yml configuration:
url_root: https://github.com/dev-cafe/autocmake/raw/master/
docopt: "--lapacke Find and link to LAPACKE [default: False]."
define: "'-DENABLE_LAPACKE={0}'.format(arguments['--lapacke'])"
fetch:
- "%(url_root)modules/find/find_libraries.cmake"
- "%(url_root)modules/find/find_include_files.cmake"
math_libs.cmake¶
Detects and links to BLAS and LAPACK libraries.
Variables used:
MATH_LIB_SEARCH_ORDER, example: set(MATH_LIB_SEARCH_ORDER MKL ESSL OPENBLAS ATLAS ACML SYSTEM_NATIVE)
ENABLE_STATIC_LINKING
ENABLE_BLAS
ENABLE_LAPACK
BLAS_FOUND
LAPACK_FOUND
BLAS_LANG
LAPACK_LANG
BLAS_TYPE
LAPACK_TYPE
ENABLE_64BIT_INTEGERS
CMAKE_HOST_SYSTEM_PROCESSOR
BLACS_IMPLEMENTATION
MKL_FLAG
Variables defined:
MATH_LIBS
BLAS_FOUND
LAPACK_FOUND
Variables modified:
CMAKE_EXE_LINKER_FLAGS
Environment variables used:
MATH_ROOT
BLAS_ROOT
LAPACK_ROOT
MKL_ROOT
MKLROOT
autocmake.yml configuration:
docopt:
- "--blas=<BLAS> Detect and link BLAS library (auto or off) [default: auto]."
- "--lapack=<LAPACK> Detect and link LAPACK library (auto or off) [default: auto]."
- "--mkl=<MKL> Pass MKL flag to the Intel compiler and linker and skip BLAS/LAPACK detection (sequential, parallel, cluster, or off) [default: off]."
- "--scalapack=<SCALAPACK_LIBRARIES> Link line for ScaLAPACK libraries [default: ]"
- "--blacs=<BLACS_IMPLEMENTATION> Implementation of BLACS for MKL ScaLAPACK (openmpi, intelmpi, sgimpt) [default: openmpi]"
define:
- "'-DENABLE_BLAS={0}'.format(arguments['--blas'])"
- "'-DENABLE_LAPACK={0}'.format(arguments['--lapack'])"
- "'-DMKL_FLAG={0}'.format(arguments['--mkl'])"
- "'-DMATH_LIB_SEARCH_ORDER=\"MKL;ESSL;OPENBLAS;ATLAS;ACML;SYSTEM_NATIVE\"'"
- "'-DBLAS_LANG=Fortran'"
- "'-DLAPACK_LANG=Fortran'"
- "'-DSCALAPACK_LIBRARIES=\"{0}\"'.format(arguments['--scalapack'])"
- "'-DBLACS_IMPLEMENTATION=\"{0}\"'.format(arguments['--blacs'])"
warning: "the math_libs.cmake module is deprecated and will be removed in future versions"
mpi.cmake¶
Enables MPI support.
Variables used:
ENABLE_MPI
MPI_FOUND
Variables modified (provided the corresponding language is enabled):
CMAKE_Fortran_FLAGS
CMAKE_C_FLAGS
CMAKE_CXX_FLAGS
autocmake.yml configuration:
docopt: "--mpi Enable MPI parallelization [default: False]."
define: "'-DENABLE_MPI={0}'.format(arguments['--mpi'])"
omp.cmake¶
Enables OpenMP support.
Variables used:
ENABLE_OPENMP
OPENMP_FOUND
Variables modified (provided the corresponding language is enabled):
CMAKE_Fortran_FLAGS
CMAKE_C_FLAGS
CMAKE_CXX_FLAGS
autocmake.yml configuration:
docopt: "--omp Enable OpenMP parallelization [default: False]."
define: "'-DENABLE_OPENMP={0}'.format(arguments['--omp'])"
profile.cmake¶
Enable profiling by appending corresponding compiler flags.
Variables modified (provided the corresponding language is enabled):
CMAKE_Fortran_FLAGS
CMAKE_C_FLAGS
CMAKE_CXX_FLAGS
autocmake.yml configuration:
docopt: "--profile Enable profiling [default: False]"
define: "'-DENABLE_PROFILING={0}'.format(arguments['--profile'])"
python_interpreter.cmake¶
Detects Python interpreter.
Variables used:
PYTHON_INTERPRETER - User-set path to the Python interpreter
Variables defined:
PYTHONINTERP_FOUND - Was the Python executable found
PYTHON_EXECUTABLE - path to the Python interpreter
PYTHON_VERSION_STRING - Python version found e.g. 2.5.2
PYTHON_VERSION_MAJOR - Python major version found e.g. 2
PYTHON_VERSION_MINOR - Python minor version found e.g. 5
PYTHON_VERSION_PATCH - Python patch version found e.g. 2
autocmake.yml configuration:
docopt: "--python=<PYTHON_INTERPRETER> The Python interpreter (development version) to use. [default: '']."
define: "'-DPYTHON_INTERPRETER=\"{0}\"'.format(arguments['--python'])"
python_libs.cmake¶
Detects Python libraries and headers. Detection is done basically by hand as the proper CMake package will not find libraries and headers matching the interpreter version.
Dependencies:
python_interpreter - Sets the Python interpreter for headers and libraries detection
Variables used:
PYTHONINTERP_FOUND - Was the Python executable found
Variables defined:
PYTHONLIBS_FOUND - have the Python libs been found
PYTHON_LIBRARIES - path to the python library
PYTHON_INCLUDE_DIRS - path to where Python.h is found
PYTHONLIBS_VERSION_STRING - version of the Python libs found (since CMake 2.8.8)
safeguards.cmake¶
Provides safeguards against in-source builds and bad build types.
Variables used:
PROJECT_SOURCE_DIR
PROJECT_BINARY_DIR
CMAKE_BUILD_TYPE
save_flags.cmake¶
Take care of updating the cache for fresh configurations.
Variables modified (provided the corresponding language is enabled):
DEFAULT_Fortran_FLAGS_SET
DEFAULT_C_FLAGS_SET
DEFAULT_CXX_FLAGS_SET
version.cmake¶
Determine program version from file “VERSION” (example: “14.1”) The reason why this information is stored in a file and not as CMake variable is that CMake-unaware programs can parse and use it (e.g. Sphinx). Also web-based hosting frontends such as GitLab automatically use the file “VERSION” if present.
Variables defined:
PROGRAM_VERSION