Confire¶
Confire is a simple but powerful configuration scheme that builds on the configuration parsers of Scapy, elasticsearch, Django and others. The basic scheme is to have a configuration search path that looks for YAML files in standard locations. The search path is hierarchical (meaning that system configurations are overloaded by user configurations, etc). These YAML files are then added to a default, class-based configuration management scheme that allows for easy development.
Features¶
- Configuration files in YAML
- Hierarchical configuration search
- Class based application defaults
- Settings pulled in from the environment
Setup¶
The easiest and usual way to install confire is to use pip:
pip install confire
To install the package from source, download the latests package tarball, unzip in a temporary directory and run the following command:
python setup.py install
As always, I highly recommend the use of a virtual environment to better manage the software dependencies for your particular code base.
Example Usage¶
Create a file called “myapp.yaml” and place it in one of the following places:
/etc/myapp.yaml
$HOME/.myapp.yaml
conf/myapp.yaml
Create some configuration values inside the file like so:
## Set application environment
debug: True
testing: False
## A simple database configuration
database:
name: mydb
host: localhost
port: 5432
user: postgres
In your code, create a file called “config.py” and add the following:
import os
from confire import Configuration
from confire import environ_setting
class DatabaseConfiguration(Configuration):
host = "localhost"
port = 5432
name = "mydb"
user = "postgres"
password = environ_setting("DATABASE_PASSWORD", required=False)
class MyAppConfiguration(Configuration):
CONF_PATHS = [
'/etc/myapp.yaml',
os.path.expanduser('~/.myapp.yaml'),
os.path.abspath('conf/myapp.yaml')
]
debug = False
testing = True
database = DatabaseConfiguration()
settings = MyAppConfiguration.load()
Now, everywhere in your code that you would like to access these settings values, simply use as follows:
from config import settings
debug = settings.get('DEBUG') or settings['DEBUG']
Voila! A complete configuration system for your application!
Next Topics¶
Here are a list of topics for more detail about how to use confire in your Python applications.
Configuring Apps with Confire¶
Let’s say that you’ve just started a new Python project - you know that this project is going to need access to a Database, possibly require an API key and API Secret, and it will definitely need some sort of debug mode so that developers can figure out what’s going on in production. These types of variables shouldn’t be hardcoded into your application, you’ll want some kind of configuration management system in your app.
So what are your options? Python has a native configuration parser that
handles .ini files similar to what you’d see on Windows machines - it’s
called configparser
, and while it works well (and even has support for
JSON files) - it is extremely basic. As a developer, you not only have to
deal with the .ini syntax, but you also have to look for the file and load
it into the parser. While the parser does handle type conversion, it has
no quick ability to add reasonable defaults. Basically, your configuration
ends up being defined by the .ini file - and this is not good, especially
if your users forget to change a particular value or leave one out all
together!
If you look at Django, they have their settings in a Python file, and the settings are Python code. This is great because now you can use any Python type as a setting. There are even reasonable defaults and some fancy import logic helps get the settings where they need to be. The problem is that you have to import that file, it has to be on your python path - so no storage of a settings file in /etc or any other reasonable place. It also means that developers have to configure Django- not just users of the app.
So we want the following things in our configuration:
- Reasonable lookup locations for config files
- Configuration type parsing from a text file
- An API that supports reasonable defaults and in-code usage
- A text based configuration that is for users not developers
This is where confire comes in. Confire uses YAML as the configuration language of choice. This is a markup format that has rich types like JSON, but is also very readable. Applications like Elasticsearch, Ruby on Rails, Travis-CI and others make use of YAML, so it’s probably already familiar to you.
Confire has a hierarchical lookup system that means it looks in the system
configuration (/etc
in *nix systems), then in a user specific place,
then in a local directory. At each level, the configuration overrides the
defaults from the other levels. Configurations are then supplied to the
developer in a friendly, Django-like way.
Project Setup¶
In your projects folder, create your app folder, let’s call it “myapp”. Then create the Python project skeleton as you would normally do, but also include a configuration directory, “conf”.
$ mkdir myapp
$ cd myapp
$ mkdirs bin tests conf docs fixtures myapp
$ touch tests/__init__.py
$ touch myapp/__init__.py
$ touch setup.py
$ cd docs
$ sphinx-quickstart
...
$ cd ..
Hopefull this is very familiar to those who develop on Linux or Mac and set up Python projects regularly. Now, assuming you’re using Git as well as virtualenv and virtualenv wrapper - let’s get our repository and env going:
$ git init
$ mkvirtualenv -a $(pwd) myapp
(myapp)$ pip install confire
(myapp)$ pip install nose
(myapp)$ pip freeze > requirements.txt
Perfect! You’re now ready to get developing your Python app. Let’s start
by getting our configuration going. Create a file called myapp.yaml
in
the conf
directory. It may be helpful to add this file to your
.gitignore
so that you don’t accidentally commit a private variable
publically. Also create a file called config.py
in your myapp
module.
Inside the myapp.yaml
file place the following, very simple code.
debug: false
testing: false
And then inside your config.py
, place the following code.
import os
from confire import Configuration
class MyAppConfiguration(Configuration):
CONF_PATHS = [
"/etc/myapp.yaml", # System configuration
os.path.expandvars("$HOME/.myapp.yaml"), # User specific configuration
os.path.abspath("conf/myapp.yaml"), # Development configuration
]
debug = True
testing = True
## Load settings immediately for import
settings = MyAppConfiguration.load()
if __name__ == "__main__":
print settings
That’s it, you now have a complete configuration system for your app! Let’s
walk through this code. Confire provides a class-based configuration API,
meaning that you simply create configuration classes and then defeine your
defaults on them at the class level (kind of like you might use Django
class-based views). Configuration classes must all extend the
confire.Configuration
base class.
Note
All configurations should be lowercase properties!
Configurations are case insensitive, but to achieve this,
the __getitem__
method lowercases all accessors!
The CONF_PATHS
class variable tells the configuration where to look
for YAML files to load. In this case, we specify three lookups that happen
in the order they’re specified - first the system, then the user directory,
then the local directory for development. You’ll notice that if the config
file is missing, no exceptions are raised.
Using Configurations in Code¶
The loaded settings immediately for import means that elsewhere in your code, all you have to do is use the following to get access to your config:
from myapp.config import settings
if settings.get("DEBUG"):
...
else:
...
Because your API has already specified reasonable defaults, you don’t have to worry about configurations being missing or unavailable!
A couple notes on using the settings in your code:
- The settings are not case sensitive, DEBUG is the same as debug. However, all properties should be stored as lowercase in the configuration subclass.
- You can access settings like so:
settings["mysetting"]
, however this will raise an exception if the setting is not available (something that really shouldn’t happen). - You can also access the settings through the get method:
settings.get("mysetting", "foo")
, which will not raise an exception on a missing setting, but instead return the supplied default orNone
. - You can also access the settings using a dot accessor method:
settings.mysetting
, which fetches the properties off the class. - Settings can be modified at runtime, but this is not recommended.
As you continue to develop, you can add settings to your config.py
as
well as your myapp.yaml
, your app development is now much smoother!
Environment Variables¶
Sometimes you don’t want your configurations to reside inside of a YAML file, saved on disk, usually when you have a secret key or a database password. Other times you don’t have access to your server’s disk, but can add ENVIRONMENTAL VARIABLES as with a hosting service like Heroku.
Confire makes it easy to specify variables that you expect to be in the
environment, using the environ_setting
function, which you can import
from the main module.
from confire import Configuration, environ_setting
class MyConfiguration(Configuration):
supersecret = environ_setting("SUPER_SECRET", None, required=True)
The function expects as a first argument, the name of the environment variable, usually an all caps, underscore separated name. You can also give a default value (in case no variable exists in the environment) as the second argument.
When the environment is initialized (not loaded) it will immediately look in the environment for the setting and store it as the default. Any settings that are in the YAML search paths will override the environment variable, so make sure that you leave ENV_VARS out of the YAML configs!
The behavior of the function depends on how it’s called, in terms of using the default and fetching from the environment:
- If it is required and the default is None, raise ImproperlyConfigured
- If it is requried and a default exists, return default
- If it is not required and default is None, return None
- If it is not required and default exists, return default
Environmental variables are usually required, hence the exception.
Note also that you can use confire exceptions and warnings in your own
code, by importing the ImproperlyConfigured
and MissingConfiguration
exception and warning.
Nested Configurations¶
Configurations are nestable in order to ensure that developers can create easily modular configurations, for example database configuations for a staging and production database or per-app settings. Nested configurations will also be loaded from a single YAML file that expects a similar nesting structure, and the configurations are loaded in a depth-first manner.
To create a nested configuration, you need a main configuration object
that supports the top-level configuration. For each nested configuration,
you simply create new Configuration
subclasses and then add them as
settings to main configuration class.
Here is the example for two different databases:
import confire
class DatabaseConfiguration(confire.Configuration):
name = None
host = "localhost"
port = 5432
user = None
pass = None
class MainConfiguration(confire.Configuration):
staging = DatabaseConfiguration()
production = DatabaseConfiguration()
settings = MainConfiguration.load()
In your YAML file, you can configure each database configuration for its specific environment:
staging:
name: "myapp-staging"
host: "localhost"
port: 5432
user: "test-user"
pass: "password"
production:
name: "myapp-production"
host: "54.21.35.141"
port: 5432
user: "user"
pass: "password"
Access to the configuration is as follows:
from myapp.config import settings
print settings.staging.host
print settings.production.host
Configurations can be nested to any depth, but it is recommended to keep them fairly shallow, to avoid deep accessor chains.
Class Documentation¶
This page documents the contents of the library.
Confire Module¶
A simple app configuration scheme using YAML and class based defaults.
Config Module¶
Confire class for specifying Confire specific optional items via a YAML configuration file format. The main configuration class provides utilities for loading the configuration from disk and iterating across all the settings. Subclasses of the Configuration specify defaults that can be updated via the configuration files.
General usage:
from confire.conf import settings mysetting = settings.get(‘mysetting’, default)
You can also get settings via a dictionary like access:
mysetting = settings[‘mysetting’]
However, this will raise an exception if the setting is not found.
Note: Keys are CASE insensitive
Note: Settings can be modified directly by settings.mysetting = newsetting however, this is not recommended, and settings should be fetched via the dictionary-like access.
-
class
confire.config.
Configuration
[source]¶ Base configuration class specifies how configurations should be handled and provides helper methods for iterating through options and configuring the base class.
Subclasses should provide defaults for the various configurations as directly set class level properties. Note, however, that ANY directive set in a configuration file (whether or not it has a default) will be added to the configuration.
Example:
class MyConfig(Configuration):
mysetting = True logpath = “/var/log/myapp.log” appname = “MyApp”The configuration is then loaded via the classmethod load:
settings = MyConfig.load()Access to properties is done two ways:
settings[‘mysetting’] settings.get(‘mysetting’, True)Note: None settings are not allowed!
-
configure
(conf={})[source]¶ Allows updating of the configuration via a dictionary of configuration terms or a configuration object. Generally speaking, this method is utilized to configure the object from a JSON or YAML parsing.
-
get
(key, default=None)[source]¶ Fetches a key from the configuration without raising a KeyError exception if the key doesn’t exist in the config, instead it returns the default (None).
-
-
confire.config.
environ_setting
(name, default=None, required=True)[source]¶ Fetch setting from the environment. The bahavior of the setting if it is not in environment is as follows:
- If it is required and the default is None, raise Exception
- If it is requried and a default exists, return default
- If it is not required and default is None, return None
- If it is not required and default exists, return default
Exceptions¶
Exceptions hierarchy for Confire
Changelog¶
The release versions that are sent to the Python package index are also tagged in Github. You can see the tags through the Github web application and download the tarball of the version you’d like. Additionally PyPI will host the various releases of confire.
The versioning uses a three part version system, “a.b.c” - “a” represents a major release that may not be backwards compatible. “b” is incremented on minor releases that may contain extra features, but are backwards compatible. “c” releases are bugfixes or other micro changes that developers should feel free to immediately update to.
Contributors¶
I’d like to personally thank the following people for contributing to confire and making it a success!
Versions¶
The following lists the various versions of confire and important details about them.
v0.2.0¶
- tag: v0.2.0
- deployment: July 31, 2014
- commit: (latest)
This release added some new features including support for environmental variables as settings defaults, ConfigurationMissing Warnings and ImproperlyConfigured errors that you can raise in your own code to warn developers about the state of configuration.
This release also greatly increased the amount of available documentation for Confire.
v0.1.1¶
- tag: v0.1.1
- deployment: July 24, 2014
- commit: bdc0488
Added Python 3.3 support thanks to @tyrannosaurus who contributed to the changes that would ensure this support for the future. I also added Python 3.3 travis testing and some other minor changes.
v0.1.0¶
- tag: v0.1.0
- deployment: July 20, 2014
- comit: 213aa5e
Initial deployment of the confire library.
About¶
There are many configuration packages available on PyPI - it seems that everyone has a different way of doing it. However, this is my prefered way, and I found that after I copy and pasted this code into more than 3 projects that it was time to add it as a dependency via PyPI. The configuration builds on what I’ve learned/done in configuring Scapy, elasticsearch, and Django - and builds on these principles:
- Configuration should not be Python (sorry Django). It’s too easy to screw stuff up, and anyway, you don’t want to deal with importing a settings file from
/etc
! - Configuration should be on a per-system basis. This means that there should be an
/etc/app.yaml
configuration file as well as a$HOME/.app.yaml
configuration file that overwrites the system defaults for a particular user. For development purposes there should also be a$(pwd)/app.yaml
file so that you don’t have to sprinkle things throughout the system if not needed. - Developers should be able to have reasonable defaults already written in code if no YAML file has been provided. These defaults should be added in an API like way that is class based and modularized.
- Accessing settings from the code should be easy.
So there you have it, with these things in mind I wrote confire and I hope you enjoy it!
Contributing¶
Confire is open source, and I would be happy to have you contribute! You can contribute in the following ways:
- Create a pull request in Github: https://github.com/bbengfort/confire
- Add issues or bugs on the bugtracker: https://github.com/bbengfort/confire/issues
- Checkout the current dev board on waffle.io: https://waffle.io/bbengfort/confire
You can contact me on Twitter if needed: @bbengfort
Name Origin¶
con · fit/kôNˈfē/
noun duck or other meat cooked slowly in its own fat.
Origin
[French] confire: to prepare
Also refers to the culinary art of pickling
I like cooking, and the thought of preparation in French culinary language appealed to me. The way I got here was to simply change the “g” in config to a “t”. A definition lookup and boom, a name!