gdeploy user guide

gdeploy is an Ansible based deployment tool. Initially gdeploy was written to install GlusterFS clusters, eventually it grew out to do lot of other things. On a given set of hosts, gdeploy can create physical volumes, volume groups, and logical volumes, install packages, subscribe to RHN channels, run shell commands, create GlusterFS volumes and lot more.

Contents

Installation

Prerequisites

gdeploy requires the following packages:

  • Python 2.x
  • Ansible >= 1.9.x
  • python-argparse
  • PyYAML
  • Jinja2

Installing Ansible

Follow instructions in the Ansible documentation on how to install Ansible, which can be found here.

Installing gdeploy

gdeploy can be installed using pre-built RPM or can be installed from source.

Installing from RPM

Latest version of gdeploy RPMs can be downloaded from here and installed

Using yum:
$ sudo yum install ./gdeploy-<version>-<release>.rpm

Using dnf:
$ sudo dnf install ./gdeploy-<version>-<release>.rpm
Installing from source

Alternatively gdeploy can be installed from source

$ git clone git@github.com:gluster/gdeploy.git
$ cd gdeploy

Make sure you have gcc and python-devel installed

$ sudo yum install gcc python-devel redhat-rpm-config
$ sudo pip install -r requirements.txt

Setup gdeploy

Run the gdeploy_setup.sh file from the root directory of gdeploy

$ cd gdeploy
$ sudo ./gdeploy_setup.sh

OR Setup manually as follows

  1. Add ansible modules to ANSIBLE_LIBRARY environment variable

    $ echo "export ANSIBLE_LIBRARY=$ANSIBLE_LIBRARY:path/to/gdeploy/modules/" >> ~/.bashrc
    

‘path/to’ will be replaced by the path on your system on which gdeploy is installed.

  1. Add ansible playbooks(inside the templates directory) to GDEPLOY_TEMPLATES environment variable

    $ echo "export GDEPLOY_TEMPLATES='path/to/gdeploy'" >> ~/.bashrc
    

‘path/to’ will be replaced by the path on your system on which gdeploy is installed.

$ source ~/.bashrc
  1. Install gdeploy using setuptools

    $ sudo python setup.py install
    

Getting Started

gdeploy works by interacting with the remote nodes by communicating via passwordless ssh connections. Passwordless ssh connections have to be created to all the nodes on which gdeploy is going to create and configure a Gluster volume.

To setup passwordless ssh to the nodes, follow the steps below:

  • Generate passphrase-less ssh-keys for the nodes which are to be used with gdeploy, running the following command:

    $ ssh-keygen -t rsa -N ''
    
  • Set up passwordless ssh access between the node running gdeploy and servers by running the following command:

    $ ssh-copy-id root@hostname
    

‘hostname’ refers to the unique IP address of the node.

You would have to run these commands for all the nodes.

Sometimes, you may encounter a “Connection Refused” error. In this case, you need to check whether the ssh service is running on your system. You may use this command to check the same:

$ systemctl status sshd.service

If the service is not running, use this command to start the service:

$ systemctl start sshd.service

Once ssh connections to all the nodes are established, we can start writing a configuration file.

That’s it! Now the machines are ready to be used with gdeploy.

Invoking gdeploy

gdeploy needs a configuration file to run. Write a configuration file, see Writing configuration file for gdeploy section below for more details.

Invoke gdeploy with configuration file as an argument:

$ gdeploy -c sample.conf

Writing configuration file for gdeploy

gdeploy configuration file is a plain text file comprising multiple sections, the sections are arranged in an order based on what needs to be achieved.

A very simple configuration file named enable-ntpd.conf which enables and starts ntpd on all the nodes looks like:

[hosts]
10.0.0.1
10.0.0.2
10.0.0.3

[service1]
action=enable
service=ntpd

[service2]
action=start
service=ntpd

Invoking gdeploy

Invoke gdeploy with configuration file as an argument:

$ gdeploy -c sample.conf

The configuration file given above will enable and start ntpd on three nodes. 10.0.0.1, 10.0.0.2, and 10.0.0.3 when the following command is invoked:

$ gdeploy -c enable-ntpd.conf

INFO: The ‘ntp’ package has to be installed on both the nodes in order for this configuration file to run. This can be done using the command “dnf install ntp”.

For more details on the list of all the features supported by gdeploy, refer gdeploy features topic.

Usage

gdeploy needs a configuration file to do anything useful. Refer Configuration file format for an example.

gdeploy -h will list the available options for gdeploy:

$ gdeploy -h
usage: gdeploy [-h] [-v] [-vv] [-c CONFIG_FILE] [-k]

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  -c CONFIG_FILE        Configuration file
  -k                    Keep the generated ansible utility files
  --trace               Turn on the trace messages in logs
  -vv                   verbose mode
  --addfeature FEATURE_NAME
                        Add new feature to gdeploy

gdeploy –addfeature FEATURE_NAME will create a skeleton to add a new feture to gdeploy. For more details on how to write a feature refer Developer Documentation.

Invoke gdeploy with configuration file as an argument:

$ gdeploy -c config-file

More example configuration files can be found here.

Debugging

Configuration file format

This section explains the gdeploy configuration file format. There is no rule to name the configuration files in gdeploy as long as the file name is mentioned with -c option for gdeploy. For example gluster.conf, gluster, gluster.txt, gluster.config are all valid names for a gdeploy configuration file.

gdeploy configuration file is split into two parts.

  1. Inventory
  2. Features

1. Inventory

The section is named [hosts], this is a mandatory section, hosts that are to be configured have to be listed in this section.

Hostnames or ip addresses have to be listed one per line. For example:

[hosts]
10.0.0.1
10.0.0.2
10.0.0.3

2. Features

There can be more than one feature listed in the configuration file, each separated by a newline. Every feature section has one or more variables which controls how the feature is configured/deployed. The below example has two features, firewalld and service that will be configured on all the hosts listed in the [hosts] section:

[hosts]
10.0.0.1
10.0.0.2
10.0.0.3
10.0.0.4

[firewalld]
action=add
ports=111/tcp,2049/tcp,54321/tcp
permanent=true
zone=public

[service1]
action=enable
service=ntpd

[service2]
action=restart
service=ntpd

If a feature has to be used more than once, then it has to be in different sections and numbered to make it unique as shown in the above example.

The list of available features and their complete documentation can be found in Features page.

Features

gdeploy supports a number of features, each feature does a particular task. For example setting up lvm and creating filesystem, subscribe to Red Hat Subscription Network, creating GlusterFS volumes, so on.

Each feature provides variables that can be set to fine tune the system. This document explains the available features and their tunable variables.

  1. Clients
  2. firewalld
  3. LVM
  4. Configuring NFS Ganesha
  5. Peer
  6. quota
  7. Setting up Samba and CTDB
  8. script
  9. Service
  10. shell
  11. Snapshot
  12. Enable SSL on a volume
  13. Subscription Manager
  14. systemd
  15. Updating a file
  16. Creating GlusterFS Volumes
  17. yum

Maintainer

Developer Documentation

With gdeploy, the user get to remotely configure a lot of features in multiple machines, without worrying dealing with the complexities of writing Ansible Playbooks, modules, etc. It is the duty of the developer to worry about all these things. Adding a new feature to gdeploy is relatively easy, but comes with the cost of writing playbooks and modules(if necessary) for Ansible. So this guide assumes that the developer is comfortable with Python and has got the basic working knowledge of Ansible.

gdeploy - Developer setup

To setup the development environment, refer Installing from source.

Proceed further once you are done with the setup.

gdeploy architecture

gdeploy is a very lightweight, simple tool that efficiently make use of Ansible, hiding the major complexities of it from the user, that does numerous operations on remote machines sequentially. To do this, gdeploy divides the logic to 3 parts:

  1. Ansible module that implements the desired logic to be executed in the remote machine.

  2. Ansible playbook that makes use of the already present Ansible module and specifies how a particular operation is to be performed.

  3. A section independent from both the above sections that reads the user configuration file, parses it accordingly, sets default values for some if not provided by the user, and then populate all these data in variables that the Ansible playbook will then use. More about these variable can be found in the Ansible documentation about them

    For this, gdeploy provide what we call the sections or features(Features). Each feature will have n-number of options for which the user will specify the value.

Adding your first feature to gdeploy

In order to add a new feature to Ansible, the developer has to make sure the three above mentioned components or sections are written properly and works as intended.

The development process can be started in the following order:

1. Write the Ansible module, if needed.
2. Write Ansible playbooks that does the necessary operations in the
   right order.
3. Add a feature with a suitable name matching your requirement to the
   gdeploy framework.

** Add a feature to gdeploy **

Use the gdeploy option –addfeature for this. To add a feature names “myfeature”:

$ gdeploy –addfeature myfeature

This will create a directory <l>myfeature</l> under <l>features</l> directory. If you look inside this directory, you will see that gdeploy has created 3 files for you: an init file, a JSON file and a python script. You need just edit the JSON file and the python script to make gdeploy do what you want. The JSON file is used by gdeploy to validate the user configuration provided for your feature(in this case, ‘myfeature’). The necessary option for every feature in gdeploy is the option named ‘action’. Specify each one of your feature’s action inside the action hash in the JSON file. Each of these action keys will have a list names ‘options’ which will specify the options that is to be provided corresponding to each of these actions. This is the JSON file written to implement the snapshot feature in gdeploy.

Once your JSON is ready, the next big task is to create playbooks to run for each of these actions. This is where we cannot help you much. Writing playbooks and modules depends on your feature. So put your Python and Ansible skills to good use and write some cool playbooks. Playbooks should go under the directory <l>playbooks</l> under the working directory and modules should go under the directory <l>modules</l> under the working directory. Once your playbooks are in, add these playbook file names to the file defaults.py, just because it is cleaner.

Now you just have to let gdeploy know which playbook corresponds to which feature action. This is where the python script comes into picture. There should be a function corresponding to each feature action inside this Python script. The function name should be in the format ‘myfeature_actionname’. You need just edit the dummy method names provided inside the script. I am sure you will figure it out. It is pretty straight forward. As you will see inside the scripts, each function will have a parameter being passed, ‘section_dict’. This dictionary holds the as keys and values, the options and their corresponding values provided by the user in her configuration file under the section ‘myfeature’. Just print it out and see for yourself if you are happy with the format. You can modify the keys and values in the dictionary as per your needs. Each function should return 2 parameters: One is the modified or not modified section_dict and other is the playbook to be run for that particular section. Just edit the YML_NAME and let the defaults be.

Testing your code

Guidelines for contribution

Design

Testsuite

Frequently Asked Questions

  1. Why do we need gdeploy, when Ansible is available?
  2. How does gdeploy help in setting up GlusterFS clusters?
  3. Does gdeploy help in installing GlusterFS packages?
  4. Is gdeploy only for installing and deploying GlusterFS?
  5. Can I run arbitrary scripts using gdeploy?
  6. My gdeploy run is failing with Module Error, why?

Why do we need gdeploy, when Ansible is available?

gdeploy enables configuration and provisioning of GlusterFS and the file access protocols using configurations and tunables which are tested and recommended by the maintainers. This enables a system administrator to have an easy way to create consistent and repeatable deployment paths.

How does gdeploy help in setting up GlusterFS clusters?

Installation, configuration and provisioning of a GlusterFS deployment involves a sequence of steps to be executed in the proper order. This would include deployment-specific detail such as:

  1. Setting up PV, VG, LV (thinpools if necessary).
  2. Peer probing the nodes.
  3. Using the CLI based volume creation steps

gdeploy provides a simple way to complete the steps and include specifics such as configuring volume options and such.

Does gdeploy help in installing GlusterFS packages?

gdeploy has a configuration workflow design which enables it to be used for package installation, either from upstream builds or, from a specific vendor provided content distribution mechanism viz. Red Hat’s CDN

Is gdeploy only for installing and deploying GlusterFS?

While gdeploy is intended to streamline the administrator experience during installation and deployment of GlusterFS, it can be used to install other packages, custom scripts and modules for configuration. The hc.conf is an example of how gdeploy can enable the set-up and configuration for a HyperConverged stack using GlusterFS.

Refer hc.conf for an example for things gdeploy can achieve.

Can I run arbitrary scripts using gdeploy?

Yes. Scripts which aid and extend the deployment setup can be configured to run from gdeploy.

See the script module. Refer hc.conf for an example for script module usage.

My gdeploy run is failing with Module Error, why?

The error is due to the Ansible version installed. This is possibly because you might be using Ansible 2.0. gdeploy currently supports 1.9.x versions of Ansible.

Examples

Using gdeploy to create a 1x3 Gluster Volume

To create 1*3 gluster volume we would need three bricks which may or may not be on the same machine. It is recommended that these three bricks reside on different machines.

Step 1:

Create the following configuration file:

[hosts]
10.70.43.127
10.70.42.190
10.70.42.232

[backend-setup]
devices=/dev/vdb
vgs=1_3_gluster
pools=pool1
lvs=lv2
mountpoints=/mnt/data1
brick_dirs=/mnt/data1/1

[peer]
manage=probe

[volume]
action=create
volname=volume1
replica=yes
replica_count=3
force=yes


[clients]
action=mount
volname=volume1
hosts=192.168.122.19
fstype=glusterfs
client_mount_points=/mnt/client_mount

Step 2:

Save the file by giving it some name e.g. ‘1_3_volume.conf’. Invoke gdeploy and run the file using:

$gdeploy -c 1_3_volume.conf

Step 3:

Check whether a gluster volume has been created by running the below command:

$gluster vol info
_images/volume_info.png

Step 4:

Now you can start writing to the volume using your client machine (192.168.122.19 in our case) by traversing to the path you have mentioned under “client_mount” using the following command:

$ sudo touch f1 f2 f3

This command will create three files under the directory /mnt/client_mount

You can also check whether the files have been created and replicated thrice inside the directory /mnt/data1/1 on the remote nodes by running the command:

$ ls
_images/1x3_vol.png

We can see that the files have been successfully replicated on all the three nodes.

Using gdeploy to create 2x2 gluster volume

To create 2x2 gluster volume, you would need four bricks which may or may not be on the same machine. It is recommended that that these four bricks reside on different machines.

Step 1:

Create the following configuration file:

[hosts]
10.70.43.127
10.70.42.190
10.70.42.232
10.70.43.67

[backend-setup]
devices=/dev/vdb
mountpoints=/gluster/brick1
brick_dirs=/gluster/brick1/one

[volume]
action=create
volname=sample_volume
replica=yes
replica_count=2
force=yes

[clients]
action=mount
hosts=192.168.122.19
fstype=glusterfs
client_mount_points=/mnt/random_client

Step 2:

Save the file by giving it some name e.g. ‘2x2-gluster-volume.conf’. Invoke gdeploy and run the file using:

$ gdeploy -c 2x2-gluster-volume.conf

Step 3:

To check whether a gluster volume has been created by running the below command:

$ gluster vol info
_images/2x2_volume.png

Step 3:

Now you can start writing to the volume using your client machine (192.168.122.19 in our case) by traversing to the path you have mentioned under “random_client” using the following command:

$ sudo touch 1 2 3 4 5

This command will create five files under the directory /home/poo/random_client.

You can also check whether the files have been created and replicated thrice inside the directory /gluster/brick1/one on the remote nodes by running the command:

$ ls
_images/writing_to_gluster.png

We can see that the files have been successfully replicated on all the four nodes.

Write a config file to do the backend setup

[backend-setup] section is used configure the disks on all the hosts mentioned in the [hosts] section. If the disks names varies from host to host then [backend-setup:<hostname>/<ip>] can be used to do setup backend on the particular host.

Step 1:

Create an empty file and give it any arbitrary name and add the following lines to it:

# This is a mandatory section, and hostnames/ip-address are listed one per line.

[hosts]
10.70.43.127
10.70.42.190
10.70.42.232
10.70.43.67

# Backend setup for all the hosts listed inside [hosts] section

[backend-setup]
devices=/dev/vdb
mountpoints=/gluster/brick1
brick_dirs=/gluster/brick1/one

# Backend setup for 10.70.46.77 with default gdeploy generated names for
# Volume Groups and Logical Volumes. Volume names will be GLUSTER_vg1,
# GLUSTER_vg2...
#
# [backend-setup:10.70.43.127]
# devices=vdb

# Backend setup for remaining 3 hosts in the `hosts' section with custom names
# for Volumes Groups and Logical Volumes.
#
# [backend-setup:10.70.46.{130,32,110}]
# devices=vdb,vdc,vdd
# vgs=vg1,vg2,vg3
# pools=pool1,pool2,pool3
# lvs=lv1,lv2,lv3
# mountpoints=/mnt/data1,/mnt/data2,/mnt/data3
# brick_dirs=/mnt/data1/1,/mnt/data2/2,/mnt/data3/3

Step 2:

Invoke gdeploy and run the file using:

$ gdeploy -c backend-setup.conf

Step 3:

To see if the GLUSTER_vg1 (which is the default name for gluster volume group) has been mounted on the desired directory or not. You can either run mount or df -h:

$ mount

You’ll see something like this.

_images/backend-setup.png

To see volume groups a.k.a vgs, run the following command:

$ vgs
_images/vgs.png

To check physical volume, run the following command:

$ pvs
_images/pvs.png

And to check logical volume, run the following command:

$ lvs
_images/lvs.png

We can see that volume groups and logical volumes has been successfully created.

How to start and stop services

‘service’ section lets you start, stop, enable, disable, restart, or reload services. Multiple service names can be provided as a comma separated list as shown in configuration file below.

For a hands-on walkthrough of the process, let us take a look at how we can start services such as glusterd, httpd, ntp through the config file.

Here, we’ll learn how to start services such as glusterd, httpd, ntp through config file.

Step 1:

Create an empty file and give it any arbitrary name. For the purpose of this demonstration, let’s call our file starting_services.conf. Add the following lines to your newly created config file:

# This is a mandatory section, and hostnames/ip-address are listed one per line.
# IP address for host shown here is for demonstration, don't forget to change it
# to a valid IP address in your network.

[hosts]
10.209.69.106

# To start services

[service]
action=start
service=glusterd,httpd

# To stop services
#
# [service]
# action=stop
# service=glusterd,httpd

# To disable services
#
# [service]
# action=disable
# service=glusterd,httpd

# To restart services
#
# [service]
# action=restart
# service=glusterd,httpd

# To reload services
#
# [service]
# action=reload
# servcie=glusterd,httpd

Step 2:

Invoke gdeploy and run the file using:

gdeploy -c starting_services.conf

Step 3:

To check the status of glusterd service, run the following command:

$ systemctl status glusterd.service
_images/status_glusterd.png

To check the status of httpd service, run the following command:

$ systemctl status httpd.service
_images/status_httpd.png

As we can see the gluster and httpd services has been started.

Set/unset volume options on an existing volume

Often, we are required to customize our volume for different use cases. Setting those options enhances performance and prepares the volume for our desired task. In order to do so, we set different options on the volume using the ‘key’ parameter in our configuration file.

This tutorial will take you through just that. It’s intended to show you how you can set different volume options on an existing Gluster volume. (To create a Gluster volume, please refer to 1x3-gluster-volume or 2x2-gluster-volume).

Setting the options on an existing volume

Step 1:

Create an empty file and give it any arbitrary name. For the purpose of this demonstration, let’s call our file set_options_vol.conf. Add the following lines to your newly created config file:

# The config file sets configuration options for the already existing volume.

# A volume can be created and its volume options can be set at the time of creation.

# The volume section takes key value pairs. The number of keys should match
# the number of values.

# 'action' option specifies what action id to be performed in the volume.
# The choices are: [create, delete, add-brick, remove-brick, rebalance,
# set].

[volume]
action=set
volname=10.70.42.190:sample_volume
key=cluster.nufa,performance.cache-size,cluster.data-self-heal-algorithm
value=on,256MB,full

Step 2:

Invoke gdeploy and run the file using:

$ gdeploy -c set_options_vol.conf
_images/running_gdeploy.png

Step 3:

To verify if these options are indeed set on the volume, run the following command:

$ gluster vol info
_images/verify_volume_options.png

We can see that those options have been set on the volume.

Resetting the options on the existing volume

What if you want to reset or unset some options that you no longer require? There is no unset option per se, but you can set them back to different values to unset them. This configuration will show you how to do it. Change the values in the value parameter and the rest of it will be taken care of by gdeploy based on the config file.

Step 1:

Let’s call your reset configuration file reset_options_vol.conf. Add the following lines to reset _options_vol.conf:

# This config resets options for the volume

[volume]
action=set
volname=10.70.42.190:sample_volume
key=cluster.nufa,features.lock-heal
value=off,off

Step 2:

Invoke gdeploy and run the following command:

$ gdeploy -c reset_options_vol.conf

Step 3:

To verify if options have been reset, run the following command:

$ gluster vol info

You’ll see that the desired settings have been applied on the volume.

Installing packages from yum repositories

yum section allows you to install or remove packages using yum package manager.

Note :

Make sure that your system is registered with subscription manager before trying to install packages otherwise you’ll get an error while following the steps below.

Step 1:

Create an empty file and give it any arbitrary name. For the purpose of this demonstration, let’s call our file install_packages.conf. Add the following lines to your newly created config file:

# To install package(s):

# Make sure you have the appropriate values in all the placeholders shown in this configuration file.
# These values are just for demonstration purposes.

 [yum]
 action=install
 repos=<reponames>
 packages=vi,glusterfs
 gpgcheck=no
 update=no

# Explanation of the above parameters

# packages
# --------

# This takes a comma separate list of values that are packages names you
# wish to install.

# gpgcheck
# --------

# gpgcheck is set to `yes` by default. You can override it
# by setting it to `no` as illustrated above.

# update
# -------

# By default,  gdeploy runs `yum update` before installation. To disable
# this behaviour, set update=no as shown above. The default value is `yes`.

# To remove package(s):
# [yum]
# action=remove
# packages=vi

Step 2:

As always, to invoke gdeploy run the following command:

$ gdeploy -c install_packages.conf

How to disable repos

To disable enabled repos, use the action ‘disable-repos’. The required repos should be passed as value to repos option.

NOTE : If repos are not provided all the enabled repos will be disabled.

Step 1:

Create an empty file and give it any arbitrary name. For the purpose of this demonstration, let’s call our file disable.conf. Add the following lines to your newly created config file:

[RH-subscription]
action=disable-repos
repos=fancy_repo1,fancy,repo2

Step 2:

Invoke gdeploy and run the following command:

$ gdeploy -c disable.conf

Quota setup on an existing volume

Here, we will be setting up quota on an existing volume.

Step 1:

Create an empty ‘.conf’ file e.g. ‘quota.conf’ and add the following to it:

#
# Usage:
#       gdeploy -c quota.conf
#
# This config enables and sets up quota limit for the specified volume
#

[quota]
action=enable
volname=10.70.41.236:1x2_vol

#You can skip the above if quota is already enabled on the volume

[quota]
action=limit-usage
volname=10.70.41.236:1x2_vol
path=/
size=20MB

‘1x2_vol’ is the name of our volume and 10.70.41.236 is one of the hosts / nodes in the cluster.

Step 2:

Run this file using the following command:

$gdeploy -c quota.conf
_images/12.png

Step 3:

You can check whether quota has been set using the command:

$gluster vol quota 1x2_vol list

This command should be run on the machine where the volume exists. ‘1x2_vol’ is the name of our volume.

_images/22.png

Enabling and disabling quota

This document is intended to demonstrate how one can enable and disable quota on a Gluster volume.

Step 1:

Create a ‘.conf’ file with the following contents:

[quota]
action=enable
volname=10.70.41.236:1x2_vol

Here, ‘1x2_vol’ is the name of our volume and 10.70.41.236 is one of the hosts / nodes in the cluster.

We’ll name this file ‘enable_quota.conf’.

Step 2:

Run this using:

gdeploy -c enable_quota.conf
_images/1.png

Step 3:

You can check whether quota has been enabled by checking volume info using:

$gluster vol info

This command can be run on any of the machines on which the volume resides.

_images/2.png

We can see that quota has been enabled on this volume.

One may follow the following steps to disable quota on this volume.

Step 1:

Create a ‘.conf’ file with the following contents:

[quota]
action=disable
volname=10.70.41.236:1x2_vol

Here, ‘1x2_vol’ is the name of our volume and 10.70.41.236 is one of the hosts / nodes in the cluster.

We’ll name this file ‘disable_quota.conf’.

Step 2:

Run this using:

gdeploy -c disable_quota.conf
_images/3.png

Step 3:

You can check whether quota has been disabled by checking volume info using:

$gluster vol info

This command can be run on any of the machines on which the volume resides.

_images/4.png

Create a Gluster volume and set up quota

Here, we will see how we can create a 1x2 replica volume and then set a size limit on one of the directories within it. A 1x2 replica volume means that we would need 2 bricks and each file will have 2 replicas, one on each brick.

As a recommended practice, our bricks reside on separate machines. We have used two VMs for our two bricks in our case, and these have IPs 10.70.41.236 and 10.70.42.253.

Step 1:

Create an empty ‘.conf’ file e.g. ‘1x2volume.conf’ on the machine where you have gdeploy installed and add the following lines to it:

# This does backend setup first and then creates the volume using the
# setup bricks.


[hosts]
10.70.41.236
10.70.42.253

# Common backend setup for 2 of the hosts.
[backend-setup]
devices=vda
mountpoints=/mnt/data
brick_dirs=/mnt/data/1

# If backend-setup is different for each host
# [backend-setup:192.168.122.109]
# devices=sdb
# brick_dirs=/gluster/brick/brick1
#
# [backend-setup:192.168.122.227]
# devices=sda,sdb,sdc
# brick_dirs=/gluster/brick/brick{1,2,3}
#
[peer]
manage=probe

[volume]
action=create
volname=1x2_vol
replica=yes
replica_count=2
force=yes

[clients]
action=mount
volname=1x2_vol
hosts=10.70.41.236
fstype=glusterfs
client_mount_points=/glusterfs

#Enabling quota for this volume
[quota]
action=enable
volname=10.70.41.236:1x2_vol

# This will set up a quota limit for the specified path on the volume
[quota]
action=limit-usage
volname=10.70.41.236:1x2_vol
path=/sample_directory
size=10MB

path refers to a directory on the client mount point i.e. a directory inside /glusterfs in our case. We created a directory named “sample_directory” inside “/glusterfs” on our client machine 10.70.41.236 using the “mkdir” command. We are setting a size limit on this directory.

Step 2:

Invoke gdeploy and run the file using:

$gdeploy -c 1x2volume.conf

‘1x2volume.conf’ is the name of our configuration file.

_images/11.png

_images/21.png

_images/31.png

_images/41.png

_images/5.png

_images/6.png

Step 3:

You can check whether a gluster volume is created by running the following command on any or all of the nodes:

$gluster vol info
_images/7.png

Here, we can also see that quota has been enabled.

Step 4:

Let’s check whether the size limit for our directory “sample_directory” has been set. One can check quota attributes on a volume using the command:

$gluster vol quota 1x2_vol list

Here, 1x2_vol is the name of our volume.

_images/8.png

Step 5:

You can test the volume by creating a file and see whether it is getting replicated. On your client machine (10.70.41.236 in our case), traverse to the path you have mentioned under “client_mount_points” (e.g. ‘cd /glusterfs’) and create a file using the following command:

$touch sample.txt

This command will create a file named as “sample.txt” under the directory “/glusterfs”. You may create this file on any of the directories under “/glusterfs”,we have created it in the topmost one.

You can check whether the file has been replicated twice by traversing to the path “/mnt/data1/1” on both the nodes and running the command:

$ls

You will see two copies of your file in total, on the bricks.

You have successfully setup a 1x2 Gluster volume using gdeploy and set a size limit on one of the directories on it.

Set a 5GB limit on a directory using quota

Here, we will see how to set a 5GB limit on a directory within our volume using quota.

Step 1:

Create the following ‘.conf’ file:

#Enabling quota for this volume
[quota]
action=enable
volname=10.70.41.236:1x2_vol

# This will set up a quota limit for the specified path on the volume
[quota]
action=limit-usage
volname=10.70.41.236:1x2_vol
path=/main_dir
size=5GB

Step 2:

Run the file using:

$gdeploy -c 5gbquota.conf

Here, ‘5gbquota.conf’ is the name of our configuration file created in Step 1.

_images/13.png

Step 3:

You can check whether quota is enabled on your desired volume by checking volume information:

$gluster vol info

This command needs to be run on the any or all of the machines on which the volume resides.

Step 4:

To check whether the size limit of 5GB has been set, we run the command:

$gluster vol quota 1x2_vol list

This command gives us a detailed description of quota settings applied on our volume.

_images/23.png

Creating a volume and setting a tuning profile on it

This document will walk you through how you can create a Gluster volume and set a profile on it. Profiles are directories of files that contain settings to enhance performance of a volume. There are many profiles that come with Red Hat Gluster Storage and these are tailored for different workloads. One can also define or create a new profile. As profiles aid in performance tuning (improving system performance), they are also called as “tuning profiles”.

Pre-defined profiles can be found here as subdirectories: /etc/tune-profiles.

For instance, /etc/tune-profiles/virtual-guest contains all the files and settings for the virtual-guest profile, which is a profile that sets performance options for virtual machines.

The following steps will illustrate how to create a volume and set a tuning profile on it.

Step 1:

Create the following configuration file:

[hosts]
10.70.41.236
10.70.42.253

# Common backend setup for 2 of the hosts.
[backend-setup]
devices=vda
mountpoints=/mnt/data
brick_dirs=/mnt/data/1

# If backend-setup is different for each host
# [backend-setup:192.168.122.109]
# devices=sdb
# brick_dirs=/gluster/brick/brick1
#
# [backend-setup:192.168.122.227]
# devices=sda,sdb,sdc
# brick_dirs=/gluster/brick/brick{1,2,3}
#
[peer]
manage=probe

[volume]
action=create
volname=1x2_vol
replica=yes
replica_count=2
force=yes

[clients]
action=mount
volname=1x2_vol
hosts=10.70.41.236
fstype=glusterfs
client_mount_points=/glusterfs

#The above section creates the volume. The below section will apply a profile to it.

[tune-profile]
rhgs-sequential-io

#This will set the profile 'rhgs-sequential-io'.

Step 2:

Invoke gdeploy and run this using:

$gdeploy -c tune_profile.conf

where “tune_profile.conf” is the name of our configuration file created in Step 1.

Step 3:

Check whether this has been applied using:

$tuned-adm list

This command, when run on any of the hosts / cluster nodes, will return you the list of available profiles along with the current active profile. In our case, the current active profile would be ‘rhgs-sequential-io’.

Setting a tuning profile on an existing volume

NFS Ganesha setup end-to-end

Here we’ll see how we can setup NFS Ganesha using gdeploy. We’ll be writing a configuration file for the end-to-end setup, right from creating a volume, subscribing to channels and installing the right packages. This configuration file will also create a high availability cluster and export the volume.

Step 1:

Create an empty .conf file with the following:

[hosts]
dhcp37-102.lab.eng.blr.redhat.com
dhcp37-103.lab.eng.blr.redhat.com

[backend-setup]
devices=/dev/vdb
vgs=vg1
pools=pool1
lvs=lv1
mountpoints=/mnt/brick

# Subscribe to necessary channels
[RH-subscription1]
action=register
username=<username>
password=<password>
pool=<pool>

[RH-subscription2]
action=disable-repos
repos=

[RH-subscription3]
action=enable-repos
repos=rhel-7-server-rpms,rh-gluster-3-for-rhel-7-server-rpms,rh-gluster-3-nfs-for-rhel-7-server-rpms,rhel-ha-for-rhel-7-server-rpms

#Installing nfs-ganesha
[yum]
action=install
repolist=
gpgcheck=no
update=no
packages=glusterfs-ganesha

#Enabling the firewall service and configuring necessary ports
[firewalld]
action=add
ports=111/tcp,2049/tcp,54321/tcp,5900/tcp,5900-6923/tcp,5666/tcp,16514/tcp,662/tcp,662/udp
services=glusterfs,nlm,nfs,rpc-bind,high-availability,mountd,rquota

#This will create a volume. Skip this section if your volume already exists
[volume]
action=create
volname=ganesha
transport=tcp
replica_count=2
force=yes

#Creating a high availability cluster and exporting the volume
[nfs-ganesha]
action=create-cluster
ha-name=ganesha-ha-360
cluster-nodes=dhcp37-102.lab.eng.blr.redhat.com,dhcp37-103.lab.eng.blr.redhat.com
vip=10.70.44.121,10.70.44.122
volname=ganesha

Step 2:

Run this file using:

$gdeploy -c nfs_ganesha1.conf

where nfs_ganesha1.conf is the name of our configuration file saved in Step 1.

Step 3:

To see if your volume has been exported, you may run this command on any or all of the nodes:

$showmount -e localhost

Unexporting a volume and destroying an NFS-Ganesha HA Cluster

Here, we’ll see how we can unexport a volume and destroy a high availability cluster.

Step 1:

Create the following configuration file:

[hosts]
dhcp37-102.lab.eng.blr.redhat.com
dhcp37-103.lab.eng.blr.redhat.com

# To un-export the volume:

[nfs-ganesha1]
action=unexport-volume
volname=ganesha

# To destroy the high availability cluster

[nfs-ganesha2]
action=destroy-cluster
cluster-nodes=dhcp37-102.lab.eng.blr.redhat.com,dhcp37-103.lab.eng.blr.redhat.com

‘ganesha’ is the name of our volume.

Step 2:

Run this file using:

$gdeploy -c ganesha_destroy.conf

Here, “ganesha_destroy.con” is the name of our configuration file created in Step 1.

Step 3:

Now, when you run this command on any or all of the nodes in the cluster, you will not see any mounts for nfs-ganesha:

$showmount -e localhost

You have successfully unexported the volume and destroyed the HA cluster.

Indices and tables