CloudSlang Documentation

Contents:

Overview

Contents:

The CloudSlang Project

CloudSlang is a flow-based orchestration tool for managing deployed applications. It allows you to rapidly automate your DevOps and everyday IT operations use cases using ready-made workflows or create custom workflows using a YAML-based DSL.

The CloudSlang project is composed of three main parts: the CloudSlang Orchestration Engine, the CloudSlang language and the ready-made CloudSlang content.

CloudSlang Orchestration Engine

The CloudSlang Orchestration Engine is packaged as a lightweight Java .jar file and can therefore be embedded into existing Java projects.

The engine can support additional workflow languages by adding a compiler that translates the workflow DSL into the engine’s generic workflow execution plans.

CloudSlang Language

The CloudSlang language is a YAML-based DSL for writing workflows. Using CloudSlang you can define a workflow in a structured, easy-to-understand format that can be run by an embedded instance of the CloudSlang Orchestration Engine or the stand-alone CloudSlang CLI.

The CloudSlang language is simple and elegant, yet immensely powerful at the same time.

There are two main types of CloudSlang content, operations and flows. An operation contains an action, which can be written in Python or Java. Operations perform the “work” part of the workflow. A flow contains steps, which stitch together the actions performed by operations, navigating and passing data from one to the other based on operation results and outputs. Flows perform the “flow” part of the workflow.

CloudSlang Ready-Made Content

Although writing your own CloudSlang content is easy, in many cases you don’t even need to write a single line of code to leverage the power of CloudSlang. The CloudSlang team has already written a rich repository of ready-made content to perform common tasks as well as content that integrates with many of today’s hottest technologies, such as Docker and CoreOS. And, the open source nature of the project means that you’ll be able to reuse and repurpose content shared by the community.

CloudSlang Features

CloudSlang and its orchestration engine are:

• Process Based: allowing you to define the ‘how’ and not just the ‘what’ to better control the runtime behavior of your workflows.
• Agentless: there are no agents to set up and manage on all your machines. Instead, workflows use remote APIs to run tasks.
• Scalable: execution logic and distribution are optimized for high throughput and are horizontally scalable.
• Embeddable: the CloudSlang Orchestration Engine is distributed as a standard java library, allowing you to embed it and run CloudSlang from your own applications.
• Content Rich: you can build your own flows, or just use CloudSlang ready-made content.

Get Started

It’s easy to get started running CloudSlang flows, especially using the CLI and ready-made content.

Download, Unzip and Run

• Download the CLI with content zip file.
• Unzip the archive.
• Run the CloudSlang executable in the cslang-cli/bin folder.
• At the prompt enter:

run --f ../content/io/cloudslang/base/print/print_text.sl --i text=Hi

• The CLI will run the ready-made print_text operation that will print the value passed to the variable text to the screen.

Set up python for CloudSlang cli

To use an external python in cloudslang-cli, set the use.jython.expressions property to false in cslang-cli/configuration/cslang.properties.

Important

By default, the property use.jython.expressions=true is set to true pointing to use Jython.

1. Unzip the downloaded python 3.8.7 package to any directory. Notice the python-3.8.7 folder after unzipping.

2. Navigate to cslang-cli/configuration and open the cslang.properties file using any text editor.

3. Specify a property python.path to point it to the following locations based on your Operating System:

   • Windows: <unzipped_location>/python-3.8.7
   • Linux: <unzipped_location>/python-3.8.7/bin

4. Additionally for Linux environment, perform the following steps to grant permissions:

   1. Open the terminal shell at /python-3.8.7/bin folder, and then run the following command:

      ln -sf python3 python
      

   2. Navigate back to /python-3.8.7 folder, and then run the following command. The permission to access the folder is granted.

      chmod -R 755 bin
      

Docker

There are two CloudSlang Docker images. One (cloudslang/cloudslang) is a lightweight image meant to get you running CloudSlang flows as quickly as possible. The other image (cloudslang/cloudslang-dev) adds the tools necessary to develop CloudSlang flows.

This image includes:

• Java
• CloudSlang CLI
• CloudSlang content

To get and run the image: docker pull cloudslang/cloudslang

To run a flow with a CloudSlang prompt:

• docker run -it cloudslang/cloudslang
• At the prompt enter: run --f ../content/io/cloudslang/base/print/print_text.sl --i text=Hi

Or, to run the flow without the prompt:

docker run --rm cloudslang/cloudslang run --f ../content/io/cloudslang/base/print/print_text.sl --i text=first_flow

The CLI will run the ready-made print_text operation that will print the value passed to the variable text to the screen.

This image includes:

• Java
• CloudSlang CLI
• CloudSlang content
• Python
• Pip
• Vim
• Emacs
• SSH
• Git
• Atom
• language-cloudslang Atom package

To get the image: docker pull cloudslang/cloudslang-dev

Next Steps

Now that you’ve run your first CloudSlang file, you might want to:

• Watch a video lesson on how to author CloudSlang content using Atom.
• Learn how to write a print operation yourself using the Hello World example.
• Learn about the language features using the New Hire Tutorial.
• Learn about the language in detail using the CloudSlang Reference.
• See an overview of the ready-made content.
• Learn about the ready-made content.
• Learn about embedding CloudSlang or the CloudSlang Orchestration Engine into your existing application.
• Learn about the architecture of CloudSlang and the CloudSlang Orchestration Engine.

FAQ

What is CloudSlang?

What are the use cases and focus for CloudSlang?

What is the difference between orchestration and configuration management?

How is CloudSlang different than configuration management tools like Chef, Puppet, Salt and Ansible? Can I use it with these tools?

What is the quickest way to try out CloudSlang?

Which languages are supported by CloudSlang?

What other technologies does CloudSlang integrate with?

---

What is CloudSlang?

CloudSlang is an open source project to automate your development and operations use cases using ready-made workflows. CloudSlang uses a process-based approach to orchestrating popular technologies, such as Docker and CoreOS in an agentless manner. You can use the ready-made CloudSlang content or define your own custom workflows that are reusable, shareable and easy to understand.

What are the use cases and focus for CloudSlang?

CloudSlang can orchestrate a wide variety of technologies. Currently, our ready-made content focuses on popular DevOps technologies, such as Docker and CoreOS.

What is the difference between orchestration and configuration management?

CloudSlang is not a traditional PaaS framework, although it can be used to orchestrate one. PaaS platforms such as OpenShift or Cloud Foundry focus primarily on common application stacks and architectures, and are designed to improve developer productivity by making it easy to develop and deploy new, simple applications.

CloudSlang, is designed to orchestrate complex, non-trivial, process-based workflows. For example, CloudSlang content allows you to integrate with the OpenShift or Cloud Foundry (Stackato) PaaS platforms to orchestrate application lifecycle creation.

How is CloudSlang different than configuration management tools like Chef, Puppet, Salt and Ansible? Can I use it with these tools?

Configuration management (CM) tools like Chef, Puppet, Salt and Ansible are great for configuring individual servers and preparing them for service. Given a server and a desired state, they will make sure to take all the required steps to configure that server so that it ends up in the desired state.

As a Runbook Automation open source product, CloudSlang will allow you automate many uses cases, such as application or server provisioning. It will take all the steps required to realize that application stack. This includes provisioning infrastructure resources on the cloud (compute, storage and network), assigning the right roles to each provisioned VM, configuring this CM (which is typically done by CM tools), injecting the right pieces of information into each tier, starting them up in the right order, continuously monitoring the instances of each tier, healing on failure and scaling tiers when needed.

CloudSlang can indeed integrate with CM tools as needed for configuring individual VMs, and in fact this a best practice. For example, CloudSlang provides ready-made content for integrating with Chef.

What is the quickest way to try out CloudSlang?

Follow the directions on the CloudSlang website or head over to the Get Started section to download CloudSlang and run your first CloudSlang content.

Which languages are supported by CloudSlang?

Python and Java operations are supported natively in CloudSlang.

What other technologies does CloudSlang integrate with?

CloudSlang was built out of the box to work with your favorite technologies. For example, not only are OpenStack and Docker technologies are supported, but also configuration management tools like Chef. There’s also support for bash (for *nix systems) and PowerShell (for Windows) and basic operations for REST or SOAP. To see a complete list of technologies that CloudSlang integrates with, see the ready-made content repository.

We’re not looking to replace great tools, we work with them. So many of the tools you’re used to working with are already supported by CloudSlang. The CloudSlang team, along with its growing open source community, are constantly expanding the list of tools we work with, so if you’re favorite tool isn’t supported yet, there’s a good chance it will be soon. Of course, we encourage and support contributions from the community. (In fact, this very answer you’re reading was contributed by a member of the community.)

CloudSlang

Contents:

Content

Ready-made CloudSlang content is hosted on GitHub in the cloud-slang-content repository. The repository contains CloudSlang content written by the CloudSlang team as well as content contributed by the community.

The cloud-slang-content repository contains ready-made CloudSlang flows and operations for many common tasks as well as content that integrates with several other systems.

The repository may contain some beta content. Beta content is not verified or tested by the CloudSlang team. Beta content is named with the beta_ prefix. The community is encouraged to assist in setting up testing environments for the beta content.

For more information on the content contained in the repository, see the docs page.

Running CloudSlang Content

The simplest way to get started running ready-made CloudSlang content is to download and run the pre-packaged cslang-cli-with-content file as described in the Get Started section.

Alternatively, you can build the CLI from source and download the content separately. To build the CLI yourself and for more information on using the CLI, see the CLI section.

Note

When using a locally built CLI you may need to include a classpath to properly reference ready-made content. For information on using classpaths, see Run with Dependencies.

Running Content Dependent on Java Actions

Some of the content is dependent on Java actions from the cs-actions repository. CloudSlang uses Maven to manage these dependencies. When executing an operation that declares a dependency, the required Maven project and all the resources specified in its pom’s dependencies will be resolved and downloaded (if necessary).

Note

Maven version used in the CLI, Builder and cs-actions is 3.3.9 with JRE version 8u282-8.52.0.23. There might be some issues while building the Java Actions with other versions.

Running Content Dependent on External Python Modules

Some of the content is dependent on external python modules. To run this content follow the instructions found in the python_action section of the DSL Reference.

Contributing Content

We welcome and encourage community contributions to CloudSlang. Please see the contribution section to familiarize yourself with the Contribution Guidelines and Project Roadmap before contributing.

Hello World

The following is a simple example to give you an idea of how CloudSlang is structured and can be used to ensure your environment is set up properly.

Prerequisites

This example uses the CloudSlang CLI to run a flow. See the CloudSlang CLI section for instructions on how to download and run the CLI.

Although CloudSlang files can be composed in any text editor, using a modern code editor with support for YAML syntax highlighting is recommended. See CloudSlang Editors for instructions on how to download, install and use the CloudSlang language package for Atom.

Code files

Download the code or use the following instructions:

Create a folder examples and then another folder hello_world inside the examples folder. In the hello_world folder, create two new CloudSlang files, hello_world.sl and print.sl.

You should now have the following folder structure:

• examples

  • hello_world

    • hello_world.sl
    • print.sl

Copy the code below into the corresponding files.

hello_world.sl

namespace: examples.hello_world

flow:
  name: hello_world

  workflow:
    - sayHi:
        do:
          print:
            - text: "'Hello, World'"
        navigate:
          - SUCCESS: SUCCESS

  results:
    - SUCCESS

print.sl

namespace: examples.hello_world

operation:
  name: print

  inputs:
    - text

  python_action:
    script: print text

  results:
    - SUCCESS

Run

Start the CLI and enter the following command at the cslang> prompt:

run --f <path_to_files>/examples/hello_world/hello_world.sl --cp <path_to_files>/examples/hello_world

Note

Use forward slashes in the file paths.

The output will look similar to this:

- sayHi
Hello, World
Flow : hello_world finished with result : SUCCESS
Execution id: 101600001, duration: 0:00:00.790

Explanation

The CLI runs the flow contained in the file passed to it using the --f flag, namely hello_world.sl. The --cp flag is used to specify the classpath where the flow’s dependencies can be found. In our case, the flow refers to the print operation, so we must add its location to the classpath.

Note

If you are using a CLI without the content folder, specifying the classpath in this instance is not necessary.

The flow named hello_world begins its workflow. The workflow has one step named sayHi which calls the print operation. The flow passes the string "Hello, World" to the text input of the print operation. The print operation performs its python_action, which is a simple Python script that prints the input, and then returns a result of SUCCESS. Since the flow does not contain any more steps the flow finishes with a result of SUCCESS.

More

For a more comprehensive walkthrough of the CloudSlang language’s features, see the tutorial.

YAML Overview

Before writing CloudSlang code it helps to have a good working knowledge of YAML. YAML is a human friendly, data serialization language that has become a popular choice for configuration files and other hand-crafted data files. CloudSlang uses YAML to define its flows and operations.

This section contains a brief overview of the common YAML syntax and the best practices for writing it. See the full YAML specification for more information.

Basics

The contents of a YAML file define a single data structure (graph) that is composed of nested nodes (mappings, sequences and scalars). Well crafted YAML files are easy to read, and can have comments in the right places to explain things better.

YAML tries to be human friendly, by minimizing the need for special syntax characters when the data is simple. The most common YAML special characters in YAML are:

• : - Between key/value pairs
• - - Denotes a sequence entry
• # - Starts a comment

Note

You should be aware that YAML also supports very complex data, and has some special characters that you need to be aware of. Even if you never need to use those features, YAML will fail to parse if you accidentally use a special character incorrectly. This is pretty easily avoided and covered more below.

If you are familiar with the popular data language, JSON, then YAML should be easy to learn. Effectively YAML can be thought of as JSON with less syntax (quotes, brackets, etc), although YAML also supports the JSON style syntax. In fact, YAML is a strict superset of JSON.

Here are some basic YAML facts and guidelines:

• Structure is usually scoped by indentation
• Line comments can be used almost anywhere
• Strings values rarely need quotes
• YAML has 5 string quoting styles

Indentation Scoping

Much like the Python programming language, YAML uses indentation to denote a change in scope level. This means that leading whitespace is syntactically significant. Indentation is always achieved using spaces. Tabs are not allowed.

While any number of spaces can be used for a given scope, it a best practice to always use 2 spaces. This makes the YAML be more consistent and readable.

Note

The - characters at the start of a sequence entry count as indentation.

Example: a CloudSlang step (in this case named divider) contains do, publish and navigate keys

- divider:
    do:
      divide:
        - dividend: ${input1}
        - divisor: ${input2}
    publish:
      - answer: ${quotient}
    navigate:
      - ILLEGAL: FAILURE
      - SUCCESS: printer

YAML calls the indentation style “block” and the JSON style “flow”. Flow style can be used at any point within the block style. Flow style doesn’t need quoting either. It is a best practice to only use flow style for small structures on a single line.

Example: above document using flow style

- divider:
    do:
      divide:
        - dividend: ${input1}
        - divisor: ${input2}
    publish:
      - answer: ${quotient}
    navigate: [{ILLEGAL: FAILURE}, {SUCCESS: printer}]

Mappings (Hashes, Objects, Dictionaries)

Mappings (maps) are a set of key/value pairs. Each key and value is separated be a colon (:). The colon must be followed by a whitespace character (space or newline). The value can be a scalar (string/number) value, a newly indented mapping or sequence.

Example: a CloudSlang step’s navigate key is mapped to a mapping of results and their targets

navigate:
  - ILLEGAL: FAILURE
  - SUCCESS: printer

Sequences (Lists, Arrays)

Sequences (seqs) are denoted with a hyphen and a space (-) preceding each entry.

Example: a CloudSlang flow’s possible results are defined using a list mapped to the results key

results:
  - ILLEGAL
  - SUCCESS

Scalars (Strings, Numbers, Values)

Scalars are single values. They are usually strings but (like JSON) can also be numbers, booleans or null values. If a value is quoted, it is always a string. If unquoted it is inspected to be something else, but defaults to being a string.

Strings can be denoted in several ways: unquoted, single quoted and double quoted. The best method for any given string depends on its content.

While most strings should be left unquoted, quotes are required for these cases:

• The string starts with a special character:
  • One of !#%@&*`?|>{[ or -.
• The string starts or ends with whitespace characters.
• The string contains : or # character sequences.
• The string ends with a colon.
• The value looks like a number or boolean (123, 1.23, true, false, null) but should be a string.

Multi-line strings can be written using a pipe (|) to preserve line breaks or a greater than symbol (>) where each line break will be converted to a space. Multi-line strings can also use the unquoted or quoted styles above, but it is best practice to avoid that.

The double-quoted style is the only style that can support any character string, using escape sequences like \n, \\, and \"). Single quoted strings only have one escape sequence: two single quotes ('') are used to put a single quote inside the single quoted string.

Example: a name of a CloudSlang flow is defined using the unquoted style

flow:
  name: hello_world

Example: a string value is passed to a CloudSlang operation using the double quoted style

- sayHi:
    do:
      print:
        - text: "Hello, World\n"

Example: the pipe is used in CloudSlang to indicate a multi-line Python script

python_action:
  script: |
    if divisor == '0':
      quotient = 'division by zero error'
    else:
      quotient = float(dividend) / float(divisor)

Note

Learning the scalar styles and their specifics will help you write YAML files that are clear and concise.

Comments

Comments begin with the # symbol following a whitespace character or beginning of line.

# This is a line comment
flow:       # Flow definition (trailing comment)
  name: hello_world # This flow is called 'hello_world'

Validate Your YAML

You can use an online YAML validator, such as the ones found here:

• YAML Lint
• Code Beautify

Conclusion

YAML is a simple yet complete data language. This means that most of the time, simple things are simple. You just need to be aware that some things have special meaning to YAML that you might not expect.

If you need more help, there are lots of resources about YAML on the web. You may want to check out the YAML Reference Card.

DSL Reference

CloudSlang is a YAML (version 1.2) based language for describing a workflow. Using CloudSlang you can easily define a workflow in a structured, easy-to-understand format that can be run by the CloudSlang Orchestration Engine (Score). CloudSlang files can be run by the CloudSlang CLI or by an embedded instance of Score using the Slang API.

This reference begins with a brief introduction to CloudSlang files and their structure, then continues with a brief explanation of CloudSlang expressions and variable contexts. Finally, there are alphabetical listings of the CloudSlang keywords and functions. See the examples section for the full code examples from which many of the code snippets in this reference are taken.

CloudSlang Files

CloudSlang files are written using YAML. The recommended extension for CloudSlang flow and operation file names is .sl, but .sl.yaml and .sl.yml will work as well. CloudSlang system properties file names end with the .prop.sl extension.

Since CloudSlang is YAML-based, proper indentation is crucial. For more information, see the YAML Overview.

There are four types of CloudSlang files:

• flow - contains a list of steps and navigation logic that calls operations or subflows
• operation - contains an action that runs a script or method
• decision - contains decision logic without an action
• system properties - contains a list of system property keys and values

The following properties are for all types of CloudSlang files. For properties specific to flow, operation, or system properties files, see their respective sections below.

Property	Required	Default	Value Type	Description	More Info
namespace	no	–	string	namespace of the file	namespace
imports	no	–	list of key:value pairs	files to import	imports
extensions	no	–	–	information to be ignored by the compiler	extensions

Naming

Names that are interpreted by YAML as non-string types (e.g. booleans, numbers) cannot be used without enclosing them in quotes (') to force YAML to recognize them as strings.

Namespace Names

Namespaces can be named using alphanumeric characters (a-z, A-Z and 0-9), underscores (_) and dashes (-) while using a period (.) as a delimiter.

Since namespaces reflect the folder structure where their respective files are found, they cannot be named using names that are invalid as Windows or Linux folder names.

Namespaces are found in:

• system property fully qualified names
• flow, operation, decision and system properties namespaces
• import values
• step references

Variable Names

Variable names in CloudSlang files cannot contain localized characters. They can be named using alphanumeric characters (a-z, A-Z and 0-9) and an underscore (_), but may not begin with a number.

CloudSlang variable names must conform to both Python’s naming constraints as well as Java’s naming constraints.

Variable name rules apply to:

• inputs
• outputs
• published variables
• loop variables

Other Names

All other names can be named using alphanumeric characters (a-z, A-Z and 0-9).

Since flow, operation and decision names must match the names of their respective files, they cannot be named using names that are invalid as Windows or Linux file names.

These rules apply to:

• import section aliases
• flow, operation and decision names
• step names
• result names
• navigation keys
• break keys

Uniqueness and Case Sensitivity

Inputs, outputs, results, publish values, fully qualified system properties and fully qualified executable names must be unique and are validated as case insensitive. When using any of the above, they must be referred to using the case in which they were declared.

Encoding

When using the CLI or Build Tool, CloudSlang will use the encoding found in the CLI configuration file or Build Tool configuration file for input values respectively. If no encoding is found in the configuration file, the CLI or Build Tool will use UTF-8.

Structure

The general structure of CloudSlang files is outlined here. Some of the properties that appear are optional. All CloudSlang keywords, properties and concepts are explained in detail below.

Flow file

• namespace
• imports
• flow
  • name
  • inputs
    • required
    • default
    • private
    • sensitive
  • workflow
    • step(s)
      • do
      • publish
      • navigate
    • iterative step
      • loop
        • for
        • do
        • publish
        • break
      • navigate
    • parallel step
      • parallel_loop
        • for
        • do
      • publish
      • navigate
    • on_failure
  • outputs
    • value
    • sensitive
  • results
• extensions

Operation file

• namespace
• operation
  • name
  • inputs
    • required
    • default
    • private
    • sensitive
  • python_action
    • script
  • java_action
    • gav
    • class_name
    • method_name
  • outputs
    • value
    • sensitive
  • results
• extensions

Decision file

• namespace
• decision
  • name
  • inputs
    • required
    • default
    • private
    • sensitive
  • outputs
    • value
    • sensitive
  • results
• extensions

System properties file

• namespace
• properties
  • value
  • sensitive
• extensions

Expressions

Many CloudSlang keys map to either an expression or literal value.

Literal Values

Literal values are denoted as they are in standard YAML. Numbers are interpreted as numerical values and strings may be written unquoted, single quoted or double quoted.

Example: literal values

literal_number: 4
literal_unquoted_string: cloudslang
literal_single_quoted_string: 'cloudslang'
literal_double_quoted_string: "cloudslang"

Note

Where expressions are allowed as values (input defaults, output and result values, etc.) and a literal string value is being used, you are encouraged to use a quoted style of literal string.

Example: recommended style for literal strings

flow:
  name: flow_name #expression not allowed - unquoted literal string

  workflow:
    - step1:
        do:
          print:
            - text: "hello" #expression allowed - quoted literal string

Standard Expressions

Expressions are preceded by a dollar sign ($) and enclosed in curly brackets ({}).

Example: expressions

- expression_1: ${4 + 7}
- expression_2: ${some_input}
- expression_3: ${get('input1', 'default_input')}

Expressions with Special Characters

Expressions that contain characters that are considered special characters in YAML must be enclosed in quotes or use YAML block notation. If using quotes, use the style of quotes that are not already used in the expression. For example, if your expression contains single quotes ('), enclose the expression using double quotes (").

Example: escaping special characters

- expression1: "${var1 + ': ' + var2}"
- expression2: >
    ${var1 + ': ' + var2}
- expression3: |
    ${var1 + ': ' + var2}

Maps

To use a map where an expression is allowed use the default property.

Example: passing a map using the default property

- map1:
    default: {a: 1, b: c}
- map2:
    default: {'a key': 1, b: c}

It is also possible to use two sets of quotes and an expression marker, but the approach detailed above is the recommended one.

Example: passing a map using the expression marker and quotes

- map3: "${{'a key': 1, 'b': 'c'}}"
- map4: >
    ${{'a key': 1, 'b': 'c'}}

Contexts

Throughout the execution of a flow, its steps, operations, decisions and subflows there are different variable contexts that are accessible. Which contexts are accessible depends on the current section of the flow, operation or decision.

The table below summarizes the accessible contexts at any given location in a flow, operation or decision.

Contexts/ Location	Context Passed To Executable	Flow Context	Operation/ Decision Context	Action Outputs Context	Subflow/ Operation/ Outputs Context	Step Arguments	Branched Step Output Values	Already Bound Values
flow inputs	Yes							Yes
flow outputs		Yes						Yes
operation/ decision inputs	Yes							Yes
operation/ decision outputs			Yes	Yes				Yes
operation/ decision results			Yes	Yes
step arguments		Yes						Yes
step publish					Yes	Yes	Yes - using branches_context	Yes
step navigation					Yes	Yes
action inputs			Yes

Keywords (A-Z)

branches_context

May appear in the publish section of a parallel step.

As branches of a parallel_loop complete, values that have been output and the branch’s result get placed as a dictionary into the branches_context list. The list is therefore in the order the branches have completed.

A specific value can be accessed using the index representing its branch’s place in the finishing order and the name of the variable or the branch_result key.

Example - retrieves the name variable from the first branch to finish

publish:
  - first_name: ${branches_context[0]['name']}

More commonly, the branches_context is used to aggregate the values that have been published by all of the branches.

Example - aggregates name values into a list

publish:
  - name_list: ${map(lambda x:str(x['name']), branches_context)}

branch_result

May appear in the publish section of a parallel step.

As branches of a parallel_loop complete, branch results get placed into the branches_context list under the branch_result key.

Example - aggregates branch results

publish:
  - branch_results_list: ${map(lambda x:str(x['branch_result']), branches_context)}

break

The key break is a property of a loop. It is mapped to a list of results on which to break out of the loop or an empty list ([]) to override the default breaking behavior for a list. When the operation or subflow of the iterative step returns a result in the break’s list, the iteration halts and the iterative step’s navigation logic is run.

If the break property is not defined, the loop will break on results of FAILURE by default. This behavior may be overriden so that iteration will continue even when a result of FAILURE is returned by defining alternate break behavior or mapping the break key to an empty list ([]).

Example - loop that breaks on result of CUSTOM

loop:
  for: value in range(1,7)
  do:
    custom_op:
      - text: ${str(value)}
  break:
    - CUSTOM
navigate:
  - CUSTOM: print_end

Example - loop that continues even on result of FAILURE

loop:
  for: value in range(1,7)
  do:
    custom_op:
      - text: ${str(value)}
  break: []

class_name

The key class_name is a property of a java_action. It is mapped to the name of the Java class where an annotated @Action resides.

decision

The key decision is mapped to the properties which make up the decision contents.

Property	Required	Default	Value Type	Description	More Info
name	yes	–	string	name of the decision	name
inputs	no	–	list	decision inputs	inputs
outputs	no	–	list	decision outputs	outputs
results	yes	–	list	possible decision results	results

Example - decision that compares two values

decision:
  name: compare

  inputs:
    - x
    - y

  outputs:
    - sum: ${str(int(x) + int(y))}

  results:
    - EQUAL: ${x == y}
    - LESS_THAN: ${int(x) < int(y)}
    - GREATER_THAN

default

The key default is a property of an input name. It is mapped to an expression value.

The expression’s value will be passed to the flow operation or decision if no other value for that input parameter is explicitly passed or if the input’s private parameter is set to true. Passing an empty string (''), null, or an expression that evaluates to None is the same as not passing any value at all and will not override the default value.

Example - default values

inputs:
  - str_literal:
      default: "default value"
  - int_exp:
      default: ${str(5 + 6)}
  - from_variable:
      default: ${variable_name}
  - from_system_property:
      default: $ { get_sp('system.property.key') }

A default value can also be defined inline by entering it as the value mapped to the input parameter’s key.

Example - inline default values

inputs:
  - str_literal: "default value"
  - int_exp: ${str(5 + 6)}
  - from_variable: ${variable_name}
  - from_system_property: $ { get_sp('system.property.key') }

do

The key do is a property of a step name, a loop, or a parallel_loop. It is mapped to a property that references an operation or flow.

Calls an operation or flow and passes in relevant arguments.

The operation or flow may be called in several ways:

• by referencing the operation or flow by name when it is in the default namespace (the same namespace as the calling flow)
• by using a fully qualified name, for example, path.to.operation.op_name
  • a path is recognized as a fully qualified name if the prefix (before the first .) is not a defined alias
• by using an alias defined in the flow’s imports section followed by the operation or flow name (e.g alias_name.op_name)
• by using an alias defined in the flow’s imports section followed by a continuation of the path to the operation or flow and its name (e.g alias_name.path.cont.op_name)

For more information, see the Operation Paths example.

Arguments are passed to a step using a list of argument names and optional mapped expressions. The step must pass values for all inputs found in the called operation, decision or subflow that are required and don’t have a default value. Passing an empty string (''), null, or an expression that evaluates to None is the same as not passing any value at all.

Argument names should be different than the output names found in the operation, decision or subflow being called in the step.

Argument names must conform to the rules for valid variable names.

An argument name without an expression will take its value from a variable with the same name in the flow context. Expression values will supersede values bound to flow inputs with the same name. To force the operation, decision or subflow being called to use it’s own default value, as opposed to a value passed in via expression or the flow context, omit the variable from the calling step’s argument list.

For a list of which contexts are available in the arguments section of a step, see Contexts.

Example - call to a divide operation with list of mapped step arguments

do:
  divide:
    - dividend: ${input1}
    - divisor: ${input2}

Example - force an operation to use default value for punctuation input

flow:
  name: flow

  inputs:
      - punctuation: "!"

  workflow:
    - step1:
        do:
          punc_printer:
            - text: "some text"
            #- punctuation
            #commenting out the above line forces the operation to use its default value (".")
            #leaving it in would cause the operation to take the value from the flow context ("!")

operation:
  name: operation
  inputs:
    - text
    - punctuation: "."
  python_action:
    script: |
      print text + punctuation

extensions

The key extensions is mapped to information that the compiler will ignore and can therefore be used for various purposes.

Example - a flow that contains an extensions section

namespace: examples.extensions

flow:
  name: flow_with_extensions_tag

  workflow:
    - noop_step:
      do:
        noop: []

extensions:
  - some_key:
      a: b
      c: d
  - another

flow

The key flow is mapped to the properties which make up the flow contents.

A flow is the basic executable unit of CloudSlang. A flow can run on its own or it can be used by another flow in the do property of a step.

Property	Required	Default	Value Type	Description	More Info
name	yes	–	string	name of the flow	name
inputs	no	–	list	inputs for the flow	inputs
workflow	yes	–	list of steps	container for workflow steps	workflow
outputs	no	–	list	list of outputs	outputs
results	no	(SUCCESS / FAILURE )	list	possible results of the flow	results

Example - a flow that performs a division of two numbers

flow:
  name: division

  inputs:
    - input1
    - input2

  workflow:
    - divider:
        do:
          divide:
            - dividend: ${input1}
            - divisor: ${input2}
        publish:
          - answer: ${quotient}
        navigate:
          - ILLEGAL: ILLEGAL
          - SUCCESS: printer
    - printer:
        do:
          print:
            - text: ${input1 + "/" + input2 + " = " + answer}
        navigate:
          - SUCCESS: SUCCESS

  outputs:
    - quotient: ${answer}

  results:
    - ILLEGAL
    - SUCCESS

for

The key for is a property of a loop or an parallel_loop.

loop: for

A for loop iterates through a list or a map.

The iterative step will run once for each element in the list or key in the map.

Loop variables must conform to the rules for valid variable names.

Iterating through a list

When iterating through a list, the for key is mapped to an iteration variable followed by in followed by a list, an expression that evaluates to a list, or a comma delimited string.

Example - loop that iterates through the values in a list

- print_values:
    loop:
      for: value in [1,2,3]
      do:
        print:
          - text: ${str(value)}

Example - loop that iterates through the values in a comma delimited string

- print_values:
    loop:
      for: value in "1,2,3"
      do:
        print:
          - text: ${value}

Example - loop that iterates through the values returned from an expression

- print_values:
    loop:
      for: value in range(1,4)
      do:
        print:
          - text: ${str(value)}

Iterating through a map

When iterating through a map, the for key is mapped to iteration variables for the key and value followed by in followed by a map or an expression that evaluates to a map.

Example - step that iterates through the values returned from an expression

- print_values:
    loop:
      for: k, v in map
      do:
        print2:
          - text1: ${k}
          - text2: ${v}

parallel_loop: for

A parallel for loop loops in parallel branches over the items in a list.

The parallel step will run one branch for each element in the list.

The for key is mapped to an iteration variable followed by in followed by a list or an expression that evaluates to a list.

Example - step that loops in parallel through the values in a list

- print_values:
    parallel_loop:
      for: value in values_list
      do:
        print_branch:
          - ID: ${value}

gav

The key gav is a property of a java_action. It is mapped to the group:artifact:version of the Maven project in which an annotated Java @Action resides.

Upon operation execution, the Maven project and all the required resources specified in its pom’s dependencies will be resolved and downloaded (if necessary).

Example - referencing Maven artifact using gav

java_action:
  gav: io.cloudslang.content:cs-xml:0.0.2
  class_name: io.cloudslang.content.mail.actions.SendMailAction
  method_name: execute

imports

The key imports is mapped to the files to import as follows:

• key - alias
• value - namespace of file to be imported

Specifies the file’s dependencies, operations and subflows, by the namespace defined in their source file and the aliases they will be referenced by in the file.

Using an alias is one way to reference the operations and subflows used in a flow’s steps. For all the ways to reference operations and subflows used in a flow’s steps, see the do keyword and the Operation Paths example.

Import aliases must conform to the rules for valid names.

Example - import operations and sublflow into flow

imports:
  ops: examples.utils
  subs: examples.subflows

flow:
  name: hello_flow

  workflow:
    - print_hi:
        do:
          ops.print:
            - text: "Hi"
    - run_subflow:
        do:
          subs.division:
            - input1: "5"
            - input2: "3"

In this example, the ops alias refers to the `examples.utils namespace. This alias is used in the print_hi step to refer to the print operation, whose source file defines its namespace as examples.utils. Similarly, the subs alias refers to the examples.subflows namespace. The subs alias is used in the run_subflow step to refer to the division subflow, whose source file defines its namespace as examples.subflows.

inputs

The key inputs is a property of a flow, operation or decision. It is mapped to a list of input names. Each input name may in turn be mapped to its properties or an input expression.

Inputs are used to pass parameters to flows, operations or decisions. Non-private input names for a specific flow, operation or decision must be different than the output names of the same flow, operation or decision.

Input values must evaluate to type string.

For a list of which contexts are available in the inputs section of a flow, operation or decision, see Contexts.

Input names must conform to the rules for valid variable names and uniqueness.

Property	Required	Default	Value Type	Description	More info
required	no	true	boolean	is the input required	required
default	no	–	expression	default value of the input	default
private	no	false	boolean	if true, the default value always overrides values passed in	private
sensitive	no	transitive sensitivity or false	boolean	is the input sensitive	sensitive

Example - several inputs

inputs:
  - input1:
      default: "default value"
      private: true
  - input2
  - input3: "default value"
  - input4: ${'input1 is ' + input1}
  - password:
      sensitive: true

java_action

The key java_action is a property of an operation. It is mapped to the properties that define where an annotated Java @Action resides.

Property	Required	Default	Value Type	Description	More info
gav	yes	–	string	group:artifact:version	gav
class_name	yes	–	string	fully qualified Java class name	class_name
method_name	no	–	string	Java method name	method_name

Example - CloudSlang call to a Java action

namespace: io.cloudslang.base.mail

operation:
  name: send_mail

  inputs:
    - hostname
    - port
    - from
    - to
    - subject
    - body

  java_action:
    gav: io.cloudslang.content:cs-xml:0.0.2
    class_name: io.cloudslang.content.mail.actions.SendMailAction
    method_name: execute

  results:
    - SUCCESS: ${ returnCode == '0' }
    - FAILURE

Existing Java Actions

There are many existing Java actions which are bundled with the CloudSlang CLI. The source code for these Java actions can be found in the cs-actions repository.

Adding a New Java Action

To add a new Java action:

• Write an annotated Java method
• Release to remote Maven repository
• Reference Maven artifact

Write an Annotated Java Method

Create a Java method that conforms to the signature public Map<String, String> doSomething(paramaters) and use the following annotations from com.hp.oo.sdk.content.annotations:

• @Action: specifies action information

  • name: name of the action
  • outputs: array of @Output annotations
  • responses: array of @Response annotations

• @Output: action output name

• @Response: action response

  • text: name of the response
  • field: result to be checked
  • value: value to check against
  • matchType: type of check
  • responseType: type of response
  • isDefault: whether or not response is the default response
  • isOnFail: whether or not response is the failure response

• @Param: action parameter

  • value: name of the parameter
  • required: whether or not the parameter is required

Values are passed to a Java action from an operation using CloudSlang inputs that match the annotated parameters.

Values are passed back from the Java action to an operation using the returned Map<String, String>, where the map’s elements each correspond to a name:value that matches a CloudSlang output.

Example - Java action

package com.example.content.actions;

import com.hp.oo.sdk.content.annotations.Action;
import com.hp.oo.sdk.content.annotations.Output;
import com.hp.oo.sdk.content.annotations.Param;
import com.hp.oo.sdk.content.annotations.Response;
import com.hp.oo.sdk.content.plugin.ActionMetadata.MatchType;

import java.util.Map;
import java.util.HashMap;

public class SaySomething {

      @Action(name = "Example Test Action",
              outputs = {
                      @Output("message")
              },
              responses = {
                      @Response(text = "success", field = "message", value = "fail", matchType = MatchType.COMPARE_NOT_EQUAL),
                      @Response(text = "failure", field = "message", value = "fail", matchType = MatchType.COMPARE_EQUAL, isDefault = true, isOnFail = true)
              }
      )
      public Map<String, String> speak(@Param(value = "text", required = true) String text) {
          Map<String, String> results = new HashMap<>();

          System.out.println("I say " + text);

          results.put("message", text);

          return  results;
      }
}

Release to remote Maven repository

Note

Maven version used in the CLI, Builder and cs-actions is 3.3.9 with JRE version 8u282-8.52.0.23. There might be some issues while building the Java Actions with other versions.

Use Maven to package the project containing the Java action method and release it to the remote repository defined in the CLI’s configuration file.

Below is an example pom.xml file that can be used for your Maven project.

Example - sample pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.example.content</groupId>
    <artifactId>action-example</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>jar</packaging>
    <name>${project.groupId}:${project.artifactId}</name>
    <description>Test Java action</description>
    <dependencies>
        <dependency>
            <groupId>com.hp.score.sdk</groupId>
            <artifactId>score-content-sdk</artifactId>
            <version>1.10.6</version>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.1</version>
                <configuration>
                    <source>1.7</source>
                    <target>1.7</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

Reference Maven artifact

Reference your Maven artifact using the gav key in the java_action section of your operation.

Upon the operation’s first execution, the Maven project and all the required resources specified in its pom’s dependencies will be resolved and downloaded.

loop

The key loop is a property of an iterative step’s name. It is mapped to the iterative step’s properties.

For each value in the loop’s list the do will run an operation or subflow. If the returned result is in the break list, or if break does not appear and the returned result is FAILURE, or if the list has been exhausted, the step’s navigation will run.

Property	Required	Default	Value Type	Description	More Info
for	yes	–	variable in list	iteration logic	for
do	yes	–	operation or subflow call	the operation or subflow this step will run iteratively	do operation flow
publish	no	–	list of key:value pairs	operation or subflow outputs to aggregate and publish to the flow level	publish outputs
break	no	–	list of results	operation or subflow results on which to break out of the loop	break

Example: loop that breaks on a result of custom

- custom3:
    loop:
      for: value in "1,2,3,4,5"
      do:
        custom3:
          - text: ${value}
      break:
        - CUSTOM
    navigate:
      - CUSTOM: aggregate
      - SUCCESS: skip_this

method_name

The key method_name is a property of a java_action. It is mapped to the name of the Java method where an annotated @Action resides.

name

The key name is a property of flow, operation or decision. It is mapped to a value that is used as the name of the flow or operation.

The name of a flow, operation or decision may be used when called from a flow’s step.

The name of a flow, operation or decision must match the name of the file in which it resides, excluding the extension.

The name must conform to the rules for names and uniqueness.

Example - naming the flow found in the file division_flow.sl

name: division_flow

namespace

The key namespace is mapped to a string value that defines the file’s namespace.

The namespace of a file may be used by a flow to import dependencies.

Example - definition a namespace

namespace: examples.hello_world

Example - using a namespace in an imports definition

imports:
  ops: examples.hello_world

Namespace values must conform to the rules described in Namespace Names. For more information about choosing a file’s namespace, see the CloudSlang Content Best Practices section.

Note

If the imported file resides in a folder that is different from the folder in which the importing file resides, the imported file’s directory must be added using the --cp flag when running from the CLI (see Run with Dependencies).

navigate

The key navigate is a property of a step name. It is mapped to a list of key:value pairs where the key is the received result and the value is the target step, flow result or on_failure.

Defines the navigation logic for a standard step, an iterative step or a parallel step. The flow will continue with the step or flow result whose value is mapped to the result returned by the called operation or subflow.

The default navigation rules, when no explicit navigate section is declared, are as follows:

Result	Step location	on_failure present	Navigation
SUCCESS	Not last non-on_failure step	–	Next step
SUCCESS	Last non-on_failure step	–	SUCCESS result of the flow
FAILURE	–	Yes	on_failure step
FAILURE	–	No	FAILURE result of the flow

The default navigation only applies when a step calls an operation or subflow that returns a result of either SUCCESS or FAILURE. If the operation or subflow can return a custom result or always returns only SUCCESS or only FAILURE then default navigation will not apply.

Note

Operations which don’t explicitly return any results always return the result SUCCESS.

All possible results returned by the called operation or subflow must be handled.

For a standard step the navigation logic runs when the step is completed.

For an iterative step the navigation logic runs when the last iteration of the step is completed or after exiting the iteration due to a break.

For a parallel step the navigation logic runs after the last branch has completed. If any of the branches returned a result of FAILURE, the flow will navigate to the step or flow result mapped to FAILURE. Otherwise, the flow will navigate to the step or flow result mapped to SUCCESS. Note that the only results of an operation or subflow called in a parallel_loop that are evaluated are SUCCESS and FAILURE. Any other results will be evaluated as SUCCESS.

For a list of which contexts are available in the navigate section of a step, see Contexts.

A navigation key’s name must conform to the rules for names and uniqueness.

Example - ILLEGAL result will navigate to flow’s FAILURE result and SUCCESS result will navigate to step named ‘printer’

navigate:
  - SUCCESS: printer
  - ILLEGAL: ILLEGAL
  - FAILURE: on_failure

on_failure

The key on_failure is a property of a workflow. It is mapped to a step.

Defines the step, which when using default navigation, is the target of a FAILURE result returned from an operation or flow. The on_failure step can also be reached by mapping one of a step’s navigation keys to on_failure. If a step’s navigation explicitly maps to on_failure, but there is no on_failure step defined in the flow, the flow ends with a result of FAILURE.

The on_failure step must be the last step in the flow.

The on_failure step cannot contain a navigation section. It always causes the flow to end with a result of FAILURE.

Example - failure step which calls a print operation to print an error message

- on_failure:
  - failure:
      do:
        print:
          - text: ${error_msg}

Example - explicitly navigating to the on_failure step

- go_to_failure:
    do:
      some_operation:
        - input1
    navigate:
      - SUCCESS: SUCCESS
      - FAILURE: on_failure

operation

The key operation is mapped to the properties which make up the operation contents.

Property	Required	Default	Value Type	Description	More Info
name	yes	–	string	name of the operation	name
inputs	no	–	list	operation inputs	inputs
python_action	no	–	script key	operation logic	python_action
java_action			map	operation logic	java_action
outputs	no	–	list	operation outputs	outputs
results	no	SUCCESS	list	possible operation results	results

Example - operation that adds two inputs and outputs the answer

operation:
  name: add

  inputs:
    - left
    - right

  python_action:
    script: ans = int(left) + int(right)

  outputs:
    - out: ${str(ans)}

  results:
    - SUCCESS

outputs

The key outputs is a property of a flow, operation or decision. It is mapped to a list of output variable names. Each output name may in turn be mapped to its properties or an output expression. Output expressions must evaluate to strings.

Defines the parameters a flow, operation or decision exposes to possible publication by a step. The calling step refers to an output by its name.

Output names for a specific flow, operation or decision must be different than the non-private input names of the same flow, operation or decision.

Output values must evaluate to type string.

For a list of which contexts are available in the outputs section of a flow, operation or decision, see Contexts.

Output identifiers must conform to the rules for valid variable names and uniqueness.

Property	Required	Default	Value Type	Description	More info
value	no	–	expression	value of the output	value
sensitive	no	transitive sensitivity or false	boolean	is the output sensitive	sensitive

Example - various types of outputs

outputs:
  - existing_variable
  - output2: ${some_variable}
  - output3: ${str(5 + 6)}
  - password:
      value: ${password}
      sensitive: true

parallel_loop

The key parallel_loop is a property of a parallel step’s name. It is mapped to the parallel step’s properties.

For each value in the loop’s list a branch is created and the do will run an operation or subflow. When all the branches have finished, the parallel step’s publish and navigation will run.

Property	Required	Default	Value Type	Description	More Info
for	yes	–	variable in list	loop values	for
do	yes	–	operation or subflow call	operation or subflow this step will run in parallel	do operation

Example: loop that breaks on a result of custom

- print_values:
    parallel_loop:
      for: value in values
      do:
        print_branch:
          - ID: ${value}
    publish:
        - name_list: ${map(lambda x:str(x['name']), branches_context)}
    navigate:
        - SUCCESS: print_list
        - FAILURE: FAILURE

private

The key private is a property of an input name. It is mapped to a boolean value.

A value of true will ensure that the input parameter’s default value will not be overridden by values passed into the flow, operation or decision. An input set as private: true must also declare a default value. If private is not defined, values passed in will override the default value.

Example - default value of text input parameter will not be overridden by values passed in

inputs:
  - text:
      default: "default text"
      private: true

properties

The key properties is mapped to a list of key:value pairs that define one or more system properties. Each system property name may in turn be mapped to its properties or a value.

System property names (keys) can contain alphanumeric characters (a-z, A-Z and 0-9), underscores (_) and hyphens (-). The names must conform to the rules for uniqueness.

System property values are retrieved using the get_sp() function.

Note

System property values that are non-string types (numeric, list, map, etc.) are converted to string representations. A system property may have a value of null.

Property	Required	Default	Value Type	Description	More info
value	no	–		value of the property	value
sensitive	no	false	boolean	is the property sensitive	sensitive

Example - system properties file

namespace: examples.sysprops

properties:
  - host: 'localhost'
  - port: 8080
  - password:
      value: 'pwd'
      sensitive: true

An empty system properties file can be defined using an empty list.

Example - empty system properties file

namespace: examples.sysprops

properties: []

publish

The key publish is a property of a step name, a loop or a parallel_loop. It is mapped to a list of key:value pairs where the key is the published variable name and the value is an expression, usually involving an output received from an operation or flow.

For a list of which contexts are available in the publish section of a step, see Contexts.

Publish names must conform to the rules for valid variable names and uniqueness.

Standard publish

In a standard step, publish binds an expression, usually involving an output from an operation or flow, to a variable whose scope is the current flow and can therefore be used by other steps or as the flow’s own output.

Example - publish the quotient output as ans

- division1:
    do:
      division:
        - input1: ${dividend1}
        - input2: ${divisor1}
    publish:
      - ans: ${quotient}

Iterative publish

In an iterative step the publish mechanism is run during each iteration after the operation or subflow has completed, therefore allowing for aggregation.

Example - publishing in an iterative step to aggregate output: add the squares of values in a range

- aggregate:
    loop:
      for: value in range(1,6)
      do:
        square:
          - to_square: ${str(value)}
          - sum
      publish:
        - sum: ${str(int(sum) + int(squared))}

Parallel publish

In a parallel step the publish mechanism defines the step’s aggregation logic, generally making use of the branches_context construct.

After all branches of a parallel step have completed, execution of the flow continues with the publish section. The expression of each name:value pair is evaluated and published to the flow’s scope. The expression generally makes use of the branches_context construct to access the values published by each of the parallel loop’s branches and their results using the branch_result key.

For a list of which contexts are available in the publish section of a step, see Contexts.

For more information, see the Parallel Loop example.

Example - publishing in an parallel step to aggregate output

- print_values:
    parallel_loop:
      for: value in values_list
      do:
        print_branch:
          - ID: ${value}
    publish:
        - name_list: ${map(lambda x:str(x['name']), branches_context)}

Example - extracting information from a specific branch

- print_values:
    parallel_loop:
      for: value in values_list
      do:
        print_branch:
          - ID: ${value}
    publish:
        - first_name: ${branches_context[0]['name']}

Example - create a list of branch results

- print_values:
    parallel_loop:
      for: value in values
      do:
        print_branch:
          - ID: ${ value }
    publish:
      - branch_results_list: ${map(lambda x:str(x['branch_result']), branches_context)}

python_action

The key python_action is a property of an operation. It is mapped to a script property that contains the actual Python script.

results

The key results is a property of a flow, operation or decision.

The results of a flow, operation or decision can be used by the calling step for navigation purposes.

A result name must conform to the rules for names and uniqueness. Additionally, a result cannot be named on_failure.

Note

The only results of an operation, decision or subflow called in a parallel_loop that are evaluated are SUCCESS and FAILURE. Any other results will be evaluated as SUCCESS.

ROI value

The key ROI is an in-built property that can be associated to the step navigation of a flow to assess the saved investment by moving from manual to automated steps. When executing CloudSlang or OO native flows (flows created using OO Desktop Studio), OO Central computes these ROI values to report the business benefit of executing the entire flow.

Example - assign an ROI value to the step navigation

- namespace: ROI_value
  flow:
  name: FlowROI
  workflow:
   - sleep:
     do:
       io.cloudslang.base.utils.sleep:
         - seconds: '2'
     navigate:
       - SUCCESS:
           next_step: SUCCESS
           ROI: '4'
       - FAILURE: on_failure
    results:
       - FAILURE
       - SUCCESS

run_id

The run_id returns the run ID of the current execution.

Example - assign a flow with run_id parameter

- namespace: run_id
    flow:
    name: run_id_usage
    workflow:
     - add_numbers:
       do:
         io.cloudslang.base.math.add_numbers:
          - value1: 'integer1'
          - value2: 'integer2'
          - run_id: '${run_id}'
        navigate:
          - SUCCESS: SUCCESS
          - FAILURE: on_failure
        results:
          - FAILURE
          - SUCCESS

Flow Results

In a flow, the key results is mapped to a list of result names.

Defines the possible results of the flow. By default a flow has two results, SUCCESS and FAILURE. The defaults can be overridden with any number of user-defined results.

When overriding, the defaults are lost and must be redefined if they are to be used.

All result possibilities must be listed. When being used as a subflow all flow results must be handled by the calling step.

Example - a user-defined result

results:
  - SUCCESS
  - ILLEGAL
  - FAILURE

Operation and Decision Results

In an operation or decision the key results is mapped to a list of key:value pairs of result names and boolean expressions.

Defines the possible results of the operation or decision. By default, if no results exist, the result of an operation is SUCCESS. A decision does not have any default results.

The first result in the list whose expression evaluates to true, or does not have an expression at all, will be passed back to the calling step to be used for navigation purposes.

If results are present, the list must include exactly one default ending result which is not mapped to anything (- result) or is mapped to the value true (- result: true).

All operation or decision results must be handled by the calling step.

For a list of which contexts are available in the results section of an operation or decision, see Contexts.

Example - three user-defined results

results:
  - POSITIVE: ${polarity == '+'}
  - NEGATIVE: ${polarity == '-'}
  - NEUTRAL

required

The key required is a property of an input name. It is mapped to a boolean value.

A value of false will allow the flow or operation to be called without passing the input parameter. If required is not defined, the input parameter defaults to being required.

Required inputs must receive a value or declare a default value. Passing an empty string (''), null, or an expression that evaluates to None to a required input is the same as not passing any value at all.

Example - input2 is optional

inputs:
  - input1
  - input2:
      required: false

script

The key script is a property of python_action. It is mapped to a value containing a Python script.

All variables in scope at the conclusion of the Python script must be serializable. If non-serializable variables are used, remove them from scope by using the del keyword before the script exits.

Note

CloudSlang uses the Jython implementation of Python 2.7. For information on Jython’s limitations, see the Jython FAQ.

Example - action with Python script that divides two numbers

name: divide

inputs:
  - dividend
  - divisor

python_action:
  script: |
    if divisor == '0':
      quotient = 'division by zero error'
    else:
      quotient = float(dividend) / float(divisor)

outputs:
  - quotient: ${str(quotient)}

results:
  - ILLEGAL: ${quotient == 'division by zero error'}
  - SUCCESS

Note

Single-line Python scripts can be written inline with the script key. Multi-line Python scripts can use the YAML pipe (|) indicator as in the example above.

Importing External Python Packages

There are three approaches to importing and using external Python modules:

• Installing packages into the python-lib folder
• Editing the executable file
• Adding the package location to sys.path

Installing Packages into the python-lib Folder

Prerequisites: Python 3.8.7 and pip.

You can download Python (version 3.8.7) from here. that includes pip by default. If you already have Python but don’t have pip, see the pip documentation for installation instructions.

1. Edit the requirements.txt file in the python-lib folder, which is found at the same level as the bin folder that contains the CLI executable.

   • If not using a pre-built CLI, you may have to create the python-lib folder and requirements.txt file.

2. Enter the Python package and all its dependencies in the requirements file.

   • See the pip documentation for information on how to format the requirements file (see example below).

3. Run the following command from inside the python-lib folder: pip install -r requirements.txt -t.

   Note

   If your machine is behind a proxy you will need to specify the proxy using pip’s --proxy flag.

4. Import the package as you normally would in Python from within the action’s script:

python_action:
  script: |
    from pyfiglet import Figlet
    f = Figlet(font='slant')
    print f.renderText(text)

Example - requirements file

pyfiglet == 0.7.2
setuptools

Note

If you have defined a JYTHONPATH environment variable, you will need to add the python-lib folder’s path to its value.

Editing the Executable File

1. Open the executable found in the bin folder for editing.
2. Change the Dpython.path key’s value to the desired path.
3. Import the package as you normally would in Python from within the action’s script.

Adding the Package Location to sys.path

1. In the action’s Python script, import the sys module.
2. Use sys.path.append() to add the path to the desired module.
3. Import the module and use it.

Example - takes path as input parameter, adds it to sys.path and imports desired module

inputs:
  - path
python_action:
  script: |
    import sys
    sys.path.append(path)
    import module_to_import
    print module_to_import.something()

Importing Python Scripts

To import a Python script in a python_action:

1. Add the Python script to the python-lib folder, which is found at the same level as the bin folder that contains the CLI executable.
2. Import the script as you normally would in Python from within the action’s script.

Note

If you have defined a JYTHONPATH environment variable, you will need to add the python-lib folder’s path to its value.

sensitive

The key sensitive is a property of an input, output or system property name. It is mapped to a boolean value.

The sensitivity of an input or output is transitive, and is therefore determined by its sensitive property and by the sensitivity of variables used in its related value expression.

The “sensitive” property works for both flow input and step input.

Values that are sensitive will not be printed in logs, events or in outputs of the CLI and Build Tool.

Example - two sensitive inputs

inputs:
  - input1:
      default: "default value"
      sensitive: true
  - input1plus:
      default: ${ get("input1") + "something else" }

Example - two sensitive outputs

outputs:
  - output1:
      value: ${output1}
      sensitive: true
  - output2: ${already_sensitive_value}

Example - a sensitive system property

properties:
  - password:
      value: 'pwd'
      sensitive: true

step

A name of a step which is a property of workflow.

A step name must conform to the rules for names and uniqueness. Additionally, a step cannot be named on_failure.

Every step which is not declared with the on_failure keyword must be reachable from another step.

step input

Property	Required	Default	Value Type	Description	More info
value	no	–	expression	value of the step input	value
sensitive	no	transitive sensitivity or false	boolean	is the step input sensitive	sensitive

There are several types of steps:

• standard
• iterative
• parallel

Example - step with two arguments, one of which contains a default value

- divider:
    do:
      some_op:
        - host
        - port: '25'

Standard Step

A standard step calls an operation or subflow once.

The step name is mapped to the step’s properties.

Property	Required	Default	Value Type	Description	More Info
do	yes	–	operation or subflow call	the operation or subflow this step will run	do flow operation
publish	no	–	list of key:value pairs	operation outputs to publish to the flow level	publish, outputs
navigate	no	FAILURE: on_failure or flow finish SUCCESS: next step	list of key:value pairs	navigation logic from operation or flow results	navigation results

Example - step that performs a division of two inputs, publishes the answer and navigates accordingly

- divider:
    do:
      divide:
        - dividend: ${input1}
        - divisor: ${input2}
    publish:
      - answer: ${quotient}
    navigate:
      - ILLEGAL: FAILURE
      - SUCCESS: printer

Iterative Step

An iterative step calls an operation or subflow iteratively, for each value in a list.

The step name is mapped to the iterative step’s properties.

Property	Required	Default	Value Type	Description	More Info
loop	yes	–	key	container for loop properties	for
navigate	no	FAILURE: on_failure or flow finish SUCCESS: next step	key:value pairs	navigation logic from break or the result of the last iteration of the operation or flow	navigation results

Example - step prints all the values in value_list and then navigates to a step named “another_step”

- print_values:
    loop:
      for: value in value_list
      do:
        print:
          - text: ${value}
    navigate:
      - SUCCESS: another_step
      - FAILURE: FAILURE

Parallel Step

A parallel step calls an operation or subflow in parallel branches, for each value in a list.

The step name is mapped to the parallel step’s properties.

Property	Required	Default	Value Type	Description	More Info
parallel_loop	yes	–	key	container for parallel loop properties	parallel_loop
publish	no	–	list of key:values	values to aggregate from parallel branches loop properties	publish
navigate	no	FAILURE: on_failure or flow finish SUCCESS: next step	key:value pairs	navigation logic	navigation results

Example - step prints all the values in value_list in parallel and then navigates to a step named “another_step”

- print_values:
    parallel_loop:
      for: value in values_list
      do:
        print_branch:
          - ID: ${value}
    publish:
        - name_list: ${map(lambda x:str(x['name']), branches_context)}
    navigate:
        - SUCCESS: another_step
        - FAILURE: FAILURE

value

The key value is a property of an step input, output or system property name. In an output, the key is mapped to an expression value. In a system property, the key is mapped to a valid system property value.

The value key is most often used in conjunction with the sensitive key. Otherwise, an output or system property’s value can be defined inline by mapping it to the output or system property’s name.

Example - output values

outputs:
  - password:
      value: ${password}
      sensitive: true
  - another_output: ${op_output}

Example - system property values

properties:
  - props.password:
      value: 'pwd'
      sensitive: true
  - props.another_property: 'prop value'

workflow

The key workflow is a property of a flow. It is mapped to a list of the workflow’s steps.

Defines a container for the steps, their published variables and navigation logic.

The first step in the workflow is the starting step of the flow. From there the flow continues sequentially by default upon receiving results of SUCCESS, to the flow finish or to on_failure upon a result of FAILURE, or following whatever overriding navigation logic that is present.

Property | Required | Default | Value Type | Description | More Info
on_failure	no	–	step	default navigation target for FAILURE	on_failure step

Example - workflow that divides two numbers and prints them out if the division was legal

workflow:
  - divider:
      do:
        divide:
          - dividend: ${input1}
          - divisor: ${input2}
      publish:
        - answer: ${quotient}
      navigate:
        - ILLEGAL: FAILURE
        - SUCCESS: printer
  - printer:
      do:
        print:
          - text: ${input1 + "/" + input2 + " = " + answer}

Functions (A-Z)

check_empty()

May appear in the value of an input, output, publish or result expression.

The function in the form of check_empty(expression1, expression2) returns the value associated with expression1 if expression1 does not evaluate to None. If expression1 evaluates to None the function returns the value associated with expression2.

Example - usage of check_empty to check operation output in a flow

flow:
  name: flow
  inputs:
    - in1
  workflow:
    - step1:
        do:
          operation:
            - in1
        publish:
          - pub1: ${check_empty(out1, 'x marks the spot')}
          #if in1 was not 'x' then out1 is 'not x' and pub1 is therefore 'not x'
          #if in1 was 'x' then out1 is None and pub1 is therefore 'x marks the spot'
  outputs:
    - pub1

operation:
  name: operation
  inputs:
    - in1
  python_action:
    script: |
      out1 = 'not x' if in1 != 'x' else None
  outputs:
    - out1

cs_regex

May appear in the value of an input and expression.

Example - usage of ``regex`` function in expression editor to match e-mail addresses

Apply a regular expression on the selected text as in the following example:

cs_regex(selection, regex, split_lines)

Wherein, selection: content to be modified, regex: the regular expression, and split_lines (Optional) whereby you can split selection into lines and apply regex on each line. If not present, apply regex on the entire selection.

Input example for cs_regex :

selection="test@test.com, t3$~!@1231.123, uuu!@@j@.com, 1#_h!@jkl.co.uk, good-email_me@microfocus.oo, test_me%+-@microfocus13.co.uk"
regex="\\b[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}\\b"

Output sample of the result:

cs_regex(selection, “\b[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+.[a-zA-Z]{2,}\b”) = [”test@test.com”, “good-email_me@microfocus.oo”, “test_me%+-@microfocus13.co.uk”]

cs_json_query

May appear in the value of an input and expression.

Example - usage of ``json_path`` function in an expression editor to find out the bicycle’s color

JSON selection example

selection =  "{
  "store": {
     "book": [
      {
        "category": "history",
        "author": "Arnold Joseph Toynbee",
        "title": "A Study of History",
        "price": 5.50
      },
      {
        "category": "poem",
        "author": "Aneirin",
        "title": "Y Gododdin",
        "price": 17.00
      },
      {
        "category": "fiction",
        "author": "James Joyce",
        "title": "Finnegans Wake",
        "isbn": "9788804677628",
        "price": 15.99
      },
      {
        "category": "fiction",
        "author": "Emily Bronte",
        "title": "Wuthering Heights",
        "isbn": "0-486-29256-8",
        "price": 3.30
      }
    ],
    "bicycle": {
      "color": "purple",
      "price": 25.45
    }
  },
  "expensive": 10
}"

Input example for json_path :

json_path="$.store.bicycle.color"

Output sample of the result:

cs_json_query(selection, "$.store.bicycle.color") = "purple"

cs_xpath_query

May appear in the value of an input and expression.

Example - usage of ``xpath`` function in an expression editor to find out the title of all books

xpath selection example

 selection =
"<root>
   <store>
     <book>
       <category>history</category>
       <author>Arnold Joseph    Toynbee</author>
       <title>A Study of History</title>
       <price>5.5</price>
     </book>
     <book>
       <category>poem</category>
       <author>Aneirin</author>
       <title>Y Gododdin</title>
       <price>17</price>
     </book>
     <book>
       <category>fiction</category>
       <author>James Joyce</author>
       <title>Finnegans Wake</title>
       <isbn>9788804677628</isbn>
       <price>15.99</price>
     </book>
     <book>
       <category>fiction</category>
       <author>Emily Bronte</author>
       <title>Wuthering Heights</title>
       <isbn>0-486-29256-8</isbn>
       <price>3.3</price>
     </book>
     <bicycle>
       <color>purple</color>
       <price>25.45</price>
     </bicycle>
   </store>
   <expensive>10</expensive>
 </root>"

Input example for xpath :

xpath=".//title"

Output sample of the result:

cs_xpath_query(selection, ".//title")= <title>A Study of History</title> <title>Y Gododdin</title> <title>Finnegans Wake</title> <title>Wuthering Heights</title>

cs_append

May appear in the value of an input and expression.

Example - usage of ``cs_append`` function in an expression editor to append a string to another string value

selection = "hello"

Input example for cs_append :

cs_append(selection, to_append)

Wherein, selection is the content to append with and to_append is the value of append.

In this example, selection = "hello" and to_append = "world".

Output sample of the result:

cs_append(selection, to_append)="helloworld"

cs_extract_number

May appear in the value of an input and expression.

Example - usage of ``cs_extract_number`` function to extract a number from text

selection = "aei5ou634"

Scenario 1: Input example for cs_extract_number:

cs_extract_number(selection)

Wherein, selection is the text to count.

Scenario 1: Output sample of the result:

cs_extract_number(selection)=5

Scenario 2: Input example for cs_extract_number:

cs_extract_number(selection, [n_th])

Wherein, selection is the text to count, and n_th is the parameter to return the nth number from the given text.

Scenario 2: Output sample of the result:

cs_extract_number(selection, 2)=634

Note

The parameter n_th is optional. It returns the required occurrence of the number from the given sequence. If not specified, the first number will be extracted.

cs_substring

May appear in the value of an input and expression.

Example - usage of ``cs_substring`` function to return the ``substring`` from the text

selection = "helloworld"

Scenario 1: Input example for cs_substring(selection, start, [end]):

Note

The parameters start and end should be constants.

cs_substring(selection, 5)

Wherein, selection is the text to extract the substring and 5 is the parameter to extract the substring.

Scenario 1: Output sample of the result:

cs_substring(selection, 5)="world"

Scenario 2: Input example for cs_substring(selection, start, [end]):

cs_substring(selection, 5, 7)

Wherein, selection is the text to extract the substrings, and 5 and 7 are the parameters to extract the substrings.

Scenario 2: Output sample of the result:

cs_substring(selection, 5, 7)="wo"

Note

The parameter end is optional. It identifies the position of the string. If not present, the extraction evaluates until the end of ‘selection’ text.

cs_to_lower

May appear in the value of an input and expression.

Example - usage of ``cs_to_lower`` function in an expression editor to convert the text to lower case

selection = "aeIOU"

Input example for cs_to_lower :

cs_to_lower(selection)

Wherein, selection is the content to convert to lowercase.

Output sample of the result:

cs_to_lower(selection)="aeiou"

cs_to_upper

May appear in the value of an input and expression.

Example - usage of ``cs_to_upper`` function in an expression editor to convert the text to upper case

selection = "aeIOU"

Input example for cs_to_upper :

cs_to_upper(selection)

Wherein, selection is the content to convert to lowercase.

Output sample of the result:

cs_to_upper(selection)="AEIOU"

cs_prepend

May appear in the value of an input and expression.

Example - usage of ``cs_prepend`` function in an expression editor to prepend the text with the required value

selection = "world"

Input example for cs_prepend :

cs_prepend(selection, to_prepend)

Wherein, selection is the text to prepend, and to_prepend is the parameter to prepend the selected content.

Output sample of the result:

cs_prepend(selection, to_prepend)="helloworld"

cs_replace

May appear in the value of an input and expression.

Example - usage of ``cs_replace`` function to return the replaced value of the text

selection = "helloworldworldworld"

Scenario 1: Input example for cs_replace(old_val, new_val, [count]):

Note

The parameter count is optional and if used it should be a constant.

It counts the occurrences of replaceable content. If present, only the required number of occurrences will be replaced, else all the occurrences will be replaced.

cs_replace(selection, old_val, new_val)

Wherein, selection is the text to replace, and old_val is the parameter of an old value and new_val is the parameter to identify the new value.

In this example, old_val = "world" and new_val = "people".

Scenario 1: Output sample of the result:

cs_replace(selection, old_val, new_val)=hellopeoplepeoplepeople

Scenario 2: Input example for cs_replace(selection, old_val, new_val, 2):

cs_replace(selection, old_val, new_val, 2)

Where:

• selection: text to replace.
• old_val: parameter to identify the old value.
• new_val: parameter to replace the old value with new value.
• 2: parameter to replace the first two occurrences of old value.

Scenario 2: Output sample of the result:

cs_replace(selection, old_val, new_val, 2)="hellopeoplepeopleworld"

cs_round

May appear in the value of an input and expression.

Example - usage of ``cs_round`` function in an expression editor to round off the decimal number to its nearest value

selection = 2.4

Input example for cs_round :

cs_round(selection)

Wherein, selection is the number to round off to the nearest value.

Output sample of the result:

cs_round(selection)=2

Note

• if the decimal value of an integer in the selection is less than (< ) .5 , then the output will return the lower integer value. In this example, it will be 2
• if the decimal value of an integer in the selection is more than (> ) .5 , then the output will return the next higher integer value. In this example, it will be 3

get()

May appear in the value of an input, output, publish or result expression.

The function in the form of get('key') returns the value associated with key if the key is defined. If the key is undefined the function returns None.

The function in the form of get('key', 'default_value') returns the value associated with key if the key is defined and its value is not None. If the key is undefined or its value is None the function returns the default_value.

Example - usage of get function in inputs and outputs

inputs:
  - input1:
      required: false
  - input1_safe:
      default: ${get('input1', 'default_input')}
      private: true

workflow:
  - step1:
      do:
        print:
          - text: ${input1_safe}
      publish:
        - some_output: ${get('output1', 'default_output')}

outputs:
  - some_output

get_sp()

May appear in the value of an input, step argument, publish, output or result expression.

The function in the form of get_sp('key', 'default_value') returns the value associated with the system property named key if the key is defined and its value is not null. If the key is undefined or its value is null the function returns the default_value. The key is the fully qualified name of the system property, meaning the namespace (if there is one) of the file in which it is found followed by a dot . and the name of the key.

System property values are always strings or null. Values of other types (numeric, list, map, etc.) are converted to string representations.

System properties are not enforced at compile time. They are assigned at runtime.

Note

If multiple system properties files are being used and they contain a system property with the same fully qualified name, the property in the file that is loaded last will overwrite the others with the same name.

Example - system properties file

namespace: examples.sysprops

properties:
  - host: 'localhost'
  - port: 8080

Example - system properties used as input values

inputs:
  - host: ${get_sp('examples.sysprops.hostname')}
  - port: ${get_sp('examples.sysprops.port', '8080')}

To pass a system properties file to the CLI, see Run with System Properties.

Examples

The following simplified examples demonstrate some of the key CloudSlang concepts.

• Example 1 - User-defined Navigation and Publishing Outputs
• Example 2 - Default Navigation
• Example 3 - Subflow
• Example 4 - Loops
• Example 5 - Parallel Loop
• Example 6 - Operation Paths

Each of the examples below can be run by doing the following:

1. Create a new folder.
2. Create new CloudSlang(.sl) files and copy the code into them.
3. Use the CLI to run the flow.

For more information on getting set up to run flows, see the CloudSlang CLI and Hello World Example sections.

Example 1 - User-defined Navigation and Publishing Outputs

This example is a full working version from which many of the example snippets above have been taken. The flow takes in two inputs, divides them and prints the answer. In case of a division by zero, the flow does not print the output of the division, but instead ends with a user-defined result of ILLEGAL.

Download code

Flow - division.sl

namespace: examples.divide

flow:
  name: division

  inputs:
    - input1
    - input2

  workflow:
    - divider:
        do:
          divide:
            - dividend: ${input1}
            - divisor: ${input2}
        publish:
          - answer: ${quotient}
        navigate:
          - ILLEGAL: ILLEGAL
          - SUCCESS: printer
    - printer:
        do:
          print:
            - text: ${input1 + "/" + input2 + " = " + str(answer)}
        navigate:
          - SUCCESS: SUCCESS

  outputs:
    - quotient: ${answer}

  results:
    - ILLEGAL
    - SUCCESS

Operations - divide.sl

namespace: examples.divide

operation:
  name: divide

  inputs:
    - dividend
    - divisor

  python_action:
    script: |
      if divisor == '0':
        quotient = 'division by zero error'
      else:
        quotient = str(float(dividend) / float(divisor))

  outputs:
    - quotient

  results:
    - ILLEGAL: ${quotient == 'division by zero error'}
    - SUCCESS

Operation - print.sl

namespace: examples.divide

operation:
  name: print

  inputs:
    - text

  python_action:
    script: print text

  results:
    - SUCCESS

Example 2 - Default Navigation

In this example the flow takes in two inputs, one of which determines the success of its first step.

• If the first step succeeds, the flow continues with the default navigation sequentially by performing the next step. That step returns a default result of SUCCESS and therefore skips the on_failure step, ending the flow with a result of SUCCESS.
• If the first step fails, the flow moves to the on_failure step by default navigation. When the on_failure step is done, the flow ends with a default result of FAILURE.

Download code

Flow - nav_flow.sl

namespace: examples.defaultnav

flow:
  name: nav_flow

  inputs:
    - navigation_type
    - email_recipient

  workflow:
    - produce_default_navigation:
        do:
          produce_default_navigation:
            - navigation_type

    # default navigation - go to this step on success
    - do_something:
        do:
          something:

    # default navigation - go to this step on failure
    - on_failure:
      - send_error_mail:
          do:
            send_email_mock:
              - recipient: ${email_recipient}
              - subject: "Flow failure"

Operation - produce_default_navigation.sl

namespace: examples.defaultnav

operation:
  name: produce_default_navigation

  inputs:
    - navigation_type

  python_action:
    script: |
      print 'Default navigation based on input of - ' + navigation_type

  results:
    - SUCCESS: ${navigation_type == 'success'}
    - FAILURE

Operation - something.sl

namespace: examples.defaultnav

operation:
  name: something

  python_action:
      script: |
        x = 'Doing something important'
        print x

  results:
    - FAILURE: ${x == 'important thing not done'}
    - SUCCESS

Operation - send_email_mock.sl

namespace: examples.defaultnav

operation:
  name: send_email_mock

  inputs:
    - recipient
    - subject

  python_action:
    script: |
      print 'Email sent to ' + recipient + ' with subject - ' + subject

Example 3 - Subflow

This example uses the flow from Example 1 as a subflow. It takes in four numbers (or uses default ones) to call division_flow twice. If either division returns the ILLEGAL result, navigation is routed to the on_failure step and the flow ends with a result of FAILURE. If both divisions are successful, the on_failure step is skipped and the flow ends with a result of SUCCESS.

Note

To run this flow, the files from Example 1 should be placed in the same folder as this flow file or use the --cp flag at the command line.

Download code

Flow - master_divider.sl

namespace: examples.divide

flow:
  name: master_divider

  inputs:
    - dividend1: "3"
    - divisor1: "2"
    - dividend2: "1"
    - divisor2: "0"

  workflow:
    - division1:
        do:
          division:
            - input1: ${dividend1}
            - input2: ${divisor1}
        publish:
          - ans: ${quotient}
        navigate:
          - SUCCESS: division2
          - ILLEGAL: failure_step
    - division2:
        do:
          division:
            - input1: ${dividend2}
            - input2: ${divisor2}
        publish:
          - ans: ${quotient}
        navigate:
          - SUCCESS: SUCCESS
          - ILLEGAL: failure_step
    - on_failure:
      - failure_step:
          do:
            print:
              - text: ${ans}

Example 4 - Loops

This example demonstrates the different types of values that can be looped on and various methods for handling loop breaks.

Download code

Flow - loops.sl

namespace: examples.loops

flow:
  name: loops

  inputs:
    - sum:
        default: '0'
        private: true

  workflow:
    - fail3a:
        loop:
          for: value in [1,2,3,4,5]
          do:
            fail3:
              - text: ${str(value)}
        navigate:
          - SUCCESS: fail3b
          - FAILURE: fail3b
    - fail3b:
        loop:
          for: value in [1,2,3,4,5]
          do:
            fail3:
              - text: ${str(value)}
          break: []
    - custom3:
        loop:
          for: value in "1,2,3,4,5"
          do:
            custom3:
              - text: ${value}
          break:
            - CUSTOM
        navigate:
          - CUSTOM: aggregate
          - SUCCESS: skip_this
    - skip_this:
        do:
          print:
            - text: "This will not run."
        navigate:
          - SUCCESS: aggregate
    - aggregate:
        loop:
          for: value in range(1,6)
          do:
            print:
              - text: ${str(value)}
              - sum
          publish:
            - sum: ${str(int(sum) + int(out))}
          break: []
        navigate:
          - SUCCESS: print
    - print:
        do:
          print:
            - text: ${sum}
        navigate:
          - SUCCESS: SUCCESS

Operation - fail3.sl

namespace: examples.loops

operation:
  name: fail3

  inputs:
    - text

  python_action:
    script: print text

  results:
    - FAILURE: ${int(text) == 3}
    - SUCCESS

Operation - custom3.sl

namespace: examples.loops

operation:
  name: custom3

  inputs:
    - text

  python_action:
    script: print text

  results:
    - CUSTOM: ${int(text) == 3}
    - SUCCESS

Operation - print.sl

namespace: examples.loops

operation:
  name: print

  inputs:
    - text

  python_action:
    script: print text

  outputs:
    - out: ${text}

  results:
    - SUCCESS

Example 5 - Parallel Loop

This example demonstrates the usage of a parallel loop including aggregation.

Download code

Flow - parallel_loop_aggregate.sl

namespace: examples.parallel

flow:
  name: parallel_loop_aggregate

  inputs:
  - values: "1 2 3 4"

  workflow:
    - print_values:
        parallel_loop:
          for: value in values.split()
          do:
            print_branch:
              - ID: ${str(value)}
        publish:
          - name_list: "${', '.join(map(lambda x : str(x['name']), branches_context))}"
          - first_name: ${branches_context[0]['name']}
          - last_name: ${branches_context[-1]['name']}
          - total: "${str(sum(map(lambda x : int(x['num']), branches_context)))}"
        navigate:
          - SUCCESS: SUCCESS

  outputs:
    - name_list
    - first_name
    - last_name
    - total

  results:
    - SUCCESS

Operation - print_branch.sl

namespace: examples.parallel

operation:
  name: print_branch

  inputs:
    - ID

  python_action:
    script: |
        name = 'branch ' + str(ID)
        print 'Hello from ' + name

  outputs:
    - name
    - num: ${ID}

Example 6 - Operation Paths

This example demonstrates the various ways to reference an operation or subflow from a flow step.

This example uses the following folder structure:

• examples
  • paths
    • flow.sl
    • op1.sl
    • folder_a
      • op2.sl
    • folder_b
      • op3.sl
      • folder_c
        • op4.sl

Download code

Flow - flow.sl

namespace: examples.paths

imports:
  alias: examples.paths.folder_b

flow:
  name: flow

  workflow:
    - default_path:
        do:
          op1:
            - text: "default path"
        navigate:
          - SUCCESS: fully_qualified_path
    - fully_qualified_path:
        do:
          examples.paths.folder_a.op2:
            - text: "fully qualified path"
        navigate:
          - SUCCESS: using_alias
    - using_alias:
        do:
          alias.op3:
            - text: "using alias"
        navigate:
          - SUCCESS: alias_continuation
    - alias_continuation:
        do:
          alias.folder_c.op4:
            - text: "alias continuation"
        navigate:
          - SUCCESS: SUCCESS

  results:
    - SUCCESS

Operation - op1.sl

namespace: examples.paths

operation:
  name: op1

  inputs:
    - text

  python_action:
    script: print text

Operation - op2.sl

namespace: examples.paths.folder_a

operation:
  name: op2

  inputs:
    - text

  python_action:
    script: print text

Operation - op3.sl

namespace: examples.paths.folder_b

operation:
  name: op3

  inputs:
    - text

  python_action:
    script: print text

Operation - op4.sl

namespace: examples.paths.folder_b.folder_c

operation:
  name: op4

  inputs:
    - text

  python_action:
    script: print text

Tests

CloudSlang tests are written to test CloudSlang content and are run during the build process by the CloudSlang Build Tool.

Wrapper Flows

Test cases either test a flow or operation directly or use a wrapper flow that calls the flow or operation to be tested.

Wrapper flows are often used to set up an environment before the test runs and to clean up the environment after the test. They are also sometimes necessary for complex tests of a flow or operation’s outputs.

Wrapper flows are written in CloudSlang using the .sl extension and use the normal flow syntax.

Test Suites

Test suites are groups of tests that are only run if the build declares them as active. Test suites are often used to group tests that require a certain environment that may or may not be present in order to run. When the environment is present the suite can be activated and when it is not present the tests will not run.

Tests declare which test suites they are a part of, if any, using the testSuites property.

If no test suites are defined for a given test case, the test will run unless !default is passed to the CloudSlang Build Tool.

Note

When using Linux, the exclamation mark (!) needs to be escaped with a preceding backslash (\). So, to ignore default tests, pass \!default to the CloudSlang Build Tool.

Test Case Syntax

CloudSlang test files are written in YAML with the .inputs.yaml extension and contain one or more test cases.

Each test case begins with a unique key that is the test case name. The name is mapped to the following test case properties:

Property	Required	Value Type	Description
inputs	no	list of key:value pairs	inputs to pass to the flow or operation being tested
systemPropertiesFile	no	string	path to the system properties file for the flow or operation - ${project_path} can be used for specifying a path relative to the project path, for example, systemPropertiesFile: ${project_path}\content\base\properties.yaml
description	no	string	description of test case
testFlowPath	yes	string	qualified name of the flow, operation or wrapper flow to test
testSuites	no	list	list of suites this test belongs to
outputs	no	list of key:value pairs	expected output values of the flow, operation or wrapper flow being tested
result	no	flow or operation result	expected result of the flow, operation or wrapper flow being tested
throwsException	no	boolean	whether or not to expect an exception

Note

The outputs parameter does not need to test all of a flow or operation’s outputs.

Example - test cases that test the match_regex operation

testMatchRegexMatch:
  inputs:
    - regex: 'a+b'
    - text: aaabc
  description: Tests that match_regex.sl operation finishes with MATCH for specified regex/text
  testFlowPath: io.cloudslang.base.strings.match_regex
  outputs:
    - match_text: 'aaab'
  result: MATCH

testMatchRegexMissingInputs:
  inputs:
    - text: HELLO WORLD
  description: Tests that match_regex.sl operation throws an exception when a required input is missing
  testFlowPath: io.cloudslang.base.strings.match_regex
  outputs:
    - match_text: ''
  throwsException: true

Run Tests

To run test cases use the CloudSlang Build Tool. Test cases are not run by the CloudSlang CLI.

Best Practices

The following is a list of best practices for authoring CloudSlang files. Many of these best practices are checked when using the CloudSlang Build Tool.

CloudSlang Content Best Practices

• The namespace for a file matches the suffix of the file path in which the file resides, for example, the send_mail operation is found in the cloudslang-content/io/cloudslang/base/mail folder, so it uses the namespace io.cloudslang.base.mail.

• Namespaces should be comprised of only lowercase alphanumeric characters (a-z and 0-9), underscores (_), periods(.) and hyphens (-).

• A flow or operation has the same name as the file it is in.

• Each file has one flow or one operation.

• Flows and operations reside together in the same folders.

• System properties files do not reside in the same folder together with flows and operations.

• Steps call subflows or operations using their fully qualified name or an alias created in the imports section, even when the subflow or operation resides in the same folder as the calling flow.

• Identifiers (flow names, operation names, input names, etc.) are written:

  • In snake_case, lowercase letters with underscores (_) between words, in all cases other than inputs to a Java @Action.
  • In camelCase, starting with a lowercase letter and each additional word starting with an uppercase letter appended without a delimiter, for inputs to a Java @Action.
  • Results are written in ALL_CAPS.

• Assign only relevant default values. For example, 8080 is a good candidate for a port number, but john_doe is probably not a good candidate for a user name.

• String values are written in one of the YAML quoted styles (' or ") or block styles (| or >). For more information, see YAML Overview - Scalars.

• Flow and operation files begin with a commented description and list of annotated inputs, outputs and results (see CloudSlang Comments Style Guide).

  • Optional parameters, default and valid values are noted.
  • Examples are provided when useful.

CloudSlang Tests Best Practices

• Tests are contained in a directory with a folder structure identical to the structure of the directory they are testing.
• Tests for a particular CloudSlang file are written in a file with the same name, but with the .inputs.yaml extension, for example, the flow print_text.sl is tested by tests in print_text.inputs.yaml.
• Wrapper flows reside in the same folder as the tests call them.
• String values are written in one of the YAML quoted styles (' or ") or block styles (| or >). For more information, see YAML Overview - Scalars.

Note

In future releases some of the above best practices may be required by the CloudSlang compiler.

CloudSlang Inputs Files Best Practices

• The name of an inputs file ends with .inputs.yaml.

CloudSlang Comments Style Guide

All CloudSlang flows and operations should begin with a documentation block that describes the flow or operation, and lists the inputs, outputs and results.

Note

The CloudSlang IntelliJ plugin will warn you in case there of missing documentation for the operation/flow, inputs, outputs, results descriptions.

A flow or operation’s documentation may be viewed from the CLI using the inspect command.

• Documentation blocks begin with a line containing #!! and nothing else.

• Documentation blocks end with a line containing #!!# and nothing else.

• Each line of the documentation begins with #!.

• Lines in the documentation block that do not begin with #! will not be considered as part of the documentation and will not display when the file is inspected.

• The @description tag is the only mandatory tag.

• The other possible tags are:

  • @prerequisites
  • @input <input_name>
  • @output <output_name>
  • @result <result_name>

####################################################################################################################
#!!
#! @description: This operation does something fantastic.
#!
#! @prerequisites: Some Python module.
#!
#! @input input_1: First input.
#! @input input_2: Second input.
#!                 Valid: 'true', 'false'
#!                 Default: 'true'
#! @input input_3: Third input.
#!                 Example: 'someone@mailprovider.com'
#!                 Optional
#! @input input_4: Fourth input.
#!                 Format: space delimited list of strings
#!
#! @output output_1: First output.
#!
#! @result SUCCESS: Operation executed successfully.
#! @result FAILURE: An error occurred while trying to complete the operation.
#!!#
####################################################################################################################

Description

• Written as a sentence, beginning with a capital letter and ending with a period.
• Written in the present tense, for example, “Prints text.”.

Prerequisites

• Flows and operations that assume prerequisites should declare them.

Inputs, Outputs and Results

• Fields appear in the same order as they appear in the code.
• Description begins with a lowercase letter (unless a proper name or capitalized acronym) and does not end with a period.
• Usage of the words “the” and “a” are strongly discouraged, especially at the beginning of the description.
• Proper names and acronyms that are normally capitalized are capitalized, for example, HTTP, Docker, ID.

Inputs and Outputs

• Written in the present tense, for example, “true if job exists”.
• Non-required fields contain the “Optional” label.
• Additional labels are “Default:”, “Example:”, “Valid:” and “Format:”.

Results

• Actions written in the past tense, for example, “error occurred”. States written in the present tense, for example, “application is up”.
• Default results which do not require any additional explanation are omitted.

Recurring Fields

• Fields that appear often with the same meaning should have the same name and description across flows and operations. However, if the meaning is specific to the flow or operation, the field description may be different. Some examples are:

  • FAILURE - otherwise
  • error_message - error message if error occurred
  • command - command to execute

CLI

There are several ways to get started with the CloudSlang CLI.

Download and Run Pre-built CLI

Prerequisites : To run the CloudSlang CLI, Java JRE version 7 or higher is required.

1. Download the CLI with content zip file.
2. Locate the downloaded file and unzip the archive. The decompressed file contains:
   • A folder named cslang-cli with the CLI tool and its necessary dependencies.
   • A folder named content with ready-made CloudSlang flows and operations.
   • A folder named python-lib.
3. Navigate to the folder cslang-cli\bin\.
4. Run the executable:
   • For Windows : cslang.bat.
   • For Linux : bash cslang.

Set up python for CloudSlang CLI

To use an external python in cloudslang-cli, set the use.jython.expressions property to false in cslang-cli/configuration/cslang.properties.

Important

By default, the property use.jython.expressions=true is set to true pointing to use Jython.

1. Unzip the downloaded python 3.8.7 package to any directory. Notice the python-3.8.7 folder after unzipping.
2. Navigate to cslang-cli/configuration and open the cslang.properties file using any text editor.
3. Specify a property python.path to point it to the following locations based on your Operating System:

• Windows: <unzipped_location>/python-3.8.7
• Linux: <unzipped_location>/python-3.8.7/bin

1. Additionally for Linux environment, perform the following steps to grant permissions:

   1. Open the terminal shell at /python-3.8.7/bin folder, and then run the following command:

      ln -sf python3 python
      

   2. Navigate back to /python-3.8.7 folder, and then run the following command. The permission to access the folder is granted.

      chmod -R 755 bin
      

Download, Build and Run CLI

Prerequisites:

• To build the CloudSlang CLI, Java JDK version 7 or higher and Maven version 3.0.3 or higher are required.

Perform the following tasks:

1. Git clone (or GitHub fork and then clone) the source code.
2. Using the Command Prompt, navigate to the project root directory.
3. Build the project by running mvn clean install.
4. After the build finishes, navigate to the cloudslang-cli\target\cloudslang\bin folder.
5. Run the executable:
   • For Windows : cslang.bat.
   • For Linux : bash cslang.

Download and Install the npm Package

Prerequisites:

• To download the package, Node.js is required.
• To run the CloudSlang CLI, Java JRE version 7 or higher is required.

To install:

1. For Windows, open the command prompt, and then issue the following command:

npm install -g cloudslang-cli

2. For Linux, the sudo command may be required:

sudo npm install -g cloudslang-cli

3. Next, enter the cslang command at any command prompt.

Docker Image

There are two CloudSlang Docker images available:

1. cloudslang/cloudslang is a lightweight image meant to get you running CloudSlang flows as quickly as possible.
2. cloudslang/cloudslang-dev adds the tools necessary to develop CloudSlang flows.

cloudslang/cloudslang

This image includes:

• Java
• CloudSlang CLI
• CloudSlang content

To get the image: docker pull cloudslang/cloudslang

To run a flow with a CloudSlang prompt:

• docker run -it cloudslang/cloudslang
• At the prompt enter: run --f ../content/io/cloudslang/.../flow.sl --i input1=value1

Or, to run the flow without the prompt:

docker run --rm cloudslang/cloudslang run --f ../content/io/cloudslang/.../flow.sl --i input1=value1

cloudslang/cloudslang-dev

This image includes:

• Java
• CloudSlang CLI
• CloudSlang content
• Python
• Pip
• Vim
• Emacs
• SSH
• Git
• Atom
• language-cloudslang Atom package

To get the image: docker pull cloudslang/cloudslang-dev

Configure the CLI

The CLI can be configured using the configuration file found at cslang-cli/configuration/cslang.properties.

Some of the configuration items are listed in the table below:

Configuration key	Default value	Description
log4j.configuration	file:${app.home}/configuration/logging/log4j.properties	Location of logging configuration file
cslang.encoding	utf-8	Character encoding for input values and input files
maven.home	${app.home}/maven/apache-maven-x.y.z	Location of CloudSlang Maven repository home directory
maven.settings.xml.path	${app.home}/maven/conf/settings.xml	Location of Maven settings file
cloudslang.maven.repo.local	${app.home}/maven/repo	Location of local repository
cloudslang.maven.repo.remote.url	http://repo1.maven.org/maven2	Location of remote Maven repository
cloudslang.maven.plugins.remote.url	http://repo1.maven.org/maven2	Location of remote Maven plugins

Logging Configuration

The CLI’s logging can be configured using the logging configuration file. The location of the logging configuration file is defined in the CLI’s configuration file.

Maven Configuration

The CLI uses Maven to manage Java action dependencies. There are several Maven configuration properties found in the CLI’s configuration file. To configure Maven to use a remote repository other than Maven Central, edit the values for cloudslang.maven.repo.remote.url and cloudslang.maven.plugins.remote.url. Additionally, you can edit the proxy settings in the file found at maven.settings.xml.path.

Maven Troubleshooting

It is possible that the CLI’s Maven repository can become corrupted (if running the first time, behind a proxy). In such a case, delete the entire repo folder found at the location indicated by the cloudslang.maven.repo.local key in the CLI’s configuration file and rerun the flow.

To use the default cloudslang.maven.repo.local behind a proxy, please see the Troubleshooting guide, where a more detailed explanation on how to fix any maven dependencies issues that might appear.

Use the CLI

When a flow is run, the entire directory in which the flow resides is scanned recursively (including all subfolders) for files with a valid CloudSlang extension. All of the files found are compiled by the CLI. If the --cp flag is used, all of the directories listed there will be scanned and compiled recursively as well.

Note

Use forward slashes (/) in all file paths, even on Windows, because back slashes (\) can be interpreted as special characters.

Run a Flow or Operation

To run a flow or operation located at c:/.../your_flow.sl, use the --f flag to specify the location of the flow to be run:

cslang>run --f c:/.../your_flow.sl

Run with Inputs

From the Command Line

If the flow or operation takes in input parameters, use the --i flag and a comma-separated list of key=value pairs:

cslang>run --f c:/.../your_flow.sl --i input1=root,input2=25

Commas (,) can be used as part of input values by escaping them with a backslash (\).

cslang>run --f c:/.../your_flow.sl --i list=1\,2\,3

To use inputs that include spaces, enclose the entire input list in quotes ("):

cslang>run --f c:/.../your_flow.sl --i "input1=Hello World, input2=x"

Double quotes (") can be used as part of quoted input values by escaping them with a backslash (\). When using a quoted input list, spaces between input parameters will be trimmed.

To pass the value “Hello” World to a flow:

cslang>run --f c:/.../your_flow.sl --i "input1=\"Hello\" World"

Using an Inputs File

Alternatively, inputs made be loaded from a file. Inputs files are written in flat YAML, containing a map of names to values. Inputs files end with the .yaml or .yml extensions. It is a best practice for the name of an inputs file to end with .inputs.yaml. If multiple inputs files are being used and they contain an input with the same name, the input in the file that is loaded last will overwrite the others with the same name.

Inputs files can be loaded automatically if placed in a folder located at cslang-cli/configuration/inputs. If the flow requires an inputs file that is not loaded automatically, use the --if flag and a comma-separated list of file paths. Inputs passed with the --i flag will override the inputs passed using a file.

Example - same inputs passed to flow using command line and inputs file

Inputs passed from the command line - run command

cslang>run --f C:/.../your_flow.sl --i "input1=simple text,input2=comma\, text,input3=\"quoted text\""

Inputs passed using an inputs file - run command

cslang>run --f C:/.../your_flow.sl --if C:/.../inputs.yaml

Inputs passed using an inputs file - inputs.yaml file

input1: simple text
input2: comma, text
input3: '"quoted text"'

Run with Dependencies

If the flow requires dependencies they can be added to the classpath using the --cp flag with a comma-separated list of dependency paths. If no cp flag is present, the cslang-cli/content folder is added to the classpath by default. If there is no --cp flag and no cslang-cli/content folder, the running flow or operation’s folder is added to the classpath by default.

cslang>run --f c:/.../your_flow.sl --i input1=root,input2=25 --cp c:/.../yaml

Run with System Properties

A system properties file is a type of CloudSlang file that contains a list of system property keys and values. If multiple system properties files are being used and they contain a system property with the same fully qualified name, the property in the file that is loaded last will overwrite the others with the same name.

System property names (keys) can contain alphanumeric characters (A-Z, a-z, 0-9), underscores (_) and hyphens (-). For more information on the structure of system properties files see the CloudSlang Files and properties sections of the DSL Reference.

System property files can be loaded automatically if placed in a folder or subfolder within cslang-cli/configuration/properties. If the flow or operation requires a system properties file that is not loaded automatically, use the --spf flag and a comma-separated list of file paths.

cslang>run --f c:/.../your_flow.sl --spf c:/.../yaml

Example - system properties file

namespace: examples.sysprops

properties:
  - host: 'localhost'
  - port: 8080

Note

System property values that are non-string types (numeric, list, map, etc.) are converted to string representations. A system property may have a value of null.

An empty system properties file can be defined using an empty list.

Example: empty system properties file

namespace: examples.sysprops

properties: []

Run in Non-Interactive Mode

A flow can be run without first starting up the CLI using the non-interactive mode.

From a shell prompt:

Windows

>cslang.bat run --f c:/.../your_flow.sl

Linux

>cslang run --f c:/.../your_flow.sl

Change the Verbosity Level

The CLI can run flows and operations at several levels of verbosity.

To change the verbosity level, use the --v flag.

Verbosity level	Printed to the screen	Syntax
default	step names and top-level outputs	no flag or --v default
quiet	top-level outputs	--v quiet
debug	default + each step’s published variables	--v or --v debug

Run in quiet mode:

cslang>run --f c:/.../your_flow.sl --v quiet

Run in debug mode:

cslang>run --f c:/.../your_flow.sl --v

Inspect a Flow or Operation

To view a flow or operation’s description, inputs, outputs and results use the inspect command.

cslang>inspect c:/.../your_flow.sl

List System Properties

To list the properties contained in a system properties file use the list command.

cslang>list c:/.../your_properties.prop.sl

Other Commands

Some of the available commands are:

• env --setAsync - Sets the execution mode to be synchronous (false) or asynchronous (true). By default the execution mode is synchronous, meaning only one flow can run at a time.

cslang>env --setAsync true

• inputs - Lists the inputs of a given flow.

cslang>inputs --f c:/.../your_flow.sl

• cslang --version - Displays the version of the CLI being used.

cslang>cslang --version

Execution Log

The execution log is saved at cslang-cli/logs/execution.log. The log file stores all the events that have been fired, and therefore allows for tracking a flow’s execution.

Maven Log

Log files of Maven activity are saved at cslang-cli/logs/maven/. Each artifact’s activity is stored in a file named with the convention <group>_<artifact>_<version>.log.

History

The CLI history is saved at cslang-cli/cslang-cli.history.

Help

To get a list of available commands, enter help at the CLI cslang> prompt. For further help, enter help and the name of the command.

Build Tool

The CloudSlang Build Tool checks the syntactic validity of CloudSlang files, their adherence to many of the best practices and runs their associated tests.

Running the CloudSlang Build Tool performs the following steps, each of which is written to the console:

1. Displays the active project, content and test paths.
2. Displays a list of the active test suites.
3. Compiles all CloudSlang files found in the content directory and all of its subfolders.
   • If there is a compilation error, it is displayed and the build terminates.
4. Compiles all CloudSlang test flows found in the test directory and all of its subfolders.
5. Parses all test cases files found in the test directory and all of its subfolders.
6. Runs all test cases found in the test case files that have no test suite or have a test suite that is active.
7. Displays the test cases that were skipped.
8. Reports the build’s status.
   • If the build fails, a list of failed test cases are displayed.

Sample Builder Run

11:08:12 [INFO]
11:08:12 [INFO] ------------------------------------------------------------
11:08:12 [INFO] Building project: C:\CloudSlang\test_code\build_tool
11:08:12 [INFO] Content root is at: C:\CloudSlang\test_code\build_tool\content
11:08:12 [INFO] Test root is at: C:\CloudSlang\test_code\build_tool\test
11:08:12 [INFO] Active test suites are: [default]
11:08:12 [INFO] Validate description: true
11:08:12 [INFO]
11:08:12 [INFO] Loading...
11:08:17 [INFO]
11:08:17 [INFO] ------------------------------------------------------------
11:08:17 [INFO] Building project: build_tool
11:08:17 [INFO] ------------------------------------------------------------
11:08:17 [INFO]
11:08:17 [INFO] --- compiling sources ---
11:08:17 [INFO] Start compiling all slang files under: C:\CloudSlang\test_code\build_tool\content
11:08:17 [INFO] 1 .sl files were found
11:08:17 [INFO]
11:08:17 [INFO] Compiled: 'build_tool.content.operation' successfully
11:08:17 [INFO] Successfully finished Compilation of: 1 Slang files
11:08:17 [INFO]
11:08:17 [INFO] --- compiling tests sources ---
11:08:17 [INFO] Start compiling all slang files under: C:\CloudSlang\test_code\build_tool\test
11:08:17 [INFO] 0 .sl files were found
11:08:17 [INFO]
11:08:17 [INFO] Compiled: 'build_tool.content.operation' successfully
11:08:17 [INFO]
11:08:17 [INFO] --- parsing test cases ---
11:08:17 [INFO] Start parsing all test cases files under: C:\CloudSlang\test_code\build_tool\test
11:08:17 [INFO] 1 test cases files were found
11:08:17 [INFO]
11:08:17 [INFO] --- running tests ---
11:08:17 [INFO] Found 2 tests
11:08:17 [INFO] Running test: testOperationFailure - Tests that operation.sl finishes with FAILURE
11:08:23 [ERROR] Test case failed: testOperationFailure - Tests that operation.sl finishes with FAILURE
      Expected result: FAILURE
      Actual result: SUCCESS
11:08:23 [INFO] Running test: testOperationSuccess - Tests that operation.sl finishes with SUCCESS
11:08:23 [INFO] Test case passed: testOperationSuccess. Finished running: build_tool.content.operation with result: SUCCESS
11:08:23 [INFO] ------------------------------------------------------------
11:08:23 [INFO] Following 1 test cases passed:
11:08:23 [INFO] - testOperationSuccess
11:08:23 [INFO]
11:08:23 [INFO] ------------------------------------------------------------
11:08:23 [INFO] Following 1 executables have tests:
11:08:23 [INFO] - build_tool.content.operation
11:08:23 [INFO]
11:08:23 [INFO] ------------------------------------------------------------
11:08:23 [INFO] Following 0 executables do not have tests:
11:08:23 [INFO]
11:08:23 [INFO] ------------------------------------------------------------
11:08:23 [INFO] 100% of the content has tests
11:08:23 [INFO] Out of 1 executables, 1 executables have tests
11:08:23 [INFO] 1 test cases passed
11:08:23 [ERROR]
11:08:23 [ERROR] ------------------------------------------------------------
11:08:23 [ERROR] BUILD FAILURE
11:08:23 [ERROR] ------------------------------------------------------------
11:08:23 [ERROR] CloudSlang build for repository: "C:\CloudSlang\test_code\build_tool" failed due to failed tests.
11:08:23 [ERROR] Following 1 tests failed:
11:08:23 [ERROR] - Test case failed: testOperationFailure - Tests that operation.sl finishes with FAILURE

              Expected result: FAILURE
              Actual result: SUCCESS
11:08:23 [ERROR]

Download the Build Tool

The CloudSlang Build Tool can be downloaded from here.

Configure the Build Tool

The Build Tool can be configured using the configuration file found at cslang-builder/configuration/cslang.properties.

Configuration key	Default value	Description
cslang.encoding	utf-8	Character encoding for input values and input files
maven.home	${app.home}/maven/apache-maven-x.y.z	Location of CloudSlang Maven repository home directory
maven.settings.xml.path	${app.home}/maven/conf/settings.xml	Location of Maven settings file
cloudslang.maven.repo.local	${app.home}/maven/repo	Location of local repository
cloudslang.maven.repo.remote.url	http://repo1.maven.org/maven2	Location of remote Maven repository
cloudslang.maven.plugins.remote.url	http://repo1.maven.org/maven2	Location of remote Maven plugins
cloudslang.test.case.report.location	${app.home}/report	Location of test case report

Maven Configuration

The Build Tool uses Maven to manage Java action dependencies. There are several Maven configuration properties found in the Build Tool’s configuration file. To configure Maven to use a remote repository other than Maven Central, edit the values for cloudslang.maven.repo.remote.url and cloudslang.maven.plugins.remote.url. Additionally, you can edit the proxy settings in the file found at maven.settings.xml.path.

Maven Troubleshooting

It is possible that the Build Tool’s Maven repository can become corrupted. In such a case, delete the entire repo folder found at the location indicated by the cloudslang.maven.repo.local key in the Build Tool’s configuration file and rerun the builder.

Use the Build Tool

The CloudSlang Build Tool builds projects. A project consists of a folder that contains the CloudSlang content and a folder containing the tests for the content.

By default the build tool will look for a folder named content and a folder named test in the project folder to use as the content and test folders respectively. If they are present in the project folder, they do not have to be passed to the build tool.

To use the CloudSlang Build Tool with default settings, run the cslang-builder executable from the command line and pass the path to the project folder.

<builder path>\cslang-builder\bin>cslang-builder.bat <project path>

To use the CloudSlang Build Tool with specific settings, run the cslang-builder executable from the command line and pass the following arguments:

Argument	Default	Description
-pr	current folder	project root folder
-cr	<project root>/content	content root folder
-tr	<project root>/test	test root folder
-ts	none	list of test suites to run - use !default to skip tests that are not included in a test suite
-cov	off	whether or not test coverage data should be output
-des	off	whether or not to validate that all inputs, outputs and results have descriptions
-cs	off	whether or not to validate that all inputs, outputs and results follow the checkstyle rules
-co	–	only run tests from active suites that are affected by the items found in the given change list
-par	false	whether or not parallel test execution should be used
-th	number of available processors for the machine	number of threads for parallel runs
-rcf	–	absolute path for the run configuration properties file

Dynamic Parameters

Parameter	Description
-Dtest.case.timeout.in.minutes	number of minutes to wait before test case timeout

Note

To skip tests not included in a test suite when using Linux, the exclamation mark (!) needs to be escaped with a preceding backslash (\). So, to ignore default tests, pass \!default.

Note

Test coverage is calculated as a percentage of flows and operations for which tests exist, regardless of how much of each flow or operation is covered by the test. Additionally, a flow or operation will be considered covered even if its test’s suite did not run during the current build. The mere existence of a test for a flow or operation is enough to consider it as covered.

Change List

The change list is used in conjunction with the -co flag to run only the tests from active suites that are affected by the items in the list.

The list is a file in which each line contains an executable’s fully qualified name.

Example change list file

io.cloudslang.amazon.aws.ec2.images.add_launch_permissions_to_image_in_region
io.cloudslang.amazon.aws.ec2.images.create_image_in_region
io.cloudslang.amazon.aws.ec2.images.deregister_image_in_region
io.cloudslang.amazon.aws.ec2.images.describe_images_in_region
io.cloudslang.amazon.aws.ec2.images.get_launch_permission_for_image_in_region
io.cloudslang.amazon.aws.ec2.images.remove_launch_permissions_from_image_in_region
io.cloudslang.amazon.aws.ec2.images.reset_launch_permissions_on_image_in_region

Run Configuration Properties

The builder can refer to a run configuration properties file with the following key=value pairs. The values in the configuration file override values passed using command line arguments.

key	value type	description
test.coverage	boolean	whether or not test coverage data should be output
test.suites.run.mode.unspecified	parallel | sequential	mode to run tests not specified by values below
test.suites.active	comma delimited list	list of test suites to run - use !default to skip tests that are not included in a test suite
test.suites.sequential	comma delimited list	test suites to run sequentially
test.suites.parallel	comma delimited list	test suites to run in parallel
test.parallel.thread.count	integer	number of threads for parallel runs default: number of available processors for the machine

Build Tool Log

The builder log is saved at cslang-builder/logs/builder.log.

Maven Log

Log files of Maven activity are saved at cslang-builder/logs/maven/. Each artifact’s activity is stored in a file named with the convention <group>_<artifact>_<version>.log.

Maven Content Compiler

The CloudSlang Maven Content Compiler can be used to compile CloudSlang source files and receive indications of errors without using the CloudSlang CLI or Build tool.

The CloudSlang Maven Content Compiler is an artifact in the cloud-slang module. It extends the Plexus Compiler Project in order to leverage the use of the existing maven-compiler-plugin.

To use the compiler, make the artifact available in the classpath when the Compiler Plugin runs. This is achieved by adding a dependency when declaring the plugin in your project’s pom.xml.

The example below shows how to use the compiler:

<project>
   [...]
   <build>
    [...]
    <plugins>
      [...]
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.5.1</version>
        <configuration>
          <compilerId>cloudslang</compilerId>
        </configuration>
        <dependencies>
          <dependency>
            <groupId>io.cloudslang.lang</groupId>
            <artifactId>cloudslang-content-maven-compiler</artifactId>
            <version><any_version></version>
          </dependency>
        </dependencies>
      </plugin>
      [...]
    <plugins>
    [...]
   <build>
   [...]
</project>

CloudSlang Editors

Although CloudSlang files can be composed in any text editor, using a modern code editor with support for syntax highlighting is recommended.

Atom

The language-cloudslang Atom package includes CloudSlang syntax highlighting and many code snippets.

Download, Install and Configure Atom

1. Download and install Atom.
2. Download and install the CloudSlang language package.

• From the Atom UI: File > Settings > Install and search for language-cloudslang
• From the command line: apm install language-cloudslang

Note

If you are behind a proxy server you may need to configure Atom as described in their package manager documentation.

3. Reload (View > Reload) or restart Atom.
4. Files saved with the .sl extension will be recognized within Atom as CloudSlang files.

Snippets

To use the snippets start typing the snippet name and press enter when it appears on the screen.

The following snippets are provided:

Keyword	Description
flow	template for a flow file
operation	template for an operation file
decision	template for an decision file
properties	template for a system properties file
java_action	template for a Java action
python_action	template for a Python action
input	template for simple input name and value
input with properties	template for an input with all possible properties
output	template for an output name and value
output with properties	template for an output with all possible properties
result	template for a result name and value
run_id	template for the run ID of the current execution
cs_functions	template supporting cs_function. See Functions.
ROI	template to define the ROI from an automated step
publish	template for a published variable name and value
import	template for an import alias name and namespace
navigate	template for a result mapped to a navigation target
step	template for a standard step
on_failure	template for an on_failure step
for	template for an iterative step
parallel_loop	template for a parallel step
property	template for a system property
property with properties	template for a system property with all possible properties
@input	template for input documentation
@description	template for file description documentation
@prerequisites	template for prerequisite documentation
@output	template for output documentation
@result	template for result documentation

Atom Troubleshooting

For troubleshooting Atom issues, see the Atom documentation and discussion board.

IntelliJ

The CloudSlang Plugin for IntelliJ package includes:

• CloudSlang file type support (.sl, .sl.yaml, .sl.yml, .prop.sl)
• Live templates (e.g: flow, operation, input, output, step, for, java_action etc.)
• Completion support for CloudSlang keywords
• Syntax highlighting
• CloudSlang file validation and error highlighting

Installation

1. Install the CloudSlang IntelliJ Plugin in IntelliJ using the Plugin Installation Wizard from File > Settings > Plugins > Browse repositories… and search for: CloudSlang plugin.
2. Click Apply button of the Settings dialog.

Note

Following the system prompt that appears, restart IntelliJ IDEA to activate the installed plugin, or postpone it, at your choice.

Developer

Contents:

Developer Overview

This section contains a brief overview of how CloudSlang and the CloudSlang Orchestration Engine (Score) work. For more detailed information see the Score API and Slang API sections.

The CloudSlang Orchestration Engine is an engine that runs workflows. Internally, the workflows are represented as ExecutionPlans. An ExecutionPlan is essentially a map of IDs and ExecutionSteps. Each ExecutionStep contains information for calling an action method and a navigation method.

When an ExecutionPlan is triggered it executes the first ExecutionStep’s action method and navigation method. The navigation method returns the ID of the next ExecutionStep to run. Execution continues in this manner, successively calling the next ExecutionStep’s action and navigation methods, until a navigation method returns null to indicate the end of the flow.

CloudSlang plugs into the CloudSlang Orchestration Engine (Score) by compiling its workflow and operation files into Score ExecutionPlans and then triggering them. Generally, when working with CloudSlang content, all interaction with Score goes through the Slang API, not the Score API.

CloudSlang

Embedded CloudSlang

CloudSlang content can be run from inside an existing Java application using Maven and Spring by embedding the CloudSlang Orchestration Engine (Score) and interacting with it through the Slang API.

Embed CloudSlang in a Java Application

• Add the Score and CloudSlang dependencies to the project’s pom.xml file in the <dependencies> tag.

<dependency>
  <groupId>io.cloudslang</groupId>
  <artifactId>score-all</artifactId>
  <version>0.3.48</version>
</dependency>

<dependency>
  <groupId>io.cloudslang.lang</groupId>
  <artifactId>cloudslang-all</artifactId>
  <version>1.0.18</version>
</dependency>

<dependency>
  <groupId>com.h2database</groupId>
  <artifactId>h2</artifactId>
  <version>1.3.175</version>
</dependency>

• Add Score and CloudSlang configuration to your Spring application context xml file.

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:score="http://www.cloudslang.io/schema/score"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
                        http://www.springframework.org/schema/beans/spring-beans.xsd
                        http://www.cloudslang.io/schema/score
                        http://www.cloudslang.io/schema/score.xsd">

<bean class="io.cloudslang.lang.api.configuration.SlangSpringConfiguration"/>

  <score:engine />

  <score:worker uuid="-1"/>

</beans>

• Get the Slang bean from the application context xml file and interact with it using the Slang API.

ApplicationContext applicationContext =
                new ClassPathXmlApplicationContext("/META-INF/spring/cloudSlangContext.xml");

Slang slang = applicationContext.getBean(Slang.class);

slang.subscribeOnAllEvents(new ScoreEventListener() {
    @Override
    public void onEvent(ScoreEvent event) {
        System.out.println(event.getEventType() + " : " + event.getData());
    }
});

Slang API

The Slang API allows a program to interact with the CloudSlang Orchestration Engine (Score) using content authored in CloudSlang. What follows is a brief discussion of the API using a simple example that compiles and runs a flow while listening for the events that are fired during the run.

Example

Code

Java Class - CloudSlangEmbed.java

package io.cloudslang.example;

import io.cloudslang.score.events.ScoreEvent;
import io.cloudslang.score.events.ScoreEventListener;
import io.cloudslang.lang.api.Slang;
import io.cloudslang.lang.compiler.SlangSource;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

import java.io.File;
import java.io.IOException;
import java.io.Serializable;
import java.net.URISyntaxException;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Set;

public class CloudSlangEmbed {
    public static void main(String[] args) throws URISyntaxException, IOException{
        ApplicationContext applicationContext =
                new ClassPathXmlApplicationContext("/META-INF/spring/cloudSlangContext.xml");

        Slang slang = applicationContext.getBean(Slang.class);

        slang.subscribeOnAllEvents(new ScoreEventListener() {
            @Override
            public void onEvent(ScoreEvent event) {
                System.out.println(event.getEventType() + " : " + event.getData());
            }
        });

        File flowFile = getFile("/content/hello_world.sl");
        File operationFile = getFile("/content/print.sl");

        Set<SlangSource> dependencies = new HashSet<>();
        dependencies.add(SlangSource.fromFile(operationFile));

        HashMap<String, Serializable> inputs = new HashMap<>();
        inputs.put("input1", "Hi. I'm inside this application.\n-CloudSlang");

        slang.compileAndRun(SlangSource.fromFile(flowFile), dependencies, inputs,
                new HashMap<String, Serializable>());
    }

    private static File getFile(String path) throws URISyntaxException {
        return new File(CloudSlangEmbed.class.getResource(path).toURI());
    }
}

Flow - hello_world.sl

namespace: resources.content

imports:
  ops: resources.content

flow:
  name: hello_world

  inputs:
    - input1

  workflow:
    - sayHi:
        do:
          ops.print:
            - text: "input1"

Operation - print.sl

namespace: resources.content

operation:
  name: print

  inputs:
    - text

  python_action:
    script: print text

  results:
    - SUCCESS

Discussion

The program begins by creating the Spring application context and getting the Slang bean. In general, most of the interactions with Score are transmitted through the reference to this bean.

ApplicationContext applicationContext =
          new ClassPathXmlApplicationContext("/META-INF/spring/cloudSlangContext.xml");

Slang slang = applicationContext.getBean(Slang.class);

Next, the subscribeOnAllEvents method is called and passed a new ScoreEventListener to listen to all the Score and Slang events that are fired.

slang.subscribeOnAllEvents(new ScoreEventListener() {
    @Override
    public void onEvent(ScoreEvent event) {
        System.out.println(event.getEventType() + " : " + event.getData());
    }
});

The ScoreEventListener interface defines only one method, the onEvent method. In this example the onEvent method is overridden to print out the type and data of all events it receives.

The API also contains a method subscribeOnEvents, which takes in a set of the event types to listen for and a method unSubscribeOnEvents, which unsubscribes the listener from all the events it was listening for.

Next, the two content files, containing a flow and an operation respectively, are loaded into File objects.

File flowFile = getFile("/content//hello_world.sl");
File operationFile = getFile("/content/print.sl");

These File objects will be used to create the two SlangSource objects needed to compile and run the flow and its operation.

A SlangSource object is a representation of source code written in CloudSlang along with the source’s name. The SlangSource class exposes several static methods for creating new SlangSource objects from files, URIs or arrays of bytes.

Next, a set of dependencies is created and the operation is added to the set.

Set<SlangSource> dependencies = new HashSet<>();
dependencies.add(SlangSource.fromFile(operationFile));

A flow containing many operations or subflows would need all of its dependencies loaded into the dependency set.

Next, a map of input names to values is created. The input names are as they appear under the inputs key in the flow’s CloudSlang file.

HashMap<String, Serializable> inputs = new HashMap<>();
inputs.put("input1", "Hi. I'm inside this application.\n-CloudSlang");

Finally, the flow is compiled and run by providing its SlangSource, dependencies, inputs and an empty map of system properties.

slang.compileAndRun(SlangSource.fromFile(flowFile), dependencies,
        inputs, new HashMap<String, Serializable>());

An operation can be compiled and run in the same way.

Although we compile and run here in one step, the process can be broken up into its component parts. The Slang interface exposes a method to compile a flow or operation without running it. That method returns a CompilationArtifact which can then be run with a call to the run method.

A CompilationArtifact is composed of a Score ExecutionPlan, a map of dependency names to their ExecutionPlans and a list of CloudSlang Inputs.

A CloudSlang Input contains its name, expression and the state of all its input properties (e. g. required).

Slang Events

CloudSlang uses Score and it’s own extended set of Slang events. Slang events are comprised of an event type string and a map of event data that contains all the relevant event information mapped to keys defined in the org.openscore.lang.runtime.events.LanguageEventData class. All fired events are logged in the execution log file.

Events that contain SensitiveValue s will have the sensitive data replaced by the ******** placeholder.

Event types from CloudSlang are listed in the table below along with the event data each event contains.

All Slang events contain the data in the following list. Additional event data is listed in the table below alongside the event type. The event data map keys are enclosed in square brackets - [KEYNAME].

• [DESCRIPTION]
• [TIMESTAMP]
• [EXECUTIONID]
• [PATH]
• [STEP_TYPE]
• [STEP_NAME]
• [TYPE]

Type [TYPE]	Usage		Event Data
EVENT_INPUT_START	Input binding started for flow or operation		[INPUTS]
EVENT_INPUT_END	Input binding finished for flow or operation		[BOUND_INPUTS]
EVENT_STEP_START	Step started
EVENT_ARGUMENT_START	Argument binding started for step		[ARGUMENTS]
EVENT_ARGUMENT_END	Step arguments resolved		[BOUND_ARGUMENTS]
EVENT_OUTPUT_START	Output binding | | [executableResults] started for | | [executableOutputs] flow or operation | | [actionReturnValues]
EVENT_OUTPUT_END	Output binding | | [OUTPUTS] finished for | | [RESULT] flow or operation | | [EXECUTABLE_NAME]
EVENT_OUTPUT_START	Output binding | | [operationReturnValues] started for step | | [stepNavigationValues] | | [stepPublishValues]
EVENT_OUTPUT_END	Output binding finished for step		[nextPosition] [RESULT] [OUTPUTS]
EVENT_EXECUTION_FINISHED	Execution finished running		[RESULT] [OUTPUTS]
EVENT_ACTION_START	Before action invocation		[TYPE] [CALL_ARGUMENTS]
EVENT_ACTION_END	After successful action invocation		[RETURN_VALUES]
EVENT_ACTION_ERROR	Exception in action execution		[EXCEPTION]
EVENT_SPLIT_BRANCHES	parallel loop expression bound		[BOUND_PARALLEL_LOOP_EXPRESSION]
EVENT_BRANCH_START	Parallel loop branch created		[splitItem] [refId]
EVENT_BRANCH_END	Parallel loop branch ended		[branchReturnValues]
EVENT_JOIN_BRANCHES_START	Parallel loop output binding started		[stepNavigationValues] [stepAggregateValues]
EVENT_JOIN_BRANCHES_END	Parallel loop output binding finished		[nextPosition] [RESULT] [OUTPUTS]
SLANG_EXECUTION_EXCEPTION	Exception in previous step		[IS_BRANCH] [executionIdContext] [systemContext] [EXECUTION_CONTEXT]

Engine

Embedded Engine

The CloudSlang Orchestration Engine (Score) can be embedded inside an existing Java application using Maven and Spring. Interaction with Score is done through the Score API.

Embed Score in a Java Application

• Add the Score dependencies to the project’s pom.xml file in the <dependencies> tag.

<dependency>
  <groupId>io.cloudslang</groupId>
  <artifactId>score-all</artifactId>
  <version>0.3.48</version>
</dependency>

<dependency>
  <groupId>com.h2database</groupId>
  <artifactId>h2</artifactId>
  <version>1.3.175</version>
</dependency>

• Add Score configuration to your Spring application context xml file.

<beans xmlns="http://www.springframework.org/schema/beans"
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xmlns:score="http://www.openscore.org/schema/score"
            xsi:schemaLocation="http://www.springframework.org/schema/beans
            http://www.springframework.org/schema/beans/spring-beans.xsd
            http://www.openscore.org/schema/score
            http://www.openscore.org/schema/score.xsd">

  <score:engine/>

  <score:worker uuid="-1"/>

  <bean class="io.cloudslang.example.ScoreEmbed"/>
</beans>

• Interact with Score using the Score API.

package io.cloudslang.example;

import org.apache.log4j.Logger;
import io.cloudslang.score.api.*;
import io.cloudslang.score.events.EventBus;
import io.cloudslang.score.events.EventConstants;
import io.cloudslang.score.events.ScoreEvent;
import io.cloudslang.score.events.ScoreEventListener;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

import java.io.Serializable;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Set;

public class ScoreEmbed {
  @Autowired
  private Score score;

  @Autowired
  private EventBus eventBus;

  private final static Logger logger = Logger.getLogger(ScoreEmbed.class);
  private ApplicationContext context;
  private final Object lock = new Object();

  public static void main(String[] args) {
      ScoreEmbed app = loadApp();
      app.registerEventListener();
      app.start();
  }

  private static ScoreEmbed loadApp() {
      ApplicationContext context = new ClassPathXmlApplicationContext("/META-INF/spring/scoreContext.xml");
      ScoreEmbed app = context.getBean(ScoreEmbed.class);
      app.context  = context;
      return app;
  }

  private void start() {
      ExecutionPlan executionPlan = createExecutionPlan();
      score.trigger(TriggeringProperties.create(executionPlan));
      waitForExecutionToFinish();
      closeContext();
  }

  private void waitForExecutionToFinish() {
      try {
          synchronized(lock){
              lock.wait(10000);
          }
      } catch (InterruptedException e) {
          logger.error(e.getStackTrace());
      }
  }

  private static ExecutionPlan createExecutionPlan() {
      ExecutionPlan executionPlan = new ExecutionPlan();

      executionPlan.setFlowUuid("1");

      ExecutionStep executionStep0 = new ExecutionStep(0L);
      executionStep0.setAction(new ControlActionMetadata("io.cloudslang.example.controlactions.ConsoleControlActions", "printMessage"));
      executionStep0.setActionData(new HashMap<String, Serializable>());
      executionStep0.setNavigation(new ControlActionMetadata("io.cloudslang.example.controlactions.NavigationActions", "nextStepNavigation"));
      executionStep0.setNavigationData(new HashMap<String, Serializable>());
      executionPlan.addStep(executionStep0);

      ExecutionStep executionStep1 = new ExecutionStep(1L);
      executionStep1.setAction(new ControlActionMetadata("io.cloudslang.example.controlactions.ConsoleControlActions", "printMessage"));
      executionStep1.setActionData(new HashMap<String, Serializable>());
      executionStep1.setNavigation(new ControlActionMetadata("io.cloudslang.example.controlactions.NavigationActions", "nextStepNavigation"));
      executionStep1.setNavigationData(new HashMap<String, Serializable>());
      executionPlan.addStep(executionStep1);

      ExecutionStep executionStep2 = new ExecutionStep(2L);
      executionStep2.setAction(new ControlActionMetadata("io.cloudslang.example.controlactions.ConsoleControlActions", "failed"));
      executionStep2.setActionData(new HashMap<String, Serializable>());
      executionStep2.setNavigation(new ControlActionMetadata("io.cloudslang.example.controlactions.NavigationActions", "endFlow"));
      executionStep2.setNavigationData(new HashMap<String, Serializable>());
      executionPlan.addStep(executionStep2);

      return executionPlan;
  }

  private void registerEventListener() {
      Set<String> handlerTypes = new HashSet<>();
      handlerTypes.add(EventConstants.SCORE_FINISHED_EVENT);
      handlerTypes.add(EventConstants.SCORE_FAILURE_EVENT);
      eventBus.subscribe(new ScoreEventListener() {
          @Override
          public void onEvent(ScoreEvent event) {
              logger.info("Listener " + this.toString() + " invoked on type: " + event.getEventType() + " with data: " + event.getData());
              synchronized (lock) {
                  lock.notify();
              }
          }
      }, handlerTypes);
  }

  private void closeContext() {
      ((ConfigurableApplicationContext) context).close();
  }
}

Score API

The Score API allows a program to interact with the CloudSlang Orchestration Engine (Score). This section describes some of the more commonly used interfaces and methods from the Score API.

ExecutionPlan

An ExecutionPlan is a map of IDs and steps, called ExecutionSteps, representing a workflow for Score to run. Normally, the ID of the first step to be run is 0.

ExecutionSteps can be added to the ExecutionPlan using the addStep(ExecutionStep step) method.

The starting step of the ExecutionPlan can be set using the setBeginStep(Long beginStep) method.

ExecutionStep

An ExecutionStep is the a building block upon which an ExecutionPlan is built. It consists of an ID representing its position in the plan, control action information and navigation action information. As each ExecutionStep is reached, its control action method is called followed by its navigation action method. The navigation action method returns the ID of the next ExecutionStep to be run in the ExecutionPlan or signals the plan to stop by returning null. The ID of an ExecutionStep must be unique among the steps in its ExecutionPlan.

The control action method and navigation action methods can be set in the ExecutionStep using the following methods, where a ControlActionMetadata object is created using string values of the method’s fully qualified class name and method name:

• setAction(ControlActionMetadata action)
• setNavigation(ControlActionMetadata navigationMetadata)

Action Method Arguments

Both the control action and navigation action are regular Java methods which can take arguments. They are invoked by reflection and their arguments are injected by the Score engine, so there is no API or naming convention for them. But there are some names that are reserved for special use.

There are several ways Score can populate an action method’s arguments:

• From the execution context that is passed to the TriggeringProperties when the ExecutionPlan is triggered.

  When a method such as public void doSomething(String argName) is encountered, Score will attempt to populate the argument argName with a value mapped to the key argName in the execution context. If the key argName does not exist in the map, the argument will be populated with null.

• From data values set in the ExecutionSteps during the creation of the ExecutionPlan.

  Data can be set using the setActionData and setNavigationData methods.

• From reserved argument names.

There are some argument names that have a special meaning when used as control action or navigation action method arguments:

• executionRuntimeServices - Score will populate this argument with the ExecutionRuntimeServices object.

public void doWithServices(ExecutionRuntimeServices executionRuntimeServices)

• executionContext - Score will populate this argument with the context tied to the ExecutionPlan during its triggering through the TriggeringProperties.

public void doWithContext(Map<String, Serializable> executionContext)

If an argument is present in both the ExecutionStep data and the execution context, the value from the execution context will be used.

Action Method Return Values

• Control action methods are void and do not return values.
• Navigation action methods return a value of type Long, which is used to determine the next ExecutionStep. Returning null signals the ExecutionPlan to finish.

Score Interface

The Score interface exposes methods for triggering and canceling executions.

Triggering New Executions

The trigger(TriggeringProperties triggeringProperties) method starts an execution with a given ExecutionPlan and the additional properties found in the TriggeringProperties object. The method returns the ID of the new execution.

By default the first executed step will be the execution plan’s start step, and the execution context will be empty.

Canceling Executions

The cancelExecution(Long executionId) method requests to cancel (terminate) a given execution. It is passed the ID that was returned when triggering the execution that is now to be canceled.

Note

The execution will not necessarily be stopped immediately.

TriggeringProperties

A TriggeringProperties object is sent to the Score interface’s trigger method when the execution begins.

The TriggeringProperties object contains:

• An ExecutionPlan to run.
• The ExecutionPlan’s dependencies, which are ExecutionPlans themselves.
• A map of names and values to be added to the execution context.
• A map of names and values to be added to the ExecutionRuntimeServices.
• A start step value, which can cause the ExecutionPlan to start from a step that is not necessarily its defined begin step.

The TriggeringProperties class exposes methods to create a TriggeringProperties object from an ExecutionPlan and then optionally set the various other properties.

ExecutionRuntimeServices

The ExecutionRuntimeServices provide a way to communicate with Score during the execution of an ExecutionPlan. During an execution, after each ExecutionStep, the engine will check the ExecutionRuntimeServices to see if there have been any requests made of it and will respond accordingly. These services can be used by a language written on top of Score, as CloudSlang does, to affect the runtime behavior.

The ExecutionRuntimeServices can be injected into an ExecutionStep’s action or navigation method’s arguments by adding the ExecutionRuntimeServices executionRuntimeServices parameter to the method’s argument list.

Some of the services provided by ExecutionRuntimeServices are:

• Events can be added using the addEvent(String eventType, Serializable eventData) method.
• Execution can be paused using the pause() method.
• Errors can be set using the setStepErrorKey(String stepErrorKey) method.
• Branches can be added using the addBranch(Long startPosition, String flowUuid, Map<String, Serializable> context) method or the addBranch(Long startPosition, Long executionPlanId, Map<String, Serializable> context, ExecutionRuntimeServices executionRuntimeServices) method.
• Requests can be made to change the ExecutionPlan that is running by calling the requestToChangeExecutionPlan(Long executionPlanId) method.

EventBus

The EventBus allows you to subscribe and unsubscribe listeners for events.

Listeners must implement the ScoreEventListener interface which consists of a single method – onEvent(ScoreEvent event).

To subscribe a listener for certain events, pass a set of the events to listen for to the subscribe(ScoreEventListener eventHandler, Set<String> eventTypes) method.

The event types are defined in the EventConstants class.

To unsubscribe a listener from all the events it was listening for call the unsubscribe(ScoreEventListener listener) method.

ScoreEvent

A ScoreEvent is comprised of a string value corresponding to its type and a map containing the event data, which can be accessed using the getEventType() and getData() methods respectively.

Score Events

The CloudSlang Orchestration Engine (Score) defines two events that may be fired during execution. Each event is comprised of a string value corresponding to its type and a map containing the event data.

Event Types:

• SCORE_FINISHED_EVENT
• SCORE_FAILURE_EVENT

Event Data Keys:

• IS_BRANCH
• executionIdContext
• systemContext
• EXECUTION_CONTEXT

A language built upon Score can add events during runtime using the ExecutionRuntimeServices API. An example of this usage can be seen in CloudSlang’s addition of Slang events.

Architecture

Overview

To be run by the CloudSlang Orchestration Engine (Score), a CloudSlang source file must undergo a process to transform it into a Score ExecutionPlan using the SlangCompiler.

Precompilation

In the precompilation process, the source file is loaded, along with its dependencies if necessary, and parsed. The CloudSlang file’s YAML structure is translated into Java maps by the YamlParser using snakeyaml. The parsed structure is then modeled to Java objects representing the parts of a flow and operation by the SlangModeller and the ExecutableBuilder. The result of this process is an object of type Executable.

Compilation

The resulting Executable object, along with its dependent Executable objects, are then passed to the ScoreCompiler for compilation. An ExecutionPlan is created from the Executable using the ExecutionPlanBuilder. The ExecutionPlanBuilder uses the ExecutionStepFactory to manufacture the appropriate Score ExecutionStep objects and add them to the resulting ExecutionPlan, which is then packaged with its dependent ExecutionPlan objects into a CompilationArtifact.

Running

Now that the CloudSlang source has been fully transformed into an ExecutionPlan it can be run using Score. The ExecutionPlan and its dependencies are extracted from the CompilationArtifact and used to create a TriggeringProperties object. A RunEnvironment is also created and added to the TriggeringProperties context. The RunEnvironment provides services to the ExecutionPlan as it runs, such as keeping track of the context stack and next step position.

Treatment of Flows and Operations

Generally, CloudSlang treats flows and operations similarly.

Flows and operations both:

• Receive inputs, produce outputs, and have navigation logic.
• Can be called by a flow’s step.
• Are compiled to ExecutionPlans that can be run by Score.

Scoped Contexts

As execution progresses from flow to operation to action, the step data (inputs, outputs, etc.) that is in scope changes. These contexts are stored in the contextStack of the RunEnvironment and get pushed onto and popped off as the scope changes.

There are three types of scoped contexts:

• Flow context
• Operation context
• Action context

Value Types

Each Context stores its data in a Map<String, Value> named variables, where Value declares the isSensitive() method and is one of three value types:

• SimpleValue
• SensitiveValue
• PyObjectValue

SimpleValue is used for non-sensitive inputs, outputs and arguments.

SensitiveValue is used for sensitive inputs, outputs and arguments. Calling the toString() method on a SensitiveValue will return the SENSITIVE_VALUE_MASK (********) instead of its content. During runtime, a SensitiveValue is decrypted upon usage and then encrypted again.

PyObjectValue is an interface which extends Value, adding the isAccessed() method. An object of this type is a (Javassist) proxy, which extends a given PyObject instance and implements the PyObjectValue interface. Value method calls are delegated to an inner Value instance, which can be either a SimpleValue or SensitiveValue. PyObject method calls are delegated to an inner PyObject, the original one this object is extending. PyObject method calls also change an accessed flag to true. This flag indicates whether the value was used in a Python script.

Value types, SimpleValue or SensitiveValue are propagated automatically from inputs to arguments and Python expression evaluation outputs. An argument or output is sensitive if at least one part of it is sensitive. For example, the result of a + b or some_func(a) will be sensitive if a is sensitive. Before running a Python expression all the arguments which are passed to it are converted to PyObjectValue. When the expression finishes, all the arguments are checked. If at least one sensitive argument was used the output will be sensitive as well.

As opposed to expressions, the output types of Java and Python operations, are not propagated automatically to the operation’s outputs. Doing so would cause all outputs of an operation to be sensitive every time at least one input was sensitive. Instead, none of the operation’s action’s data appears in the logs and a content author explicitly marks an operation’s outputs as sensitive when needed. This approach ensures that sensitive data is hidden at all times while still allowing for full control over which operation outputs are sensitive and which are not.

Types of ExecutionSteps

As flows and operations are compiled, they are broken down into a number of ExecutionSteps. These steps are built using their corresponding methods in the ExecutionStepFactory.

There are five types of ExecutionSteps used to build a CloudSlang ExecutionPlan:

• Start
• End
• Begin Step
• End Step
• Action

An operation’s ExecutionPlan is built from a Start step, an Action step and an End step.

A flow’s ExecutionPlan is built from a Start step, a series of Begin Step steps and End Step steps, and an End step. Each step’s ExecutionSteps hand off the execution to other ExecutionPlan objects representing operations or subflows.

RunEnvironment

The RunEnvironment provides services to the ExecutionPlan as it is running. The different types of execution steps read from, write to and update the environment.

The RunEnvironment contains:

• callArguments - call arguments of the current step
• returnValues - return values for the current step
• nextStepPosition - position of the next step
• contextStack - stack of contexts of the parent scopes
• parentFlowStack - stack of the parent flows’ data
• executionPath - path of the current execution
• systemProperties - system properties
• serializableDataMap - serializable data that is common to the entire run

Engine Architecture

The CloudSlang Orchestration Engine (Score) is built from two main components, an engine and a worker. Scaling is achieved by adding additional workers and/or engines.

Engine

The engine is responsible for managing the workers and interacting with the database. It does not hold any state information itself.

The engine is composed of the following components:

• Orchestrator: Responsible for creating new executions, canceling existing executions, providing the status of existing executions and managing the split/join mechanism.
• Assigner: Responsible for assigning workers to executions.
• Queue: Responsible for storing execution information in the database and responding with messages to polling workers.

Worker

The worker is responsible for doing the actual work of running the execution plans. The worker holds the state of an execution as it is running.

The worker is composed of the following components:

• Worker Manager: Responsible for retrieving messages from the queue and placing them in the in-buffer, delegating messages to the execution service, draining messages from the out-buffer to the orchestrator and updating the engine as to the worker’s status.
• Execution Service: Responsible for executing the execution steps, pausing and canceling executions, splitting executions and dispatching relevant events.

Database

The database is composed of the following tables categorized here by their main functions:

• Execution tracking:
  • RUNNING_EXECUTION_PLANS: full data of an execution plan and all of its dependencies
  • EXECUTION_STATE: run statuses of an execution
  • EXECUTION_QUEUE_1: metadata of execution message
  • EXECUTION_STATES_1 and EXECUTION_STATES_2: full payloads of execution messages
• Splitting and joining executions:
  • SUSPENDED_EXECUTIONS: executions that have been split
  • FINISHED_BRANCHES: finished branches of a split execution
• Worker information:
  • WORKER_NODES: info of individual workers
  • WORKER_GROUPS: info of worker groups
• Recovery:
  • WORKER_LOCKS: row to lock on during recovery process
  • VERSION_COUNTERS: version numbers for testing responsiveness

Typical Execution Path

In a typical execution the orchestrator receives an ExecutionPlan along with all that is needed to run it in a TriggeringProperties object through a call to the Score interface’s trigger method. The orchestrator inserts the full ExecutionPlan with all of its dependencies into the RUNNING_EXECUTION_PLANS table. An Execution object is then created based on the TriggeringProperties and an EXECUTION_STATE record is inserted indicating that the execution is running. The Execution object is then wrapped into an ExecutionMessage. The assigner assigns the ExecutionMessage to a worker and places the message metadata into the EXECUTION_QUEUE_1 table and its Payload into the active EXECUTION_STATES table.

The worker manager constantly polls the queue to see if there are any ExecutionMessages that have been assigned to it. As ExecutionMessages are found, the worker acknowledges that they were received, wraps them as SimpleExecutionRunnables and submits them to the execution service. When a thread is available from the execution service’s pool the execution will run one step (control action and navigation action) at a time until there is a reason for it to stop. There are various reasons for a execution to stop running on the worker and return to the engine including: the execution is finished, is about to split or it is taking too long. Once an execution is stopped it is placed on the out-buffer which is periodically drained back to the engine.

If the execution is finished, the engine fires a SCORE_FINISHED_EVENT and removes the execution’s information from all of the execution tables in the database.

Splitting and Joining Executions

Before running each step, a worker checks to see if the step to be run is a split step. If it is a split step, the worker creates a list of the split executions. It puts the execution along with all its split executions into a SplitMessage which is placed on the out-buffer. After draining, the orchestrator’s split-join service takes care of the executions until they are to be rejoined. The service places the parent execution into the SUSPENDED_EXECUTIONS table with a count of how many branches it has been split into. Executions are created for the split branches and placed on the queue. From there, they are picked up as usual by workers and when they are finished they are added to the FINISHED_BRANCHES table. Periodically, a job runs to see if the number of branches that have finished are equal to the number of branches the original execution was split into. Once all the branches are finished the original execution can be placed back onto the queue to be picked up again by a worker.

Recovery

The recovery mechanism allows Score to recover from situations that would cause a loss of data otherwise. The recovery mechanism guarantees that each step of an execution plan will be run, but does not guarantee that it will be run only once. The most common recovery situations are outlined below.

Lost Worker

To prevent the loss of data from a worker that is no longer responsive the recovery mechanism does the following. Each worker continually reports their active status to the engine which stores a reporting version number for the worker in the WORKER_NODES table. Periodically a recovery job runs and sees which workers’ reported version numbers are outdated, indicating that they have not been reporting back. The non-responsive workers’ records in the queue get reassigned to other workers that pick up from the last known step that was executed.

Worker Restart

To prevent the loss of data from a worker that has been restarted additional measures must be taken. The restarted worker will report that it is active, so the recovery job will not know to reassign the executions that were lost when it was restarted. Therefore, every time a worker has been started an internal recovery is done. The worker’s buffers are cleaned and the worker reports to the engine that it is starting up. The engine then checks the queue to see if that worker has anything that’s already on the queue. Whatever is found is passed on to a different worker while the restarted one finishes starting up before polling for new messages.

Contributions

GitHub Repositories

The CloudSlang project consists of the following repositories on GitHub with the dependencies depicted in the diagram below.

Dependency diagram

---

• score - CloudSlang Orchestration Engine (Score)
  • dependency-management
  • engine
  • package
  • runtime-management
  • score-api
  • score-samples
  • score-tests
  • worker
• cloud-slang - CloudSlang and the CLI
  • build
  • cloudslang-all
  • cloudslang-cli
  • cloudslang-commons
  • cloudslang-compiler
  • cloudslang-content-maven-compiler
  • cloudslang-content-verifier
  • cloudslang-entities
  • cloudslang-runtime
  • cloudslang-spi
  • cloudslang-tests

• cloud-slang-content - CloudSlang flows and operations
  • ci-env
  • configuration/properties/io/cloudslang
  • content/io/cloudslang
    • amazon
      • aws
        • ec2
    • base
      • cmd
      • comparisons
      • datetime
      • examples
      • filesystem
      • http
      • json
      • lists
      • mail
      • maps
      • math
      • network
      • os
      • print
      • remote_file_transfer
      • scripts
      • ssh
      • strings
      • utils
      • xml
    • chef
    • ci
      • circleci
    • consul
    • coreos
    • couchbase
    • digital_ocean
      • v2
    • docker
    • git
    • google
      • compute
    • hashicorp
      • vault
    • haven_on_demand
    • heroku
    • itsm
      • service_now
    • jenkins
    • marathon
    • maven
    • microfocus
      • dca
    • microsoft
      • azure
    • new_relic
      • servers
    • openshift
    • openstack
    • slack
    • stackato
    • twilio
      • sms
    • vmware
      • vcenter
    • (other integrations to be added as new folders)
• cs-actions - Java @Action classes used by CloudSlang
  • cs-amazon
  • cs-azure
  • cs-commons
  • cs-couchbase
  • cs-microfocus-dca
  • cs-database
  • cs-date-time
  • cs-google
  • cs-http-client
  • cs-json
  • cs-lists
  • cs-mail
  • cs-powershell
  • cs-rft
  • cs-ssh
  • cs-utilities
  • cs-vmware
  • cs-xml
• score-content-sdk - SDK for developing Java @Actions
  • src/main/java/com/hp/oo/sdk/content
    • annotations
    • plugin
      • ActionMetadata
• test-functional - Global functional tests for CLI and builder
• CloudSlang-Docker-Image - CloudSlang Docker image
• CloudSlang.github.io - CloudSlang website
• docs - CloudSlang documentation
• atom-cloudslang-package - Atom package for CloudSlang support
• cloudslang-cli - npm cloudslang-cli
• cs-intellij-plugin - CloudSlang Intellij Plugin
• cs-content-generator - Tool to convert Java Actions to .sl files
• cs-content-packager - Tool to package CloudSlang content

Contribution Guide

We welcome and encourage community contributions to CloudSlang. Please familiarize yourself with the Contribution Guidelines and Project Roadmap before contributing.

There are many ways to help the CloudSlang project:

• Report issues
• Fix issues
• Improve the documentation

Contributing Code

The best way to directly collaborate with the project contributors is through GitHub: https://github.com/CloudSlang.

• If you want to contribute to our code by either fixing a problem or creating a new feature, please open a GitHub pull request.
• If you want to raise an issue such as a defect, an enhancement request or a general issue, please open a GitHub issue.

All patches from all contributors get reviewed.

After a pull request is made, other contributors will offer feedback. If the patch passes review, a maintainer will accept it with a comment.

When a pull request fails testing, the author is expected to update the pull request to address the failure until it passes testing and the pull request merges successfully.

At least one review from a maintainer is required for all patches (even patches from maintainers).

Content contributions which require environments that are difficult to setup may be accepted as beta content. Beta content is not verified or tested by the CloudSlang team. Beta content is named with the beta_ prefix. The community is encouraged to assist in setting up testing environments for the beta content.

See the contributing.md file in the relevant repository for additional guidelines specific to that repository.

Developer’s Certificate of Origin

All contributions must include acceptance of the DCO:

Developer Certificate of Origin Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors. 660 York Street, Suite 102, San Francisco, CA 94110 USA

Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

Developer’s Certificate of Origin 1.1

By making a contribution to this project, I certify that:

1. The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or
2. The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or
3. The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.
4. I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved.

Sign your work

To accept the DCO, simply add this line to each commit message with your name and email address (git commit -s will do this for you):

Signed-off-by: Jane Example <jane@example.com>

For legal reasons, no anonymous or pseudonymous contributions are accepted.

Pull Requests

We encourage and support contributions from the community. No fix is too small. We strive to process all pull requests as soon as possible and with constructive feedback. If your pull request is not accepted at first, please try again after addressing the feedback you received.

To make a pull request you will need a GitHub account. For help, see GitHub’s documentation on forking and pull requests.

Normally, all pull requests must include tests that validate your change. Occasionally, a change will be very difficult to test. In those cases, please include a note in your commit message explaining why tests are not included.

Conduct

Whether you are a regular contributor or a newcomer, we care about making this community a safe place for you.

We are committed to providing a friendly, safe and welcoming environment for all regardless of their background and the extent of their contributions.

Please avoid using nicknames that might detract from a friendly, safe and welcoming environment for all. Be kind and courteous.

Those who insult, demean or harass anyone will be excluded from interaction. In particular, behavior that excludes people in socially marginalized groups will not be tolerated.

We welcome discussion about creating a welcoming, safe and productive environment for the community. If you have any questions, feedback or concerns please let us know. (info@cloudslang.io)

Tutorial

Contents:

Lesson 1 - Introduction and Setup

Goal

In this lesson we’ll outline our overall goals for this tutorial and set up an environment to write and run flows.

Overview

In this tutorial we will build a flow that represents the process a new hire must go through to get set up to work. We will build the flow one piece at a time with the goal of highlighting the features of CloudSlang. We recommend you follow along with the process, writing the flows and operations we will build on your own machine and running them using the CloudSlang CLI. To do so, you’ll need a text editor to create the CloudSlang files and the CloudSlang CLI to run them.

YAML

CloudSlang is a YAML-based language so it’s important to know a bit about YAML before getting started. If you’re new to YAML, you can take a look at the YAML Overview section of the CloudSlang documentation to familiarize yourself with its main structure. This tutorial will also include YAML Notes to guide you through potential trouble areas.

Copy/Pasting Code

Because proper indentation is so important in YAML, take care to indent pasted code examples to their proper indentation levels. The general rules for indentation can be found in the structured outlines of CloudSlang files found in the CloudSlang Files section of the DSL Reference.

If you are unsure what the indentation level is for a particular code snippet, you can take a look at where it fits into the rest of the code in the New Code - Complete section at the bottom of each lesson or by downloading the lesson’s code.

Prerequisites

This tutorial uses the CloudSlang CLI to run flows. See the CloudSlang CLI section of the documentation for instructions on how to download and run the CLI.

Although CloudSlang files can be composed in any text editor, using a modern code editor with support for syntax highlighting is recommended. See CloudSlang Editors for instructions on how to download, install and use the CloudSlang language package for Atom.

More Information

For more information on any of the topics covered in this tutorial, see the CloudSlang documentation.

Flows, Operations and Decisions

Let’s begin our study of the CloudSlang language by discussing the three types of CloudSlang executable constructs: flows, operations and decisions.

Generally, CloudSlang treats flows, operations and decisions similarly. Flows, operations and decisions can all receive inputs, produce outputs, return results and can be called by a flow’s step.

But flows, operations and decisions serve different purposes.

An operation contains an action, which can be written in Python or Java. Operations perform the “work” part of the workflow.

A flow contains steps, which stitch together the actions performed by operations (or subflows), navigating and passing data from one to the other based on operation results and outputs. Flows perform the “flow” part of the workflow.

A decision is very similar to an operation, but without an action.

Here is a diagram of the flow, operation and decision structure we will be building in this tutorial.

Setup

We’ll start writing CloudSlang code in the next lesson. But before we do that we’ll setup our folder structure to get ready.

Create a folder named tutorials. We’ll store our flows and operations in this folder. Since we’re going to have some general content as well as content that is specific to our use case, let’s create two subfolders under tutorials called base and hiring.

We’ll start off with just one flow and one operation. In the next two lessons we’ll create a file named new_hire.sl in the hiring folder and in the base folder we’ll create a file named print.sl. The file new_hire.sl will hold our flow and print.sl will hold our first operation.

Your file structure will look like this:

• tutorials
  • base
    • print.sl
  • hiring
    • new_hire.sl

Note

If your editor requires it for syntax highlighting, you can also use the .sl.yaml and .sl.yml extensions.

Up Next

In the next lesson we’ll write and run our first operation.

Lesson 2 - First Operation

Goal

In this lesson we’ll write our first operation. We’ll learn the basic structure of a simple operation by writing one that simply prints out a message.

Get Started

First, we need to create the print.sl file in the base folder so we can start writing the print operation. The print operation is as simple as they get. It just takes in a input and prints it out using Python.

Namespace

All CloudSlang files start with a namespace which mirrors the folder structure in which the files are found. In our case we’ve put print.sl in the tutorials/base folder so our namespace should reflect that.

namespace: tutorials.base

The namespace can be used by flows that call this operation.

For more information, see namespace in the DSL reference.

Operation Name

Each operation begins with the operation key which will map to the contents of the operation body. The first part of that body is a key:value pair defining the name of the operation. The name of the operation must be the same as the name of the file it is stored in.

operation:
  name: print

Note

YAML Note: Indentation is very important in YAML. It is used to indicate scope. In the example above, you can see that name: print is indented under the operation key to denote that it belongs to the operation. Always use an identical number of spaces to indent. Never use tabs. For more information, see the YAML Overview.

For more information, see operation in the DSL reference.

Inputs

After the name, if the operation takes any inputs, they are listed under the inputs key. In our case we’ll need to take in the text we want to print. We’ll name our input text.

inputs:
  - text

Note

YAML Note: The inputs key maps to a list of inputs. In YAML, a list is signified by prepending a hyphen (-) and a space to each item.

The values for the inputs are either passed via the CloudSlang CLI, as we do below in this lesson, or from a step in a flow, as we will do in the next lesson.

Inputs can also have related parameters, such as required, default, private and sensitive. We will discuss these parameters in lessons 8 - Input Parameters.

For more information, see inputs, required, default, private and sensitive in the DSL reference.

Action

Finally, we’ve reached the core of the operation, the action. There are two types of actions in CloudSlang, Python-based actions and Java-based actions.

We’ll start off by creating a Python action that simply prints the text that was input. To do so, we add python_action and script keys that map to the action contents.

python_action:
  script: print text

Note

CloudSlang uses the Jython implementation of Python 2.7. For information on Jython’s limitations, see the Jython FAQ.

Python scripts that need 3rd party packages may import them using the procedures described in lesson 14 - 3rd Party Python Packages.

For more information, see python_action in the DSL reference.

The usage of Java-based actions is beyond the scope of this tutorial. For more information, see the java_action in the DSL reference.

Run It

That’s it. Our operation is all ready. Our next step will be to create a flow that uses the operation we just wrote, but we can actually just run the operation as is.

To do so, save the operation file, fire up the CloudSlang CLI and enter the following at the prompt to run your operation:

run --f <folder path>/tutorials/base/print.sl --i text=Hi

You should see the input text printed out to the screen.

For more information, see Use the CLI in the DSL reference.

Download the Code

Lesson 2 - Complete code

Up Next

In the next lesson we’ll write a flow that will call the print operation.

New Code - Complete

print.sl

namespace: tutorials.base

operation:
  name: print

  inputs:
    - text

  python_action:
    script: print text

Lesson 3 - First Flow

Goal

In this lesson we’ll write a simple flow that will call the print operation. We’ll learn about the main components that make up a flow.

Get Started

First, we need to create a new_hire.sl file in the hiring folder so we can start writing the new hire flow. We’ll build it one step at a time. So for now, all it will do is call the print operation we wrote in the previous lesson.

Namespace

Just like in our operation file, we need to start the flow file with a namespace. Since we’re storing new_hire.sl in the tutorials/hiring folder the namespace must reflect that.

namespace: tutorials.hiring

For more information, see namespace in the DSL reference.

Imports

After the namespace you can list the namespace of any CloudSlang files that you will need to reference in your flow. Our flow will need to reference the operation in print.sl, so we’ll add the namespace from that file, tutorials.base, to the optional imports key. We map an alias that we will use as a reference in the flow to the namespace we are importing. Let’s call the alias base.

imports:
  base: tutorials.base

Now we can use base.print to refer to the print operation in a step. We’ll do that in a moment.

For more information, see imports in the DSL reference.

For ways to refer to an operation or subflow without creating an alias, see the CloudSlang DSL Reference and the Operation Paths example.

Flow Name

Each flow begins with the flow key which will map to the contents of the flow body. The first part of that body is a key:value pair defining the name of the flow. The name of the flow must be the same as the name of the file it is stored in.

flow:
  name: new_hire

For more information, see flow in the DSL reference.

Steps

The next part of our flow will be the workflow. The workflow key maps to a list of all the steps in the flow. We’ll start off with just one step, the one that calls our print operation. Each step in a workflow starts with a key that is its name. We’ll call our step print_start.

workflow:
  - print_start:

For more information, see workflow in the DSL reference.

A step can contain several parts, but we’ll start with a simple step with the only required part, the do key. We want to call the print operation. In this case we’ll reference it using the alias we created up in the flow’s imports section. Also, we’ll have to pass any required inputs to the operation. In our case, there’s one input named text which we’ll add to a list under the operation call and pass it a value.

do:
  base.print:
    - text: "Starting new hire process"
navigate:
  - SUCCESS: SUCCESS

In addition to the required do, a step can also contain the optional publish and navigate keys. Here we added a navigate section. We’ll explain more about publish and navigate a little later in lessons 5 - Default Navigation and 7 - Custom Navigation respectively.

For more information, see do, publish and navigate in the DSL reference.

Run It

Now our flow is all ready to run. To do so, save the file and enter the following at the prompt.

run --f <folder path>/tutorials/hiring/new_hire.sl --cp <folder path>/tutorials/base

Note

The --cp flag is used to add folders where the flow’s dependencies are found to the classpath. For more information, see Run with Dependencies in the DSL reference.

You should see the name of the step and the string sent to the print operation printed to the screen.

Download the Code

Lesson 3 - Complete code

Up Next

In the next lesson we’ll write a more complex operation that also returns outputs and results.

New Code - Complete

new_hire.sl

namespace: tutorials.hiring

imports:
  base: tutorials.base

flow:
  name: new_hire

  workflow:
    - print_start:
        do:
          base.print:
            - text: "Starting new hire process"
        navigate:
          - SUCCESS: SUCCESS

  results:
    - SUCCESS

Lesson 4 - Outputs and Results

Goal

In this lesson we’ll write a bit of a more complex operation that returns an output and results.

Get Started

Let’s create a new file in the tutorials/hiring folder named check_availability.sl in which we’ll house an operation to check whether a given email address is available.

We’ll also start off our new operation in much the same way we did with the print operation. We’ll put in a namespace, the operation key, the name of the operation and an input.

namespace: tutorials.hiring

operation:
  name: check_availability

  inputs:
    - address

Action

This time we’ll have a little more of a complex action. The idea here is to simulate checking the availability of the given address. We’ll import and use the Python random module to get a random number between 0 and 2. If the random number we get is 0 we’ll say the requested email address is already taken.

We’ve added a commented-out line, using a Python comment (#) to print the random number that was generated. We can uncomment this line during testing to see that our operation is working as expected.

python_action:
  script: |
    import random
    rand = random.randint(0, 2)
    vacant = rand != 0
    #print rand

Note

YAML Note: Since we’re writing a multi-line Python script here we use the pipe (|) character to denote the usage of literal style block notation where all newlines will be preserved.

Outputs

In the outputs section we put any information we want to send back to the calling flow. In our case, we want to return whether the requested address was already taken. The outputs are a list of key:value pairs where the key is the name of the output and the value is the expression to be returned. In our case, we’ll just return the value in the vacant variable. Outputs must be strings so we’ll use the Python str() function to convert the value.

outputs:
  - available: ${str(vacant)}

Notice the special ${} syntax. This indicates that what is inside the braces is a CloudSlang expression. If we would have just written str(vacant), it would be understood as a string literal. We’ll see this syntax in action again in a few moments.

For more information, see Expressions in the DSL reference.

At this point we won’t be using the output value, but we will soon enough. In lesson 5 - Default Navigation we publish the available output and use it in another step.

For more information, see outputs in the DSL reference.

Results

The last section of our operation defines the results we return to the calling flow. The results are used by the navigation of the calling flow. We’ll start by using the default result values, SUCCESS and FAILURE. If the email address was available, we’ll return a result of SUCCESS, otherwise we’ll return a result of FAILURE. There always must be a default ending result that does not have an expression or explicitly maps to the value true. Here, we will use the SUCCESS result as our catchall. When the operation is run, the first result whose expression is true or empty is returned. It is therefore important to take care in the ordering of the results.

results:
  - FAILURE: ${rand == 0}
  - SUCCESS

The results are used by the calling flow for navigation purposes. You can see the default navigation rules in action in lessons 5 - Default Navigation and 6 - Handling Failure Results. And you can learn how to create custom navigation in lesson 7 - Custom Navigation.

For more information, see results in the DSL reference.

Run It

Let’s save and run this operation by itself before we start using it in our flow to make sure everything is working properly. (You might want to uncomment the line that prints out the random number while testing.) To run the operation, enter the following in the CLI:

run --f <folder path>/tutorials/hiring/check_availability.sl --i address=john.doe@somecompany.com

Run the operation a few times and make sure that both the SUCCESS and FAILURE cases are working as expected.

Download the Code

Lesson 4 - Complete code

Up Next

In the next lesson we’ll integrate our new operation into our flow, using the output and results it sends.

New Code - Complete

check_availability.sl

namespace: tutorials.hiring

operation:
  name: check_availability

  inputs:
    - address

  python_action:
    script: |
      import random
      rand = random.randint(0, 2)
      vacant = rand != 0
      #print rand

  outputs:
    - available: ${str(vacant)}

  results:
    - FAILURE: ${rand == 0}
    - SUCCESS

Lesson 5 - Default Navigation

Goal

In this lesson we’ll use the check_availability operation in our flow and use the results it returns.

Get Started

Let’s go back to new_hire.sl and add in the plumbing to run our new operation.

Inputs

First, we’ll add an inputs section to our flow between the name and the workflow. It works the same way as inputs to an operation.

inputs:
  - address

Just as with an operation, values for the inputs of a flow are either passed via the CloudSlang CLI, as we do below in this lesson, or from a step in a calling flow, as in lesson 9 - Subflows.

Inputs can also have related parameters, such as required, default, private and system_property. We will discuss these parameters in lessons 8 - Input Parameters and 13 - System Properties.

New Step

Now we can add a new step to our flow. We’ll add it right after the print_start step and call it check_address. Here, since both the flow and the operation are in the same folder, the do section does not need to use an alias or path to reference the operation like we needed with the print operation in the print_start step.

Note

The best practice is to use an alias or fully qualified name when calling an operation (the same applies for subflows and decisions) even when it resides in the same folder as the calling flow. For simplicity, we will not follow this practice in this tutorial.

- check_address:
    do:
      check_availability:
        - address
    publish:
      - availability: ${available}

First note that in the check_address step, the address input name is not given an explicit value, as the text input is given in the print_start step. It’s not necessary here because the address input name in the check_availability operation matches the address input name in the flow. The value input to the flow will get passed to the operation input with the same name.

Publish

Also notice that we’ve added the publish section to this step. This is the spot where we publish the outputs returned from the operation to the flow’s scope. In our case, the check_availability operation returns an output named available and we publish it to the flow’s scope under the name availability. We’ll use the availability variable momentarily in an input expression in another of the flow’s steps.

For more information, see publish in the DSL reference.

Input Expression

We’ll add one more step to our flow for now to demonstrate the default navigation behavior. This new step calls the print operation again to print out whether the email address that was provided is available. We pass a string which contains a Python expression to the text input. Note that we are using the published output from the previous step along with some of the flow input variables in the expression.

- print_finish:
    do:
      base.print:
        - text: "${'Availability for address ' + address + ' is: ' + availability}"

Notice the extra set of quotes ("") around the expression. They are necessary to escape the colon (:) which has special meaning in YAML.

Wire Navigation

Each step can declare navigation to explicitly say which step should be the next one to be run in the flow.

The default navigation rules, when no explicit navigate section is declared, are as follows:

Result	Step location	on_failure present	Navigation
SUCCESS	Not last non-on_failure step	–	Next step
SUCCESS	Last non-on_failure step	–	SUCCESS result of the flow
FAILURE	–	Yes	on_failure step
FAILURE	–	No	FAILURE result of the flow

The default navigation only applies when a step calls an operation or subflow that returns a result of either SUCCESS or FAILURE. If the operation or subflow can return a custom result or always returns only SUCCESS or only FAILURE then default navigation will not apply.

Note

Operations which don’t explicitly return any results always return the result SUCCESS.

So what does that mean for us? We can use the default navigation for the check_address step. However, since the print operation will always return a SUCCESS result, we need to add some navigation for the print_start and print_finish steps. The navigate section takes a list of received results and maps them to step names or flow results. In print_start we’ll map the SUCCESS result to the next step. And in print_finish we’ll map the SUCCESS result to the SUCCESS result of the flow.

- print_start:
    do:
      base.print:
        - text: "Starting new hire process"
    navigate:
      - SUCCESS: check_address

...

- print_finish:
    do:
      base.print:
        - text: "${'Availability for address ' + address + ' is: ' + availability}"
    navigate:
      - SUCCESS: SUCCESS

Run It

Let’s save our files and run the flow and see what happens based on the output and results of the generate_user_mail operation. Once again, make sure to run it a few times so we can see what happens when the operation returns a result of SUCCESS and what happens when the result is FAILURE.

run --f <folder path>/tutorials/hiring/new_hire.sl --cp <folder path>/tutorials --i address=john.doe@somecompany.com

When the check_availability operation returns a result of SUCCESS the flow continues with the next step and prints out the availability message. However, when the check_availability operation returns a result of FAILURE the flow ends immediately with a result of FAILURE. This is the default navigation behavior.

Download the Code

Lesson 5 - Complete code

Up Next

In the next lesson we’ll see one way to handle FAILURE results.

New Code - Complete

new_hire.sl

namespace: tutorials.hiring

imports:
  base: tutorials.base

flow:
  name: new_hire

  inputs:
    - address

  workflow:
    - print_start:
        do:
          base.print:
            - text: "Starting new hire process"
        navigate:
          - SUCCESS: check_address

    - check_address:
        do:
          check_availability:
            - address
        publish:
          - availability: ${available}

    - print_finish:
        do:
          base.print:
            - text: "${'Availability for address ' + address + ' is: ' + availability}"
        navigate:
          - SUCCESS: SUCCESS

Lesson 6 - Handling Failure Results

Goal

In this lesson we’ll learn one strategy for handling results of FAILURE using the default navigation.

Get Started

Let’s continue where we left off in the new_hire.sl flow and add some code to deal with the case when the check_availability operation returns a result of FAILURE.

Failure Handling

There is special syntax that can be used for handling FAILURE results by default. We wrap a step inside the on_failure key. Let’s add this functionality after the print_finish step.

- on_failure:
  - print_fail:
      do:
        base.print:
          - text: "${'Failed to create address: ' + address}"

Now, when any step receives a result of FAILURE from its operation the flow will navigate to the on_failure step by default.

For more information, see on_failure in the DSL reference.

Run It

We can save and run the flow using the exact command we used in the last lesson. This time, however, things should work slightly differently.

run --f <folder path>/tutorials/hiring/new_hire.sl --cp <folder path>/tutorials --i address=john.doe@somecompany.com

In the case of the check_availability operation returning a result of SUCCESS we expect the flow to behave exactly as it did before. Notice that this means it will know not to run the on_failure step without us adding any navigation instructions. This is part of the default navigation behavior.

In the case of the check_availability operation returning a result of FAILURE the flow will no longer terminate immediately with a result of FAILURE. Instead, the flow will continue by running the on_failure step, which in our case prints out an error message.

Download the Code

Lesson 6 - Complete code

Up Next

In the next lesson we’ll see how to achieve a similar outcome using custom navigation.

New Code - Complete

new_hire.sl

namespace: tutorials.hiring

imports:
  base: tutorials.base

flow:
  name: new_hire

  inputs:
    - address

  workflow:
    - print_start:
        do:
          base.print:
            - text: "Starting new hire process"
        navigate:
          - SUCCESS: check_address

    - check_address:
        do:
          check_availability:
            - address
        publish:
          - availability: ${available}
        navigate:
          - UNAVAILABLE: print_fail
          - AVAILABLE: SUCCESS

    - print_finish:
        do:
          base.print:
            - text: "${'Availability for address ' + address + ' is: ' + availability}"
        navigate:
          - SUCCESS: SUCCESS

    - on_failure:
      - print_fail:
          do:
            base.print:
              - text: "${'Failed to create address: ' + address}"

Lesson 7 - Custom Navigation

Goal

In this lesson we’ll learn how to override the default navigation, declaring our own results and navigating based on them.

Get Started

Once again, we’ll continue working on the flow in new_hire.sl. This time though, we’ll try to replicate the default navigation behavior using explicit custom navigation. To do that we’ll have to change check_availability.sl as well.

Custom Results

Let’s change the results of the check_availability operation to more closely reflect what is actually going on. Instead of using the default SUCCESS and FAILURE we’ll create our own result labels named AVAILABLE and UNAVAILABLE.

results:
  - UNAVAILABLE: ${rand == 0}
  - AVAILABLE

Custom Navigation

Now that we’ve customized the result labels, the flow doesn’t know what to do upon receipt of these results. Instead of relying on the default navigation, we’ll have to explicitly tell the flow what to do next. For now, we’ll just replicate what the flow would have done in the default navigation. We have to add navigation logic to the step for all possible results. We do so under the navigate key. Each possible result is a list item mapped to the step that should be navigated to when returned.

- check_address:
    do:
      check_availability:
        - address
    publish:
      - availability: ${available}
    navigate:
      - UNAVAILABLE: print_fail
      - AVAILABLE: print_finish

For more information, see navigate in the DSL reference.

Run It

This time, not only can we save the file and then run the flow using the exact command we used in the last lesson, but we expect everything to work exactly as it did before.

run --f <folder path>/tutorials/hiring/new_hire.sl --cp <folder path>/tutorials --i address=john.doe@somecompany.com

Download the Code

Lesson 7 - Complete code

Up Next

In the next lesson we’ll look at how to use the various properties of inputs.

New Code - Complete

new_hire.sl

namespace: tutorials.hiring

imports:
  base: tutorials.base

flow:
  name: new_hire

  inputs:
    - address

  workflow:
    - print_start:
        do:
          base.print:
            - text: "Starting new hire process"
        navigate:
          - SUCCESS: check_address

    - check_address:
        do:
          check_availability:
            - address
        publish:
          - availability: ${available}
        navigate:
          - UNAVAILABLE: print_fail
          - AVAILABLE: print_finish

    - print_finish:
        do:
          base.print:
            - text: "${'Availability for address ' + address + ' is: ' + availability}"
        navigate:
          - SUCCESS: SUCCESS

    - on_failure:
      - print_fail:
          do:
            base.print:
              - text: "${'Failed to create address: ' + address}"

check_availability.sl

namespace: tutorials.hiring

operation:
  name: check_availability

  inputs:
    - address

  python_action:
    script: |
      import random
      rand = random.randint(0, 2)
      vacant = rand != 0
      #print vacant

  outputs:
    - available: ${str(vacant)}

  results:
    - UNAVAILABLE: ${rand == 0}
    - AVAILABLE

Lesson 8 - Input and Output Parameters

Goal

In this lesson we’ll learn how to change the way inputs and outputs behave using input and output properties.

Get Started

It’s time to create a new operation. Create a new file in the tutorials/hiring folder called generate_user_email.sl. In here we’ll create an operation that takes in some user information and produces an email address for that user. We’ll write it so that it will take in which attempt this is at creating an email address for this user. That way we can use it in conjunction with our check_availability operation. Eventually, we’ll generate an address, check its availability, and if it’s unavailable we’ll do it all over again. The following code does not present any new concepts. We will use it as a starting point for a discussion on input properties.

namespace: tutorials.hiring

operation:
  name: generate_user_email

  inputs:
    - first_name
    - middle_name
    - last_name
    - domain
    - attempt

  python_action:
    script: |
      attempt = int(attempt)
      if attempt == 1:
        address = first_name[0:1] + '.' + last_name + '@' + domain
      elif attempt == 2:
        address = first_name + '.' + last_name[0:1] + '@' + domain
      elif attempt == 3 and middle_name != '':
        address = first_name + '.' + middle_name[0:1] + '.' + last_name + '@' + domain
      else:
        address = ''
      # print address

  outputs:
    - email_address: ${address}

  results:
    - FAILURE: ${address == ''}
    - SUCCESS

Test

You can save the file and test that the operation is working as expected by using the following command:

run --f <folder path>/tutorials/hiring/generate_user_email.sl --i first_name=john,middle_name=e,last_name=doe,domain=somecompany,attempt=1

It may help to uncomment the print line to see what is being output. Change the value for attempt in the run command and see what happens.

Add to Flow

Let’s add a step in the new_hire flow to call our new operation. That will allow us to demonstrate how input properties affect the way variables are passed to operations.

Between the print_start step and check_address step we’ll put our new step named generate_address.

- generate_address:
    do:
      generate_user_email:
        - first_name
        - middle_name
        - last_name
        - domain
        - attempt
    publish:
      - address: ${email_address}

We’ll also have to change the inputs of the flow to reflect our new addition. We can remove the address from the flow inputs since we’ll now be getting the address from the generate_user_email operation and publishing it in the generate_address step. Instead, we need to add the inputs necessary for the generate_user_email operation to the flow’s inputs section.

inputs:
  - first_name
  - middle_name
  - last_name
  - domain
  - attempt

We also have to fix the navigation of the print_start step.

- print_start:
    do:
      base.print:
        - text: "Starting new hire process"
    navigate:
      - SUCCESS: generate_address

One last thing to tidy up is the failure message, which no longer receives an address that was not created.

- on_failure:
  - print_fail:
      do:
        base.print:
          - text: "Failed to create address"

At this point everything is set up to go. We can save the file and run the flow as long as we pass all the necessary arguments.

run --f <folder path>/tutorials/hiring/new_hire.sl --cp <folder path>/tutorials --i first_name=john,middle_name=e,last_name=doe,domain=somecompany.com,attempt=1

Required

By default all flow and operation inputs are required. We can change that behavior by setting the required property of an input to false. Let’s make the middle_name optional. We’ll have to set its required property to false in both the flow’s inputs and the generate_user_email operation’s inputs.

flow:
  name: new_hire

  inputs:
    - first_name
    - middle_name:
        required: false
    - last_name
    - domain
    - attempt

operation:
  name: generate_user_email

  inputs:
    - first_name
    - middle_name:
        required: false
    - last_name
    - domain
    - attempt

Note

YAML Note: Don’t forget to add a colon (:) to the input name before adding its properties.

For more information, see required in the DSL reference.

Default

We can also make an input optional by providing a default value. If no value is passed for an input that declares the default property, the default value is used instead. In our case, we can set the generate_user_email operation’s middle_name to default to the empty string.

operation:
  name: generate_user_email

  inputs:
    - first_name
    - middle_name:
        required: false
        default: ""
    - last_name
    - domain
    - attempt

Now the flow can be run after saving the files without providing a value for the middle name.

run --f <folder path>/tutorials/hiring/new_hire.sl --cp <folder path>/tutorials --i first_name=john,last_name=doe,domain=somecompany.com,attempt=1

For more information, see default in the DSL reference.

Private

The default value is only used if another value is not passed to the operation. But sometimes we want to force the default value to be the one used, even if a different value is passed from a flow. Let’s do that to the domain input of the generate_user_email operation. To do so, we set the input’s private parameter to true. We’ll also have to set a default value for the input.

operation:
  name: generate_user_email

  inputs:
    - first_name
    - middle_name:
        required: false
        default: ""
    - last_name
    - domain:
        default: "acompany.com"
        private: true
    - attempt

We can save the file and then run the flow using the same command as above. You’ll notice that no matter what is passed to the domain input, acompany.com is what ends up in the email address. That’s exactly what we want, but obviously there is no reason to pass values to the domain variable anymore. So let’s just remove it from the flow inputs and the generate_address step.

flow:
  name: new_hire

  inputs:
    - first_name
    - middle_name:
        required: false
    - last_name
    - attempt

- generate_address:
    do:
      generate_user_email:
        - first_name
        - middle_name
        - last_name
        - attempt
    publish:
      - address: ${email_address}

For more information, see private in the DSL reference.

Sensitive

Finally, we can mark both inputs and outputs as sensitive. When a variable is marked as sensitive, its value will not be printed in logs, events or in outputs of the CLI and Build Tool.

In the check_availability operation, let’s create a temporary password if an email address is available. We’ll just add a few lines to our script to randomly generate a short password if the address is available. In the outputs section, we’ll mark that password as sensitive. Notice, that when we add a sensitive property to an output we have to add a value property as well.

python_action:
  script: |
    import random
    import string
    rand = random.randint(0, 2)
    vacant = rand != 0
    # print vacant
    if vacant == True:
      password = ''.join(random.choice(string.ascii_uppercase + string.digits) for _ in range(6))
    else:
      password = ''

outputs:
  - available: ${str(vacant)}
  - password:
      value: ${password}
      sensitive: true

You can now run the check_availability operation and see how the password output is not printed to the screen.

run --f <folder path>/tutorials/hiring/check_availability.sl --i address=john.doe@somecompany.com

In the new_hire flow we’ll add password to the publish section of the check_address step to be used later on.

- check_address:
    do:
      check_availability:
        - address
    publish:
      - availability: ${available}
      - password
    navigate:
      - UNAVAILABLE: print_fail
      - AVAILABLE: print_finish

For more information, see sensitive in the DSL reference.

Run It

Now we can save the file and run the flow without passing the domain. We can also leave out the middle name if we want, but we can also leave it in.

run --f <folder path>/tutorials/hiring/new_hire.sl --cp <folder path>/tutorials --i first_name=john,last_name=doe,attempt=1

Download the Code

Lesson 8 - Complete code

Up Next

In the next lesson we’ll see how to use subflows.

New Code - Complete

new_hire.sl

namespace: tutorials.hiring

imports:
  base: tutorials.base

flow:
  name: new_hire

  inputs:
    - first_name
    - middle_name:
        required: false
    - last_name
    - attempt

  workflow:
    - print_start:
        do:
          base.print:
            - text: "Starting new hire process"
        navigate:
          - SUCCESS: generate_address

    - generate_address:
        do:
          generate_user_email:
            - first_name
            - middle_name
            - last_name
            - attempt
        publish:
          - address: ${email_address}

    - check_address:
        do:
          check_availability:
            - address
        publish:
          - availability: ${available}
          - password
        navigate:
          - UNAVAILABLE: print_fail
          - AVAILABLE: print_finish

    - print_finish:
        do:
          base.print:
            - text: "${'Availability for address ' + address + ' is: ' + availability}"
        navigate:
          - SUCCESS: SUCCESS

    - on_failure:
      - print_fail:
          do:
            base.print:
              - text: "Failed to create address"

generate_user_email.sl

namespace: tutorials.hiring

operation:
  name: generate_user_email

  inputs:
    - first_name
    - middle_name:
        required: false
        default: ""
    - last_name
    - domain:
        default: "acompany.com"
        private: true
    - attempt

  python_action:
    script: |
      attempt = int(attempt)
      if attempt == 1:
        address = first_name[0:1] + '.' + last_name + '@' + domain
      elif attempt == 2:
        address = first_name + '.' + last_name[0:1] + '@' + domain
      elif attempt == 3 and middle_name != '':
        address = first_name + '.' + middle_name[0:1] + '.' + last_name + '@' + domain
      else:
        address = ''
      # print address

  outputs:
    - email_address: ${address}

  results:
    - FAILURE: ${address == ''}
    - SUCCESS

check_availability.sl

namespace: tutorials.hiring

operation:
  name: check_availability

  inputs:
    - address

  python_action:
    script: |
      import random
      import string
      rand = random.randint(0, 2)
      vacant = rand != 0
      # print vacant
      if vacant == True:
        password = ''.join(random.choice(string.letters) for _ in range(6))
      else:
        password = ''

  outputs:
    - available: ${str(vacant)}
    - password:
        value: ${password}
        sensitive: true
  results:
    - UNAVAILABLE: ${rand == 0}
    - AVAILABLE

Lesson 9 - Subflows

Goal

In this lesson we’ll learn how to use subflows.

Get Started

We’ll start by creating a new file in the tutorials/hiring folder called create_user_email.sl to hold our subflow. A subflow is a flow itself and therefore it follows all the regular flow syntax.

Move Code

The first thing we’ll do is steal a bunch of the code that currently sits in new_hire.sl. Let’s take everything up until the workflow key and copy it into the new flow and make a couple of changes. First, we won’t need the imports, so we can just delete them. Next, we’ll change the name of the flow to create_user_email. That should do it for this section.

namespace: tutorials.hiring

flow:
  name: create_user_email

  inputs:
    - first_name
    - middle_name:
        required: false
    - last_name
    - attempt

Next let’s create a workflow section and copy the generate_address and check_address steps into it.

workflow:
  - generate_address:
      do:
        generate_user_email:
          - first_name
          - middle_name
          - last_name
          - attempt
      publish:
        - address: ${email_address}

  - check_address:
      do:
        check_availability:
          - address
      publish:
        - availability: ${available}
        - password
      navigate:
        - UNAVAILABLE: print_fail
        - AVAILABLE: print_finish

Fix Up Subflow

Now we have to reroute our navigation, add flow outputs and flow results.

Let’s start with adding the flow results. We’ll have our flow return one of three result options.

1. CREATED - everything went smoothly and a new, available address was created
2. UNAVAILABLE - an address was generated, but it wasn’t available
3. FAILURE - an address was not even generated.

results:
  - CREATED
  - UNAVAILABLE
  - FAILURE

Now we can reroute the steps’ navigation to point to the flow results we just defined.

For the generate_address step, whose operation returns SUCCESS or FAILURE, we can route SUCCESS to the next step and FAILURE to the FAILURE result of the flow.

- generate_address:
    do:
      generate_user_email:
        - first_name
        - middle_name
        - last_name
        - attempt
    publish:
      - address: ${email_address}
    navigate:
      - SUCCESS: check_address
      - FAILURE: FAILURE

For the check_address step, whose operation returns UNAVAILABLE or AVAILABLE, we can route UNAVAILABLE to the UNAVAILABLE result of the flow and AVAILABLE to the CREATED result of the flow.

- check_address:
    do:
      check_availability:
        - address
    publish:
      - availability: ${available}
      - password
    navigate:
      - UNAVAILABLE: UNAVAILABLE
      - AVAILABLE: CREATED

Finally, we can pass along the outputs published in the steps as flow outputs.

outputs:
  - address
  - password
  - availability

Test It

At this point the subflow is ready and we can test it by running it as we would any other flow. Save the file and run it a few times while playing with the attempt input to make sure all three possible results are being returned at some point.

run --f <folder path>/tutorials/hiring/create_user_email.sl --cp <folder path>/tutorials --i first_name=john,last_name=doe,attempt=1

Fix Up Parent Flow

Finally, let’s make changes to our original flow so that it makes use of the subflow we just created.

First let’s replace the two steps we took out with one new step that calls the subflow instead of an operation. You may have noticed that both flows and operations take inputs, return outputs and return results. That allows us to use them almost interchangeably. We’ve run both flows and operations using the CLI. Now we see that we can call them both from steps as well.

Delete the generate_address and check_address steps. We’ll now replace them with a new step called create_email_address. It will pass along the flow inputs, publish the necessary outputs and wire up the appropriate navigation.

- create_email_address:
    do:
      create_user_email:
        - first_name
        - middle_name
        - last_name
        - attempt
    publish:
      - address
      - password
    navigate:
      - CREATED: print_finish
      - UNAVAILABLE: print_fail
      - FAILURE: print_fail

All that’s left now is to change the text of the messages sent in the print_finish and print_fail steps to better reflect what is happening.

- print_finish:
    do:
      base.print:
        - text: "${'Created address: ' + address + ' for: ' + first_name + ' ' + last_name}"
    navigate:
      - SUCCESS: SUCCESS

- on_failure:
  - print_fail:
      do:
        base.print:
          - text: "${'Failed to create address for: ' + first_name + ' ' + last_name}"

Run It

Now we can save the files and run the parent flow, which will also run the subflow. Once again, you should run it a few times and play with the attempt input to make sure all the possible outcomes are occurring at some point.

run --f <folder path>/tutorials/hiring/new_hire.sl --cp <folder path>/tutorials --i first_name=john,last_name=doe,attempt=1

Download the Code

Lesson 9 - Complete code

Up Next

In the next lesson we’ll change our new step to include a loop which will retry the email creation several times if necessary.

New Code - Complete

new_hire.sl

namespace: tutorials.hiring

imports:
  base: tutorials.base

flow:
  name: new_hire

  inputs:
    - first_name
    - middle_name:
        required: false
    - last_name
    - attempt

  workflow:
    - print_start:
        do:
          base.print:
            - text: "Starting new hire process"
        navigate:
          - SUCCESS: create_email_address

    - create_email_address:
        do:
          create_user_email:
            - first_name
            - middle_name
            - last_name
            - attempt
        publish:
          - address
          - password
        navigate:
          - CREATED: print_finish
          - UNAVAILABLE: print_fail
          - FAILURE: print_fail

    - print_finish:
        do:
          base.print:
            - text: "${'Created address: ' + address + ' for: ' + first_name + ' ' + last_name}"
        navigate:
          - SUCCESS: SUCCESS

    - on_failure:
      - print_fail:
          do:
            base.print:
              - text: "${'Failed to create address for: ' + first_name + ' ' + last_name}"

create_user_email

namespace: tutorials.hiring

flow:
  name: create_user_email

  inputs:
    - first_name
    - middle_name:
        required: false
    - last_name
    - attempt

  workflow:
    - generate_address:
        do:
          generate_user_email:
            - first_name
            - middle_name
            - last_name
            - attempt
        publish:
          - address: ${email_address}
        navigate:
          - SUCCESS: check_address
          - FAILURE: FAILURE

    - check_address:
        do:
          check_availability:
            - address
        publish:
          - availability: ${available}
          - password
        navigate:
          - UNAVAILABLE: UNAVAILABLE
          - AVAILABLE: CREATED

  outputs:
    - address
    - password
    - availability

  results:
    - CREATED
    - UNAVAILABLE
    - FAILURE

Lesson 10 - For Loop

Goal

In this lesson we’ll learn how to use a for loop to create an iterative step.

Get Started

The idea here is to continually try the create_user_email subflow until it either creates an available address or fails. To do so, we should be able to leave the subflow as is and just work on the create_email_address step in new_hire.sl.

Loop Syntax

An iterative step looks very similar to a standard step that only runs once. To transform our create_email_address step into one that loops we’ll add the loop key along with a loop expression and indent the do and publish sections. For now, we’ll loop over a list of numbers.

- create_email_address:
    loop:
      for: attempt in [1,2,3,4]
      do:
        create_user_email:
          - first_name
          - middle_name
          - last_name
          - attempt: ${str(attempt)}
      publish:
        - address
        - password
    navigate:
      - CREATED: print_finish
      - UNAVAILABLE: print_fail
      - FAILURE: print_fail

Note

YAML Note: A list can be written using bracket ([]) notation instead of using indentation and hyphens (-).

For each item in our list the attempt loop variable is assigned the value and then passed to an iteration of the subflow call. All inputs must be strings. Therefore we convert the attempt value to a string using the Python str() function.

Since we’re assigning a value to attempt in the loop and not using it as flow input we can delete it from the flow’s input list.

For more information, see loop, for and publish in the DSL reference.

Default Behavior

We can save the file and run the flow now and see how the loop works. It won’t quite do what we want yet, but it will demonstrate what a loop’s default behavior is. Play around a bit with passing the optional middle name and not passing it to see what happens. Also try removing the last item in the loop expression’s list.

run --f <folder path>/tutorials/hiring/new_hire.sl --cp <folder path>/tutorials --i first_name=john,middle_name=e,last_name=doe

The first thing you’ll notice is that the subflow is being run several times. This is what our loop has done. Next, you’ll notice that depending on whether you’ve passed a middle name and how big the loop list is, different things will happen. This is due to the default behavior of loops and our create_user_email subflow.

By default a loop exits when either the list it is looping on has been exhausted or the operation or subflow called returns a result of FAILURE. This will explain the following cases:

case	middle_name	list	iterations	flow result
1	no	[1,2,3,4]	3	FAILURE
2	yes	[1,2,3,4]	4	FAILURE
3	no	[1,2,3]	3	FAILURE
4	yes	[1,2,3]	3	FAILURE or SUCCESS

For all cases: For attempt 1 and 2 the create_user_email subflow runs it will return a result of either CREATED or UNAVAILABLE because the generate_user_email operation will return a result of SUCCESS. Since neither of those are FAILURE, the loop will continue to run.

Case 1: Since the middle_name is not present, the generate_user_email operation will return a result of FAILURE when the 3 is passed to its attempt input. The loop exits on the FAILURE result by default and goes to its navigate section which forwards it to print_fail. Since print_fail is the on_failure step, it ends the flow with a result of FAILURE.

Case 2: This case is very similar to the previous one. The only difference is that the generate_user_email operation will return a result of FAILURE when the 4, not 3, is passed to its attempt input.

Case 3: This case is even more similar to the first case. The first case never got to the 4th iteration of the loop, so we can expect that it we removed the 4th item from the list the same thing will happen.

Case 4: This time we have a middle_name so the create_user_email subflow will run successfully all three times, returning results of either CREATED or UNAVAILABLE. Since neither of those are FAILURE, the loop will only exit when the list is exhausted. At that point the result from the last iteration of the step will be used by the navigation to see where the flow goes next. If the last iteration’s result is CREATED, the print_finish step will run and the flow will end with a result of SUCCESS. If the last iteration’s result is UNAVAILABLE, the print_fail step will run and the flow will end with a result of FAILURE.

Custom Break

Now that we understand what happens in the default case, let’s put in a custom break so the loop will do what we want it to. We want the loop to stop when we’ve either found a suitable email address or something has gone wrong, so we’ll add a break key with a list of results we want to break on, which in our case is CREATED or FAILURE.

- create_email_address:
    loop:
      for: attempt in [1,2,3,4]
      do:
        create_user_email:
          - first_name
          - middle_name
          - last_name
          - attempt
      publish:
        - address
        - password
      break:
        - CREATED
        - FAILURE
    navigate:
      - CREATED: print_finish
      - UNAVAILABLE: print_fail
      - FAILURE: print_fail

In a case where we want the loop to continue no matter what happens, we would have to override the default break on a result of failure by mapping the break key to an empty list ([]).

The published address variable will contain the address value from the last iteration of the loop. We can use at the same way published variables are used in regular steps. However, when using loops, you often want to aggregate the published output. We will do that in the next lesson.

For more information, see break in the DSL reference.

List Types

One last thing we can change to improve our flow is the loop’s list. Right now we’re using a literal list, but we can use any Python expression that results in a list instead. So here we can substitute [1,2,3,4] with range(1,5). We could also use a comma delimited strings which would be split automatically into a list.

Run It

Everything should be working as expected now. We can save our file and run the flow with or without a middle name. To test a result of FAILURE it’s best not to pass a middle name and run the flow several times.

run --f <folder path>/tutorials/hiring/new_hire.sl --cp <folder path>/tutorials --i first_name=john,last_name=doe

Download the Code

Lesson 10 - Complete code

Up Next

In the next lesson we’ll write another loop and aggregate the information that is output.

New Code - Complete

new_hire.sl

namespace: tutorials.hiring

imports:
  base: tutorials.base

flow:
  name: new_hire

  inputs:
    - first_name
    - middle_name:
        required: false
    - last_name

  workflow:
    - print_start:
        do:
          base.print:
            - text: "Starting new hire process"
        navigate:
          - SUCCESS: create_email_address

    - create_email_address:
        loop:
          for: attempt in range(1,5)
          do:
            create_user_email:
              - first_name
              - middle_name
              - last_name
              - attempt: ${str(attempt)}
          publish:
            - address
            - password
          break:
            - CREATED
            - FAILURE
        navigate:
          - CREATED: print_finish
          - UNAVAILABLE: print_fail
          - FAILURE: print_fail

    - print_finish:
        do:
          base.print:
            - text: "${'Created address: ' + address + ' for: ' + first_name + ' ' + last_name}"
        navigate:
          - SUCCESS: SUCCESS

    - on_failure:
      - print_fail:
          do:
            base.print:
              - text: "${'Failed to create address for: ' + first_name + ' ' + last_name}"

Lesson 11 - Loop Aggregation

Goal

In this lesson we’ll learn how to aggregate output from a loop.

Get Started

We’ll create a new step to simulate ordering equipment. Internally it will randomly decide whether a piece of equipment is available or not. Then we’ll run that step in a loop from the main flow and record the cost of the ordered equipment and which items were unavailable. Create a new file named order.sl in the tutorials/hiring folder to house the new operation we’ll write and get the new_hire.sl file ready because we’ll need to add a step to the main flow.

Operation

The order operation, as we’ll call it, looks very similar to our check_availability operation. It uses a random number to simulate whether a given item is available. If the item is available, it will return the amount spent as one output and the not_ordered output will be empty. If the item is unavailable, it will return 0 for the spent output and the name of the item in the not_ordered output.

namespace: tutorials.hiring

operation:
  name: order

  inputs:
    - item
    - price

  python_action:
    script: |
      print 'Ordering: ' + item
      import random
      rand = random.randint(0, 2)
      available = rand != 0
      not_ordered = item + ';' if rand == 0 else ''
      spent = 0 if rand == 0 else price
      if rand == 0: print 'Unavailable'

  outputs:
    - not_ordered
    - spent: ${spent}

  results:
    - UNAVAILABLE: ${rand == 0}
    - AVAILABLE

Step

Now let’s go back to our flow and create a step, between create_email_address and print_finish, to call our operation in a loop. This time we’ll loop through a map of items and their prices, named, order_map that we’ll define at the flow level in a few moments. We use the Python eval() function to turn a string into a Python dictionary that we can loop over.

- get_equipment:
    loop:
      for: item, price in eval(order_map)
      do:
        order:
          - item
          - price: ${str(price)}
          - missing: ${all_missing}
          - cost: ${total_cost}
      break: []

Notice the missing and cost variables. These are not for inputs in the order operation. That operation only takes the item and price inputs. We will be using missing and cost together with some flow-level variables to perform the loop aggregation.

Also notice how we’ve added a break which maps to an empty list of break results. This is necessary because the order operation does not contain a result of FAILURE which is the default for breaking out of a loop.

Now let’s create those flow-level variables in the flow’s inputs section. Each time through the loop we want to aggregate the data that the order operation outputs. We’ll create two variables, all_missing and total_cost, for this purpose, defining them as private and giving them default values to start with.

Also, we’ll declare another variable called order_map that will contain the map we’re looping on.

inputs:
  - first_name
  - middle_name:
      required: false
  - last_name
  - all_missing:
      default: ""
      required: false
      private: true
  - total_cost:
      default: '0'
      private: true
  - order_map:
      default: '{"laptop": 1000, "docking station": 200, "monitor": 500, "phone": 100}'

Now we can perform the aggregation. In the get_equipment step’s publish section, we’ll add the values output from the order operation (not_ordered and spent) to the step arguments we just created in the get_equipment step (missing and cost) and publish them back to the flow-level variables (all_missing and total_cost). This will run for each iteration after the operation has completed, aggregating all the data. For example, each time through the loop the cost is updated with the current total_cost. Then the order operation runs and a spent value is output. That spent value is added to the step’s cost variable and published back into the flow-level total_cost for each iteration of the get_equipment step.

publish:
  - all_missing: ${missing + not_ordered}
  - total_cost: ${str(int(cost) + int(spent))}

Finally we have to rewire all the navigation logic to take into account our new step.

We need to change the create_email_address step to forward successful email address creations to get_equipment.

navigate:
  - CREATED: get_equipment
  - UNAVAILABLE: print_fail
  - FAILURE: print_fail

And we need to add navigation to the get_equipment step. We’ll always go to print_finish no matter what happens.

navigate:
  - AVAILABLE: print_finish
  - UNAVAILABLE: print_finish

Finish

The last thing left to do is print out a finish message that also reflects the status of the equipment order.

- print_finish:
    do:
      base.print:
        - text: >
            ${'Created address: ' + address + ' for: ' + first_name + ' ' + last_name + '\n' +
            'Missing items: ' + all_missing + ' Cost of ordered items: ' + total_cost}
    navigate:
      - SUCCESS: SUCCESS

Run It

We can save the files, run the flow and see that the ordering takes place, the proper information is aggregated and then it is printed.

run --f <folder path>/tutorials/hiring/new_hire.sl --cp <folder path>/tutorials --i first_name=john,middle_name=e,last_name=doe

Download the Code

Lesson 11 - Complete code

Up Next

In the next lesson we’ll see how to write a decision.

New Code - Complete

new_hire.sl

namespace: tutorials.hiring

imports:
  base: tutorials.base

flow:
  name: new_hire

  inputs:
    - first_name
    - middle_name:
        required: false
    - last_name
    - all_missing:
        default: ""
        required: false
        private: true
    - total_cost:
        default: '0'
        private: true
    - order_map:
        default: '{"laptop": 1000, "docking station": 200, "monitor": 500, "phone": 100}'

  workflow:
    - print_start:
        do:
          base.print:
            - text: "Starting new hire process"
        navigate:
          - SUCCESS: create_email_address

    - create_email_address:
        loop:
          for: attempt in range(1,5)
          do:
            create_user_email:
              - first_name
              - middle_name
              - last_name
              - attempt: ${str(attempt)}
          publish:
            - address
            - password
          break:
            - CREATED
            - FAILURE
        navigate:
          - CREATED: get_equipment
          - UNAVAILABLE: print_fail
          - FAILURE: print_fail

    - get_equipment:
        loop:
          for: item, price in eval(order_map)
          do:
            order:
              - item
              - price: ${str(price)}
              - missing: ${all_missing}
              - cost: ${total_cost}
          publish:
            - all_missing: ${missing + not_ordered}
            - total_cost: ${str(int(cost) + int(spent))}
          break: []
        navigate:
          - AVAILABLE: print_finish
          - UNAVAILABLE: print_finish

    - print_finish:
        do:
          base.print:
            - text: >
                ${'Created address: ' + address + ' for: ' + first_name + ' ' + last_name + '\n' +
                'Missing items: ' + all_missing + ' Cost of ordered items: ' + total_cost}
        navigate:
          - SUCCESS: SUCCESS

    - on_failure:
      - print_fail:
          do:
            base.print:
              - text: "${'Failed to create address for: ' + first_name + ' ' + last_name}"

order.sl

namespace: tutorials.hiring

operation:
  name: order

  inputs:
    - item
    - price

  python_action:
    script: |
      print 'Ordering: ' + item
      import random
      rand = random.randint(0, 2)
      available = rand != 0
      not_ordered = item + ';' if rand == 0 else ''
      spent = 0 if rand == 0 else price
      if rand == 0: print 'Unavailable'

  outputs:
    - not_ordered
    - spent: ${str(spent)}

  results:
    - UNAVAILABLE: ${rand == 0}
    - AVAILABLE

Lesson 12 - Decisions

Goal

In this lesson we’ll write a decision. We’ll learn the how to use a decision by creating one that will determine if a given requirement is met.

Get Started

First, we’ll create a contains.sl file in the base folder. As we’ll see, a decision is very similar to an operation. The only real difference is that it cannot contain an action.

The contains decision will determine if a given substring is contained within a given container string.

Decision

From what we already know, a decision should be pretty self explanatory. So let’s just dive in.

namespace: tutorials.base

decision:
  name: contains

  inputs:
    - container:
        default: ""
        required: false
    - sub

  results:
    - CONTAINS: ${container.find(sub) >= 0}
    - DOES_NOT_CONTAIN

Just about everything above should be familiar. The only new thing is the decision keyword which replaces what would have been the operation keyword in an operation. Other than that, the decision has a namespace, name, inputs and results. A decision can have outputs as well, but we don’t use them here.

In terms of function, the decision returns a result of CONTAINS when sub is found in container and a result of DOES_NOT_CONTAIN otherwise.

Call from Flow

Now let’s call the decision from a flow. Unsurprisingly, a decision is called in the exact same way an operation or subflow would be called.

In new_hire.sl we’ll add a step right after get_equipment and call it check_min_reqs. That step will call our decision and navigate accordingly.

- check_min_reqs:
    do:
      base.contains:
        - container: ${all_missing}
        - sub: 'laptop'
    navigate:
      - DOES_NOT_CONTAIN: print_finish
      - CONTAINS: print_warning

We pass the all_missing string to the decision to check if it contains the word 'laptop'. We’ll say the if the new hire didn’t get a laptop we need to print a warning.

Clean Up

Finally, to get everything working properly we need to reroute the navigation of get_equipment add a print_warning step.

The get_equipment navigation should now always point to check_min_reqs.

navigate:
  - AVAILABLE: check_min_reqs
  - UNAVAILABLE: check_min_reqs

And we’ll add a simple print_warning step.

- print_warning:
    do:
      base.print:
        - text: >
            ${first_name + ' ' + last_name +
            ' did not receive all the required equipment'}
    navigate:
      - SUCCESS: print_finish

Now let’s review the possible scenarios.

1. A laptop was ordered: get_equipment navigates to check_min_reqs which returns a result of DOES_NOT_CONTAIN, therefore navigating to print_finish and then ending the flow. The output is exactly as it was before.
2. A laptop was not ordered: get_equipment navigates to check_min_reqs which returns a result of CONTAINS, therefore navigating to print_warning and then print_finish by default navigation and finally ending the flow. The output is as it was before, plus the warning is printed.

Run It

We can save the files and run the flow a few times to see that the warning is printed when appropriate and nothing is changed otherwise.

run --f <folder path>/tutorials/hiring/new_hire.sl --cp <folder path>/tutorials --i first_name=john,middle_name=e,last_name=doe

Download the Code

Lesson 12 - Complete code

Up Next

In the next lesson we’ll see how to use existing content in your flows.

New Code - Complete

new_hire.sl

namespace: tutorials.hiring

imports:
  base: tutorials.base

flow:
  name: new_hire

  inputs:
    - first_name
    - middle_name:
        required: false
    - last_name
    - all_missing:
        default: ""
        required: false
        private: true
    - total_cost:
        default: '0'
        private: true
    - order_map:
        default: '{"laptop": 1000, "docking station": 200, "monitor": 500, "phone": 100}'

  workflow:
    - print_start:
        do:
          base.print:
            - text: "Starting new hire process"
        navigate:
          - SUCCESS: create_email_address

    - create_email_address:
        loop:
          for: attempt in range(1,5)
          do:
            create_user_email:
              - first_name
              - middle_name
              - last_name
              - attempt: ${str(attempt)}
          publish:
            - address
            - password
          break:
            - CREATED
            - FAILURE
        navigate:
          - CREATED: get_equipment
          - UNAVAILABLE: print_fail
          - FAILURE: print_fail

    - get_equipment:
        loop:
          for: item, price in eval(order_map)
          do:
            order:
              - item
              - price: ${str(price)}
              - missing: ${all_missing}
              - cost: ${total_cost}
          publish:
            - all_missing: ${missing + not_ordered}
            - total_cost: ${str(int(cost) + int(spent))}
          break: []
        navigate:
          - AVAILABLE: check_min_reqs
          - UNAVAILABLE: check_min_reqs

    - check_min_reqs:
        do:
          base.contains:
            - container: ${all_missing}
            - sub: 'laptop'
        navigate:
          - DOES_NOT_CONTAIN: print_finish
          - CONTAINS: print_warning

    - print_warning:
        do:
          base.print:
            - text: >
                ${first_name + ' ' + last_name +
                ' did not receive all the required equipment'}
        navigate:
          - SUCCESS: print_finish

    - print_finish:
        do:
          base.print:
            - text: >
                ${'Created address: ' + address + ' for: ' + first_name + ' ' + last_name + '\n' +
                'Missing items: ' + all_missing + ' Cost of ordered items: ' + total_cost}
        navigate:
          - SUCCESS: SUCCESS

    - on_failure:
      - print_fail:
          do:
            base.print:
              - text: "${'Failed to create address for: ' + first_name + ' ' + last_name}"

contains.sl

namespace: tutorials.base

decision:
  name: contains

  inputs:
    - container:
        default: ""
        required: false
    - sub

  results:
    - DOES_NOT_CONTAIN: ${container.find(sub) == -1}
    - CONTAINS

Lesson 13 - Existing Content

Goal

In this lesson we’ll learn how to easily integrate ready-made content into our flow.

Get Started

Instead of printing that our flow has completed, let’s send an email to HR to let them know that the new hire’s email address has been created and notify them as to the status of the new hire’s equipment order. If you’re using a pre-built CLI you’ll have a folder named content that contains all of the ready-made content. If you’ve built the CLI from the source code, you’ll have to get the content mentioned below from the GitHub repository and point to the right location when running the flow.

Ready-Made Operation

We’ll use the send_mail operation which is found in the base/mail folder. All ready-made content begins with a commented explanation of its purpose and its inputs, outputs and results.

Here’s the documentation for the send_mail operation:

########################################################################################################################
#!!
#! @description: This operation sends SMTP email.
#!
#! @input hostname: The hostname or ip address of the SMTP server.
#! @input port: The port of the SMTP service.
#! @input from: From email address.
#! @input to: A delimiter separated list of email address(es) or recipients where the email will be sent.
#! @input cc: Optional - A delimiter separated list of email address(es) or recipients, to be placed in the CC.
#!            Default: ''
#! @input bcc: Optional - A delimiter separated list of email address(es) or recipients, to be placed in the BCC.
#!             Default: ''
#! @input subject: The email subject. If a subject spans on multiple lines, it is formatted to a single one.
#! @input body: The body of the email.
#! @input html_email: The value should be true if the email is in rich text/html format.
#!                    The value should be false if the email is in plain text format.
#!                    Valid values: 'true', 'false'.
#!                    Default: 'true'.
#! @input read_receipt: Optional - The value should be true if read receipt is required, else false.
#!                                 Valid values: 'true', 'false'.
#!                                 Default: 'false'
#! @input attachments: Optional - A delimited separated list of files to attach (must be full path).
#!                     Default: ''
#! @input username: Optional - If SMTP authentication is needed, the username to use.
#!                  Default: ''
#! @input password: Optional - If SMTP authentication is needed, the password to use.
#!                  Default: ''
#! @input character_set: Optional - The character set encoding for the entire email which includes subject, body,
#!                                  attached file name and the attached file.
#!                                  Valid values: 'UTF-8', 'UTF-16', 'UTF-32', 'EUC-JP', 'ISO-2022-JP',
#!                                                'Shift_JIS', 'Windows-31J'.
#!                                  Default: 'UTF-8'
#! @input content_transfer_encoding: Optional - The content transfer encoding scheme (such as 7bit, 8bit, base64,
#!                                              quoted-printable, etc) for the entire email which includes subject,
#!                                              body, attached file name and the attached file.
#!                                              Valid values: 'quoted-printable', 'base64', '7bit',
#!                                                            '8bit', 'binary', 'x-token'.
#!                                              Default: 'base64'
#! @input delimiter: Optional -  A delimiter to separate the email recipients and the attachments.
#!                               Default: ''
#! @input enable_TLS: Optional - Specify if the connection should be TLS enabled or not.
#!                               Default: 'false'
#!
#! @output return_result: That will contain the 'Sent Mail Successfully' if the mail was sent successfully.
#! @output return_code: '0' if success, '-1' otherwise.
#! @output exception: The exception message if the operation goes to failure.
#!
#! @result SUCCESS: Succeeds if mail was sent successfully and the return_code = '0'.
#! @result FAILURE: There was an error while trying to sent the mail and the return code = '-1'.
#!!#
########################################################################################################################

We could get this information by opening the operation from the ready-made content folder or by running inspect on the flow.

inspect <content folder path>/io/cloudslang/base/mail/send_mail.sl

When calling the operation, we’ll need to pass values for all the arguments listed in the documentation that are not optional.

Imports

First, we’ll need to set up an import alias for the new operation since it doesn’t reside where our other operations and subflows do.

imports:
  base: tutorials.base
  mail: io.cloudslang.base.mail

For more information, see imports in the DSL reference.

Step

Then, all we really need to do is create a step in our flow that will call the send_mail operation. Let’s put it right after the print_finish operation. We need to pass a host, port, from, to, subject and body. You’ll need to substitute the values in angle brackets (<>) to work for your email host. Notice that the body value is taken directly from the print_finish step with two slight changes. First, we turned the \n into a <br> since the html_email input defaults to true. Second, we added the temporary password published by the create_email_address step.

- send_mail:
    do:
      mail.send_mail:
        - hostname: "<host>"
        - port: "<port>"
        - from: "<from>"
        - to: "<to>"
        - subject: "${'New Hire: ' + first_name + ' ' + last_name}"
        - body: >
            ${'Created address: ' + address + ' for: ' + first_name + ' ' + last_name + '<br>' +
            'Missing items: ' + all_missing + ' Cost of ordered items: ' + str(total_cost) + '<br>' +
            'Temporary password: ' + password}
    navigate:
      - FAILURE: FAILURE
      - SUCCESS: SUCCESS

Run It

We can save the files, run the flow and check that an email was sent with the proper information.

run --f <folder path>/tutorials/hiring/new_hire.sl --cp <folder path>/tutorials,<content folder path>/io/cloudslang/base --i first_name=john,last_name=doe

Download the Code

Lesson 13 - Complete code

Up Next

In the next lesson we’ll see how to use system properties to send values to input variables.

New Code - Complete

new_hire.sl

namespace: tutorials.hiring

imports:
  base: tutorials.base
  mail: io.cloudslang.base.mail

flow:
  name: new_hire

  inputs:
    - first_name
    - middle_name:
        required: false
    - last_name
    - all_missing:
        default: ""
        required: false
        private: true
    - total_cost:
        default: '0'
        private: true
    - order_map:
        default: '{"laptop": 1000, "docking station": 200, "monitor": 500, "phone": 100}'

  workflow:
    - print_start:
        do:
          base.print:
            - text: "Starting new hire process"
        navigate:
          - SUCCESS: create_email_address

    - create_email_address:
        loop:
          for: attempt in range(1,5)
          do:
            create_user_email:
              - first_name
              - middle_name
              - last_name
              - attempt: ${str(attempt)}
          publish:
            - address
            - password
          break:
            - CREATED
            - FAILURE
        navigate:
          - CREATED: get_equipment
          - UNAVAILABLE: print_fail
          - FAILURE: print_fail

    - get_equipment:
        loop:
          for: item, price in eval(order_map)
          do:
            order:
              - item
              - price: ${str(price)}
              - missing: ${all_missing}
              - cost: ${total_cost}
          publish:
            - all_missing: ${missing + not_ordered}
            - total_cost: ${str(int(cost) + int(spent))}
          break: []
        navigate:
          - AVAILABLE: check_min_reqs
          - UNAVAILABLE: check_min_reqs

    - check_min_reqs:
        do:
          base.contains:
            - container: ${all_missing}
            - sub: 'laptop'
        navigate:
          - DOES_NOT_CONTAIN: print_finish
          - CONTAINS: print_warning

    - print_warning:
        do:
          base.print:
            - text: >
                ${first_name + ' ' + last_name +
                ' did not receive all the required equipment'}
        navigate:
          - SUCCESS: print_finish

    - print_finish:
        do:
          base.print:
            - text: >
                ${'Created address: ' + address + ' for: ' + first_name + ' ' + last_name + '\n' +
                'Missing items: ' + all_missing + ' Cost of ordered items: ' + total_cost}
        navigate:
          - SUCCESS: send_mail

    - send_mail:
       do:
         mail.send_mail:
           - hostname: "<host>"
           - port: "<port>"
           - from: "<from>"
           - to: "<to>"
           - subject: "${'New Hire: ' + first_name + ' ' + last_name}"
           - body: >
               ${'Created address: ' + address + ' for: ' + first_name + ' ' + last_name + '<br>' +
               'Missing items: ' + all_missing + ' Cost of ordered items: ' + total_cost + '<br>' +
               'Temporary password: ' + password}
       navigate:
         - FAILURE: FAILURE
         - SUCCESS: SUCCESS

    - on_failure:
      - print_fail:
          do:
            base.print:
              - text: "${'Failed to create address for: ' + first_name + ' ' + last_name}"

Lesson 14 - System Properties

Goal

In this lesson we’ll learn how to use system properties to set the values of inputs.

Get Started

We’ll need to create a system properties file that contains the values we want to use for the inputs. Let’s create a properties folder under tutorials and in there create a file named bcompany.prop.sl. We’ll also need to use the system properties somewhere. We’ll use them in the new_hire.sl and generate_user_email.sl files.

System Properties File

A system properties file ends with the .prop.sl extension and can include a namespace. A system properties file also contains the properties keyword which is mapped to a list of key:value pairs that define system property names and values.

Here’s what the contents of our system properties file looks like:

namespace: tutorials.properties

properties:
  - domain: bcompany.com
  - hostname: <host>
  - port: '25'
  - system_address: <test@test.com>
  - hr_address: <test@test.com>

You’ll need to substitute the values in angle brackets (<>) to work for your email host.

Note

All system property values are interpreted as strings. So in our case, even if the port is a numeric value, it’s value when used as a system property will be a string representation. For example, entering a value of 25 will create a system property whose value is '25'.

For more information, see properties in the DSL Reference and Run with System Properties in the CLI documentation.

Retrieve Values

Now we’ll use the system properties to place values in our inputs and step arguments. We retrieve system property values using the get_sp() function. We’ll do this in two places.

Note

The get_sp() function can also be used to retrieve system property values in publish, output and result expressions.

First, we’ll use a system property in the inputs of generate_user_email by calling the get_sp() function in the default property of the the domain input. The get_sp() function will retrieve the value associated with the property defined by the fully qualified name in its first argument. If no such property is found, the function will return the second argument.

inputs:
  - first_name
  - middle_name:
      required: false
      default: ""
  - last_name
  - domain:
      default: ${get_sp('tutorials.properties.domain', 'acompany.com')}
      private: true
  - attempt

The second place we’ll use system properties is in the new_hire flow. Here we’ll retrieve the system properties in the arguments of the send_mail step we created last lesson. We’ll use the get_sp() function to get the hostname, port, from and to default values from the system properties file.

- send_mail:
    do:
      mail.send_mail:
        - hostname: ${get_sp('tutorials.properties.hostname')}
        - port: ${get_sp('tutorials.properties.port')}
        - from: ${get_sp('tutorials.properties.system_address')}
        - to: ${get_sp('tutorials.properties.hr_address')}
        - subject: "${'New Hire: ' + first_name + ' ' + last_name}"
        - body: >
            ${'Created address: ' + address + ' for: ' + first_name + ' ' + last_name + '<br>' +
            'Missing items: ' + missing + ' Cost of ordered items: ' + total_cost + '<br>' +
            'Temporary password: ' + password}
    navigate:
      - FAILURE: FAILURE
      - SUCCESS: SUCCESS

For more information, see get_sp() in the DSL Reference.

Run It

We can save the files and run the flow to see that the values are being taken from the system properties file we specify. If we want to swap out the values with another set, all we have to do is point to a different system properties file.

run --f <folder path>/tutorials/hiring/new_hire.sl --cp <folder path>/tutorials,<content folder path>/base --i first_name=john,last_name=doe --spf <folder path>/tutorials/properties/bcompany.prop.sl

For more information on running with a system properties file, see Run with System Properties in the CLI documentation.

Download the Code

Lesson 14 - Complete code

Up Next

In the next lesson we’ll see how to use 3rd Python packages in your operation’s actions.

New Code - Complete

new_hire.sl

namespace: tutorials.hiring

imports:
  base: tutorials.base
  mail: io.cloudslang.base.mail

flow:
  name: new_hire

  inputs:
    - first_name
    - middle_name:
        required: false
    - last_name
    - all_missing:
        default: ""
        required: false
        private: true
    - total_cost:
        default: '0'
        private: true
    - order_map:
        default: '{"laptop": 1000, "docking station": 200, "monitor": 500, "phone": 100}'

  workflow:
    - print_start:
        do:
          base.print:
            - text: "Starting new hire process"
        navigate:
          - SUCCESS: create_email_address

    - create_email_address:
        loop:
          for: attempt in range(1,5)
          do:
            create_user_email:
              - first_name
              - middle_name
              - last_name
              - attempt: ${str(attempt)}
          publish:
            - address
            - password
          break:
            - CREATED
            - FAILURE
        navigate:
          - CREATED: get_equipment
          - UNAVAILABLE: print_fail
          - FAILURE: print_fail

    - get_equipment:
        loop:
          for: item, price in eval(order_map)
          do:
            order:
              - item
              - price: ${str(price)}
              - missing: ${all_missing}
              - cost: ${total_cost}
          publish:
            - all_missing: ${missing + not_ordered}
            - total_cost: ${str(int(cost) + int(spent))}
          break: []
        navigate:
          - AVAILABLE: check_min_reqs
          - UNAVAILABLE: check_min_reqs

    - check_min_reqs:
        do:
          base.contains:
            - container: ${all_missing}
            - sub: 'laptop'
        navigate:
          - DOES_NOT_CONTAIN: print_finish
          - CONTAINS: print_warning

    - print_warning:
        do:
          base.print:
            - text: >
                ${first_name + ' ' + last_name +
                ' did not receive all the required equipment'}
        navigate:
          - SUCCESS: print_finish

    - print_finish:
        do:
          base.print:
            - text: >
                ${'Created address: ' + address + ' for: ' + first_name + ' ' + last_name + '\n' +
                'Missing items: ' + all_missing + ' Cost of ordered items: ' + total_cost}
        navigate:
          - SUCCESS: send_mail

    - send_mail:
        do:
          mail.send_mail:
            - hostname: ${get_sp('tutorials.properties.hostname')}
            - port: ${get_sp('tutorials.properties.port')}
            - from: ${get_sp('tutorials.properties.system_address')}
            - to: ${get_sp('tutorials.properties.hr_address')}
            - subject: "${'New Hire: ' + first_name + ' ' + last_name}"
            - body: >
                ${'Created address: ' + address + ' for: ' + first_name + ' ' + last_name + '<br>' +
                'Missing items: ' + all_missing + ' Cost of ordered items:' + total_cost + '<br>' +
                'Temporary password: ' + password}
        navigate:
          - FAILURE: FAILURE
          - SUCCESS: SUCCESS

    - on_failure:
      - print_fail:
          do:
            base.print:
              - text: "${'Failed to create address for: ' + first_name + ' ' + last_name}"

generate_user_email.sl

namespace: tutorials.hiring

operation:
  name: generate_user_email

  inputs:
    - first_name
    - middle_name:
        required: false
        default: ""
    - last_name
    - domain:
        default: ${get_sp('tutorials.properties.domain', 'acompany.com')}
        private: true
    - attempt

  python_action:
    script: |
      attempt = int(attempt)
      if attempt == 1:
        address = first_name[0:1] + '.' + last_name + '@' + domain
      elif attempt == 2:
        address = first_name + '.' + last_name[0:1] + '@' + domain
      elif attempt == 3 and middle_name != '':
        address = first_name + '.' + middle_name[0:1] + '.' + last_name + '@' + domain
      else:
        address = ''
      # print address

  outputs:
    - email_address: ${address}

  results:
    - FAILURE: ${address == ''}
    - SUCCESS

bcompany.prop.sl

namespace: tutorials.properties

properties:
  - domain: bcompany.com
  - hostname: <host>
  - port: '25'
  - system_address: <test@test.com>
  - hr_address: <test@test.com>

Note

You need to substitute the values in angle brackets (<>) to work for your email host.

Lesson 15 - 3rd Party Python Packages

Goal

In this lesson we’ll learn how to import 3rd party Python packages to use in an operation’s python_action.

Get Started

In this lesson we’ll be installing a 3rd party Python package. In order to do so you’ll need to have Python and pip installed on your machine. You can download Python (version 3.8.7) from here which includes pip by default. If you already have Python but don’t have pip installed on your machine, see the pip documentation for installation instructions.

We’ll also need to add a requirements.txt file to a python-lib folder which is at the same level as the bin folder that the CLI executable resides in. If you downloaded a pre-built CLI the requirements.txt file is already there and we will be appending to its contents.

The folder structure where the CLI executable is should look something like this (other folders omitted for simplicity):

• cslang-cli
  • bin
    • cslang
    • cslang.bat
  • content
    • CloudSlang ready-made content
  • lib
    • includes all the Java .jar files for the CLI
  • python-lib
    • requirements.txt

And finally, we’ll need a new file, fancy_text.sl in the tutorials/hiring folder, to house a new operation.

Requirements

In the requirements.txt file we’ll list all the Python packages we need for our project. In our case we’ll add a package that will allow us to create large lettered strings using ordinary screen characters. The package is called pyfiglet. A quick search on PyPI tells us that the current version (at the time this tutorial was written) is 0.7.2, so we’ll use that one. We also need to install setuptools since pyfiglet depends on it. Each package we need takes up one line in our requirements.txt file.

setuptools
pyfiglet == 0.7.2

Installing

Now we need to use pip to download and install our packages.

To do so, run the following command from the python-lib directory:

pip install -r requirements.txt -t .

Note

If your machine is behind a proxy you’ll need to specify the proxy using pip’s --proxy flag.

If everything has gone well, you should now see the pyfiglet package’s files in the python-lib folder along with the setuptools files.

Operation

Next, let’s write an operation that will let us turn normal text into something fancy using pyfiglet. All we need to do is import pyfiglet as we would normally do in Python and use it. We also have to do a little bit of work to turn the regular string we get from calling renderText into something that will look right in our HTML email.

namespace: tutorials.hiring

operation:
  name: fancy_text

  inputs:
    - text

  python_action:
    script: |
      from pyfiglet import Figlet
      f = Figlet(font='slant')
      fancy = '<pre>' + f.renderText(text).replace('\n','<br>').replace(' ', '&nbsp') + '</pre>'

  outputs:
    - fancy

Note

CloudSlang uses the Jython implementation of Python 2.7. For information on Jython’s limitations, see the Jython FAQ.

Step

Now we can create a step in the new_hire flow to send some text to the fancy_text operation and publish the output so we can use it in our email. We’ll put the new step between print_finish and send_mail.

- fancy_name:
    do:
      fancy_text:
        - text: ${first_name + ' ' + last_name}
    publish:
      - fancy_text: ${fancy}
    navigate:
      - SUCCESS: send_mail

Use It

Finally, we need to change the body of the email to include our new fancy text.

- send_mail:
    do:
      mail.send_mail:
        - hostname: ${get_sp('tutorials.properties.hostname')}
        - port: ${get_sp('tutorials.properties.port')}
        - from: ${get_sp('tutorials.properties.system_address')}
        - to: ${get_sp('tutorials.properties.hr_address')}
        - subject: "${'New Hire: ' + first_name + ' ' + last_name}"
        - body: >
            ${fancy_text + '<br>' +
            'Created address: ' + address + ' for: ' + first_name + ' ' + last_name + '<br>' +
            'Missing items: ' + all_missing + ' Cost of ordered items: ' + total_cost + '<br>' +
            'Temporary password: ' + password}
    navigate:
      - FAILURE: FAILURE
      - SUCCESS: SUCCESS

Run It

We can save the files and run the flow. When the email is sent it should include the new fancy text we added to it.

run --f <folder path>/tutorials/hiring/new_hire.sl --cp <folder path>/tutorials,<content folder path>/base --i first_name=john,last_name=doe --spf <folder path>/tutorials/properties/bcompany.prop.sl

Download the Code

Lesson 15 - Complete code

Up Next

In the next lesson we’ll see how to use a parallel loop.

New Code - Complete

new_hire.sl

namespace: tutorials.hiring

imports:
  base: tutorials.base
  mail: io.cloudslang.base.mail

flow:
  name: new_hire

  inputs:
    - first_name
    - middle_name:
        required: false
    - last_name
    - all_missing:
        default: ""
        required: false
        private: true
    - total_cost:
        default: '0'
        private: true
    - order_map:
        default: '{"laptop": 1000, "docking station": 200, "monitor": 500, "phone": 100}'

  workflow:
    - print_start:
        do:
          base.print:
            - text: "Starting new hire process"
        navigate:
          - SUCCESS: create_email_address

    - create_email_address:
        loop:
          for: attempt in range(1,5)
          do:
            create_user_email:
              - first_name
              - middle_name
              - last_name
              - attempt: ${str(attempt)}
          publish:
            - address
            - password
          break:
            - CREATED
            - FAILURE
        navigate:
          - CREATED: get_equipment
          - UNAVAILABLE: print_fail
          - FAILURE: print_fail

    - get_equipment:
        loop:
          for: item, price in eval(order_map)
          do:
            order:
              - item
              - price: ${str(price)}
              - missing: ${all_missing}
              - cost: ${total_cost}
          publish:
            - all_missing: ${missing + not_ordered}
            - total_cost: ${str(int(cost) + int(spent))}
          break: []
        navigate:
          - AVAILABLE: check_min_reqs
          - UNAVAILABLE: check_min_reqs

    - check_min_reqs:
        do:
          base.contains:
            - container: ${all_missing}
            - sub: 'laptop'
        navigate:
          - DOES_NOT_CONTAIN: print_finish
          - CONTAINS: print_warning

    - print_warning:
        do:
          base.print:
            - text: >
                ${first_name + ' ' + last_name +
                ' did not receive all the required equipment'}
        navigate:
          - SUCCESS: print_finish

    - print_finish:
        do:
          base.print:
            - text: >
                ${'Created address: ' + address + ' for: ' + first_name + ' ' + last_name + '\n' +
                'Missing items: ' + all_missing + ' Cost of ordered items: ' + total_cost}
        navigate:
          - SUCCESS: fancy_name

    - fancy_name:
        do:
          fancy_text:
            - text: ${first_name + ' ' + last_name}
        publish:
          - fancy_text: ${fancy}
        navigate:
          - SUCCESS: send_mail

    - send_mail:
        do:
          mail.send_mail:
            - hostname: ${get_sp('tutorials.properties.hostname')}
            - port: ${get_sp('tutorials.properties.port')}
            - from: ${get_sp('tutorials.properties.system_address')}
            - to: ${get_sp('tutorials.properties.hr_address')}
            - subject: "${'New Hire: ' + first_name + ' ' + last_name}"
            - body: >
                ${fancy_text + '<br>' +
                'Created address: ' + address + ' for: ' + first_name + ' ' + last_name + '<br>' +
                'Missing items: ' + all_missing + ' Cost of ordered items: ' + total_cost + '<br>' +
                'Temporary password: ' + password}
        navigate:
          - FAILURE: FAILURE
          - SUCCESS: SUCCESS

    - on_failure:
      - print_fail:
          do:
            base.print:
              - text: "${'Failed to create address for: ' + first_name + ' ' + last_name}"

fancy_text.sl

namespace: tutorials.hiring

operation:
  name: fancy_text

  inputs:
    - text

  python_action:
    script: |
      from pyfiglet import Figlet
      f = Figlet(font='slant')
      fancy = '<pre>' + f.renderText(text).replace('\n','<br>').replace(' ', '&nbsp') + '</pre>'

  outputs:
    - fancy

Lesson 16 - Parallel Loop

Goal

In this lesson we’ll learn how to loop in parallel. When looping in parallel, a new branch is created for each value in a list and the action associated with the step is run for each branch in parallel.

Get Started

We’ll be creating a new flow that will call the new_hire flow we’ve built in previous lessons as a subflow. Let’s begin by creating a new file named hire_all.sl in the tutorials/hiring folder for our new flow. Also, we’ll need the new_hire.sl because we’re going to make some minor changes to that as well. And finally, we’ll pass our flow inputs using a file, so let’s create a tutorials/inputs folder and add a hires.yaml file.

Outputs

Since we’ll be using the new_hire flow as a subflow, it will be helpful if we add some flow outputs for a parent flow to make use of. We’ll simply add an outputs section at the bottom of our flow to output a bit of information. This outputs section is quite a distance from the flow key, so be extra careful to place it at the proper indentation.

outputs:
  - address
  - final_cost: ${total_cost}

Parent Flow

Our new hire_all flow is going to take in a list of names of people being hired and will call the new_hire flow for each one of them. It will be looping in parallel, so all the new_hire flows will be running simultaneously.

In hire_all.sl we can start off as usual by declaring a namespace, specifying the imports and taking in the inputs, which in our case is a list of names.

namespace: tutorials.hiring

imports:
  base: tutorials.base

flow:
  name: hire_all

  inputs:
    - names_list

  workflow:

Loop Syntax

A parallel loop looks pretty similar to a normal for loop, but with a few key differences.

Let’s create a new step named process_all in which we’ll do our looping. Each branch of the loop will call the new_hire flow.

- process_all:
    parallel_loop:
      for: name in eval(names_list)
      do:
        new_hire:
          - first_name: ${name["first"]}
          - middle_name: ${name.get("middle","")}
          - last_name: ${name["last"]}

As you can see, so far it is almost identical to a regular for loop, except the loop key has been replaced by parallel_loop.

The names_list input will be a list of dictionaries containing name information with the keys first, middle and last. For each name in names_list the new_hire flow will be called and passed the corresponding name values. The various branches running the new_hire flow will run in parallel and the rest of the flow will continue only after all the branches have completed.

For more information, see parallel_loop in the DSL reference.

Publish

Next we perform aggregation in the publish section in a similar manner to what we do in a normal for loop (as we did in lesson 11 - Loop Aggregation). Publish occurs only after all branches have completed.

In most cases the publish will make use of the branches_context list. This is a list that is populated with all of the outputs from all of the branches. For example, in our case, branches_context[0] will contain keys address and final_cost, corresponding to the values output by the first branch to complete. Similarly, branches_context[1] will contain the keys address and final_cost mapped to the values output by the second branch to complete.

There is no way to predict the order in which branches will complete, so the branches_context is rarely accessed using a particular index. Instead, Python expressions are used to extract the desired aggregations.

- process_all:
    parallel_loop:
      for: name in eval(names_list)
      do:
        new_hire:
          - first_name: ${name["first"]}
          - middle_name: ${name.get("middle","")}
          - last_name: ${name["last"]}
    publish:
      - email_list: "${', '.join(filter(lambda x : x != '', map(lambda x : str(x['address']), branches_context)))}"
      - cost: "${str(sum(map(lambda x : x['final_cost'], branches_context)))}"

In our case we use the map(), filter() and sum() Python functions to create a list of all the email addresses that were created and a sum of all the equipment costs.

Let’s look a bit closer at one of the publish aggregations to better understand what’s going on. Each time a branch of the parallel loop is finished running the new_hire subflow it publishes a final_cost value. Each of those individual final_cost values gets added to the branches_context list at index n, where n indicates the order the branches finish in, under the final_cost key. So, if we were to loop through the branches_context we would find at branches_context[n][final_cost] the final_cost value that was published by the nth new_hire subflow to finish running. Instead of looping through the branches_context, we use a Python lambda expression in conjunction with the map function to extract just the values of the final_cost from each branches_context[n][final_cost] to a new list. Finally, we use the Python sum function to add up all the extracted values in our new list and publish that value as cost.

For more information, see publish and branches_context in the DSL reference.

For more information on the Python constructs used here, see lambda, map and sum in the Python documentation.

Navigate

Navigation also works a bit differently in a parallel loop. If any of the branches return a result of FAILURE the flow will follow the navigation path of FAILURE. Otherwise, the flow will follow the SUCCESS navigation path.

Here we’ll add navigation logic that mimics the default behavior. If any one of our branches returns a result of FAILURE because an email address was not generated or there was a problem sending an email, then the flow will navigate to the print_failure step. Otherwise, it will navigate to the print_success step.

- process_all:
    parallel_loop:
      for: name in eval(names_list)
      do:
        new_hire:
          - first_name: ${name["first"]}
          - middle_name: ${name.get("middle","")}
          - last_name: ${name["last"]}
    publish:
      - email_list: "${', '.join(filter(lambda x : x != '', map(lambda x : str(x['address']), branches_context)))}"
      - cost: "${str(sum(map(lambda x : x['final_cost'], branches_context)))}"
    navigate:
      - SUCCESS: print_success
      - FAILURE: print_failure

Input File

We’ll use an input file to send the flow our list of names. An input file is very similar to a system properties file. It is written in plain YAML and therefore ends with the .yaml extension.

Here is the contents of our hires.yaml input file that we created in the tutorials/inputs folder.

names_list: '[{"first": "joe", "middle": "p", "last": "bloggs"}, {"first": "jane", "last": "doe"}, {"first": "juan", "last": "perez"}]'

The file contains a names_list key that maps to a stringified version of a list of name information. Remember, all inputs must be strings, so here we must use a string as well.

For more information, see Using an Inputs File in the CLI documentation.

Steps

Finally, we have to add the steps we referred to in the navigation section. We can put them right after the process_all step.

- print_success:
    do:
      base.print:
        - text: >
            ${"All addresses were created successfully.\nEmail addresses created: "
            + email_list + "\nTotal cost: " + cost}
    navigate:
      - SUCCESS: SUCCESS

- on_failure:
    - print_failure:
        do:
          base.print:
            - text: >
                ${"Some addresses were not created or there is an email issue.\nEmail addresses created: "
                + email_list + "\nTotal cost: " + cost}

Run It

We can save the files and run the flow. It’s a bit harder to track what has happened now because there are quite a few things happening at once. On careful inspection you will see that each step in the new_hire flow, and in each of its subflows, is run for each of the people in the names_list input.

run --f <folder path>/tutorials/hiring/hire_all.sl --cp <folder path>/tutorials,<content folder path>/base --if <folder path>/tutorials/inputs/hires.yaml --spf <folder path>/tutorials/properties/bcompany.prop.sl

Download the Code

Lesson 16 - Complete code

New Code - Complete

new_hire.sl

namespace: tutorials.hiring

imports:
  base: tutorials.base
  mail: io.cloudslang.base.mail

flow:
  name: new_hire

  inputs:
    - first_name
    - middle_name:
        required: false
    - last_name
    - all_missing:
        default: ""
        required: false
        private: true
    - total_cost:
        default: '0'
        private: true
    - order_map:
        default: '{"laptop": 1000, "docking station": 200, "monitor": 500, "phone": 100}'

  workflow:
    - print_start:
        do:
          base.print:
            - text: "Starting new hire process"
        navigate:
          - SUCCESS: create_email_address
    - create_email_address:
        loop:
          for: attempt in range(1,5)
          do:
            create_user_email:
              - first_name
              - middle_name
              - last_name
              - attempt: ${str(attempt)}
          publish:
            - address
            - password
          break:
            - CREATED
            - FAILURE
        navigate:
          - CREATED: get_equipment
          - UNAVAILABLE: print_fail
          - FAILURE: print_fail
    - get_equipment:
        loop:
          for: item, price in eval(order_map)
          do:
            order:
              - item
              - price: ${str(price)}
              - missing: ${all_missing}
              - cost: ${total_cost}
          publish:
            - all_missing: ${missing + not_ordered}
            - total_cost: ${str(int(cost) + int(spent))}
          break: []
        navigate:
          - AVAILABLE: check_min_reqs
          - UNAVAILABLE: check_min_reqs
    - check_min_reqs:
        do:
          base.contains:
            - container: ${all_missing}
            - sub: 'laptop'
        navigate:
          - DOES_NOT_CONTAIN: print_finish
          - CONTAINS: print_warning
    - print_warning:
        do:
          base.print:
            - text: >
                ${first_name + ' ' + last_name +
                ' did not receive all the required equipment\n'}
        navigate:
          - SUCCESS: print_finish
    - print_finish:
        do:
          base.print:
            - text: >
                ${'Created address: ' + address + ' for: ' + first_name + ' ' + last_name + '\n' +
                'Missing items: ' + all_missing + ' Cost of ordered items: ' + total_cost}
        navigate:
          - SUCCESS: fancy_name
    - fancy_name:
        do:
          fancy_text:
            - text: ${first_name + ' ' + last_name}
        publish:
          - fancy_text: ${fancy}
        navigate:
          - SUCCESS: send_mail
    - send_mail:
        do:
          mail.send_mail:
            - hostname: ${get_sp('tutorials.properties.hostname')}
            - port: ${get_sp('tutorials.properties.port')}
            - from: ${get_sp('tutorials.properties.system_address')}
            - to: ${get_sp('tutorials.properties.hr_address')}
            - subject: "${'New Hire: ' + first_name + ' ' + last_name}"
            - body: >
                ${fancy_text + '<br>' +
                'Created address: ' + address + ' for: ' + first_name + ' ' + last_name + '<br>' +
                'Missing items: ' + all_missing + ' Cost of ordered items: ' + total_cost + '<br>' +
                'Temporary password: ' + password}
        navigate:
          - FAILURE: FAILURE
          - SUCCESS: SUCCESS
    - on_failure:
      - print_fail:
          do:
            base.print:
              - text: "${'Failed to create address for: ' + first_name + ' ' + last_name}"

  outputs:
    - address
    - final_cost: ${total_cost}

hire_all.sl

namespace: tutorials.hiring

imports:
  base: tutorials.base

flow:
  name: hire_all

  inputs:
    - names_list

  workflow:
    - process_all:
        parallel_loop:
          for: name in eval(names_list)
          do:
            new_hire:
              - first_name: ${name["first"]}
              - middle_name: ${name.get("middle","")}
              - last_name: ${name["last"]}
        publish:
          - email_list: "${', '.join(filter(lambda x : x != '', map(lambda x : str(x['address']), branches_context)))}"
          - cost: "${str(sum(map(lambda x : int(x['final_cost']), branches_context)))}"
        navigate:
          - SUCCESS: print_success
          - FAILURE: print_failure

    - print_success:
        do:
          base.print:
            - text: >
                ${"All addresses were created successfully.\nEmail addresses created: "
                + email_list + "\nTotal cost: " + cost}
        navigate:
          - SUCCESS: SUCCESS

    - on_failure:
        - print_failure:
            do:
              base.print:
                - text: >
                    ${"Some addresses were not created or there is an email issue.\nEmail addresses created: "
                    + email_list + "\nTotal cost: " + cost}

hires.yaml

names_list: '[{"first": "joe", "middle": "p", "last": "bloggs"}, {"first": "jane", "last": "doe"}, {"first": "juan", "last": "perez"}]'

Troubleshooting

Contents:

Troubleshooting Guide

How to fix the “Failed to download Maven resources” message?

---

How to fix the “Failed to download Maven resources” message?

Note: If you are running cslang-cli, close it (otherwise the changes in settings.xml will not be applied).

1. In the cslang-cli or cslang-builder folder, go to /maven/conf/ and open settings.xml.
2. Change the proxies tags from:

<proxies>
    <proxy>
        <id>http-proxy</id>
        <active>false</active> <!--set to active if behind a proxy-->
        <protocol>http</protocol>
        <host>proxy.example.com</host>
        <port>8080</port>
        <username/>
        <password/>
        <nonProxyHosts>www.google.com|*.example.com</nonProxyHosts>
    </proxy>
    <proxy>
        <id>https-proxy</id>
        <active>false</active> <!--set to active if behind a proxy-->
        <protocol>https</protocol>
        <host>proxy.example.com</host>
        <port>8080</port>
        <username/>
        <password/>
        <nonProxyHosts>www.google.com|*.example.com</nonProxyHosts>
    </proxy>
</proxies>

to

<proxies>
    <proxy>
        <id>http-proxy</id>
        <active>true</active> <!--change this value to 'active'-->
        <protocol>http</protocol>
        <host>add.your.proxy**</host> <!--change this value to your proxy host-->
        <port>8080</port> <!--change this value to your proxy port-->
        <username/> <!--change this value to your proxy username (only if required)-->
        <password/> <!--change this value to your proxy password (only if required)-->
        <nonProxyHosts>www.google.com|*.example.com</nonProxyHosts> <!--add any hosts that should be ignored-->
    </proxy>
    <proxy>
        <id>https-proxy</id>
        <active>active</active> <!--set to active if behind a proxy-->
        <protocol>https</protocol>
        <host>add.your.proxy**</host> <!--change this value to your proxy host-->
        <port>8080</port> <!--change this value to your proxy port-->
        <username/> <!--change this value to your proxy username (only if required)-->
        <password/> <!--change this value to your proxy password (only if required)-->
        <nonProxyHosts>www.google.com|*.example.com</nonProxyHosts> <!--add any hosts that should be ignored-->
    </proxy>
</proxies>

3. Check the folder `/maven/repo/io/cloudslang/content/cs-package` (cs-package will be the dependency that failed to download, delete that folder, as it contains invalid contents and it will not be replaced if you retry the execution.

4. Once all the changes have been applied, open the CLI again and run the operation/flow, that has previously failed. You should see the following message Completed downloading maven resources for cs-example-0.0.1 and the execution will run normally.

Note

In case you still get the same error, there might be a problem with your proxy configuration (double check, just in case).