Red Hat Ansible Automation Platform (AAP) or Ansible Tower has become de facto Automation Tool for deploying the Automation projects to automate the System Administration tasks such as configuration management, application deployment, provisioning, orchestration, and continuous delivery.
Although AAP and Ansible Tower are designed to run on large scale Infrastructure environment but sometime running complex Ansible playbook tasks via AAP or Ansible Tower instance on such large scale infrastructure environment or running those playbooks which contains CPU or memory intensive tasks could toll on these Automation tools performance. When AAP or Ansible Tower start lagging in performance you can see large number of Ansible playbook jobs are in waiting queue or jobs are taking lot of time in completion, to counter such issues I have shared some simple steps through which System Admin can optimize the performance of AAP or Ansible Tower.
Although there are numbers of method either from ansible playbook coding or from AAP or Ansible Tower UI itself you can optimize playbook jobs execution performance but in this article, I am only covering some of below methods which can significantly improve these Automation tools performance:
- Using of Ansible job Fork and Slicing
- Enabling of pipeling and timeout options
- Enabling Fact Storage or Fact cache in AAP or Ansible Tower job templates
Using of Fork and Slicing in in AAP or Ansible Tower job templates
Ansible Fork: Ansible Fork is equal to one process of Ansible Controller instance which utilizes its CPU and memory. By default, Ansible use 5 fork value which means each Ansible task in a playbook can be run on 5 endpoints at a time but say in case you want to run the ansible playbook on 100 endpoints at a time for reducing the runtime of job you can increase the fork value to 100.
Now there are two ways to increase the fork value in Ansible. One method is if you are developing your own ansible playbook, you can configure ansible.cfg file and define the fork value there as an example below:
Another method is to set fork value in AAP or Ansible Tower job templates as well which is easiest way, and you need not to change your existing Ansible configuration in your code repository like GitHub repo.
Example of Fork value set in AAP or Ansible Tower job templates:
Ansible Job Slicing: Contrary to fork method Ansible job slicing allows you to run Ansible playbook job in multiple Ansible controller instances simultaneously. By using slicing method AAP or Ansible Tower ensure that no single Ansible Controller instance should be overloaded as it distributes the jobs between multiple Ansible Controller instances and hence it speeds up the job execution as well. Ansible playbook job which is sliced as per the value sets in AAP or Ansible Tower job template it creates a numbers of job template nodes which is invoked by Ansible Workflow template and each job template node is equal to Ansible Controller instance on which job runs, as shown in below example:
Below is the example of Ansible Job template where you can define job slicing value and also you can see the instance groups name as well on which jobs can be distributed as per the value set in job slicing:
Enabling of pipeling and timeout options
Pipeling: As Ansible Controller instance use SSH for connecting with endpoints for executing several operations in background like copying the files, scripts and running the commands it can also increase the job runtime in AAP or Ansible Tower therefore enabling pipeline reduces this overload. Pipeline or SSH Pipelining is a new method of speeding up the SSH connection with endpoints by reducing the multiple SSH connections for executing the task. If you want to enable SSH pipeline for your developed playbook you can enable it in ansible.cfg configuration file:
There is also an alternative way to enable SSH Pipeline in AAP and Ansible Tower itself which will be applicable on all added hosts or endpoints inventory. To enable SSH Pipeline in AAP or Ansible Tower Go to Resources > Inventories > Click on your account Inventory and under inventory variable add below variable:
ansible_pipelining: ‘true’
Below is an example of AAP and Ansible Tower inventory variable:
AAP:
Ansible Tower:
SSH Timeout: SSH Timeout variable allows you to keep the SSH connection active with endpoints as per the defined time it helps in reducing the other Ansible job wait time in AAP and Ansible Tower if any Ansible job which needs to run on busy endpoints you can also increase the timeout value through this variable:
AAP:
Ansible Tower:
Enabling Fact Storage or Fact cache in AAP or Ansible Tower job templates
Facts are endpoints system variables information (such as OS information, hostname, network information, hardware information etc.) which is sometime needed by ansible playbook for running the further tasks based on gathered facts. In AAP or Ansible Tower some Ansible playbooks need gather facts (gather_facts: True) variable to capture the endpoints system information by running the setup module hidden task in background but using gather_facts variable on busy endpoint can increase the ansible playbook job runtime hence for addressing this issue AAP and Ansible Tower allows you to enable fact storage or fact cache in ansible job templates so that next time whenever any Ansible playbook job need endpoints facts variable it need not to gather it again from endpoints in this place it can take the input from stored facts variable.
Below are the examples of enabling Fact Storage or Fact cache in AAP or Ansible Tower job templates:
AAP:
Ansible Tower:
Below is the example of endpoint stored or cached facts variables:
Wrap up!
In this article you have read that there are two ways to tune the values and configure the variables for boosting the speed of Ansible jobs or improving the performance of AAP and Ansible Tower but one thing you need to note that if you have defined the tunable value and variables in your Automation project ansible.cfg configuration file in your GitHub repo and also same values and variables are configured in AAP and Ansible Tower as well in this case AAP and Ansible Tower values and variables will be superseded.
He is a lazy System Administrator who likes to Automate his task through Shell scripts and Ansible playbooks. He has contributed his many Automations work in Kyndryl and also maintain his own internal company Github repo.
His hobby is researching on new technologies and sharing this through his blog but if he is not researching or writing blog then you will find him listening music while sipping cup of coffee and enjoying the nature view.
You can find him on Twitter, Facebook, Linkedin or send him an email at webmaster@cjcheema.com