By Anuj Tuli, CTO
Keyva announces the certification of their ServiceNow App for Red Hat Ansible Tower against the Orlando release (latest release) of ServiceNow. ServiceNow announced its release of Orlando on January 23rd, 2020, which is the newest version in the long line of software updates since the company's creation.
Customers can now upgrade their ServiceNow App for Ansible Tower from previous ServiceNow Releases – London, Madrid, New York – to Orlando release seamlessly.
You can find out more about the App, and view all the ServiceNow releases it is certified against, on the ServiceNow store here: http://bit.ly/2W5tYHv
[post_title] => ServiceNow App for Red Hat Ansible Tower "NOW Certified" against Orlando release [post_excerpt] => [post_status] => publish [comment_status] => closed [ping_status] => closed [post_password] => [post_name] => servicenow-app-for-red-hat-ansible-tower-now-certified-against-orlando-release [to_ping] => [pinged] => [post_modified] => 2020-03-24 15:27:07 [post_modified_gmt] => 2020-03-24 15:27:07 [post_content_filtered] => [post_parent] => 0 [guid] => https://keyvatech.com/?p=2278 [menu_order] => 7 [post_type] => post [post_mime_type] => [comment_count] => 0 [filter] => raw ) [1] => WP_Post Object ( [ID] => 2286 [post_author] => 7 [post_date] => 2020-03-19 12:15:08 [post_date_gmt] => 2020-03-19 12:15:08 [post_content] => By Jaime Gmach, CEO As the COVID-19 situation continues to unfold, we wanted to give you a brief update about the steps we have taken to protect our employees, clients and partners. We have updated our policies and practices to provide all Evolving Solutions and Keyva constituents with the safest environment in which to perform their job in the current work climate. Our business continuity plans have been activated to ensure service continuity for our clients during times of crisis and we remain on standby to support our clients. Evolving Solutions and Keyva have assembled a cross-functional team to monitor the rapidly changing situation and address questions. This group is reviewing and adhering to our clients' updated policies and visitor restrictions. We have been in constant contact with our supply chain partners to ensure minimal or no disruption where possible. Additional safety efforts our team is taking include:By Brad Johnson, Lead DevOps Engineer
When developing automation you may be faced with challenges that are simply too complicated or tedious to accomplish with Ansible alone. There may even be cases where you are told that “it can’t be automated”. However, when you combine the abilities of Ansible and custom python using the pexpect module, then you are able to automate practically anything you can do on the command line. In this post we will discuss the basics of creating a custom Ansible module in python.
Here are a few examples of cases where you might need to create a custom module:
For the purposes of this article we will focus on the first case. When writing a traditional linux shell or bash script it simply isn’t possibly to continue your script when a command you run drops you into a new shell or new interactive interface. If these tools also provided a non-interactive mode or config/script input we would not need to do this. To overcome this situation we need to use python with pexpect. The native Ansible “expect” module provides a simple interface to this functionality and should be evaluated before writing a custom module. However, when you need more complex interactions, want specific data returned or want to provide a re-usable and simpler interface to an underlying program for to others to consume, then custom development if warranted.
In this guide I will talk about the requirements and steps needed to create your own library module. The source code with our example is located here and contains notes in the code as well. The pexpect code is intentionally complex to demonstrate some use cases.
#!/usr/bin/env python
import os
import getpass
DOCUMENTATION = '''
---
module: my_module
short_description: This is a custom module using pexpect to run commands in myscript.sh
description:
- "This module runs commands inside a script in a shell. When run without commands it returns current settings only."
options:
commands:
description:
- The commands to run inside myscript in order
required: false
options:
description:
- options to pass the script
required: false
timeout:
description:
- Timeout for finding the success string or running the program
required: false
default: 300
password:
description:
- Password needed to run myscript
required: true
author:
- Brad Johnson - Keyva
'''
EXAMPLES = '''
- name: "Run myscript to set up myprogram"
my_module:
options: "-o myoption"
password: "{{ myscript_password }}"
commands:
- "set minheap 1024m"
- "set maxheap 5120m"
- "set port 7000"
- "set webport 80"
timeout: 300
'''
RETURN = '''
current_settings: String containing current settings after last command was run and settings saved
type: str
returned: On success
logfile: String containing logfile location on the remote host from our script
type: str
returned: On success
'''
def main():
# This is the import required to make this code an Ansible module
from ansible.module_utils.basic import AnsibleModule
# This instantiates the module class and provides Ansible with
# input argument information, it also enforces input types
module = AnsibleModule(
argument_spec=dict(
commands=dict(required=False, type='list', default=[]),
options=dict(required=False, type='str', default=""),
password=dict(required=True, type='str', no_log=True),
timeout=dict(required=False, type='int', default='300')
)
)
commands = module.params['commands']
options = module.params['options']
password = module.params['password']
timeout = module.params['timeout']
try:
# Importing the modules here allows us to catch them not being installed on remote hosts
# and pass back a failure via ansible instead of a stack trace.
import pexpect
except ImportError:
module.fail_json(msg="You must have the pexpect python module installed to use this Ansible module.")
try:
# Run our pexpect function
current_settings, changed, logfile = run_pexpect(commands, options, password, timeout)
# Exit on success and pass back objects to ansible, which are available as registered vars
module.exit_json(changed=changed, current_settings=current_settings, logfile=logfile)
# Use python exception handling to keep all our failure handling in our main function
except pexpect.TIMEOUT as err:
module.fail_json(msg="pexpect.TIMEOUT: Unexpected timeout waiting for prompt or command: {0}".format(err))
except pexpect.EOF as err:
module.fail_json(msg="pexpect.EOF: Unexpected program termination: {0}".format(err))
except pexpect.exceptions.ExceptionPexpect as err:
# This catches any pexpect exceptions that are not EOF or TIMEOUT
# This is the base exception class
module.fail_json(msg="pexpect.exceptions.{0}: {1}".format(type(err).__name__, err))
except RuntimeError as err:
module.fail_json(msg="{0}".format(err))
def run_pexpect(commands, options, password, timeout=300):
import pexpect
changed = True
script_path = '/path/to/myscript.sh'
if not os.path.exists(script_path):
raise RuntimeError("Error: the script '{0}' does not exist!".format(script_path))
if script_path == '/path/to/myscript.sh':
raise RuntimeError("This module example is based on a hypothetical command line interactive program and "
"can not run. Please use this as a basis for your own development and testing.")
# Set prompt to expect with username embedded in it
# YOU MAY NEED TO CHANGE THIS PROMPT FOR YOUR SYSTEM
# My default RHEL prompt regex
prompt = r'\[{0}\@.+?\]\$'.format(getpass.getuser())
output = ""
child = pexpect.spawn('/bin/bash')
try:
# Look for initial bash prompt
child.expect(prompt)
# Start our program
child.sendline("{0} {1}".format(script_path, options))
# look for our scripts logfile prompt
# Example text seen in output: 'Logfile: /path/to/mylog.log'
child.expect(r'Logfile\:.+?/.+?\.log')
# Note that child.after contains the text of the matching regex
logfile = child.after.split()[1]
# Look for password prompt
i = child.expect([r"Enter password\:", '>'])
if i == 0:
# Send password
child.sendline(password)
child.expect('>')
# Increase timeout for longer running interactions after quick initial ones
child.timeout = timeout
try:
# Look for program internal prompt or new config dialog
i = child.expect([r'Initialize New Config\?', '>'])
# pexpect will return the index of the regex it found first
if i == 0:
# Answer 'y' to initialize new config prompt
child.sendline('y')
child.expect('>')
# If any commands were passed in loop over them and run them one by one.
for command in commands:
child.sendline(command)
i = child.expect([r'ERROR.+?does not exist', r'ERROR.+?$', '>'])
if i == 0:
# Attempt to intelligently add items that may have multiple instances and are missing
# e.g. "socket.2" may need "add socket" run before it.
# Try to allow the user just to use the set command and run add as needed
try:
new_item = child.after.split('"')[1].split('.')[0]
except IndexError:
raise RuntimeError("ERROR: unable to automatically add new item in myscript,"
" file a bug\n {0}".format(child.after))
child.sendline('add {0}'.format(new_item))
i = child.expect([r'ERROR.+?$', '>'])
if i == 0:
raise RuntimeError("ERROR: unable to automatically add new item in myscript,"
" file a bug\n {0}".format(child.after.strip()))
# Retry the failed original command after the add
child.sendline(command)
i = child.expect([r'ERROR.+?$', '>'])
if i == 0:
raise RuntimeError("ERROR: unable to automatically add new item in myscript,"
" file a bug\n {0}".format(child.after.strip()))
elif i == 1:
raise RuntimeError("ERROR: unspecified error running a myscript command\n"
" {0}".format(child.after.strip()))
# Set timeout shorter for final commands
child.timeout = 15
# If we processed any commands run the save function last
if commands:
child.sendline('save')
# Using true loops with expect statements allow us to process multiple items in a block until
# some kind of done or exit condition is met where we then call a break.
while True:
i = child.expect([r'No changes made', r'ERROR.+?$', '>'])
if i == 0:
changed = False
elif i == 1:
raise RuntimeError("ERROR: unexpected error saving configuration\n"
" {0}".format(child.after.strip()))
elif i == 2:
break
# Always print out the config data from out script and return it to the user
child.sendline('print config')
child.expect('>')
# Note that child.before contains the output from the last expected item and this expect
current_settings = child.before.strip()
# Run the 'exit' command that is inside myscript
child.sendline('exit')
# Look for a linux prompt to see if we quit
child.expect(prompt)
except pexpect.TIMEOUT:
raise RuntimeError("ERROR: timed out waiting for a prompt in myscript")
# Get shell/bash return code of myscript
child.sendline("echo $?")
child.expect(prompt)
# process the output into a variable and remove any whitespace
exit_status = child.before.split('\r\n')[1].strip()
if exit_status != "0":
raise RuntimeError("ERROR: The command returned a non-zero exit code! '{0}'\n"
"Additional info:\n{1}".format(exit_status, output))
child.sendline('exit 0')
# run exit as many times as needed to exit the shell or subshells
# This might be useful if you ran a script that put you into a new shell where you then ran some other scripts
# This is also a good example of
while True:
i = child.expect([prompt, pexpect.EOF])
if i == 0:
child.sendline('exit 0')
elif i == 1:
break
finally:
# Always try to close the pexpect process
child.close()
return current_settings, changed, logfile
if __name__ == '__main__':
main()
In order to create a module you need to put your new “mymodule.py” file somewhere in the Ansible module library path, typically the “library” directory next to your playbook or library inside your role. It’s also important to note that Ansible library modules run on the target ansible host, so if you want to use the ansible “expect” module or make a custom module with pexpect in it then you will need to install the python pexpect module on the remote host before running module. (Note: the pexpect version provided in RHEL/CentOS repos is old and will not support the Ansible “expect” module, install via pip instead for the latest version.)
Information on the library path is located here:
https://docs.ansible.com/ansible/latest/dev_guide/developing_locally.html
Your example.py file needs to be a standard file with a python shebang header and also import the ansible module. Here is a bare minimum amount of code needed for an ansible module.
#!/usr/bin/env python from ansible.module_utils.basic import AnsibleModule module = AnsibleModule(argument_spec=dict(mysetting=dict(required=False, type='str'))) try: return_value = "mysetting value is: {0}".format(module.params['mysetting']) except: module.fail_json(msg="Unable to process input variable into string") module.exit_json(changed=True, my_output=return_value)
With this example you can see how variables are passed into and out of the module. This also includes a basic exception handle for dealing with errors and allowing ansible to deal with the failure. This exception clause is too broad for normal use as it will catch and hide all errors that could happen in the try block. When you create your module you should only except error types that you anticipate to avoid hiding stack traces of unexpected errors from your logs.
Now we can add in some custom pexpect processing code. This is again a very basic example. The example code linked in this blog post has a complicated and in-depth example. This function would then be added into our try-except block in the code above.
def run_pexpect(password): import pexpect child = pexpect.spawn('/path/to/myscript.sh') child.timeout = 60 child.expect(r"Enter password\:") child.sendline(password) child.expect('Thank you') child.sendline('exit') child.expect(pexpect.EOF) exit_dialog = child.before.strip() return exit_dialog
There are some important things to note here when dealing with pexpect and Ansible.
When creating custom modules I would encourage you to give thought to making the simplest, most maintainable and modular modules possible. It can be easy to create one module/script to rule them all, but the linux concept of having one tool to do one thing well will save you rewriting chunks of code that do the same thing and also help future maintainers of the automation you create.
https://docs.ansible.com/ansible/latest/modules/expect_module.html
https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_general.html
https://pexpect.readthedocs.io/en/stable/overview.html
If you have any questions about the steps documented here, would like more information on the custom development process, or have any feedback or requests, please let us know at info@keyvatech.com.
[post_title] => Build custom Red Hat Ansible modules: pexpect [post_excerpt] => [post_status] => publish [comment_status] => closed [ping_status] => closed [post_password] => [post_name] => build-custom-red-hat-ansible-modules-pexpect [to_ping] => [pinged] => [post_modified] => 2022-01-26 13:18:26 [post_modified_gmt] => 2022-01-26 13:18:26 [post_content_filtered] => [post_parent] => 0 [guid] => https://keyvatech.com/?p=2221 [menu_order] => 10 [post_type] => post [post_mime_type] => [comment_count] => 0 [filter] => raw ) [3] => WP_Post Object ( [ID] => 2149 [post_author] => 7 [post_date] => 2020-01-22 18:13:59 [post_date_gmt] => 2020-01-22 18:13:59 [post_content] =>Kong Enterprise provides you the ability to rate limit the traffic for various objects using the Rate Limiting Advanced Plugin. In the example below, we will rate limit a service fronted by Kong Enterprise.
We will use our existing Kong Enterprise on RHEL 7 environment. The installation process for this environment is documented here.
First lets make sure we have an existing service we can use. If your environment needs to have a service created, you can also check out our blog on how to do so here.
We will also be using the RBAC controls and the user we set up in our blog post. If you have not yet setup RBAC you can learn how to do so here.
1) Create a service that we can use for this example
Log in to the Kong portal at https://<kong_FQDN_or_IP>:8445 and navigate to your chosen Workspace -> Services -> New Service
Fill in the fields for Service Name, Host, Path, Port and other fields as necessary
You can also run the step of creating a Service via the command line in the format below:
curl -i -X POST --url http://<kong_FQDN_or_IP>:8001/services --data 'name=DemoService' --data 'url=myurl.com'
Check to make sure the Service was created successfully by navigating through the console
Or running the following command line:
curl -i -X GET --url "http://<kong_FQDN_or_IP>:8001/services" --header "Kong-Admin-Token: rbac_user_token_1"
2) Next we will add a route for this service
curl -i -X POST --url "http://<kong_FQDN_or_IP>:8001/services/DemoService/routes" --data "hosts[]=mydemoexample.com" --header "Kong-Admin-Token: rbac_user_token_1"
3) Use the rate limiting plugin with our defined service
curl -i -X POST --url "http://<kong_FQDN_or_IP>:8001/services/DemoService/plugins" --data "name=rate-limiting-advanced" --data "config.sync_rate=0" --data "config.window_size=60" --data "config.limit=2" --header "Kong-Admin-Token: rbac_user_token_1"
This configuration means that the DemoService service should not be allowed to process more than 2 requests per 60 seconds period.
4) Now we will test running more than 2 requests against the DemoService service.
After running the request below more than twice
curl -i -X GET --url "http://<kong_FQDN_or_IP>:8000/" --header "Host: mydemoexample.com" --header "Kong-Admin-Token: rbac_user_token_1"
We get the following message:
HTTP/1.1 429 Too Many Requests
By controlling the volume of requests to a specific service, and by adding RBAC controls in front of it, you can secure a quasi-firewall for east-west traffic against internal networking vulnerabilities.
If you have any questions or comments on the tutorial content above, or run in to specific errors not covered here, please feel free to reach out to info@keyvatech.com
[post_title] => Kong Enterprise - How to Setup the Rate Limiting Advanced Plugin [post_excerpt] => [post_status] => publish [comment_status] => closed [ping_status] => closed [post_password] => [post_name] => kong-enterprise-how-to-setup-the-rate-limiting-advanced-plugin [to_ping] => [pinged] => [post_modified] => 2022-01-26 13:18:35 [post_modified_gmt] => 2022-01-26 13:18:35 [post_content_filtered] => [post_parent] => 0 [guid] => https://keyvatech.com/?p=2149 [menu_order] => 11 [post_type] => post [post_mime_type] => [comment_count] => 0 [filter] => raw ) [4] => WP_Post Object ( [ID] => 2133 [post_author] => 2 [post_date] => 2020-01-15 09:49:55 [post_date_gmt] => 2020-01-15 09:49:55 [post_content] =>If you've used the community version of the Kong API gateway, you have probably noticed that anyone that knows the server name or IP for your Kong community API gateway can access and modify existing objects including services and routes. To set up and use role-based access control Kong Enterprise version provides additional capabilities.
In this example, we will leverage the Kong Enterprise on RHEL 7 lab instance we set up earlier. You can read the install steps here.
Before getting started, please make sure enforce_rbac=on
is in the kong.conf file.
Log in to https://<Kong-Enterprise-VM-IP>:8445/login using kong_admin as the username and the password you set during the install process (this is the same password you assigned during the step of EXPORT_PASSWORD='password')
Click on Teams -> RBAC Users
Create a new user rbac_user_1 with a token of rbac_user_token_1
Make sure that enabled checkbox is checked
Add roles –> admin
Note that we are creating this user with 'admin' permissions, but not 'super-admin'. So it will have access to all endpoints, across all workspaces—except RBAC Admin API.
A new RBAC user, rbac_user_1, gets created
Now let's try and test the RBAC setup. We will use Postman (https://www.getpostman.com/) for this example.
First we will create a new Collection labeled 'Kong Enterprise' and then a new Request within that Collection called 'Get Services'.
Next, we will try to run a GET request against https://<Kong-Enterprise-VM-IP>:8445/services to list out all available services. If you don't pass any headers or credentials, you get the error notification "Invalid credentials. Token or User credentials required".
By adding the header with Kong-Admin-Token and the value of the token set in the earlier step 'rbac_user_token_1', we try to run the request again and this time it succeeds
As you can see, with RBAC enabled, Kong Enterprise provides much greater control over who can access and modify various objects. The user permissions can be tailored to suit various team needs – depending upon how granular you want access to be.
If you have any questions or comments on the tutorial content above, or run in to specific errors not covered here, please feel free to reach out to info@keyvatech.com
[post_title] => Setting up Role-Based Access Control (RBAC) with Kong Enteprise [post_excerpt] => [post_status] => publish [comment_status] => closed [ping_status] => closed [post_password] => [post_name] => setting-up-role-based-access-control-rbac-with-kong-enteprise [to_ping] => [pinged] => [post_modified] => 2020-05-03 18:02:12 [post_modified_gmt] => 2020-05-03 18:02:12 [post_content_filtered] => [post_parent] => 0 [guid] => https://keyvatech.com/?p=2133 [menu_order] => 13 [post_type] => post [post_mime_type] => [comment_count] => 0 [filter] => raw ) [5] => WP_Post Object ( [ID] => 2124 [post_author] => 7 [post_date] => 2020-01-13 18:13:03 [post_date_gmt] => 2020-01-13 18:13:03 [post_content] =>This blog walks through the installation of Kong Enterprise (via rpm) on a Red Hat Enterprise 7 Virtual Machine.
Since we will be installing Kong Enterprise in a development environment, it is recommended that you use at least 2 GB of RAM and 2 vCPUs with 20 GB of storage space for your virtual machine.
It is also recommended to set up VMware tools. In order to do that, you will need to mount the VMware tools via the VMware console, and run the following commands via SSH.
yum install perl
mkdir /mnt/cdrom
Mount /dev/cdrom /mnt/cdrom
cp /mnt/cdrom/VMwareTools-
version
.tar.gz /tmp/
tar -zxvf VMwareTools-
version
.tar.gz
/tmp/vmware-tools-distrib/./vmware-install.pl
umount /mnt/cdrom
In this tutorial, we will install the Kong Enterprise server and the required PostgreSQL database on the same server. For production environments, you can choose to install the database and application tiers on separate machines. On the Kong Enterprise server, run the following commands:
subscription-manager register
subscription-manager refresh
subscription-manager attach –auto
subscription-manager repos –list
subscription-manager repos --enable rhel-7-server-rh-common-beta-rpms
subscription-manager repos --enable rhel-7-server-rpms
subscription-manager repos --enable rhel-7-server-source-rpms
subscription-manager repos --enable rhel-7-server-rh-common-source-rpms
subscription-manager repos --enable rhel-7-server-rh-common-debug-rpms
subscription-manager repos --enable rhel-7-server-optional-source-rpms
subscription-manager repos --enable rhel-7-server-extras-rpms
sudo yum update
sudo yum install wget
sudo yum install python36
sudo pip3 install httpie
For this development instance, we will stop and disable the firewall on the local machine, and then install PostgreSQL locally:
sudo systemctl stop firewalld
sudo systemctl disable firewalld
Download PostgreSQL RPM
sudo yum install
https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
Install PostgreSQL
sudo yum install postgresql95 postgresql95-server
Initialize the PostgreSQL Database, and start it:
sudo /usr/pgsql-9.5/bin/postgresql95-setup initdb
sudo systemctl enable postgresql-9.5
sudo systemctl start postgresql-9.5
Log in to the PostgreSQL database, and create the necessary structures for Kong Enterprise installation (Note that you will want to follow naming & password standards for your organization):
sudo -i -u postgres
$ psql
$ CREATE USER kong; CREATE DATABASE kong OWNER kong; ALTER USER kong WITH password 'kong';
$ \q
$ exit
Backup the original postgresql.conf file before modification
sudo cp /var/lib/pgsql/9.5/data/postgresql.conf /var/lib/pgsql/9.5/data/postgresql.conf.orig
Update the database configuration file postgresql.conf
sudo vi /var/lib/pgsql/9.5/data/postgresql.conf
Update the postgresql.conf file with the listen_addresses entry
listen_addresses = '*'
Backup the original pg_hba.conf file before modification
sudo cp /var/lib/pgsql/9.5/data/pg_hba.conf /var/lib/pgsql/9.5/data/pg_hba.conf.orig
Update database settings in pg_hba.conf
sudo vi /var/lib/pgsql/9.5/data/pg_hba.conf
Change the IPv4 entry to the IP address and the method to md5
host all all 0.0.0.0/0 md5
Restart PostgreSQL server
sudo systemctl restart postgresql-9.5
sudo systemctl status postgresql-9.5
Let's create a new folder to store the Kong RPMs:
mkdir kong
cd kong
In order to download Kong Enterprise, please work with your Kong Partner Manager or Account Executive to get access to your specific repository. Log in with your credentials at https://bintray.com/kong
The license file is located in the folder with your company or repository name.
On a separate machine, download the license file from the Kong repository portal, and then SCP it to the target VM.
scp ~/Downloads/ex12162020.license.json root@
<Kong-Enterprise-VM-IP>
:~/kong
You can either use wget to download the kong rpm and the license files directly on the VM, or you can download the files on a jump box and transfer them to the Kong Enterprise VM. We will use wget in this example:
wget '
https://<kong-supplied-username>:<kong-supplied-password>@bintray.com/kong/kong-enterprise-edition-rpm/rpm
' -O bintray-kong-kong-enterprise-edition-rpm.repo --auth-no-challenge
Copy the repo file under /etc/yum.repos.d
sudo mv bintray-kong-kong-enterprise-edition-rpm.repo /etc/yum.repos.d/
Next we will need to get the API key from the Kong bintray portal. Once you log in to https://bintray.com/kong click on your Username -> Edit Profile -> API Key
Update the repo file that we copied earlier
sudo vi /etc/yum.repos.d/bintray-kong-kong-enterprise-edition-rpm.repo
Modify the baseurl line by adding in your username and API key
#bintray--kong-kong-enterprise-edition-rpm - packages by from Bintray
[bintray--kong-kong-enterprise-edition-rpm]
name=bintray--kong-kong-enterprise-edition-rpm
baseurl=https://<Username>:<User-API-Key>@kong.bintray.com/kong-enterprise-edition-rpm/rhel/7
gpgcheck=0
repo_gpgcheck=0
enabled=1
Install the Kong service
sudo yum install kong-enterprise-edition
Add the language settings for the user environment:
sudo vi /etc/environment
Add the following lines
LANGUAGE=en_US.utf-8
LC_ALL=en_US.UTF-8
LC_CTYPE=UTF-8
LANG=en_US.utf-8
Logout of the session, and log in again
Update the user environment with ulimit value:
vi $HOME/.bashrc
At the end of the file, add
ulimit –n 4096
Save a copy of the default Kong conf file that ships with the installation before making modifications:
cp /etc/kong/kong.conf.default /etc/kong/kong.conf
sudo vi /etc/kong/kong.conf
Update the following variables with your environment specific values:
database = postgres
pg_host = <Kong-Enterprise-VM-IP>
pg_port = 5432
pg_timeout = 5000
pg_user = kong
pg_password = kong
pg_database = kong
admin_listen = 0.0.0.0:8001, 0.0.0.0:8444 ssl
We will now move the license file under the /etc/kong folder
sudo cp ex12162020.license.json /etc/kong/license.json
Update permissions for kong user
sudo chmod -R 777 /usr/local/kong/
Important: The KONG_PASSWORD environment variable needs to be exported before running the database migration and bootstrap processes. The password defined in this variable will be used to log in to the Kong Enterprise console once it is set up:
export KONG_PASSWORD=kong
kong migrations bootstrap -c /etc/kong/kong.conf -vv
kong start -c /etc/kong/kong.conf
Run a test against the local service to make sure Kong is up and running:
curl -i -X GET --url
http://localhost:8001/services
You can access the Kong Enterprise portal here:
http://<Kong-Enterprise-VM-IP>:8002/
https://<Kong-Enterprise-VM-IP>:8445/
Troubleshooting
Even the best-made plans can occasionally go awry, but don't worry, your friends at Keyva have your back. In our experience here's a list of some issues you could encounter, and if you do, how to fix them.
1) If you are unable to open or access the portal, make sure the firewall is turned off
sudo systemctl stop firewalld
2) Error: [PostgreSQL error] failed to retrieve server_version_num: connection refused OR
Error: [PostgreSQL error] failed to retrieve server_version_num: host or service not provided, or not known
Possible Remediations:
3) "Username/Password is invalid" – for the kong admin portal
Try running the bootstrap process again and clearing the browser cache
kong migrations reset
kong migrations bootstrap -c /etc/kong/kong.conf
kong migrations bootstrap
kong reload
kong stop
kong start
Try clearing cache, and going directly to the URL https://<Kong-Enterprise-VM-IP>:8445/overview
Important: You will also need to go to the URL https://<Kong-Enterprise-VM-IP>:8444 and accept the certificate. After accepting the certificate, go to the URL https://<Kong-Enterprise-VM-IP>:8445
4) "RBAC is disabled! Configuration will not be applied until RBAC is enabled. "
Rbac enabled but still keeps showing as disabled when you go to :8002/overview
In order to use RBAC, you will need to set up the following variables in kong.conf:
enforce_rbac = on
admin_gui_auth = basic-auth
admin_gui_session_conf = { "secret":"your_secret_text" }
Reload and restart kong service
kong reload
kong stop
kong start
5) Error: /usr/local/share/lua/5.1/kong/cmd/start.lua:31: [PostgreSQL error] failed to retrieve server_version_num: FATAL: no pg_hba.conf entry for host "127.0.0.1", user "kong", database "kong", SSL off
Verify that the below line exists in pg_hba.conf file
host all all 0.0.0.0/0 md5
You can also try adding the following line to trust all endpoints
host all all 0.0.0.0/0 trust
6) Error: /usr/local/share/lua/5.1/kong/db/migrations/state.lua:291: attempt to index local 'legacy_res' (a nil value)
stack traceback:
/usr/local/share/lua/5.1/kong/db/migrations/state.lua:291: in function 'load'
/usr/local/share/lua/5.1/kong/db/init.lua:412: in function 'schema_state'
/usr/local/share/lua/5.1/kong/cmd/migrations.lua:111: in function 'cmd_exec'
/usr/local/share/lua/5.1/kong/cmd/init.lua:88: in function </usr/local/share/lua/5.1/kong/cmd/init.lua:88>
[C]: in function 'xpcall'
/usr/local/share/lua/5.1/kong/cmd/init.lua:88: in function </usr/local/share/lua/5.1/kong/cmd/init.lua:45>
/usr/local/bin/kong:9: in function 'file_gen'
init_worker_by_lua:50: in function <init_worker_by_lua:48>
[C]: in function 'xpcall'
init_worker_by_lua:57: in function <init_worker_by_lua:55>
Verify that the following environment variables are exported
export KONG_DATABASE=postgres
export KONG_PG_HOST=kong-database
Restart the PostgreSQL, and Kong service
sudo systemctl status postgresql
kong reload
kong stop
kong start
7) Error: /usr/local/share/lua/5.1/kong/cmd/migrations.lua:109: [PostgreSQL error] failed to retrieve server_version_num: host or service not provided, or not known
stack traceback:
[C]: in function 'assert'
/usr/local/share/lua/5.1/kong/cmd/migrations.lua:109: in function 'cmd_exec'
/usr/local/share/lua/5.1/kong/cmd/init.lua:88: in function </usr/local/share/lua/5.1/kong/cmd/init.lua:88>
[C]: in function 'xpcall'
/usr/local/share/lua/5.1/kong/cmd/init.lua:88: in function </usr/local/share/lua/5.1/kong/cmd/init.lua:45>
/usr/local/bin/kong:9: in function 'file_gen'
init_worker_by_lua:50: in function <init_worker_by_lua:48>
[C]: in function 'xpcall'
init_worker_by_lua:57: in function <init_worker_by_lua:55>
Verify that the following environment variable is exported
export KONG_PG_HOST=<Kong-Enterprise-VM-IP>
Restart the PostgreSQL, and Kong service
sudo systemctl status postgresql
kong reload
kong stop
kong start
8) Kong Manager portal error: "Authentication is not enabled. "
Set basic authentication variable (admin-gui-auth) configured in the kong.conf file
enforce_rbac = on
admin_gui_auth = basic-auth
admin_gui_session_conf = { "secret":"your_secret_text" }
Reload and restart kong service
kong reload
kong stop
kong start
If you have any questions or comments on the tutorial content above, or run into specific errors not covered here, please feel free to reach out to info@keyvatech.com
[post_title] => Install Kong Enteprise on Red Hat Enterprise Linux 7 [post_excerpt] => [post_status] => publish [comment_status] => closed [ping_status] => closed [post_password] => [post_name] => install-kong-enteprise-on-red-hat-enterprise-linux-7 [to_ping] => [pinged] => [post_modified] => 2020-01-15 16:44:20 [post_modified_gmt] => 2020-01-15 16:44:20 [post_content_filtered] => [post_parent] => 0 [guid] => https://keyvatech.com/?p=2124 [menu_order] => 14 [post_type] => post [post_mime_type] => [comment_count] => 0 [filter] => raw ) [6] => WP_Post Object ( [ID] => 2119 [post_author] => 7 [post_date] => 2020-01-10 10:05:33 [post_date_gmt] => 2020-01-10 10:05:33 [post_content] =>We all know that Agile is has been around for a while now. You have probably heard about it a thousand times already over the past few years. But even today there are organizations (e.g. Utilities – that prefer CapEx) that rely heavily on the waterfall model for software development – whereby all the project and implementations details are agreed to by the stakeholders upfront. In the past, this practice has proved to be risky, inflexible, too time consuming and costly at the beginning of the project. When starting any new project, even when applying all possible game theory outcomes to determine the risk and estimated effort, at some point you don’t know what you don't know. The methodology of Agile helps alleviate some of these issues. Using various methodologies, the process of Agile allows for the requirements to evolve and change, and a team comprised of developers and experts from various organizational areas work together to address the tasks as they evolve. This type of a setup is typically led by a Scrum Master that leads regular checkpoint meetings with the stakeholders, helps break down the work into smaller chunks to be picked up by the developers, and sets up timelines for completion and accountability.
There are several methodologies and frameworks that you can follow to be more Agile - for example, you can use Scrum methodology, Test-Driven Development, DevOps, Continuous Integration, Continuous Delivery, Kanban, Extreme Programming, and more. The idea is to be able to provide flexibility and avoiding lock-in to a set process or tools, and have regular checkpoints with the stakeholders so that any shifts in directly can be accommodated earlier in the cycle.
The Manifesto for Agile Software Development outlines the following four values:
It also follows the following twelve principles:
The Product Owner is one of the most important stakeholders in the Agile process. Product Owner is responsible for setting the overall strategy and direction for the deliverable that is being worked. It is quite easy to miss the big picture of what is being delivered and why, because the Scrum tasks are at a very low level. The Product Owner's role is to help make sense of all the small unrelated tasks, to deliver a product that provides business value to the organization.
Keyva has offerings available to help you with your Agile journey. You can find more information here. Please contact us if you'd like to have us review your environment and provide suggestions on what might work for you. Simply drop us a line here: info@keyvatech.com
[post_title] => Agile Methodologies - Rise & Shine [post_excerpt] => [post_status] => publish [comment_status] => closed [ping_status] => closed [post_password] => [post_name] => agile-methodologies-rise-shine [to_ping] => [pinged] => [post_modified] => 2020-03-05 20:11:34 [post_modified_gmt] => 2020-03-05 20:11:34 [post_content_filtered] => [post_parent] => 0 [guid] => https://keyvatech.com/?p=2119 [menu_order] => 15 [post_type] => post [post_mime_type] => [comment_count] => 0 [filter] => raw ) [7] => WP_Post Object ( [ID] => 2083 [post_author] => 7 [post_date] => 2020-01-08 15:14:26 [post_date_gmt] => 2020-01-08 15:14:26 [post_content] =>This blog walks through the installation of Kong Enterprise (via rpm) on a CentOS 7 Virtual Machine. You can download and install the latest version of CentOS from https://www.centos.org/download/. We will be installing Kong Enterprise in a development environment so it is recommended that you use at least 2 GB of RAM and 2 vCPUs with 20 GB of storage space.
Additionally, it is recommended to setup VMware tools. In order to do that you will need to mount the vmware tools via the VMware console, and run the following commands via SSH:
yum install perl
mkdir /mnt/cdrom
Mount /dev/cdrom /mnt/cdrom
cp /mnt/cdrom/VMwareTools-
version
.tar.gz /tmp/
tar -zxvf VMwareTools-
version
.tar.gz
/tmp/vmware-tools-distrib/./vmware-install.pl
umount /mnt/cdrom
In this tutorial, we will install the Kong Enterprise server and the required PostgreSQL on the same server. For production environments, you can choose to install the database and application tiers on separate machines. To get started, on the Kong Enterprise server, run the following commands:
sudo yum update
sudo yum install wget
sudo yum install python36
sudo pip3 install httpie
We will also create a new folder to store the Kong RPMs:
mkdir kong
In order to download Kong Enterprise you will need to work with your Kong Partner Manager or Kong Enterprise sales rep to get access to your specific repository. Log in with your credentials at https://bintray.com/kong
The license file is located in the folder with your company or repository name.
You can either use wget to download the kong rpm and the license files directly on the VM, or you can download the files on a jump box and transfer them to the Kong Enterprise VM. Steps for the latter are below.
On a separate machine, download the specific rpm for your OS and version (kong-enterprise-edition-0.36-4.el7.noarch.rpm) and the license file. In this case, we first download the files under the 'Downloads' folder on the jump box, and then SCP the files to the target VM.
scp ~/Downloads/kong-enterprise-edition-0.36-4.el7.noarch.rpm
root@<Kong-Enterprise-VM-IP>:~/kong
scp ~/Downloads/ex12162020.license.json root@
<Kong-Enterprise-VM-IP>
:~/kong
Log back into the Kong Enterprise VM via SSH:
cd /root/kong
sudo yum install kong-enterprise-edition-0.36-4.el7.noarch.rpm
sudo cp ex12162020.license.json /etc/kong/license.json
sudo yum install
https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
sudo yum --disablerepo=bintray--kong-kong-enterprise-edition-rpm install postgresql95 postgresql95-server
Initialize the PostgreSQL Database, and start it:
sudo /usr/pgsql-9.5/bin/postgresql95-setup initdb
sudo systemctl enable postgresql-9.5
sudo systemctl start postgresql-9.5
Login to the PostgreSQL database, and create the necessary structures for Kong Enterprise installation. Note: For this example, we are using kong as our username, database name, and password. You will want to choose user, database, and passwords according to the naming and complexity standards of your organization. If you need assistance with this step, your friends at Keyva are no more than a click away!
For this example, we are using kong as our username, database name, and password.
sudo -i -u postgres
$ psql
$ CREATE USER kong; CREATE DATABASE kong OWNER kong; ALTER USER kong WITH password 'kong';
$ \q
$ exit
Update database settings in pg_hba.conf:
sudo vi /var/lib/pgsql/9.5/data/pg_hba.conf
Change the IPv4 entry to the IP address and the method to md5
host all all 0.0.0.0/0 md5
Update the database configuration file postgresql.conf:
sudo vi /var/lib/pgsql/9.5/data/postgresql.conf
Add the line at the end -
listen_addresses = '*'
Restart the PostgreSQL server:
sudo systemctl restart postgresql-9.5
Save a copy of the default Kong conf file that ships with the installation before making modifications:
cp /etc/kong/kong.conf.default /etc/kong/kong.conf
Update the following variables with your environment specific values:
sudo vi /etc/kong/kong.conf
database = postgres
pg_host = <Kong-Enterprise-VM-IP>
pg_port = 5432
pg_timeout = 5000
pg_user = kong
pg_password = kong
pg_database = kong
Update the user environment with ulimit value:
vi $HOME/.bashrc
At the end of the file, add
ulimit –n 4096
Add the language settings for the user environment:
sudo vi /etc/environment
Add the following lines
LANGUAGE=en_US.utf-8
LC_ALL=en_US.UTF-8
LC_CTYPE=UTF-8
LANG=en_US.utf-8
Update permissions:
sudo chmod -R 777 /usr/local/kong/
Important: The KONG_PASSWORD environment variable needs to be exported before running the database migration and bootstrap processes. The password defined in this variable will be used to log in to the Kong Enterprise console once it is set up:
export KONG_PASSWORD=kong
kong migrations bootstrap -c /etc/kong/kong.conf
kong start -c /etc/kong/kong.conf
Run a test against the local service to make sure Kong is up and running:
curl -i -X GET --url
http://localhost:8001/services
You can access the Kong Enterprise portal here:
http://<Kong-Enterprise-VM-IP>:8002/
https://<Kong-Enterprise-VM-IP>:8445/
Even the best-made plans can occasionally go awry, but don't worry, your friends at Keyva have your back. In our experience here's a list of some issues you could encounter, and if you do, how to fix them.
1. If you are unable to open or access the portal, make sure the firewall is turned off
sudo systemctl stop firewalld
2. Error: [PostgreSQL error] failed to retrieve server_version_num: connection refused OR
Error: [PostgreSQL error] failed to retrieve server_version_num: host or service not provided, or not known
Possible Remediations:
host all all 0.0.0.0/0 trust
Make sure the following environment variables are set up and they exist prior to running the database migration and bootstrap process:
export KONG_DATABASE=postgres
export KONG_PG_HOST=<Kong-Enterprise-VM-IP>
3. "Username/Password is invalid" – for the kong admin portal
Try running the bootstrap process again and clearing the browser cache
kong migrations reset
kong migrations bootstrap -c /etc/kong/kong.conf
kong migrations bootstrap
kong reload
kong stop
kong start
Try clearing cache, and going directly to the URL https://<Kong-Enterprise-VM-IP>:8445/overview
Important: You will also need to go to the URL https://<Kong-Enterprise-VM-IP>:8444 and accept the certificate. After accepting the certificate, go to the URL https://<Kong-Enterprise-VM-IP>:8445
4. "RBAC is disabled! Configuration will not be applied until RBAC is enabled. "
RBAC is enabled but still keeps showing as disabled when you go to :8002/overview
In order to use RBAC, you will need to set up the following variables in kong.conf:
enforce_rbac = on
admin_gui_auth = basic-auth
admin_gui_session_conf = { "secret":"your_secret_text" }
Reload and restart Kong service:
kong reload
kong stop
kong start
5. Error: /usr/local/share/lua/5.1/kong/cmd/start.lua:31: [PostgreSQL error] failed to retrieve server_version_num: FATAL: no pg_hba.conf entry for host "127.0.0.1", user "kong", database "kong", SSL off
Verify that the below line exists in pg_hba.conf file
host all all 0.0.0.0/0 md5
You can also try adding the following line to trust all endpoints
host all all 0.0.0.0/0 trust
6. Error: /usr/local/share/lua/5.1/kong/db/migrations/state.lua:291: attempt to index local 'legacy_res' (a nil value)
stack traceback:
/usr/local/share/lua/5.1/kong/db/migrations/state.lua:291: in function 'load'
/usr/local/share/lua/5.1/kong/db/init.lua:412: in function 'schema_state'
:111: in function 'cmd_exec'
/usr/local/share/lua/5.1/kong/cmd/init.lua:88: in function </usr/local/share/lua/5.1/kong/cmd/init.lua:88>
[C]: in function 'xpcall'
/usr/local/share/lua/5.1/kong/cmd/init.lua:88: in function </usr/local/share/lua/5.1/kong/cmd/init.lua:45>
/usr/local/bin/kong:9: in function 'file_gen'
init_worker_by_lua:50: in function <init_worker_by_lua:48>
[C]: in function 'xpcall'
init_worker_by_lua:57: in function <init_worker_by_lua:55>
Verify that the following environment variables are exported
export KONG_DATABASE=postgres
export KONG_PG_HOST=kong-database
Restart the PostgreSQL, and Kong service
sudo systemctl status postgresql
kong reload
kong stop
kong start
7. Error: /usr/local/share/lua/5.1/kong/cmd/migrations.lua:109: [PostgreSQL error] failed to retrieve server_version_num: host or service not provided, or not known
stack traceback:
[C]: in function 'assert'
/usr/local/share/lua/5.1/kong/cmd/migrations.lua:109: in function 'cmd_exec'
/usr/local/share/lua/5.1/kong/cmd/init.lua:88: in function </usr/local/share/lua/5.1/kong/cmd/init.lua:88>
[C]: in function 'xpcall'
/usr/local/share/lua/5.1/kong/cmd/init.lua:88: in function </usr/local/share/lua/5.1/kong/cmd/init.lua:45>
/usr/local/bin/kong:9: in function 'file_gen'
init_worker_by_lua:50: in function <init_worker_by_lua:48>
[C]: in function 'xpcall'
init_worker_by_lua:57: in function <init_worker_by_lua:55>
Verify that the following environment variable is exported
export KONG_PG_HOST=<Kong-Enterprise-VM-IP>
Restart the PostgreSQL, and Kong service
sudo systemctl status postgresql
kong reload
kong stop
kong start
8. Kong Manager portal error: "Authentication is not enabled. "
Set basic authentication variable (admin-gui-auth) configured in the kong.conf file
enforce_rbac = on
admin_gui_auth = basic-auth
admin_gui_session_conf = { "secret":"your_secret_text" }
Reload and restart kong service
kong reload
kong stop
kong start
If you have any questions or comments on the tutorial content above, or run in to specific errors not covered here, please feel free to reach out to info@keyvatech.com
Anuj joined Keyva from Tech Data where he was the Director of Automation Solutions. In this role, he specializes in developing and delivering vendor-agnostic solutions that avoid the “rip-and-replace” of existing IT investments. Tuli has worked on Cloud Automation, DevOps, Cloud Readiness Assessments and Migrations projects for healthcare, banking, ISP, telecommunications, government and other sectors.
During his previous years at Avnet, Seamless Technologies, and other organizations, he held multiple roles in the Cloud and Automation areas. Most recently, he led the development and management of Cloud Automation IP (intellectual property) and related professional services. He holds certifications for AWS, VMware, HPE, BMC and ITIL, and offers a hands-on perspective on these technologies.
Like what you read? Follow Anuj on LinkedIn at: https://www.linkedin.com/in/anujtuli/
[post_title] => How to Install Kong Enterprise on CentOS 7 [post_excerpt] => [post_status] => publish [comment_status] => closed [ping_status] => closed [post_password] => [post_name] => how-to-install-kong-enterprise-on-centos-7 [to_ping] => [pinged] => [post_modified] => 2020-03-02 12:34:46 [post_modified_gmt] => 2020-03-02 12:34:46 [post_content_filtered] => [post_parent] => 0 [guid] => https://keyvatech.com/?p=2083 [menu_order] => 16 [post_type] => post [post_mime_type] => [comment_count] => 0 [filter] => raw ) ) [post_count] => 8 [current_post] => -1 [before_loop] => 1 [in_the_loop] => [post] => WP_Post Object ( [ID] => 2278 [post_author] => 7 [post_date] => 2020-03-24 15:27:04 [post_date_gmt] => 2020-03-24 15:27:04 [post_content] =>
By Anuj Tuli, CTO
Keyva announces the certification of their ServiceNow App for Red Hat Ansible Tower against the Orlando release (latest release) of ServiceNow. ServiceNow announced its release of Orlando on January 23rd, 2020, which is the newest version in the long line of software updates since the company's creation.
Customers can now upgrade their ServiceNow App for Ansible Tower from previous ServiceNow Releases – London, Madrid, New York – to Orlando release seamlessly.
You can find out more about the App, and view all the ServiceNow releases it is certified against, on the ServiceNow store here: http://bit.ly/2W5tYHv
[post_title] => ServiceNow App for Red Hat Ansible Tower "NOW Certified" against Orlando release [post_excerpt] => [post_status] => publish [comment_status] => closed [ping_status] => closed [post_password] => [post_name] => servicenow-app-for-red-hat-ansible-tower-now-certified-against-orlando-release [to_ping] => [pinged] => [post_modified] => 2020-03-24 15:27:07 [post_modified_gmt] => 2020-03-24 15:27:07 [post_content_filtered] => [post_parent] => 0 [guid] => https://keyvatech.com/?p=2278 [menu_order] => 7 [post_type] => post [post_mime_type] => [comment_count] => 0 [filter] => raw ) [comment_count] => 0 [current_comment] => -1 [found_posts] => 113 [max_num_pages] => 15 [max_num_comment_pages] => 0 [is_single] => [is_preview] => [is_page] => [is_archive] => [is_date] => [is_year] => [is_month] => [is_day] => [is_time] => [is_author] => [is_category] => [is_tag] => [is_tax] => [is_search] => [is_feed] => [is_comment_feed] => [is_trackback] => [is_home] => 1 [is_privacy_policy] => [is_404] => [is_embed] => [is_paged] => 1 [is_admin] => [is_attachment] => [is_singular] => [is_robots] => [is_favicon] => [is_posts_page] => [is_post_type_archive] => [query_vars_hash:WP_Query:private] => 9c73350f649d9d90ae97980bfef9953a [query_vars_changed:WP_Query:private] => [thumbnails_cached] => [allow_query_attachment_by_filename:protected] => [stopwords:WP_Query:private] => [compat_fields:WP_Query:private] => Array ( [0] => query_vars_hash [1] => query_vars_changed ) [compat_methods:WP_Query:private] => Array ( [0] => init_query_flags [1] => parse_tax_query ) [tribe_is_event] => [tribe_is_multi_posttype] => [tribe_is_event_category] => [tribe_is_event_venue] => [tribe_is_event_organizer] => [tribe_is_event_query] => [tribe_is_past] => )