Blog & Insights

Hey everyone! I hope you're staying warm through this historic cold snap. There's no better time than the present to stay indoors and check out some upcoming virtual tech events.

The first event I'd like to mention is coming up this Thursday, Feb 18th, 2021. It's the Open Source North Speaker Series. It's free to attend but you need to register in advance. Here's a look at this Thursday's speaker agenda:

Open Source North Speaker Series

Thursday, February 18, 12:00 - 1:00 PM CST

• Jenna Pederson, AWS, Setting yourself up for failure: cultivating a culture where failure is celebrated
• Jason Scherschligt, SDG, Making the shift: projects to products
• Mark Sielaff, Facebook, Moving Fast and Delivering Quality
• Todd Gardner, TrackJS, Observable Web Applications

For details on each presentation and speaker - AND TO REGISTER, please visit https://opensourcenorth.com/speaker-series

The next event I want to mention is an event coming up with the CTO of Kong , Marco Palladino.

Marco is a dynamic speaker and excellent CTO, always a fun, informative watch.

Kong - Automatic Observability With Service Mesh

Friday, February 26, 11:00 - 12:00 PM CT

In this session you'll learn how a service mesh can observe all of our traffic in new modern applications running on both Kubernetes and virtual machines.

Key takeaways:

• Understanding the service mesh pattern and how it helps to modernize our applications out of the box.
• A live demonstration of a service mesh to extract logs, metrics and traces from our service traffic and store it in Prometheus + Grafana, Jaeger and Elasticsearch.
• Learning how to use the service mesh policies available in CNCF’s Kuma to enable observability in one click.

To register go here: Register Now

The last event I want to draw your attention to this month is focused on the OpenShift Developer Sandbox hosted by Red Hat DevNation during which you will create an account an OpenShift cluster and deploy a sample app. The dev cluster will be available for you to use thereafter for 14 days.

OpenShift Developer Sandbox

Thursday, February 18, 11:00 - 12:00 PM CST

Have you heard of the new OpenShift Developer Sandbox? Join this DevNation Tech Talk where you will be guided through the process of creating an account, creating/configuring your Developer Sandbox cluster, and deploying a sample application on OpenShift. Your OpenShift cluster will be available for your use for 14 days. If you've ever wanted to test out OpenShift, this is your chance to do it!

Produced by the Red Hat Developer team, DevNation Tech Talks are live discussions led by the Red Hat technologists who create our products. Sessions include real solutions and code to help you build with open source, plus sample projects, robust discussion, and live Q&A to help you get started.

Are you new to DevNation Tech Talks? See what you've missed.

To register for this event go here: Register for the OpenShift Sandbox Tech Talk

In this post we'll briefly explore the history of the Opsware automation portfolio and talk about modern equivalents and replacements you should be considering.  

A Brief History of Opsware  

Let’s start with defining what we are talking about in today's blog. I'm focusing specifically on the IT datacenter automation software, namely: Cloud Service Automation (CSA), Server Automation (SA), Network Automation (NA), and Operations Orchestration (OO) - a product which once had the acronym HPOO… you can't make it up! 

If we allow ourselves to hop in the way back machine, the story starts with a Bay Area startup called Loudcloud which was founded by Ben Horowitz and Marc Andreesen in 1999. Loudcloud was an infrastructure and application hosting company and developed really cool management software to manage its clients' IT infrastructure. The company went public in 2001. In 2002 Loudcloud sold its managed services business to EDS. (Ed. note: EDS briefly became HP ES in an acquisition on its ultimate voyage into the sun and to a merger with CSC, the joint company becoming known as DXC Technology in 2017.) Loudcloud rebranded as an enterprise software company called Opsware that focused on developing and selling its IT datacenter lifecycle management software. In 2007 Opsware was acquired by HP Software. In 2017 HP sold the software business to Micro Focus. This software that Loudcloud / Opsware built back in the late 1990 / early 2000s is the aforementioned suite of automation software, specifically: Server Automation (System), Network Automation (System), and Process Automation (System) - all of which were rebranded slightly after the 2007 acquisition by HP Software.  

So other than exercising some knowledge on the history of the software, why mention all of this? It's because it is truly old tech. It's been upgraded and expanded and rewritten since the early days, but it is still that kind of old school top-down management interface for IT environments with more modern amenities like the ability to write automation in YAML stapled to the side of it. At their peak these software solutions were used to manage tens of thousands of operating systems, network devices, and to automate endpoints leveraging an agent-based architecture. And it wasn't cheap! Solutions like Server Automation, Operations Orchestration and other similar market offerings (anyone remember BMC Bladelogic, now TrueSight?) were closed-source and partially responsible for the explosion of enterprise open source software. Sales teams had a number back then, if your device count was smaller than that number they knew there was no business case for you to evaluate that type of software - you just couldn't get there. A good chunk of mid-market and large, but not large-enough, IT enterprises were left with no good enterprise automation solutions.  

What Else Is Out There? 

So what happens? People start looking for (and building) their own solutions in the mid-2000s. Open source solutions start getting community adoption and IT staff are able to go way beyond things like CFEngine and are starting to adopt solutions like Chef and Puppet and learn more modern languages like Ruby. Chef and Puppet provide an early example of how to build a userbase on open source software but quickly realize no one wants to suddenly pay for things they'd been previously given for free. Licensing models change, some products go open core and paywall subsequently developed features. Far more recently, that is, in the last 10 years (geez, I am getting old)open source software supporting modern software development and hybrid cloud architectures has become the standard. And if you find yourself in a traditional IT environment or at least one with some tech debt you're looking to retire, you really owe it to yourself to look at Ansible & Terraform.  

Red Hat Ansible & HashiCorp Terraform 

Ansible began life as an open source project in 2012. Automation is written in YAML, a simple scripting language that anyone can learn and it is an agentless architecture. Ansible was acquired by Red Hat in 2015, and to their great credit, Red Hat not only left Ansible core as open source, they went and open-sourced the enterprise version Ansible Tower (the community version of which is AWX)! Awesome move for the community. Due to the commitment to open source, Red Hat's market reach, and the extraordinarily simple-to-use scripting language YAML, usage of Ansible in enterprises of all sizes has skyrocketed. If you're not using it today, you're in luck, you're a simple web search and download from having an enterprise grade solution that really acts as a jack-of-all-trades for endpoint configuration regardless of operating system running on the target. It's been used quite successfully for years at very large scale in organizations of every size.  

HashiCorp Terraform launched in the community in 2014. It has since seen massive growth as an open source project and as both SaaS-based and on-premise enterprise software solutions. Terraform is an extremely powerful tool which enables infrastructure-as-code use cases. Terraform manages external resources using what it calls providers and gives the end-user the ability to declare the end-state configuration leveraging those external providers. This declarative architecture allows for highly modular, scalable, and reusable code to configure highly complex end points, platform-as-a-service, etc.  

In practice, we see Ansible + Terraform being used in concert with code release processes as well as being front-ended by service catalogs like ServiceNow to enable a limitless variety of push button IT capabilities. Please contact us If you'd like to learn more about using Ansible or Terraform .

$ mkdir ~/openshift_windows_cluster && cd ~/openshift_windows_cluster

$ openshift-install create install-config
? Platform aws
INFO Credentials loaded from the "default" profile in file "/home/ec2-user/.aws/credentials"
? Region us-east-2
? Base Domain example.com
? Cluster Name win-test-cluster
? Pull Secret [? for help] (Paste your Pull Secret from the Red Hat web site or text file you downloaded)

$ sed -i 's/OpenShiftSDN/OVNKubernetes/g' install-config.yaml

$ cp -p install-config.yaml install-config.yaml.backup

$ openshift-install create manifests
INFO Credentials loaded from the "default" profile in file "/home/ec2-user/.aws/credentials"
INFO Consuming Install Config from target directory

$ cp -p manifests/cluster-network-02-config.yml manifests/cluster-network-03-config.yml

$ vi manifests/cluster-network-03-config.yml

apiVersion: operator.openshift.io/v1
kind: Network
metadata:
  creationTimestamp: null
  name: cluster
spec:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  externalIP:
    policy: {}
  networkType: OVNKubernetes
  serviceNetwork:
  - 172.30.0.0/16
  defaultNetwork:
    type: OVNKubernetes
    ovnKubernetesConfig:
      hybridOverlayConfig:
        hybridClusterNetwork:
        - cidr: 10.132.0.0/14
          hostPrefix: 23
status: {}

$ openshift-install create cluster
INFO Consuming Openshift Manifests from target directory
INFO Consuming Worker Machines from target directory
INFO Consuming Master Machines from target directory
INFO Consuming OpenShift Install (Manifests) from target directory
INFO Consuming Common Manifests from target directory
INFO Credentials loaded from the "default" profile in file "/home/ec2-user/.aws/credentials"
INFO Creating infrastructure resources...
INFO Waiting up to 20m0s for the Kubernetes API at https://api.win-test-cluster.example.com:6443...
INFO API v1.18.3+5302882 up
INFO Waiting up to 40m0s for bootstrapping to complete...
INFO Destroying the bootstrap resources...
INFO Waiting up to 30m0s for the cluster at https://api.win-test-cluster.example.com:6443 to initialize...
I1015 22:40:12.502855 1042 trace.go:116] Trace[1959950141]: "Reflector ListAndWatch" name:k8s.io/client-go/tools/watch/informerwatcher.go:146 (started: 2020-10-15
22:39:55.810110164 +0000 UTC m=+886.539985514) (total time: 16.692708687s):
Trace[1959950141]: [16.692655552s] [16.692655552s] Objects listed

INFO Waiting up to 10m0s for the openshift-console route to be created...
INFO Install complete!
INFO To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/home/ec2-user/openshift_windows_cluster/auth/kubeconfig'
INFO Access the OpenShift web-console here: https://console-openshift-console.apps.win-test-cluster.example.com
INFO Login to the console with user: "kubeadmin", and password: "XXXXX-XXXXX-XXXXX-XXXXX"
INFO Time elapsed: 30m48s

$ export KUBECONFIG=/home/ec2-user/openshift_windows_cluster/auth/kubeconfig
$ oc get nodes
NAME STATUS ROLES AGE VERSION
ip-10-0-128-115.us-east-2.compute.internal Ready master 1h v1.18.3+970c1b3
ip-10-0-150-141.us-east-2.compute.internal Ready worker 1h v1.18.3+970c1b3
ip-10-0-161-110.us-east-2.compute.internal Ready worker 1h v1.18.3+970c1b3
ip-10-0-186-69.us-east-2.compute.internal Ready master 1h v1.18.3+970c1b3
ip-10-0-201-57.us-east-2.compute.internal Ready master 1h v1.18.3+970c1b3
ip-10-0-220-129.us-east-2.compute.internal Ready worker 1h v1.18.3+970c1b3
$ oc version
Client Version: 4.5.14
Server Version: 4.5.14
Kubernetes Version: v1.18.3+5302882

 $ oc get network.operator cluster -o yaml

Look at the spec section of the yaml output. It should look like this.

spec:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  defaultNetwork:
    ovnKubernetesConfig:
      hybridOverlayConfig:
        hybridClusterNetwork:
        - cidr: 10.132.0.0/14
          hostPrefix: 23
    type: OVNKubernetes
serviceNetwork:
- 172.30.0.0/16

$ ssh-keygen -t rsa -b 4096 -N "" -C "example-key" -f ~/.ssh/example-key

$ aws --region us-east-2 ec2 import-key-pair --key-name "example-key" --public-key-material file://$HOME/.ssh/example-key.pub

$ wget https://github.com/openshift/windows-machine-config-bootstrapper/releases/download/v4.5.2-alpha/wni -O ~/bin/wni

$ chmod +x ~/bin/wni && mkdir windowsnodeinstaller

$ wni aws create --kubeconfig $KUBECONFIG --credentials ~/.aws/credentials --credential-account default --instance-type m5a.large --ssh-key example-key --private-key ~/.ssh/example-key --dir ./windowsnodeinstaller/
2020/10/16 20:05:13 kubeconfig source: /home/ec2-user/openshift_windows_cluster/auth/kubeconfig
2020/10/16 20:05:14 Added rule with port 5986 to the security groups of your local IP
2020/10/16 20:05:14 Added rule with port 22 to the security groups of your local IP
2020/10/16 20:05:14 Added rule with port 3389 to the security groups of your local IP
2020/10/16 20:05:14 Using existing Security Group: sg-0123456789012345
2020/10/16 20:09:41 External IP: 4.138.182.84
2020/10/16 20:09:41 Internal IP: 10.0.42.50

$ cat windowsnodeinstaller/windows-node-installer.json
{"InstanceIDs":["i-0123456789012345"],"SecurityGroupIDs":["sg-0123456789012345"]}

$ aws ec2 get-password-data --instance-id i-0123456789012345 --priv-launch-key ~/.ssh/example-key

 $ vi inventory.ini

[win]
4.138.182.84 ansible_password='YOURWINDOWSNODEPASSWORDHERE' private_ip=10.0.42.50

[win:vars]
ansible_user=Administrator
cluster_address=win-test-cluster.example.com
ansible_connection=winrm
ansible_ssh_port=5986
ansible_winrm_server_cert_validation=ignore

$ ansible win -i inventory.ini -m win_ping
4.138.182.84 | SUCCESS => {
"changed": false,
"ping": "pong"
}

$ git clone https://github.com/openshift/windows-machine-config-bootstrapper.git

$ ansible-playbook -v -i inventory.ini windows-machine-config-bootstrapper/tools/ansible/tasks/wsu/main.yaml

$ oc get nodes -o wide -l kubernetes.io/os=windows
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
ip-10-0-42-50.us-east-2.compute.internal Ready worker 29m v1.18.3 10.0.42.50 3.138.182.84 Windows Server 2019 Datacenter 10.0.17763.1518 docker://19.3.12

$ oc create -f https://raw.githubusercontent.com/keyvatech/blog_files/master/kubernetes_windows_web_server.yaml -n default

$oc rollout status deployment win-webserver -n default
deployment "win-webserver" successfully rolled out

PS C:\Users\Administrator> docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
09c8bbd2a7e8 mcr.microsoft.com/windows/servercore "powershell.exe -com…" 13 minutes ago Up 13 minutes k8s_windowswebserver_win-webserver-85b49f8677-cgqkq_default_01fe28db-5ae7-4ead-8e84-5d9d5cd2cb01_0
52d42f33de9d mcr.microsoft.com/k8s/core/pause:1.2.0 "cmd /S /C 'cmd /c p…" 16 minutes ago Up 16 minutes k8s_POD_win-webserver-85b49f8677-cgqkq_default_01fe28db-5ae7-4ead-8e84-5d9d5cd2cb01_0

$ oc rollout restart deployment/win-webserver
$ oc rollout retry deployment/win-webserver

$ oc get pods
NAME READY STATUS RESTARTS AGE
win-webserver-564d75c5f7-l4kk2 1/1 Running 0 96s
$ oc logs win-webserver-564d75c5f7-l4kk2
Listening at http://*:80/

$ oc get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 172.30.0.1 <none> 443/TCP 23h
openshift ExternalName <none> kubernetes.default.svc.cluster.local <none> 23h
win-webserver LoadBalancer 172.30.88.146 a038a9aa4571f4a7cafaf15ebf7ae270-23672059.us-east-2.elb.amazonaws.com 80:32601/TCP 35m
$ curl a038a9aa4571f4a7cafaf15ebf7ae270-23672059.us-east-2.elb.amazonaws.com
<html><body><H1>Windows Container Web Server</H1></body></html>

$ wni aws destroy --kubeconfig $KUBECONFIG --credentials ~/.aws/credentials --credential-account default --dir ./windowsnodeinstaller/

$ openshift-install destroy cluster

$ sudo yum install python3 python3-pip git
$ pip3 install --user pywinrm ansible

$ cd ~

$ wget https://mirror.openshift.com/pub/openshift-v4/clients/ocp/latest/openshift-client-linux.tar.gz

$ mkdir bin && tar -xvf openshift-client-linux.tar.gz --directory bin && mv bin/README.md ~/openshift-client-README.md

$ wget https://mirror.openshift.com/pub/openshift-v4/clients/ocp/latest/openshift-install-linux.tar.gz

$ tar -xvf openshift-install-linux.tar.gz --directory bin && mv bin/README.md ~/openshift-install-README.md

$ ansible --version
ansible 2.10.2
config file = None
configured module search path = ['/home/ec2-user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/ec2-user/.local/lib/python3.7/site-packages/ansible
executable location = /home/ec2-user/.local/bin/ansible
python version = 3.7.9 (default, Aug 27 2020, 21:59:41) [GCC 7.3.1 20180712 (Red Hat 7.3.1-9)]

$ aws --version
aws-cli/1.18.107 Python/2.7.18 Linux/4.14.193-149.317.amzn2.x86_64 botocore/1.17.31

$ oc version
Client Version: 4.5.14

$ openshift-install version
openshift-install 4.5.14
built from commit 9893a482f310ee72089872f1a4caea3dbec34f28
release image quay.io/openshift-release-dev/ocp-release@sha256:95cfe9273aecb9a0070176210477491c347f8e69e41759063642edf8bb8aceb6

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2-0-g52c56ce", GitCommit:"d7f3ccf9a5bdc96ba92e31526cf014b3de4c46aa", GitTreeState:"clean", BuildDate:"2020-09-16T15:25:59Z", GoVersion:"go1.13.4", Compiler:"gc", Platform:"linux/amd64"}

$ pip3 freeze
ansible==2.10.1
ansible-base==2.10.2
certifi==2020.6.20
cffi==1.14.3
chardet==3.0.4
cryptography==3.1.1
idna==2.10
Jinja2==2.11.2
MarkupSafe==1.1.1
ntlm-auth==1.5.0
packaging==20.4
pycparser==2.20
pyparsing==2.4.7
pywinrm==0.4.1
PyYAML==5.3.1
requests==2.24.0
requests-ntlm==1.1.0
six==1.15.0
urllib3==1.25.10
xmltodict==0.12.0

$ pip3 show pywinrm
Name: pywinrm
Version: 0.4.1
Summary: Python library for Windows Remote Management
Home-page: http://github.com/diyan/pywinrm/
Author: Alexey Diyan
Author-email: alexey.diyan@gmail.com
License: MIT license
Location: /home/ec2-user/.local/lib/python3.7/site-packages
Requires: xmltodict, requests, requests-ntlm, six

$ aws ec2 describe-regions
$ aws ec2 describe-availability-zones --region us-east-2
$ aws ec2 describe-availability-zones --all-availability-zones

$ aws configure
AWS Access Key ID [None]: YOURACCESSKEYID
AWS Secret Access Key [None]: YOURSECRETACCESSKEY
Default region name [None]: us-east-2
Default output format [None]: json

AnsibleFest 2020 digital event kicks off today. The event runs from Oct 13^th – Oct 14^th. You can find the detailed agenda for this event here – https://www.ansible.com/ansiblefest 

The event features a number of talk sessions and talk tracks, as well as live demos and Q&As. Red Hat Ansible is regarded as the de-facto standard for open source automation and orchestration. Red Hat Ansible Tower – the enterprise version of community Ansible – provides support for clustered architecture, enterprise level support from Red Hat, and other enterprise features. Ansible is used by a large number of organizations to implement use cases like infrastructure-as-code, platform as a service, DevSecOps automation and more. Leveraging one of Keyva's integrations like ServiceNow App for Red Hat Ansible Tower, and ServiceNow App for Red Hat OpenShift, you can also tie back automated remediations with your IT Service Management systems. 

Keyva has strategic partnership with Red Hat, and Keyva provides services and offerings around community and enterprise versions of Red Hat Ansible and OpenShift. You can check out some of those offerings here - https://keyvatech.com/red-hat/ or can always reach our team at: info@keyvatech.com to request additional information.  

Kong Summit digital event kicks off today. The event runs from Oct 7^th – Oct 9^th. You can find the detailed agenda for this event here – https://konghq.com/kong-summit/sessions 

The event features a number of talk sessions and talk tracks, as well as 60-minute virtual workshops with hands-on-labs. Kong is very popular open source API gateway, and Kong Enterprise provides enterprise level support for RBAC, API throttling, API monetization, Dev Portal, and many other features. With add-ons like Kong Immunity and Kong Vitals, you can autonomously identify and monitor issues. Leveraging one of Keyva's integrations like ServiceNow App for Kong, you can also tie back automated remediations with your IT Service Management systems. 

Keyva has strategic partnership with Kong, and Keyva provides services and offerings around community and enterprise versions of Kong API gateway. You can check out some of those offerings here - https://keyvatech.com/kong-enterprise/. You can always reach our team at: info@keyvatech.com to request additional information.  

By Brad Johnson, Lead DevOps Engineer

When considering infrastructure automation Terraform and Ansible are usually brought up. Both do some things really well, but also have limitations. Terraform is an infrastructure as code tool, whereas Ansible is a configuration management tool that can also do infrastructure as code. I've had people ask about how the tools compare and which one to use and when, so let's explore these tools and talk about the benefits of each. 

First, why would you use Terraform? The single most important reason is that Terraform, like Ansible, is platform agnostic. This means that if you have a hybrid or multi cloud application or service, you can use terraform to manage the infrastructure in a single repository. Cloud vendor specific solutions like AWS CloudFormation templates work well, but they are limited to using within the platform they are available in. With Terraform's ability to support multiple providers, you can do things like managing the infrastructure code definition of on-premise and AWS/GCP/Azure Cloud VMs, load balancers, DNS, or network configuration in the same set of files. Using a single common configuration language means greater flexibility in transitioning to new environments and reducing vendor lock-in. Another reason you should consider using Terraform is that, unlike Ansible, it works on the principle of understanding the current vs desired state. This means that if you do something like deploy a VM via Terraform, then later delete that block of configuration, Terraform will delete the VM. So your Terraform code is declarative of your infrastructure. With Ansible you would need to write additional code to perform an operation similar to this, as Ansible is not aware of the state from previous runs. Another benefit of Terraform is that you can see what it will do before you run it by using the 'terraform plan' command. 

However, if you already have a significant amount of infrastructure deployed, it can be time consuming to import your current environment to manage under Terraform. You can use it for new deployments without importing existing environment configurations, however it won't be able to manage those existing resources. Terraform also stores the state of what was provisioned in a state file. This means that if there are multiple people working on the code then they must run it out of a single common location with the same state file. Cloud providers can use cloud storage buckets to store the state file. The ideal solution might be using a CI or Orchestration system to run the 'terraform apply' to deploy infrastructure changes, and gating the process via approvals in ITSM. It is critical to ensure actual changes are applied from a single source of truth, like a master git branch. Also, while Terraform is extensible with custom providers ,you will need to write them in Go, which is not yet as widely used as Python. 

Now let's look at why Ansible. The best thing about Ansible is that it can handle a wide variety of configuration and deployment tasks using standard modules and it's easily extensible with Python. You can deploy a VM, use templates in case of custom configuration files, communicate with REST APIs, interact with git repos, and easily configure Linux or custom software all using already available standard modules. Building your own custom Ansible modules, which typically isn't needed given the exhaustive Ansible library, requires minimal programming effort. An example 'hello world' module only requires 4 lines of Python code. Drop the code in a 'library' directory next to your playbook and you're ready to use it. Ansible also comes with 'ansible-vault' which provides a way to store sensitive variables in encrypted yaml files in your playbook repo, which can be decrypted at runtime using a vault password. Because of these features, you can easily implement a wide variety of use cases using Ansible to achieve configuration as code. Some example cases we've used Ansible for include deployment of Linux OS hardening changes to meet security standard compliance,  configuring Apache Tomcat and Oracle Weblogic as part of application server deployment, integrating with ITSM (IT Service Management) and CMDB (Configuration Management Database) platforms, and interacting with silent installers and CLIs using Keyva built custom modules like one for Python Pexpect. 

Now, given that Ansible does not store state of the resources, you will need to write playbooks to handle removal of resources. Meaning, even if you deployed something and Ansible made sure it was 'present', to remove it you would usually need to run the same function with the named resource as 'absent'. For simple things like removing a file, this is easy and you just need to remove the code after it is run once everywhere. For more complex use cases, you can get around this limitation by writing playbooks in a way that queries existing resources into variable lists, compares to what is in Ansible, then removes the items that do not match. However, this would take additional time, is more complex, and does not account for any changes that were made on target resources manually. From a configuration, compliance and remediation standpoint, this may actually be desirable for some organizations. 

What's great about both tools is that they can work with each other. There's no reason to believe that one tool needs to own the whole process. Given their differences in scope, while they can do similar things, they are in no way replacements for the full functionality of the other. Terraform can be set up to run Ansible on a host after provisioning to do the configuration of that host. Likewise, Ansible can use the Terraform module to plan or apply a Terraform project as a step within a playbook. The Ansible module for Terraform also returns the outputs from Terraform as variables that Ansible can consume and use for further action. When designing and implementing  infrastructure-as-code in your environment, it is important to consider which tool is best suited for each part of the task. It is also imperative to consider combining Terraform with Ansible when deploying infrastructure. If you need help getting started or advice on best practices around implementing infrastructure-as-code, please reach out to info@keyvatech.com. 

By Anuj Tuli, CTO

Many organizations that use PowerBI for business insights and analytics have a need to run their reports against various data sources, including workloads that they may have residing in Amazon AWS. There can be a number of various data sources configured for AWS; this blog walks through how to set up connectivity between PowerBI and AWS Aurora MySQL Database.  

Assumptions:  

• You have access to AWS Control Tower 
• You have an account for access PowerBI SaaS (powerbi.com) and to download the client 
• You have Administrator access to the Windows machine that we will use as our MS Data Gateway server within AWS.  

First, let's look at various configurations that we need to set up on the AWS side -  

• Create a new keypair specifically for this work. This is optional. You can choose to use an existing keypair for AWS access.  
• Create Aurora DB with Dev/Test template. This is optional. You can choose to use an existing DB cluster instance. 
• If you don't have a public internet gateway set up for the VPC, we will need to set one up.  
• Create RDS instance connectivity subnet (or leverage one directly inherited via the VPC) set up with the appropriate inbound connection. 
• If you plan on connecting to your Database instance from an external client (i.e. external to private AWS subnet), then you will need to make the Database publicly accessible. This can be done by going to your RDS console, selecting the database instance, select 'Modify' button, and enable 'publicly accessible' property under Additional Networking options. 
• Capture the endpoint name of the Database instance (Reader or Writer) under the Connectivity and Security tab from the RDS console. 

Next, we will configure the PowerBI components -  

• If you don't have an account already, sign up for PowerBI SaaS (powerbi.com)  
• We will spin up a new Microsoft Windows server (we used Server 2019, with .NET framework 4.7+) within our AWS EC2 environment. This Windows server will work as a jump box, as well as the MS Data Gateway that talks to PowerBI SaaS. The AWS keypair will be used to decrypt the Windows 'Administrator' password, so we will associate the public internet gateway with this machine, so we can RDP directly onto this box and download the following components: 
  • Download and install Visual Studio 2019 x64 redistributable (https://aka.ms/vs/16/release/vc_redist.x64.exe) 
  • Download and install MySQL 8.0.21 ODBC connector (https://dev.mysql.com/downloads/connector/odbc/) 
  • Download and install MS Data Gateway installer (https://go.microsoft.com/fwlink/?LinkId=2116849&clcid=0x409) 
  • Download and install PowerBI Desktop Client (https://www.microsoft.com/en-us/download/confirmation.aspx?id=58494) 
• Once the ODBC connector is installed, we will open up the console and add the data source under 'System DSN' tab. This is where we will be defining the connection to AWS Aurora MySQL Database. Click on 'Add' and enter the Database endpoint gathered from AWS, port (default 3306) and associated credentials. You also have the option to 'Test' the connection string. 
• If the Test connection is successful, you are now ready to configure the Gateway instance in PowerBI SaaS.  
• You can use the PowerBI Desktop Client to now create reports and publish them to your PowerBI SaaS tenant. Upon configuring the Gateway instance in PowerBI SaaS, you can also control the schedule for report generation from the Aurora Datasource. 

One of the most common ODBC errors we've seen is when the ODBC connector is unable to connect to the database. This usually happens either because the public subnet for the VPC is not associated with the Windows EC2 instance, or the public accessibility flag for the database is not set.  

If you have any questions or comments on the tutorial content above, or run in to specific errors not covered here, please feel free to reach out to info@keyvatech.com 

Hey everyone! I hope you're staying warm through this historic cold snap. There's no better time than the present to stay indoors and check out some upcoming virtual tech events.

The first event I'd like to mention is coming up this Thursday, Feb 18th, 2021. It's the Open Source North Speaker Series. It's free to attend but you need to register in advance. Here's a look at this Thursday's speaker agenda:

Open Source North Speaker Series

Thursday, February 18, 12:00 - 1:00 PM CST

• Jenna Pederson, AWS, Setting yourself up for failure: cultivating a culture where failure is celebrated
• Jason Scherschligt, SDG, Making the shift: projects to products
• Mark Sielaff, Facebook, Moving Fast and Delivering Quality
• Todd Gardner, TrackJS, Observable Web Applications

For details on each presentation and speaker - AND TO REGISTER, please visit https://opensourcenorth.com/speaker-series

The next event I want to mention is an event coming up with the CTO of Kong , Marco Palladino.

Marco is a dynamic speaker and excellent CTO, always a fun, informative watch.

Kong - Automatic Observability With Service Mesh

Friday, February 26, 11:00 - 12:00 PM CT

In this session you'll learn how a service mesh can observe all of our traffic in new modern applications running on both Kubernetes and virtual machines.

Key takeaways:

• Understanding the service mesh pattern and how it helps to modernize our applications out of the box.
• A live demonstration of a service mesh to extract logs, metrics and traces from our service traffic and store it in Prometheus + Grafana, Jaeger and Elasticsearch.
• Learning how to use the service mesh policies available in CNCF’s Kuma to enable observability in one click.

To register go here: Register Now

The last event I want to draw your attention to this month is focused on the OpenShift Developer Sandbox hosted by Red Hat DevNation during which you will create an account an OpenShift cluster and deploy a sample app. The dev cluster will be available for you to use thereafter for 14 days.

OpenShift Developer Sandbox

Thursday, February 18, 11:00 - 12:00 PM CST

Have you heard of the new OpenShift Developer Sandbox? Join this DevNation Tech Talk where you will be guided through the process of creating an account, creating/configuring your Developer Sandbox cluster, and deploying a sample application on OpenShift. Your OpenShift cluster will be available for your use for 14 days. If you've ever wanted to test out OpenShift, this is your chance to do it!

Produced by the Red Hat Developer team, DevNation Tech Talks are live discussions led by the Red Hat technologists who create our products. Sessions include real solutions and code to help you build with open source, plus sample projects, robust discussion, and live Q&A to help you get started.

Are you new to DevNation Tech Talks? See what you've missed.

To register for this event go here: Register for the OpenShift Sandbox Tech Talk