Automating IBM Websphere

Peter Gatt
Aug 9, 2016

Inline with Vibrato's mission to automate everything, Alvin Oh explores how to use CHEF to automate IBM Websphere (WAS v8.5) in a Microsoft Windows environment.

CHEF is a configuration management tool that is written in Ruby and Erlang and it’s used to streamline tasks to configure servers. It can be easily integrated to popular cloud-based platforms such as Microsoft Azure, Amazon Web Services (AWS) and Google Cloud Platform to automatically provision and maintain existing servers.

But can it handle automating installation of IBM Websphere (WAS v8.5) in a Microsoft Windows environment? While the answer is YES, we're going to have to jump over a few hurdles:

  • Deciphering the WAS v8.5 response file to execute can be painful and time-consuming. Often it doesn not consist of all of the relevant features that are required.
  • By default, the WINRM listener is not configured on Windows environments. It needs to be pre-configured together with ICF (Internet Connection Firewall) to allow Chef execution on that environment.
  • Troublesome  installation process (IBM HTTP Server, WAS v8.5, Installation Manager, WAS v8.5 Fixed-Pack) that requires multiple zip-files installation onto same directory bundling WAS Application Server and Profiles.
  • Installation of WAS v8.5 as a non-administrator on Windows environment is not recommended, as execution of certain operating system policies restrictions will apply.
  • No generic CHEF cookbooks available in the supermarket ( to cater for installation and configuration of WAS v8.5. It needs significant modification to selective CHEF cookbooks to cater for this installation.

In this article, I’ll describe how we overcame these hurdles using Chef and what the recommended ways are to get IBM Websphere (WAS v8.5) to work with Windows. As well as making it a standardised repeatable process that can be executed and extended across hosted infrastructure providers, or within enterprise on-premise infrastructure.

The following diagram shows the basic high-level components and the main interactions between them:


  1. Using either the Microsoft Azure or the AWS (Amazon) platform, the Chef Server is installed on a single instance.
  2. The local workstation has chefdk (Development Development Tool) and Berkshelf installed to interact with the Chef Server via the command line.
  3. A separate node / instance is created and Chef recipes are used to install WAS 8.5 on that instance.
  4. The installation is verified using a remote desktop connection or through the WAS 8.5 web admin user interface.
  5. Hashicorp Packer is used to build standardised images for these environments

Building standardised Windows Image

Let’s begin by building a standardised Windows environment image that has the WINRM listener pre-configured with relevant ports enabled for Chef execution. Hashicorps Packer provides a nice straightforward way to achieve this as a repeatable and reliable. 

Hashicorp Packer ( is a tool for creating Operating System images for multiple platforms that’s governed from a single source configuration file. The advantages of using Packer include the creation of Operating System images that are automated and with multiple providers for portability into multiple Cloud-based or hosted platforms. If you're interested in know more checkout Hashicorp's page on the support of Packer with Microsoft Azure:

Let’s start off by creating a Packer image template, which is basically a simple .JSON file. If you compare the Azure and AWS resources in the Packer file, you'll notice that there aren’t many differences between them except for the different variables and builder formats that are used for creating images within these two environments.

AWS (Amazon) Environment (windows-2012r2-based-aws.json)

  "variables": {
    "aws_access_key"        : "",
    "aws_secret_key"        : "",
    "aws_vpc_id"            : "",
    "aws_subnet_id"         : "",
    "aws_security_group_id" : "",
    "aws_environment_tag"   : "",
    "aws_implementer_tag"   : ""
  "builders": [{
    "name"              : "win-2012r2",
    "type"              : "amazon-ebs",
    "access_key"        : "",
    "secret_key"        : "",
    "region"            : "ap-southeast-2",
    "source_ami"        : "ami-a1f7d5c2",
    "vpc_id"            : "",
    "subnet_id"         : "",
    "security_group_id" : "",
    "instance_type"     : "c3.xlarge",
    "ami_name"          : "w2012r2-winrm ",
    "user_data_file"    : "ws2012-setup.ps1",
    "tags": {
               "Implementer"     : "",
               "OperatingSystem" : "Windows 2012 R2",
               "Environment"     : "",
               "Description"     : "Based on Microsoft's Official Windows AMI"
  "provisioners": [

Microsoft Azure Environment (windows-2012r2-based-azure.json)

  "variables": {
    "sn": "",
    "ps": "",
    "sa": ""
  "builders": [
      "type": "azure",
      "publish_settings_path": "",
      "subscription_name": "",
      "storage_account": "",
      "storage_account_container": "images",
      "os_type": "Windows",
      "os_image_label": "Windows Server 2012 R2 Datacenter, February 2016",
      "location": "Central US",
      "instance_size": "Small",
      "user_image_label": "PackerMade_Windows2012R2DC"
  "provisioners": [
      "type": "azure-custom-script-extension",
      "script_path": "./ws2012-setup.ps1"

Note that I have chosen NOT to separate the provisioners in the file (ws2012-setup.ps1), as it should be standardised regardless on which environments it would build to. Packer will execute the provision file during the image creation process and that’s where we pre-configure our WINRM listeners and ports:

# Setting up WinRM
makecert.exe -sk "" -ss My -sr localMachine -pe -r -n ", O=myname" -a sha1 -eku ""
winrm quickconfig -q
winrm quickconfig -transport:http
powershell.exe -command "$cert=Get-ChildItem cert:LocalMachine/My ; New-WSManInstance winrm/config/Listener -SelectorSet @{Transport='HTTPS';Address='*'} -ValueSet @{Hostname='devops-win';CertificateThumbprint=$cert.Thumbprint;Port='5986'}"
winrm set winrm/config @{MaxTimeoutms="1800000"}
winrm set winrm/config/winrs @{MaxMemoryPerShellMB="300"}
winrm set winrm/config/service/auth @{Basic="true"}
winrm set winrm/config/service @{AllowUnencrypted="true"}

I found using a self-signed functionality (makecert) from Windows SDK is sufficient enough to configure WINRM and so it could enable PowerShell remote execution on the Windows machine. There are other ways in making it more secure by securing WINRM for HTTPS transport protocol or even upgrading to a trusted CA signed certificate. But with Windows, it can often be unclear whether you are fighting a connection or authentication problem so making it simple would help you to eliminate these issues.

# Enabled WinRM ports
netsh advfirewall firewall add rule name="winrm-t5985" protocol=TCP dir=in localport=5985 action=allow enable=yes
netsh advfirewall firewall add rule name="winrm-t5986" protocol=TCP dir=in localport=5986 action=allow enable=yes
# Restarting WinRM
net stop winrm
sc config winrm start= auto
net start winrm
# Relax the powershell execution policy
powershell.exe -Command "Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Force"

By default, WINRM uses ports 5985 (HTTP) and 5986 (HTTPS) as we enabled both to allow flexibility towards remote connectivity. The Set-ExecutionPolicy cmdlet determine which PowerShell scripts are allowed to be executed on the Windows machine. By enabling the policy to RemoteSigned, typically all downloaded scripts must be signed by a trusted publisher before it can be executed and by using the –Force parameter it would not asked to conform a confirmation.

There you have it. This is your first step towards automating IBM WebSphere (v8.5) on a Windows environment. I cannot remember how much I stressed to customers that base lining your OS images first before applying any configuration management is crucial in making environments – repeatable.  

Side-Note: If you are running a Masterless Chef (chef-solo), you might want to install the chefdk ( during the Packer image baking process. It would save time in prepping for your Windows environment in the long run.

Enabling IBM WebSphere (v.8.5) to work with Chef

In the previous guide, we talked about the general structure of using Hashicorp Packer to build our Operating System image and the relevance and importance on achieving these objectives.

The next step is a brief introduction on setting up a centralised Chef Server v12.0 that would store and execute all configuration instructions and nodes profiling information. We would follow up by discussing in details on how we managed to install and configure WAS 8.5 with the relevant Chef recipes and bootstrapping a Chef node into the management of the Chef ecosystem.

Installing and Configuring Chef Server (v12.0)

Chef recommends at least 4 cores and 4GB of RAM with 50GB local disk (minimum) to install the Chef Server. At this present moment, Chef Server 12 is only available on the ‘Linux’ platform and its free up to 25 nodes before requiring to purchase additional nodes.

Chef Server operates on a resolvable fully qualified domain name (FQDN) pertaining to an IP address. It should be reachable via an address but if it’s not the case, the domain name with the IP address of the Chef Server needs to be set by modifying the host file:

sudo vi /etc/hosts

The result should be a resolvable domain that the Chef Server can be reach anywhere in your current infrastructure.

Once the Chef Server instance is provisioned, we can install and configure the Chef Server.  As root on the server, perform the following commands (note that it uses the wget command to download the Chef Server 12 package (Red Hat Enterprise Linux 7) and configure it)

sudo rpm –ivh chef-server-core-12.6.0-1.el7.x86_64.rpm
sudo chef-server-ctl reconfigure

Next, we will need to create an admin user and an organisation. It would generate two .pem files (admin.pem and organisation.pem) that are required by the administrator workstation (the computer from which cookbooks and nodes are managed) to gain access through Knife and Berkshelf command to the Chef Server.

sudo chef-server-ctl user-create USERNAME FIRST_NAME LAST_NAME EMAIL PASSWORD -f admin.pem
sudo chef-server-ctl org-create SHORTNAME LONGNAME --association_user USERNAME -f organisation.pem

For now though, our current Chef Server 12 installation is complete.

Working on IBM WAS 8.5 chef cookbook recipes

It’s TRUE that automating WAS 8.5 can be a nightmare and you could end up deep inside a 'rabbit hole' debugging on why WAS 8.5 wasn’t installed properly if you decide to take shortcuts in the installation process.

In my experience, there are no shortcuts in installing WAS 8.5 and it would be wise to install WAS 8.5 manually first before trying to automate it in Chef, especially on Windows. IBM provides a baseline WAS 8.5 installation response file but it’s so plain that most of the WAS 8.5 features required would not be installed. The WAS 8.5 cookbook recipes require the WAS 8.5 response file as a based template to standardise the installation of WAS 8.5 on multiple environments.

a. Response file provided by IBM

<?xml version="1.0" encoding="UTF-8"?>
<agent-input clean="true" temporary="true">
<install modify='false'>
<offering id='' profile='IBM WebSphere Application Server V8.5'
  features='' installFixes='none'/>
<profile id='IBM WebSphere Application Server V8.5' installLocation='C:\Program Files\IBM\WebSphere\AppServer'>

b. Response file after successful manual Installation

 <?xml version='1.0' encoding='UTF-8'?>
        <variable name='sharedLocation' value='D:\IM\IMShared'/>
        <repository location='D:\temp\WAS_V8.5.5'/>
        <repository location='D:\temp\WS_SDK_JAVA_TEV7.0_WAS_8.5.5'/>
        <repository location='D:\temp\8.5.5-WS-WAS-FP0000007'/>
    <profile id='IBM WebSphere Application Server V8.5' installLocation='D:\WebSphere\AppServer'>
        <data key='eclipseLocation' value='D:\WebSphere\AppServer'/>
        <data key='user.import.profile' value='false'/>
        <data key='cic.selector.os' value='win32'/>
        <data key='cic.selector.arch' value='x86'/>
        <data key='' value='win32'/>
        <data key='' value='en'/>
    <install modify='false'>
        <!-- IBM WebSphere Application Server -->
        <offering profile='IBM WebSphere Application Server V8.5' id='' version='8.5.5007.20150820_2101' features='core.feature,ejbdeploy,thinclient,embeddablecontainer,' installFixes='none'/>
        <!-- IBM WebSphere SDK Java Technology Edition (Optional) -->
        <offering profile='IBM WebSphere Application Server V8.5' id='' version='7.0.4001.20130510_2103' features='' installFixes='none'/>
    <preference name='' value='${sharedLocation}'/>
    <preference name='' value='30'/>
    <preference name='' value='45'/>
    <preference name='' value='0'/>
    <preference name='offering.service.repositories.areUsed' value='true'/>
    <preference name='' value='false'/>
    <preference name='' value='false'/>
    <preference name='http.ntlm.auth.kind' value='NTLM'/>
    <preference name='http.ntlm.auth.enableIntegrated.win32' value='true'/>
    <preference name='' value='true'/>
    <preference name='' value='false'/>
    <preference name='PassportAdvantageIsEnabled' value='false'/>
    <preference name='' value='false'/>
    <preference name='' value='false'/>
    <preference name='' value='true'/>
    <preference name='' value='true'/>
    <preference name='' value='true'/>

c. Template response file using Chef coding standard

 <?xml version="1.0" encoding="UTF-8"?>
<agent-input clean="true" temporary="true">
        <repository location='<%= @WAS_REPO %>'/>
        <% if node['ibm']['was']['install_java7'] == 'true' %>
        <repository location='<%= @JAVA_REPO %>'/>
        <% end %>
    <install modify='false'>
        <offering id='<%= @OFFERING %>' profile='<%= @PROFILE %>' features='<%= @FEATURE_LIST %>' installFixes='none'/>
        <% if node['ibm']['was']['install_java7'] == 'true' %>
        <offering id='' profile='<%= @PROFILE %>' features='' installFixes='none'/>
        <% end %>
    <profile id='<%= @PROFILE %>' installLocation='<%= @INSTALL_LOCATION %>'>
        <data key='eclipseLocation' value='<%= @INSTALL_LOCATION %>'/>
        <data key='user.import.profile' value='false'/>
        <data key='cic.selector.os' value='win32'/>
        <data key='cic.selector.arch' value='x86'/>
        <data key='' value='win32'/>
        <data key='' value='en'/>
    <preference name='' value='30'/>
    <preference name='' value='45'/>
    <preference name='' value='0'/>
    <preference name='offering.service.repositories.areUsed' value='true'/>
    <preference name='' value='false'/>
    <preference name='' value='false'/>
    <preference name='http.ntlm.auth.kind' value='NTLM'/>
    <preference name='http.ntlm.auth.enableIntegrated.win32' value='true'/>
    <preference name='' value='true'/>
    <preference name='' value='false'/>
    <preference name='PassportAdvantageIsEnabled' value='false'/>
    <preference name='' value='false'/>
    <preference name='' value='false'/>
    <preference name='' value='true'/>
    <preference name='' value='true'/>
    <preference name='' value='true'/>

The solution is to manually install WAS 8.5, selecting all the features required and generate the response file prior to the completion of the installation. Hence, template it up as .erb file as baseline install for Chef. That solves the first part of the solution but even before the correct response file could be generated, but what are the installation steps to successfully install a WAS 8.5 Application Server?

Based on documentation and recommendation from IBM, there are 2 installation paths that we could undertake when installing WAS 8.5:

IM (IBM Installation Manager) -> IBM WAS 8.5 -> IBM WAS 8.5-FP (Fixed Pack) -> Profiles (Config)
IM (IBM Installation Manager) -> IHS (IBM HTTP Server) -> IBM WAS 8.5-FP (Fixed Pack) -> Profiles (Config)

The IBM HTTP Server (IHS) is a full feature web server that includes IBM WAS 8.5 as part of its package. Therefore, it depends on the features required by the organisation to which installation path they would undertake. The automation process in Chef shouldn’t differ from these installation processes that are recommended by IBM. Chef is merely a configuration management tool that governs the way each product is installed and configured.

Let’s take a closer look inside the WAS 8.5 recipes on how they are automated using Chef:

Every cookbook requires a metadata.rb file that is located at the top of every cookbook directory structure. The WAS 8.5 metadata content provides a configuration detail to the Chef Server in ensuring its node is deployed correctly. It defines the authenticity of this particular cookbook (name, maintainer, version) but the most important field is the depends field. It shows the WAS 8.5 cookbook is dependant on another cookbooks. The most important dependency WAS 8.5 cookbook requires is the 'IM' cookbook.

# cookbook-was855/metadata.rb
name             'was855'
maintainer       'ITS'
maintainer_email ''
license          'All rights reserved'
description      'Installs/Configures was'
long_description 'Installs/Configures was'
version          '0.1.8'
source_url       '' if respond_to?(:source_url)
depends          'im'
depends          'windows'
depends          's3_file'

The IM cookbook installs and configures the IBM Installation Manager that is written specifically for Windows environment. Without it, WAS 8.5 wouldn’t be able to execute installation using its response file. Every mentioned process above consists of components that have their own downloadable packages that require installation. The pain-points are the location of packages, download durations (significant in size) and idempotency.

We placed our installation packages into a S3 Bucket or similarly Windows Azure Blob Storage with security rights to prohibit unauthorised access to these packages. Hence, it’s on the same network so the downloading speed shouldn’t be compromised. Chef cookbooks are meant to be able to run on an interval set by users and only apply new changes if they have been updated. So idempotency is really key part of developing correct Chef cookbooks. 

# cookbook-im/recipes/default.rb
zip_file = s3_file "#{working_folder}/#{filename}" do
  remote_path "#{node['ibm']['im']['s3_path']}/#{filename}"
  bucket node['ibm']['im']['bucket']
  action :create
  not_if do
windows_zipfile filename do
  path "#{working_folder}/im"
  source "#{working_folder}/#{filename}"
  action :unzip
  only_if { zip_file.updated_by_last_action? }

Note the used of syntax here (only_if, updated_by_last_action, not_if) to ensure idempotency. 

With the IBM Installation Manager cookbook successfully written, the WAS 8.5 cookbook in the same vein, follows the same Chef coding standards and installation process as shown below:

# cookbook-was855/recipes/install.rb
# Prepare the silent install response file
template_file = template response_file do
  source "WASv85.#{edition}.install.xml.erb"
    FEATURE_LIST: node['ibm']['was']['features'],
    REPO_CONFIG: "#{working_folder}/#{edition}/repository.config",
    INSTALL_DIR: node['ibm']['was']['install_dir'],
    IM_SHAREDLOATION: node['ibm']['was']['im_sharedLocation']
  not_if do
# cookbook-was855/recipes/fixpack.rb
# Start the WAS installation
execute 'installFIX' do
  cwd "#{node['ibm']['was']['im_install_dir']}/eclipse/tools"
  command ".\\imcl -acceptLicense input #{response_file} -log"\
    " #{working_folder}/WASv85.was.update.log.xml"
  only_if { template_file.updated_by_last_action? }
# cookbook-was855/recipes/profile.rb
# create a profile and windows service
execute 'createWASProfile' do
   cwd "#{node['ibm']['was']['install_dir']}\\bin"
   command '.\\manageprofiles.bat -create '\
     "-profileName #{node['ibm']['was']['profile_dmgr']['profile_name']} "\
     "-profilePath \"#{node['ibm']['was']['profile_dmgr']['profile_dir']}\" "\
     "-templatePath \"#{node['ibm']['was']['profile_dmgr']['profile_template']}\" "\
     "-nodeName #{node['ibm']['was']['node_name']} "\
     "-hostName #{node['ibm']['was']['host_name']} "\
     "-cellName #{node['ibm']['was']['cell_name']} "\
     "-serverName #{node['ibm']['was']['server_name']} "\
     '-enableAdminSecurity true -defaultPorts '\
     "-adminUserName \"#{node['ibm']['was']['profile_dmgr']['admin_user']}\" "\
     "-adminPassword \"#{node['ibm']['was']['profile_dmgr']['admin_password']}\" "
   not_if do
     File.exist?(node['ibm']['was']['profile_dmgr']['profile_dir'] +
   notifies :run, 'execute[createWASService]', :immediately

The installation of WAS 8.5 runs against the response file that was generated previously and uses IBM Installation Manager to patch it to the latest stable version. Finally, the creation of a WAS profile to manage the WAS 8.5 Application Server (stop & start).

One last question remains – how does Chef know when the installation and configuration has been successfully completed?

Basically, what we are automating is a ‘SILENT’ mode install. As WAS 8.5 has many log files spreading across different directories, we found the easiest way to verify a successful installation is by checking for the .swtag files. All the above code snippets show verifying the existence of these files which if found indicate a successful installation.

The difficult part of automating WAS 8.5 in Chef is not the process nor the coding development standards but its verification and checking process in determining whether WAS 8.5 Application Server was installed properly.

The code base may look straightforward and simple but it took long hours and many failed attempts to successfully get it working on a Window environment.

Bootstrapping WAS node with Knife

With our Chef Server and WAS 8.5 recipes finalised, we can begin using Chef to bootstrap our newly created WAS node or Chef term (Chef client). Remember the two .pem keys (admin.pem and organisation.pem) that we generated from the Chef Server?

Those keys need to be used to configure Chef knife on the administrator workstation by opening up a file called knife.rb in your text editor:

# ~/.chef/knife.rb
log_level :info
log_location STDOUT
node_name 'admin'
client_key '~/.chef/admin.pem'
validation_client_name 'chef-validator'
validation_key '~/.chef/chef-validator.pem'
chef_server_url 'https://chef-server:443'
syntax_check_cache_path '~/.chef/syntax_check_cache'

In order to run chef knife from the administrator workstation to bootstrap the WAS node, chefdk ( together with knife-windows ( must be installed on a Windows machine.

*Note we found that currently knife-windows plugin does not cater for operating behind a proxy! 

Most enterprise organisations Vibrato consults, usually operates behind an Internet proxy server and chef knife-windows plugin didn’t cater for that.

I would like to give a shout out to Tristan Morgan, our Vibrato automation engineer in fixing this problem for Chef ( so the public is able to bootstrap chef nodes behind a proxy server on Windows environments with the latest knife-windows version release from Chef.

Once the knife.rb has been configured, running this command would cause the Chef client to be installed on the WAS node, register it to the Chef Server and download and run all the recipes for its defined role.

knife bootstrap <node IP address>
   --node-name <was node name>
   --ssh-user root
   --ssh-password <password>
   --run-list role[was]

As I mentioned earlier, all recipes are expected to be idempotent so re-running them any number of times won’t change the final results unless the recipes are updated or there are manual changes on the server itself those are outside Chef. In this scenario, Chef would revert back the changes to its original state as specified in the recipes.


After following this blog post, you should have a basic understanding on how I used Chef to automate the WAS 8.5 installation in a Windows environment and the process I used to create a repeatable procedure that could be adopted on multiple different environments.

One last note, there are no shortcuts. I highly reccomend to follow the instructions produced by IBM and try to install it manually first before automating it.

If all else fails and you seem to be stuck debugging in a rabbit hole – Document what you have achieved so far and restart from scratch.

If you want to learn about how Chef can help you automate, especially in an Windows environment, check them out at


Did you find this article useful? If so, explore our Vibrato blog more more helpful tips and solutions.