A Look Inside EPMVirt: Automation Scripts - buildEPMVirt

Finally, in this installment, we will look at the automation scripts. They control the actual installation and configuration of the Oracle EPM environment.

The automation scripts get deployed into /u0/automation.
There are three main automation areas:

1. Installation and configuration of the Oracle Database (RDBMS)
2. Installation and configuration of Hyperion/EPM
3. Post install configuration and setup of EPM sample applications

They are denoted by the directories:

  /u0/automation/{apps, database, epm}

We will look at each of these in more detail below and expand on the areas. 

buildEPMVirt

buildEPMVirt the main script that is run when setting up EPMVirt. buildEPMVirt is a wrapper script that calls buildEPMVirt.py in /u0/automation. buildEPMVirt.py is the script that ties all these automation areas together. Let's take a look in more detail.

There is one note to bring up first... The scripts in EPMVirt are in a rather unpolished format. They are not intended to be examples of high quality python code. Rather, they are quick and dirty scripts created in my spare time to help get EPMVirt working. Please excuse the ugly code and hacks.

buildEPMVirt can be broken down simply by this script flow:

1. extractFiles.sh - Unzips all the downloaded files
2. database/installDB.sh - Installes the Oracle Database and configures tablespaces/schemas for EPM
3. epm/installAll.sh - Runs the EPM installer using response files
4. epm/configAll.sh - Runs the EPM configuration using response files
5. apps/insertPlanDS.sh - Creates a planning datasource for the sample planning app. 
6. start all EPM services - Runs EPM start.sh
7. apps/createApps.sh - LCM Imports some sample apps

Lastly, the buildEPMVirt script checks each step for expected output and tries to detect any errors along the way. Ideally, if errors crop up, the script will fail and let you fix the issue. If the script completes, it is a fairly good indication that the install was successful.

The full listing of the buildEPMVirt.py script is below:

from subprocess import call

import os

import sys

import glob

import time

EXPECTED_EPM_INSTALL_PASS_COUNT=72

STEP_NUM = 0

STOP_NUM = 99999

if len(sys.argv) == 2:

  STEP_NUM = int(sys.argv[1])

elif len(sys.argv) == 3:

  STEP_NUM = int(sys.argv[1])

  STOP_NUM = int(sys.argv[2])

if STEP_NUM:

  print "Starting at %s, stopping at %s" % (STEP_NUM, STOP_NUM)

def IsInFile(str, file):

  if not os.path.exists(file):

    return False

  with open(file, 'r') as f:

    return str in f.read()

def CheckDatabase():

    if IsInFile("Connected to:", "/tmp/DBResults"):

      print "DB Connection OK!"

    else:

      print "Db connection failed. Exiting..."

      sys.exit(-1)

def CheckEPMInstall():

  fname = glob.glob("/u0/Oracle/Middleware/EPMSystem11R1/diagnostics/logs/install/installTool-summary*.log")[0]

  with open(fname, 'r') as f:

    contents = f.read()

  pass_count = contents.count("Pass")

  if pass_count == EXPECTED_EPM_INSTALL_PASS_COUNT:

    print "EPM Install Component Count OK."

  else:

    print "EPM Installer failed Pass check (%s vs %s). Exiting..." % (pass_count, EXPECTED_EPM_INSTALL_PASS_COUNT)

    sys.exit(-1)

def CheckEPMConfig():

  if IsInFile("Fail", "/u0/Oracle/Middleware/user_projects/epmsystem1/diagnostics/logs/config/configtool_summary.log"):

    print "EPM Config failed. Exiting..."

    sys.exit(-1)

  else:

    print "EPM Config OK..."

print "Starting at %s, stopping at %s" % (STEP_NUM, STOP_NUM)

my_step = 0

if my_step >= STEP_NUM and my_step < STOP_NUM:

  print "Running step number %s" % my_step

  print "Extracting files..."

  time.sleep(10)

  call(["/u0/automation/extractFiles.sh", ])

  if not os.path.exists("/u0/install/epm/jre"):

    print "Error Exacting Files..."

    sys.exit(-1)

my_step = 1

if my_step >= STEP_NUM and my_step < STOP_NUM:

  print "Running step number %s" % my_step

  print "Installing Database..."

  time.sleep(10)

  call(["/u0/automation/database/installDB.sh", ])

  CheckDatabase()

my_step = 2

if my_step >= STEP_NUM and my_step < STOP_NUM:

  print "Running step number %s" % my_step

  print "Installing EPM..."

  time.sleep(10)

  call(["/u0/automation/epm/installAll.sh", ])

  CheckEPMInstall()

my_step = 3

if my_step >= STEP_NUM and my_step < STOP_NUM:

  print "Running step number %s" % my_step

  print "Configuring EPM..."

  time.sleep(10)

  call(["/u0/automation/epm/configAll.sh", ])

  CheckEPMConfig()

my_step = 4

if my_step >= STEP_NUM and my_step < STOP_NUM:

  print "Running step number %s" % my_step

  print "Configuring APPS (preinstall)..."

  time.sleep(10)

  call(["/u0/automation/apps/insertPlanDS.sh", ])

my_step = 5

if my_step >= STEP_NUM and my_step < STOP_NUM:

  print "Running step number %s" % my_step

  print "Starting EPM..."

  call(["/u0/Oracle/Middleware/user_projects/epmsystem1/bin/start.sh", ])

my_step = 6

if my_step >= STEP_NUM and my_step < STOP_NUM:

  print "Running step number %s" % my_step

  print "Installing APPS..."

  time.sleep(10)

  call(["/u0/automation/apps/createApps.sh", ])

print """

Installation Complete!

"""

Essentially buildEPMVirt is just a wrapper script that ties together a few other automation scripts and does some quick error checking. To get more details we need to look at all the automation scripts listed above. Let's start with extractFiles.sh.

extractFiles.py

The extractFiles.sh script simply calls extractFiles.py while redirecting the output for logging via the "tee" command. Most of the scripts in EPMVirt use the tee command like this to capture output to a log file while displaying the output on the screen.

#!/bin/sh
python /u0/automation/extractFiles.py  2>&1 | tee /u0/automation/extractFiles.log

The basic flow of extractFiles.py is:

Loop through a list of zip files and run the unzip command. 

If a file is not found it should throw an error. This  ensures you have all the files needed to run the EPM install. Finally, there is some ugly code to extract the Oracle DB files into a separate directory than the EPM files. Specifically, EPM is extracted to /u0/install/epm while the database installer is extracted to /u0/install/oracle_db.

 Full contents of extractFiles.py

import os
from subprocess import call
import sys
import getpass 

ISTESTING = False
required = ("linuxamd64_12102_database_se2_1of2.zip",
            "linuxamd64_12102_database_se2_2of2.zip",
            "Foundation-11124-linux64-Part1.zip",
            "Foundation-11124-linux64-Part2.zip",
            "Foundation-11124-Part3.zip",
            "Foundation-11124-linux64-Part4.zip",
            "Essbase-11124-linux64.zip",
            "Apps-11124-linux64.zip",
           )
base_dir = "/u0/install/downloads"
def check_downloads():
  for file in required:
    fpath = "%s/%s" % (base_dir, file)
    if not os.path.exists(fpath):
      if not ISTESTING:
        print "Error, required file not not found, %s" % fpath
        sys.exit()
    else:
      print "Found %s" % fpath
def extract(archive, dest):
  call(["unzip", "-o", "-d", dest, archive])

if not getpass.getuser() == "oracle":
  print "Error, you must be oracle user to run this script."
  sys.exit(-1)

check_downloads()
for file in required:
  fpath = "%s/%s" % (base_dir, file)
  if not os.path.exists(fpath):
    print "Not found, skipping... %s" % fpath
    continue
  epm_path = "/u0/install/epm"
  db_path = "/u0/install/oracle_db"
  if "-111" in os.path.basename(fpath):
    extract(fpath, epm_path)
  else:
    extract(fpath, db_path)
  # get rid of the original download to save space
  if not ISTESTING:
    os.remove(fpath);

Now that we have discussed the wrapper script, buildEPMVirt, and the extractFiles.py (step 1 from above) the next two blog posts will focus on the automated database install (step 2 from above) and the automation scripts for EPM (steps 3-7 from above).