Why... Why... Why?
This blog is dedicated to documenting error resolution and other tidbits that I discover while working as a Consultant in the Oracle EPM (Hyperion) field. As much of my job revolves around issue resolution, I see an opportunity to supplement the typical troubleshooting avenues such as the Oracle Knowledgebase and Oracle Forums with more pinpointed information about specific errors as they are encountered. Beware, the information found in this blog is for informational purposes only and comes without any warranty or guarantee of accuracy.

EPMVirt: Create your own Oracle Hyperion Virtual Environment:

Sunday, January 1, 2017

A Look Inside EPMVirt: Automation Scripts - buildEPMVirt

Finally, in this installment, we will look at the automation scripts. They control the actual installation and configuration of the Oracle EPM environment.

The automation scripts get deployed into /u0/automation.
There are three main automation areas:

  1. Installation and configuration of the Oracle Database (RDBMS)
  2. Installation and configuration of Hyperion/EPM
  3. Post install configuration and setup of EPM sample applications
They are denoted by the directories:
  /u0/automation/{apps, database, epm}

We will look at each of these in more detail below and expand on the areas. 


buildEPMVirt the main script that is run when setting up EPMVirt. buildEPMVirt is a wrapper script that calls buildEPMVirt.py in /u0/automation. buildEPMVirt.py is the script that ties all these automation areas together. Let's take a look in more detail.

There is one note to bring up first... The scripts in EPMVirt are in a rather unpolished format. They are not intended to be examples of high quality python code. Rather, they are quick and dirty scripts created in my spare time to help get EPMVirt working. Please excuse the ugly code and hacks.

buildEPMVirt can be broken down simply by this script flow:
  1. extractFiles.sh - Unzips all the downloaded files
  2. database/installDB.sh - Installes the Oracle Database and configures tablespaces/schemas for EPM
  3. epm/installAll.sh - Runs the EPM installer using response files
  4. epm/configAll.sh - Runs the EPM configuration using response files
  5. apps/insertPlanDS.sh - Creates a planning datasource for the sample planning app. 
  6. start all EPM services - Runs EPM start.sh
  7. apps/createApps.sh - LCM Imports some sample apps
Lastly, the buildEPMVirt script checks each step for expected output and tries to detect any errors along the way. Ideally, if errors crop up, the script will fail and let you fix the issue. If the script completes, it is a fairly good indication that the install was successful.

The full listing of the buildEPMVirt.py script is below:
from subprocess import call
import os
import sys
import glob
import time


STOP_NUM = 99999
if len(sys.argv) == 2:
  STEP_NUM = int(sys.argv[1])
elif len(sys.argv) == 3:
  STEP_NUM = int(sys.argv[1])
  STOP_NUM = int(sys.argv[2])

  print "Starting at %s, stopping at %s" % (STEP_NUM, STOP_NUM)

def IsInFile(str, file):
  if not os.path.exists(file):
    return False
  with open(file, 'r') as f:
    return str in f.read()

def CheckDatabase():
    if IsInFile("Connected to:", "/tmp/DBResults"):
      print "DB Connection OK!"
      print "Db connection failed. Exiting..."

def CheckEPMInstall():

  fname = glob.glob("/u0/Oracle/Middleware/EPMSystem11R1/diagnostics/logs/install/installTool-summary*.log")[0]
  with open(fname, 'r') as f:
    contents = f.read()
  pass_count = contents.count("Pass")
    print "EPM Install Component Count OK."
    print "EPM Installer failed Pass check (%s vs %s). Exiting..." % (pass_count, EXPECTED_EPM_INSTALL_PASS_COUNT)

def CheckEPMConfig():
  if IsInFile("Fail", "/u0/Oracle/Middleware/user_projects/epmsystem1/diagnostics/logs/config/configtool_summary.log"):
    print "EPM Config failed. Exiting..."
    print "EPM Config OK..."

print "Starting at %s, stopping at %s" % (STEP_NUM, STOP_NUM)
my_step = 0

if my_step >= STEP_NUM and my_step < STOP_NUM:
  print "Running step number %s" % my_step
  print "Extracting files..."
  call(["/u0/automation/extractFiles.sh", ])
  if not os.path.exists("/u0/install/epm/jre"):
    print "Error Exacting Files..."

my_step = 1
if my_step >= STEP_NUM and my_step < STOP_NUM:
  print "Running step number %s" % my_step
  print "Installing Database..."
  call(["/u0/automation/database/installDB.sh", ])

my_step = 2
if my_step >= STEP_NUM and my_step < STOP_NUM:
  print "Running step number %s" % my_step
  print "Installing EPM..."
  call(["/u0/automation/epm/installAll.sh", ])

my_step = 3
if my_step >= STEP_NUM and my_step < STOP_NUM:
  print "Running step number %s" % my_step
  print "Configuring EPM..."
  call(["/u0/automation/epm/configAll.sh", ])

my_step = 4
if my_step >= STEP_NUM and my_step < STOP_NUM:
  print "Running step number %s" % my_step
  print "Configuring APPS (preinstall)..."
  call(["/u0/automation/apps/insertPlanDS.sh", ])

my_step = 5
if my_step >= STEP_NUM and my_step < STOP_NUM:
  print "Running step number %s" % my_step
  print "Starting EPM..."
  call(["/u0/Oracle/Middleware/user_projects/epmsystem1/bin/start.sh", ])

my_step = 6
if my_step >= STEP_NUM and my_step < STOP_NUM:
  print "Running step number %s" % my_step
  print "Installing APPS..."
  call(["/u0/automation/apps/createApps.sh", ])
print """
Installation Complete!
Essentially buildEPMVirt is just a wrapper script that ties together a few other automation scripts and does some quick error checking. To get more details we need to look at all the automation scripts listed above. Let's start with extractFiles.sh.


The extractFiles.sh script simply calls extractFiles.py while redirecting the output for logging via the "tee" command. Most of the scripts in EPMVirt use the tee command like this to capture output to a log file while displaying the output on the screen.
python /u0/automation/extractFiles.py  2>&1 | tee /u0/automation/extractFiles.log
The basic flow of extractFiles.py is:

Loop through a list of zip files and run the unzip command. 
If a file is not found it should throw an error. This  ensures you have all the files needed to run the EPM install. Finally, there is some ugly code to extract the Oracle DB files into a separate directory than the EPM files. Specifically, EPM is extracted to /u0/install/epm while the database installer is extracted to /u0/install/oracle_db.

 Full contents of extractFiles.py
import os
from subprocess import call
import sys
import getpass
required = ("linuxamd64_12102_database_se2_1of2.zip",
base_dir = "/u0/install/downloads"
def check_downloads():
  for file in required:
    fpath = "%s/%s" % (base_dir, file)
    if not os.path.exists(fpath):
      if not ISTESTING:
        print "Error, required file not not found, %s" % fpath
      print "Found %s" % fpath
def extract(archive, dest):
  call(["unzip", "-o", "-d", dest, archive])

if not getpass.getuser() == "oracle":
  print "Error, you must be oracle user to run this script."

for file in required:
  fpath = "%s/%s" % (base_dir, file)
  if not os.path.exists(fpath):
    print "Not found, skipping... %s" % fpath
  epm_path = "/u0/install/epm"
  db_path = "/u0/install/oracle_db"
  if "-111" in os.path.basename(fpath):
    extract(fpath, epm_path)
    extract(fpath, db_path)
  # get rid of the original download to save space
  if not ISTESTING:

Now that we have discussed the wrapper script, buildEPMVirt, and the extractFiles.py (step 1 from above) the next two blog posts will focus on the automated database install (step 2 from above) and the automation scripts for EPM (steps 3-7 from above). 

No comments:

Post a Comment