Why... Why... Why?
This blog is dedicated to documenting error resolution and other tidbits that I discover while working as a Consultant in the Oracle EPM (Hyperion) field. As much of my job revolves around issue resolution, I see an opportunity to supplement the typical troubleshooting avenues such as the Oracle Knowledgebase and Oracle Forums with more pinpointed information about specific errors as they are encountered. Beware, the information found in this blog is for informational purposes only and comes without any warranty or guarantee of accuracy.

EPMVirt: Create your own Oracle Hyperion Virtual Environment:

Sunday, March 10, 2013

Learning EPM - Level Up

I have been spending time lately focused on how to bring seasoned IT folks up to speed in Oracle EPM. I wanted to help identify specific levels of familiarity with EPM and help folks who are in one tier have resources to help go into more depth in their quest for EPM knowledge. The following post is a work in progress. The expectation is that a more polished version will eventually end up as part of a future presentation.

Levels of EPM Skills


Basic Operational Skills -
   
No expectation prior EPM knowledge
   This is good for the typical "on call" resource who is not specializing in EPM.
  1. Familiar with start and stop operations - specific to company
  2. Familiar with the architecture diagrams for each ENV - specific to company
  3. What's up and whats down - able to identify products that are not running and use the startup script to start or restart components - familiar with monitoring specific to company
Basic EPM Admin Skills
  1. Has logged in to each EPM server
  2. Has admin access to each product and has logged into each EPM module
  3. Familiar with EPM modules
    Knows the basic end user functionality of each EPM module
    Comfortable explaining end user interactions
    Can explain the importance of EPM regarding the business and why it is critical for good infrastructure support
  4. Knowledge of jobs which run in the environment - LCM exports, FDM loads, etc. - specific to company
  5. Ability to execute basic validation procedures to determine if the ENV is working as expected after planned maintenance or during issue resolution.
  6. Traversing the Oracle documentation - Read up!
    http://docs.oracle.com/cd/E17236_01/index.htm

    EPM Module Admin Guides
    example: http://docs.oracle.com/cd/E17236_01/epm.1112/esb_dbag.pdf
    EPM Module User Guides
    example: http://docs.oracle.com/cd/E17236_01/epm.1112/sv_user.pdf

    Infrastructure:
    -Support Matrix:   http://docs.oracle.com/cd/E17236_01/epm.1112/epm_install_start_here_1112200.pdf
    -Installation and Configuration
    http://docs.oracle.com/cd/E17236_01/epm.1112/epm_install_1112200.pdf
    -High Availability and Disaster Recovery
    http://docs.oracle.com/cd/E17236_01/epm.1112/epm_high_avail_11121.pdf
    -Security and SSL
    http://docs.oracle.com/cd/E17236_01/epm.1112/epm_security_11121.pdf
  7. Experience Installing and Configuring basic EPM environments
Intermediate EPM Skills
  1. Understands Opatch
    Can evaluate patch readme's
    Understands patch prerequisites and orders of patches
    Knows difference between patch sets vs one off patches.
    Understands where the patches should be applied - potentially 3 different places
       Product web
       Product app
       WebLogic admin node
  2. Understands implications of running config tool
     Resets registry heap settings
     Potentially wipes out custom changes
  3. Understands load balancing and SSL setup
    Key files:
       ssl.conf - basic SSL settings - defines location of wallet and SSL port
       mod_wl_ohs.conf - The Web->WebLogic forwarding
       httpd.conf - Web->IIS fowarding
    Files are auto generated when running web configuration in config tool - custom changes are blown away
  4. Knows what happens behind the scenes during the proces of deploying an application in config tool
  5. Can perform SSL certificate rotation based on predefined procedures
  6. Can perform password rotations
    a. database accounts
    b. EPM system accounts (admin, essadmin, jobs)
    c. windows DCOM account
  7. Familiar with oracle support - able to follow tickets from inception to resolution based on oracle's instructions.
  8. Knowledge of log files locations to troubleshoot issues.
Advanced Skills

  1. Can put together instructions for SSL implementation and rotation
  2. Can create architecture to implement high availability and the architecture can be tested to achieve high availability
  3. Create and refine DR process for EPM
  4. Troubleshooting
    Quickly understands interaction between products and can evaluate error messages from logs to quickly resolve issues
  5. Able to plan and implement a upgrade from one EPM version to another
  6. Begins to train others in the art of EPM infrastructure
EPM Guru (subjective)

  1. Ability to reach out and solve others problems
  2. Takes responsibility for infrastructure issues
  3. Multitasking - understands most EPM modules and is able to help on every front 
  4. Begins to see EPM at the "code" level - can start to evaluate why issues are occurring based on how the code is structured.
  5. Historical reference of EPM - watching EPM evolve over time and multiple versions can help bring invaluable context to situations.


Thursday, March 7, 2013

EPM Config Tool: The next hosts are unreachable.


When opening the config tool in 11.1.2.2 the utility actually tries to connect to every host in the EPM Registry. It show the error message if any hosts are unreachable.

I started down this path,

Checking closer it appears that the config tool uses the Java InetAddress isReachable() method.
Looking into the doc on the class,
http://docs.oracle.com/javase/6/docs/api/java/net/InetAddress.html


public boolean isReachable(int timeout)
                    throws IOException
Test whether that address is reachable. Best effort is made by the implementation to try to reach the host, but firewalls and server configuration may block requests resulting in a unreachable status while some specific ports may be accessible. A typical implementation will use ICMP ECHO REQUESTs if the privilege can be obtained, otherwise it will try to establish a TCP connection on port 7 (Echo) of the destination host.

I wrote up a simple test which confirmed the issue (Can't connect):

import java.net.InetAddress;
public class test
{
    public static void main(String[] args)
    throws Exception
    {

  
     InetAddress address =  InetAddress.getByName("hostname.fullyqualified.com");

     if (address == null) {
         System.out.println("Is null");
     }


     else if (! address.isReachable(3000)) {
         System.out.println("Cant connect");
     }

     else { 
         System.out.println("Host is reachable");
     }

     }


}


The host is pingable so ICMP traffic is being allowed. It's puzzling why the Java method cannot connect.

Further research on the web brings up some articles regarding why the isReachable method might fail, 
http://stackoverflow.com/questions/5126697/java-networking-issue

So this sort of pinpoints a firewall issue - Next I disabled the Windows Firewall - The method began working! 

About this time I came across John Goodwin's article while researching:
John had gone down almost the exact same path, written up nicely in his blog.

Final thoughts: 
  Why does Oracle introduce a new check in the config tool using a strange ICMP/TCP method that has nothing to do with the actual requirements of EPM? Crazy... Assuming you are implementing firewalls - you will need to include this as an additional requirement.