Why... Why... Why?
This blog is dedicated to documenting error resolution and other tidbits that I discover while working as a Consultant in the Oracle EPM (Hyperion) field. As much of my job revolves around issue resolution, I see an opportunity to supplement the typical troubleshooting avenues such as the Oracle Knowledgebase and Oracle Forums with more pinpointed information about specific errors as they are encountered. Beware, the information found in this blog is for informational purposes only and comes without any warranty or guarantee of accuracy.

EPMVirt: Create your own Oracle Hyperion Virtual Environment:

Saturday, January 12, 2013

EPM Log Files Revisited



The topic of log files can be a bit mundane, yet during troubleshooting I find myself constantly pursuing log files. The ability to navigate quickly to the correct log file is crucial for successful administration of the EPM system.

For example, a quick review of the log files on a typical EPM system might include over 100 log files:



\Oracle\Middleware\user_projects\domains\EPMSystem\servers\FoundationServices*\logs\SharedServices_Admin.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\FinancialReporting*\logs\FRLogging.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\FinancialReporting*\logs\FinancialReporting*.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\CalcMgr*\logs\CalcManager.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\CalcMgr*\logs\CalcMgr*.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\FoundationServices*\logs\FoundationServices*.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\FoundationServices*\logs\SharedServices_Admin.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\FoundationServices*\logs\SharedServices_Audit.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\FoundationServices*\logs\SharedServices_Bpmui.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\FoundationServices*\logs\SharedServices_Bpmui_Config.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\FoundationServices*\logs\SharedServices_Bpmui_Sec.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\FoundationServices*\logs\SharedServices_CMSClient.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\FoundationServices*\logs\SharedServices_Hub.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\FoundationServices*\logs\SharedServices_LCM.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\FoundationServices*\logs\SharedServices_Registry.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\FoundationServices*\logs\SharedServices_Security.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\FoundationServices*\logs\SharedServices_TaskFlow.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\FoundationServices*\logs\Workspace.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\FoundationServices*\logs\WorkspaceConfig.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\FoundationServices*\logs\WorkspaceConfig.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\RaFramework*\logs\RaFramework*.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\hfm\hfm.odl.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\hfm\HsvEventLog.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\hfm\InteropJava.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\registry\registry.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\stdout_console_servlets.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9CALC-syserr.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9CALC-sysout.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9FinancialManagementWebSvcs-syserr.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9FinancialManagementWebSvcs-sysout.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9FoundationServices-syserr.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9FoundationServices-sysout.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9FRReports-syserr.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9FRReports-sysout.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9RaFramework-syserr.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9RaFramework-sysout.log
\Oracle\Middleware\user_projects\epmsystem1\httpConfig\ohs\diagnostics\logs\OHS\ohs_component\access_log
\Oracle\Middleware\user_projects\epmsystem1\httpConfig\ohs\diagnostics\logs\OHS\ohs_component\console~OHS~*.log
\Oracle\Middleware\user_projects\epmsystem1\httpConfig\ohs\diagnostics\logs\OHS\ohs_component\ohs_component.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9aifWeb-syserr.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9aifWeb-sysout.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9FinancialManagementWebSvcs-syserr.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9FinancialManagementWebSvcs-sysout.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\ErpIntegrator*\logs\ErpIntegrator*.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\stdout_console_default.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\agent.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\configuration_messages_default.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\eiengine.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_AdministrationServlet.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_AnalyticBridgeService.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_AuthenticationService.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_AuthorizationService.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_BrowseServlet.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_CommonServices.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_CSSSynchronizer.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_DataAccessServlet.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_EventService.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_GSM.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_HarvesterService.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_iHTMLServlet.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_JobManagerServlet.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_JobService.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_LoggingService.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_LSM.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_PersonalPagesServlet.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_PublisherService.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_RepositoryService.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_SearchIndexing.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_SearchKeywordProvider.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_SearchMonitor.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_ServiceBroker.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_ServiceBroker.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_SessionManager.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_TransformerService.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\server_messages_UsageService.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\stdout_console_agent.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\FinancialReporting\FRPrintLogging.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\epma\datasync.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\epma\dimensionServer.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\epma\NetJNIBridge.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9EPMADataSynchronizer-syserr.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9EPMADataSynchronizer-sysout.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9EPMAWebTier-syserr.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9EPMAWebTier-sysout.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\FRPrintService.Err.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\FRPrintService.Out.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9RaFrameworkAgentErr.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9RaFrameworkAgentOut.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\EpmaDataSync*\logs\EpmaDataSync*.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\EpmaWebReports*\logs\EpmaWebReports*.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\AnalyticProviderServices*\logs\AnalyticProviderServices*.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\EssbaseAdminServices*\logs\easserver.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\EssbaseAdminServices*\logs\EssbaseAdminServices*.log
\Oracle\Middleware\user_projects\domains\EPMSystem\servers\Planning*\logs\Planning*.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9aps-syserr.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9aps-sysout.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9eas-syserr.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9eas-sysout.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9Planning-syserr.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\services\HyS9Planning-sysout.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\hfm\HsvEventLog.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\hfm\hfm.odl.log
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\hfm\InteropJava.log'


Let's focus on how to sift through the logs more quickly: 
Types of logs: 
  1. WebLogic Application Logs
    These logs come from a web deployment and are located in the domains folder:
    Oracle\Middleware\user_projects\domains\EPMSystem\servers
  2. EPM Application Logs
    Once the service or application server is started properly, its internal logging takes over using the Oracle ODL logging framework. These log files are typically in the diagnostics folder, but may also reside in the domains folder or elsewhere based on the ODL configuration.
    \Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs

WebLogic Service Not Starting
For web application based services check out the specific WebLogic application log under the domains folder. If the web app is not able to start correctly, chances are it is not writing the EPM application logs because the logger was not initialized or the underlying service never came up. In the Application server log you will see the server go through different states like STATE_PREPARED, etc. During these phases the application does things like setup JDBC connections to the database and tries to initialize its services based on configuration details within the HSS registry. The server is available once you see a message like "server started in PRODUCTION mode".


Workspace Errors After Logging On
Workspace has two components required to log on, the Framework Services and the Framework/Foundation Web Services. For the Web components check the domain logs for Framework and Foundation.
However, the Agent services must start correctly, Check for the message "Giving control to the ORB" in the log:
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\stdout_console_default.log
Typically, if BI will not start - some type of exception is generated during the AGENT start up in stdout_contsole*.log


Errors in a Running Service
Usually once a web service has been started the domain logs become less important because the EPM application logging starts to take over. Assuming the service has started check for the EPM Applicaiton logs in the diagnostics folder for the particular product as the WebLogic Application log usually stops giving useful messages after "server started in PRODUCTION mode".


HFM Logs
HFM has historically been an outlier with respect to log setup. The best place to look for HFM specific errors is to use the HFMErrorLogViewer utility. This utility accumulates and formats the somewhat cryptic messages left in the Windows Registry and the HFM Database for human readable display. Remember, if you have the HFM Web service on a different node than the HFM Application node - the web server HFMErrorLogViewer might have different information.

In 11.1.2.2 HFM has a WebLogic component so look in the domain logs.


FDM Logs
FDM is an IIS deployed application. It logs generic errors to the Windows Registry. During run-time specific operations like loads may generate specific log messages. The log files are viewable by selecting the log file in the FDM web menu. Additionally this log file can be found by searching for *.log within the FDM Application's share directory, prefixed with the username of the user running the operation. Additional logging regarding adapter messages can be coaxed out by setting logging levels with the adapter settings for things like ERPI.

Additional information regarding troubleshooting using the logs can be found in my other posts:
Oracle-EPM-Troubleshooting-and-Debugging-Guide-(Part-1-of-2)
Oracle-EPM-Troubleshooting-and-Debugging-Guide-(Part-2-of-2)

By at
Labels:

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)