Zum Hauptinhalt springen

Operating the data warehouse

Introduction

The data generated during daily work with ConSol CM is a valuable source of information about your business. Reporting helps you to gain insights from this data and to share these insights with the users.

The following figure illustrates the components required for reporting in ConSol CM:

CMEBIA_Architektur_EN.png

The ConSol CM database and the DWH are two separate databases or database schemes. The CMRF is responsible for transferring the data. If CM/EBIA is used, there is an internal database for user and report data.

CMRF

The CMRF (ConSol CM Reporting Framework) handles the data transfer between the ConSol CM database and the DWH database. It must be installed by either deploying it together with ConSol CM on an application server, or by running it as a standalone Java application. In both cases the communication between ConSol CM and CMRF must be configured in the properties files of the application. See CMRF.

Data warehouse

The data warehouse is a separate database which is filled with data from the ConSol CM database. The data model of the data warehouse is optimized for reporting purposes. This has the advantage that the creation of reports does not impair the performance of the ConSol CM system, even if a large amount of data has to be processed. In addition, the data warehouse can store historical data which has already been removed from the production database.

The Web Admin Suite allows to manage the data warehouse during its regular operation. This includes defining which data is needed for reporting and therefore needs to be available in the data warehouse, and transferring this data to the data warehouse database. In addition, you can create cubes on the database to facilitate report creation. A detailed description of the required steps can be found in the ConSol CM Administrator Manual.

CM/EBIA

CM/EBIA is an add-on which uses a licensed enterprise version of the third-party application Metabase. It is integrated with the Web Client using single sign-on. With CM/EBIA you can create reports based on the data from the DWH directly in the Web Client on a user-friendly graphical user interface. The created reports are shared with the regular users as dashboard widgets.

Common operational tasks

The following components are required to operate ConSol CM with a DWH:

  • ConSol CM server is running and writes data into the ConSol CM database.
  • The CMRF is running and has access to both the ConSol CM and the DWH database.
  • The DWH mode is set to Admin or Live mode. The mode determines the handling of DWH updates and tasks.

Open DWH tasks

Open DWH tasks are saved in the cmas_dwh_task table of the ConSol CM database. You can view and execute them on the Tasks page of the DWH menu in the Web Admin Suite.

Tasks are created for configuration changes, e.g. setting Transfer to DWH to "true" for an entity (data field or relation), or creating a new cube. If the DWH runs in LIVE mode and the checkbox Administrative changes are applied automatically is selected on the Tasks page, the tasks are executed automatically. Otherwise, you need to start them manually on the Tasks page. This setting is also saved in the ConSol CM system property cmas-dwh-server, autocommit.cf.changes.

Monitoring

You can see the status of the DWH on the Monitor page in the Web Admin Suite. In addition, the log files and some tables contain useful information:

  • hlp_parameter (DWH table): dwh_status indicates the status, which should be 4, see Tables in the DWH database for details.
  • int_parameter (DWH table): last-success-live-id indicates the epoch time stamp in milliseconds of the last successful live update of a transfer or update operation.
  • cmas_dwh_synchronization (CM table): dwh_status indicates the status on the ConSol CM side, cmrf_status indicates the status on the DWH side, see DwhSynchronisation.md for details.

Log files

The DWH-specific log messages are written to the cmrf.log file. There are several ways to access it:

  • Open it on the file system:
      <JBOSS_HOME>/standalone/log/cmrf.log
  • View or download it on the Log files page of the Web Admin Suite.
  • View log entries which refer to specific DWH operations by clicking the View log button of the operation on the Administration page of the DWH menu in the Web Admin Suite.

In addition, server.log file entries related to dwh-transfer-service, dwh-live-service, dwh-log-service are relevant for the DWH.

See Log files for details about configuring the logging.