Synchronization mechanism

Synchronization mode

The synchronization mode is determined when activating the data warehouse on the Administration page of the DWH menu in the Web Admin Suite. The following values are available for the DWH mode:

• Off: The data warehouse is disabled.

• Admin: The data warehouse is enabled, but the data is not updated automatically. The administrator needs to create an update action to start synchronization.

• Live: The data warehouse is enabled and the data is updated automatically. This means that the required update actions are created and executed automatically. Therefore, the data is always up-to-date without administrator actions.

If you run the DWH in LIVE mode, you can choose whether tasks, which are created after configuration changes, should be executed automatically or manually, see Open DWH tasks.

Components involved in transfer

Tables in the CM database

• cmas_dwh_operation_status
  The DWH log service writes into this table to document the DWH status. These are the info and error log entries which you can also find in the log file cmrf.log.
  
• cmas_dwh_ser_sync_object
  This table is only used in LIVE mode. CM inserts an entry into this table for a transaction which was performed in the CM database. The entry contains a serialized package which needs to be sent to CMRF. The DWH live service reads from this table.
• cmas_dwh_synchronization
  When a DWH operation is performed using the Web Admin Suite, an entry is created in this table. The column type contains one of the possible values: INITIALIZATION, REINITIALIZATION, TRANSFER, UPDATE, TASK. The DWH transfer service reads from this table.
  
• cmas_dwh_task
  An entry in this database is created when a setting which is relevant for DWH transfer has been changed, i.e. Transfer to DWH and Field value omitted from DWH history. Tasks listed in this table are visible on the Tasks page under the DWH menu in the Web Admin Suite. When a task is started (automatically or manually), a new entry in cmas_dwh_synchronization is created. When the data has been processed successfully, the entry from cmas_dwh_task is deleted.

Tables in the DWH database

• INT_CONTROL_QUEUE
  INT_CONTROL_QUEUE is used to control the processing in CMRF. Two actions are available: pause and resume. Only the JMX components write into INT_CONTROL_QUEUE. Only the CMRF service reads and deletes entries from INT_CONTROL_QUEUE. It is done during processing of other data (e.g. transfer). The CMRF service pauses processing after pause action and continues after resume action.
• INT_LIVE_QUEUE
  The DWH live service writes serialized packages into this table. CMRF reads from this table to process the LIVE data.
• INT_TRANSFER_QUEUE
  The DWH transfer service writes serialized packages into this table. CMRF reads from this table to process the TRANSFER data.
• INT_LOG_QUEUE
  This table is used for log entries written by the CMRF. Based on those entries, the DWH log service creates entries in the table cmas_dwh_operation_status. See section DWH log service.
• hlp_parameter
  This table is not directly involved in transfer but provides some useful information, most importantly:

  • parameter_name last_data_transfer: The timestamp of the end of the last DHW update or transfer in the local time of the CMRF server.

  • parameter_name dwh_status: The possible values are explained in the following table.
    

    dwh_status (ID)	Description of the value	Explanation
    0	DWH uninitialized	This is the status after application startup, when the DWH sees that the database has not been initialized. This is a theoretical value, because when the DWH is uninitialized, it does not yet exist.
    1	DWH waiting for initial transfer	This status is set during (re)initialization. After the start of the DWH application, the DWH sees that there has not been a data transfer yet. When transfer messages are sent, the DWH switches to the status 2.
    2	DWH: data transfer in progress	The DWH enters this status when the transfer action is executed. The CMRF is working on transfer messages, and the DWH ignores all incoming non-transfer messages (there may have been problems with the transmission). The DWH remains in this status until either a fatal error occurs (subsequent status will be 3) or all transfer messages have been handled (subsequent status will be either 3 or 4, depending on whether they were errors).
    3	DWH: data transfer finished unsuccessfully	All data transfer messages have been handled, but, there were some errors.
    4	DWH operational	The DWH is operational and will handle all incoming update messages from CM. This status is reached when all data transfers or updates have finished successfully.

• hlp_transfer_error:
  Keeps a copy of the error message from exceptions of the CMRF live mode since the last REINITIALIZE. Here you can look up the object_uid for which the exception occurred.
• int_parameter:
  This table show technical details about the data transfer applied to the DWH database.
  • live-serial-number: The last applied serial number during live mode.

  • live-error: Boolean. If it is false, the live mode did not abort.

  • last-success-id: Epoch time stamp in milliseconds of the last successful live mode update

  • last-success-live-id: Epoch time stamp in milliseconds of the last successful live update of a transfer or update operation. Note: This value can be lower than last-success-id, when a transfer or update task is being executed.

CM processes and services

The following processes and services are relevant for the data transfer from the ConSol CM database to the DWH . The details are explained in the following sections.

• DWH transfer service
• CMRF service
• DWH log service
• Synchronization procedures
• DWH live service