Search Configuration and Indexer Management

ConSol CM provides a powerful search mechanism for all objects involved in the business processes (customers, tickets and resources). Technically, the search is based on the Indexer, a module of ConSol CM.

The following paragraphs explain the entire topic Search in ConSol CM from an administrative point of view. Please refer to the ConSol CM User Manual for a detailed explanation about how to use the search as an engineer.

Search Modes

A ConSol CM engineer can use several search modes:

Quick Search

This is performed using the Quick Search field in the upper right-hand corner of the Web Client. The display of the results (i.e., the fields and the order of the fields in the result list) can be formatted using templates, as detailed in sections Templates for Customer Data and CM.Resource Pool - Templates for Resource Data. Please keep in mind that you can adapt the length of the result set using the system property cmweb-server-adapter, globalSearchResultSizeLimit - see System Properties for details. The result set will also be influenced by the Page Customization attribute appendWildcardAutomatically, see section Page Customization, appendWildcardAutomatically for details.

Figure 333: ConSol CM Web Client - Quick Search

Phonetic Search

For the Quick Search, a phonetic search can be configured for String fields. Please see section Configuring the Phonetic Search for the explanation of how to prepare your ConSol CM system for this feature.

With a phonetic search, the engineer cannot only find exact matches in the search results but also results which sound similar even if the spelling differs from the entered search term. The implementation is based on the Apache Commons Codec libraries.

Figure 334: Search results without phonetic search

Figure 335: Search results with phonetic search

Detailed Search

This is performed using the Detailed Search page. To open this page, click on the magnifier icon next to the Quick Search entry field.

Figure 336: ConSol CM Web Client - Detailed Search

Please keep in mind that the result list length and paging of the result list for the Detailed Search can be configured using the system properties cmweb-server-adapter, searchPageSize and cmweb-server-adapter, searchPageSizeOptions. See System Properties for an explanation.

Please refer to the ConSol CM User Manual, section Searching for Tickets, Customers and Resources to learn how to use the search functionality.

Configure the Search Result List

You, as an administrator, can configure the layout of the search result list using the annotation order-in-result. This annotation influences the columns of object-specific search result sets. For example, the Data Object Group ResellerCompanyData, belonging to the Customer Data Model ResellerModel, contains the following Data Object Group Fields:

Thus, in a search result list, e.g., a Detailed Search which shows a result set with companies of a customer group which uses the ResellerModel will contain the two columns company_name and company_number.

For a field (i.e., column in the result table) which should not be displayed in the Web Client (i.e., in the field selector for the result list and as column in the table) at all, set the value of the order-in-result annotation to 0 (zero).

Figure 337: ConSol CM Admin Tool - Setting the annotation order-in-result for a Data Object Group Field

Figure 338: ConSol CM Web Client - Search result set with the two annotated columns (order-in-result)

Notes about the Order of Search Result Columns

Since the annotation order-in-result is not the only parameter which influences the order of columns in search results (tables), please find here a detailed explanation.

In the following section, user preferences means the layout of a search result table which the engineer has configured manually, e.g., the columns which should be displayed, the order of those columns or the sorting.

Rules:

  1. The columns of a search result table can display different fields for each of the CM object types:

    1. for tickets: Custom Fields (CFs) as well as internal fields like queue, creation date, engineers
    2. for customers (contacts or companies): Data Object Group Fields as well as internal fields like object name of a customer
    3. for resources: Resource Fields as well as internal fields like object name of a resource
  2. The order-in-result annotation organizes only the order of the Custom Fields, Data Object Group Fields, and Resource Fields, not of internal fields.
  3. The internal field types have the following visibility and order by default:
    1. for ticket lists:
      1. Engineer
      2. Main customer
      3. Ticket name
      4. Ticket subject
    2. for customer (contact/company) lists:
      1. object (i.e. contact or company) name
    3. for resources lists:
      1. object (=resource) name
  4. The data fields (CFs, Data Object Groups Fields, and Resource Fields) are primarily ordered by the user preferences. Those will overwrite any other configuration.
    When, e.g., as default configuration, a CF has the annotation order-in-result = 5 and is visible, it will (case a) not be displayed if the engineer has deselected the column, and (case b) it will be displayed as column #2 if the engineer has placed it there.
  5. If no order is defined by the user preferences, the order is the ascending order of numbers from the data field annotation values order-in-result. Those numbers are evaluated globally across all data fields within all CF groups, all Data Object Groups, and all Resource Groups. Identical values of the annotation order-in-result of different data fields are theoretically possible.
    For an exactly defined order, use some convention for an entire CM installation and apply unique numbers, e.g., use high four digit values.
  6. If there are still two fields with identical values for order-in-result which should both be displayed, the data fields are alphabetically ordered based on their localized names. The locale of the engineer's browser is used.

Autocomplete Search (Search by Using Intelligent Fields)

The Autocomplete Search is performed implicitly when you start entering a word in an Autocomplete field, e.g., in company data or customer data when you create a ticket (see figures below).

Figure 339: ConSol CM Web Client - Autocomplete Search

Figure 340: ConSol CM Web Client - Suggestions for an Autocomplete Search

Starting with ConSol CM version 6.10.2, a search in all contacts of the ConSol CM database is also performed in the Ticket E-Mail Editor when an engineer starts typing in the To, Cc or Bcc field:

Figure 341: ConSol CM Web Client - Ticket E-Mail Editor: Automatic search for contacts as e-mail receivers

The number of characters which must be typed before automatic search is activated can be configured using the page customization attribute minMailInputLength (Integer, Default 1) in the scope mailTemplate.

Fields Which Can Be Searched

For Custom Fields, Data Object Group Fields and Resource Fields which should be searched, the annotation field indexed has to be set. See section The Annotation field indexed. This makes the field available for the Quick Search and for the Detailed Search.

Two types of fields are processed by the search engine:

  1. data fields and content which are/is indexed by default
  2. data fields which are annotated using the annotation field indexed

Data Fields and Content Which Are Indexed by Default

Data Fields Which Are Annotated Using the Annotation field indexed

For all data fields which have been added and configured by the ConSol CM administrators, the annotation field indexed has to be set to make the fields searchable. A field which is indexed is available in

You, as an administrator, can define the system's behavior with regards to search results. Depending on the values of the field indexed annotation, the search results for a contact might include all the tickets of the contact as well. The following table shows the implications of all possible values of the field indexed annotation for the different object types. The field indexed annotation has to be set for each individual data field, e.g., name and e-mail for a contact, zip and address for a company, priority and software module for tickets, or name and model for a resource type.

Please note that two distinct perspectives are addressed here:

  1. the search criterion, i.e., the field for which you start the search, e.g., all companies that start with ConS*. A field is made available in the search by setting the field-indexed annotation.
  2. the result set, i.e., the objects which are displayed in the result lists. The objects in the result set are defined by the value of the field-indexed annotation. Depending on this value, you might see only the objects you have searched for (e.g., only contacts) or also objects which are related to the objects you have searched for (e.g., companies of those contacts). Please see the following table for a detailed explanation.

Object type/

value of field indexed annotation

transitive unit local not indexed
TICKET
  • no difference between the three values
  • the ticket data will be available for searching
  • no other objects which are related to a ticket in any way will be retrieved
  • we recommend that you use the default value transitive
  • the ticket data will be NOT available for searching
CONTACT
  • the contact data will be available for searching
  • the tickets of the contact will also be found when you search for the contact
  • searching by company by its contact field is NOT possible
  • the contact data will be available for searching
  • searching for a company by its contact field is NOT possible
  • the contact data will be available for searching
  • no other objects which are related to a contact in any way will be retrieved
  • the contact data will be NOT available for searching
COMPANY
  • the company data will be available for searching
  • the tickets of the company will also be found when you search for the company
  • the contacts of the company will also be found when you search for the company
  • the company data will be available for searching
  • the contacts of the company will also be found when you search for the company
  • the company data will be available for searching
  • no other objects which are related to a company in any way will be retrieved
  • the company data will be NOT available for searching
RESOURCE
  • no difference between the three values
  • the resource data will be available for searching
  • no other objects which are related to a resource in any way will be retrieved
  • we recommend to use the default value transitive
  • the resource data will be NOT available for searching

Please note that if it should be possible to sort result tables (in the Web Client) according to a column (by clicking on the column header), the respective field has to be indexed!

Configuring the Phonetic Search

In order to activate the phonetic search for a data field, set the annotation phonetic to true for this field. The annotation can only be used for String fields.

Administrator Tasks Concerning the Indexer

It is important for the administrator to know how to configure ConSol CM in a way that ...

These tasks are described in the following sections.

First, some background knowledge about the Indexer, the system which manages the search in ConSol CM, is provided. This will help you, as an administrator, to look behind the scenes and understand the configuration in a better way.

Introduction to the ConSol CM Indexer

The Indexer is a module of ConSol CM which creates search indexes. For each data field that should serve as search criterion (see section Fields Which Can Be Searched above), an index is created.

The indexes are stored in a sub-directory of the data directory that you have indicated during system set-up (the data directory is stored as a system property: cmas-core-shared, data.directory). The following picture shows an example for index files of a ConSol CM installation (here used for a demo environment). The demo_Datadir is the data directory specified during set-up, and all subdirectories are created automatically by ConSol CM.

Figure 342: ConSol CM Indexer - Directory demo_Datadir

Please make sure that ...

If the index directory should be corrupted or the index is not available, it can be rebuilt or repaired. Please see the following sections for details.

Indexer and Index Management Using the Admin Tool

The Annotation field indexed

By default, the entire ticket text and all attachments are indexed. For all Custom Fields, Data Object Group Fields and Resource Fields, the fields which should be indexed have to have the annotation field-indexed. Please refer to the sections Custom Field Administration (Setting Up the Ticket Data Model), Data Object Group Field Management and GUI Design for Customer Data and CM.Resource Pool - Setting Up the Basic Resource Model for details about setting annotations to Custom Fields (ticket data), Data Object Group Fields (customer data) and Resource Fields (resource data). There are four possible values for this annotation:

A detailed explanation of the system behavior triggered by those values is provided in the table above.

Nested fields all have to have the same index type, otherwise they cannot be searched. For example, when you work with a list of structs, the list, the struct and all data fields in the struct which should be searched have to have the value transitive for the annotation field-indexed.

Indexer Management: Navigation Item Index

Usually, the index requires no manual maintenance. ConSol CM will handle everything regarding indexing automatically. There are only two cases where you have to perform manual administrative operations:

In the Admin Tool, open the navigation group Services, navigation item Index to configure and manage the Indexer. If Indexer tasks are still running, this will be indicated by an exclamation mark next to the name of the navigation group (Services) and by the number of open tasks next to the name of the navigation item (Index). In this way, even when the navigation group is not opened, you, as an administrator, can immediately notice that some tasks are open in the group Services and can then quickly identify the number of open tasks.

Figure 343: ConSol CM Admin Tool - Navigation group Services: Navigation item Index

Figure 344: ConSol CM Admin Tool - Indication of open indexer tasks

In the first line the current status of the Indexer is displayed (this is the value of the system property cmas-core-index-common, index.status):

The following operations can be performed:

If there are open tasks in the Indexer task list, the following data is displayed for each task:

Please note that data in the index is always synchronized with the ConSol CM database, i.e. during an Index update no data is deleted/removed from the index files. The index is fully usable during the synchronization process, i.e. the search operations (detail as well as quick search) can be used with their full functionality and the complete data set. Changes made after the synchronization was started are immediately reflected in the index, because these data have a higher priority than the data which is synchronized due to an index update triggered manually using the Admin Tool.

When an admin has started the index update manually (Synchronize Index), all other index tasks are removed, before this update starts.

Indexer and Index-Relevant System Properties

The following system properties are also relevant for the Indexer, see following figure. Please refer to System Properties for a detailed explanation of the Indexer system properties and the Indexer and Search Configuration section of List of System Properties by Area.

Figure 345: ConSol CM Admin Tool - System properties for the indexer

Indexer Update

For a detailed description of the ConSol CM Indexer, including the configuration of the Indexer in an environment with master and slave indexing servers, please refer to the ConSol CM Operations Manual. Here in the current section, some short information will be provided as quick introduction for CM administrators.

Indexer Update Modes

The indexer stores entities which require an index update (e.g. modified tickets, modified customers, new engineers) in a persistent store. This can be either a JMS queue of the application server or a table in the CM database. You can select the mode by setting the system property database.notification.enabled, default value in CM version 6.10 is false.

Index Persistent Store as JMS Queue

database.notification.enabled = false

The JMS queue queue/cm6-index will be used.

Index Persistent Store as Database Table

database.notification.enabled = true (should not be used with CM versions before 6.9.4.1!)

The database table cmas_index_update_serialized will be used for indexer transactions (as persistent store). The master indexing server (or the only indexing server in single-server environments) polls this database

Indexer Update Principle

The master indexing server polls the persistent store for new entries. If there are new entries, indexer tasks are created in the tables cmas_index_update_task and cmas_index_update_part. The master indexing server then executes the tasks from cmas_index_update_task and cmas_index_batch_update_task. In this way, the (master) index is updated.

On the master (or only) indexing server, administrative tasks are stored in the database table cmas_ index_administrative_task.

Every slave indexing server polls the master indexing server for updates and then updates its own / local index.

Indexer-Related System Properties

Please refer to the Indexer and Search Configuration section of List of System Properties by Area.

CM Indexer Services

For indexing, two ConSol CM services are important:

Please see also section CM Services.

Stopping index changes notifier is NOT safe. If the indexer module discovers that the notifier is stopped and there is a message that has to be sent to the persistent store, the indexer will set the index status configuration property to RED, i.e., signal that the index needs a full synchronization.

However, stopping index changes receiver is safe. After restarting, it will pick up all of the missing changes from the persistent store.

Systems with More Than One Indexing Server

When ConSol CM is run in a cluster of application servers, each cluster node holds a local index in its ${cmas.data}/index/ directory. The index gets updated on the (single) master server, which is then polled by all slave servers to update their copies. As a result, all indexing servers are eventually consistent, i.e., updates are not visible right away, usually with a small delay.

JGroups is used for indexer synchronization (see the JGroups.org web site for more information).