Administering Messaging Servers for Asynchronous Messaging

This chapter provides an overview of messaging server administration and discusses how to:

Create and assign dedicated servers.
Edit messaging server queue lists.
Delete messaging servers.
Configure messaging servers.
Set the BEA Tuxedo queue size.

Understanding Messaging Server Administration

This section discusses messaging servers, messaging server processes, and dedicated messaging servers.

Messaging Servers

The PeopleSoft messaging infrastructure is the core system upon which PeopleSoft Integration Broker is built. Before using Integration Broker for asynchronous message processing, you must configure and start the messaging server.

Note. The messaging servers and messaging server processes are used for asynchronous integrations only. If you are performing only synchronous integrations, you need not configure a messaging server.

Activating Messaging Server Domains

Pub/sub server domains are delivered inactive, and you must active them for the pub/sub system to become available.

You can activate pub/sub server domains using the Integration Broker Quick Configuration page or on the Domain Status page in the Service Operations Monitor.

Messaging Servers in the DB2 UDB OS/390 and z/OS Environments

For DB2 UDB OS/390 and z/OS environments, PeopleSoft delivers messaging servers with persistent cursors off. Therefore, all SQL statements are compiled each time they are invoked.

To change the persistent cursors setting:

In PSADMIN locate the Values for config section — Publish&Subscribe.
Set the Persistent Cursors on DB2/OS390 option. The values are:
- 0: Persistent cursors off.
- 1: Persistent cursors on.

Messaging Server Processes

Although the server processes devoted to the messaging system are all part of the larger application server domain, they comprise a distinct set of processes that aren’t involved with the ordinary transactions associated with PeopleSoft Pure Internet Architecture connections.

Six processes of two types—dispatchers and handlers—are paired to produce the messaging servers that transmit asynchronous messages throughout the messaging system. A set of three messaging servers—a publication broker, a publication contractor, and a subscription contractor—is required by PeopleSoft Integration Broker. The following table lists the generic names for the processes:

Messaging Server	Dispatcher Name	Handler Name
Publication Broker (BRK)	PSBRKDSP	PSBRKHND
Publication Contractor (PUB)	PSPUBDSP	PSPUBHND
Subscription Contractor (SUB)	PSSUBDSP	PSSUBHND

To distinguish the messaging servers, the PeopleSoft Server Administration utility (PSADMIN) includes a separate menu for administering them—the Messaging Server Administration menu. You select this menu from the PeopleSoft Domain Administration menu, as shown in the following example:

PeopleSoft Domain Administration menu

From this menu, you can create new messaging servers, edit the queue list for existing messaging servers, and delete messaging servers that are no longer needed.

Note. Although you add new messaging servers using a separate menu, you configure the messaging server processes with PSADMIN as you would any other server process.

Understanding Dedicated Messaging Servers

When you create a new application server domain, PSADMIN offers a set of messaging server processes that comprise the default messaging server set for that domain. The default messaging server set is sufficient for development, testing, or demonstrations.

You might use the default messaging server set as the only messaging server set; however, in most cases, it is insufficient. As the volume of published messages increases in a production system, it’s likely that a single messaging server set will become overloaded. To avoid potential overloads and performance degradation, create additional dedicated messaging servers to cope with an increase in message volume.

Note. Dedicated messaging servers are used only for asynchronous messaging.

When you create a new messaging server, you assign it to a particular queue using PSADMIN. If a given queue is the most active and creates performance bottlenecks, you can dedicate several messaging servers to that queue to cope with the message volume. A messaging server is capable of handling multiple message queues.

The following illustration depicts a dedicated messaging server set assigned to QUEUE_03:

Dedicated messaging server set

In this scenario, the default messaging server set (_dflt process collection) continues to process the messages in the other message queues while the dedicated messaging server set processes only the messages within a specified queue. Unless you create and configure dedicated messaging servers, the default messaging server set handles all incoming messages. Remember that a messaging server set is a collection of six messaging server processes.

Note. Before you can assign messaging servers to message queues, you must first define the message queues using PeopleSoft Application Designer.

The process for adding a dedicated messaging server includes two parts:

Creating the new messaging server.

Use the Messaging Server Administration menu in PSADMIN. This is where you specify the type of server you’re adding, name the server, and assign it to specific message queues.
Configuring the new messaging server.

When you add a new messaging server of any type, the configuration files are updated to include parameters for the new server processes. Because a messaging server consists of two server processes, when you create a new one, you’ll see two additional configuration sections in the PSADMIN domain configuration menu. They appear identical to the _dflt messaging server processes, except they have the name that you gave them in place of the _dflt. For any new messaging server processes to take effect, you must first reconfigure the domain to include the new parameters.

Note. Typically, you add multiple messaging elements simultaneously, so you should create all the elements and then reconfigure the domain once.

Considerations When Creating Dedicated Servers

When creating dedicated messaging servers, consider the following points:

There is no validation checking when you enter service operation queue names in PSADMIN. As a result, if service operation queue names are not spelled correctly and match those defined in the system, the dedicated server will not process any service operation. Instead the default server will process them.
Never split a service operation queue across domains. You don't want a situation where a service operation queue is assigned to domain A and the same service operation queue is also assigned to domain B, since both domains will try to do the same work. You want specific service operation queues for domain A and specific service operation queues for domain B.
Setting up a dedicated server consists of a creating a dedicated dispatcher and handler(s). Make sure that the number of handlers booted is sufficient to process the request volume.
If you create more than one dedicated server over different domains do not to include any service operation queues already specified for other dedicated servers of the same server type. For example, do not include service operation queue A in publication broker server X, as well as in publication broker server Y.
Verify that the BEA tuxedo queue size is large enough and correctly configured in PSADMIN.

See Setting the BEA Tuxedo Queue Size.
If you choose to set up group domain failover for dedicated servers, ensure that:
- Service operation queue sets within groups are identical.
- Service operation queue sets between groups are unique.
  
  See Setting Up Domain Failover.
When you create a messaging server, the following dispatcher parameters are populated with their default values. Verify those default settings you want to keep and those that you want to change.
- Restart period.
- Scan interval.
- Dispatcher queue maximum queue size.
- Memory queue refresh rate.
  
  See Specifying Dispatcher Parameters.

Creating and Assigning Dedicated Servers

Typically, you create one server of each type to produce a complete messaging server set dedicated to one or more service operation queues.

Note. Although a messaging server set consists of one of each of the three server types, they do not all need to be dedicated servers. For example, for a given service operation queue, you can create only a dedicated publication contractor. If you haven’t assigned a dedicated publication broker or a dedicated subscription contractor to the service operation queue, the default publication broker and subscription contractor is used.

The following example shows the Message Server Administration menu:

To create a dedicated messaging server:

From the PeopleSoft Domain Administration menu, select the Messaging Server Administration menu.
From the Messaging Server Administration menu, select the Create a new messaging server.
From the submenu that appears, select the type of server to create.

You can create a publication broker, a publication contractor, or a subscription contractor.
Enter a name to identify the new messaging server.

The name is limited to six characters; for example, PT8MSG. The name that you enter is appended to each generic server process name; for example, PSBRKDSP_PT8MSG for the broker dispatcher and PSBRKHND_PT8MSG for the broker handler.

Note. The name that you enter must be unique for the messaging server type in the current domain.
Specify the service operation queue that is handled by the new messaging server.

You must specify a service operation queue, which must already be defined in the PeopleSoft Pure Internet Architecture.

Note. The service operation queue name that you enter must exactly match the name that appears in the PeopleSoft Pure Internet Architecture. No prompt or validation occurs between PSADMIN and PeopleSoft Pure Internet Architecture definitions.

Important! Don’t specify a given service operation queue for more than one messaging server of each type in the current domain. For example, you cannot have two subscription contractors assigned to the service operation queue. Nor can you have two dispatchers assigned to the service operation queue.

After several status messages, the Messaging Server Administration menu reappears, displaying a list of the existing dedicated messaging servers for the current domain.

Editing Messaging Server Queue Lists

After you create a publication broker, publication contractor, or subscription contractor, you may need to add more service operation queues to the server’s queue list, or you may want to decrease the number of service operation queues it services to improve performance. You use the commands shown in the following example:

To modify a queue list:

From the PeopleSoft Domain Administration menu, select Messaging Server Administration menu.
From the Messaging Server Administration menu, select Edit the queue list for a messaging server.
From the list of defined servers, select the messaging server for which you want to modify the queue list.
Specify a list of the message queues that will be handled by the selected server.

You must specify at least one message queue. Multiple queue names must be entered as a list separated by commas, with no spaces; for example, HRMS_01,HRMS_02,CRM_03.

Note. The new list of message queues that you enter replaces the current list of queues for the selected messaging server. The queues that you specify must already be defined in the PeopleSoft Pure Internet Architecture.

After several status messages, the Messaging Server Administration menu reappears, displaying the updated messaging server listing.

Deleting Messaging Servers

Sometimes a previously created messaging server is no longer needed. Rather than allow the server to consume valuable system resources, you should remove it from the domain.

To delete a messaging server from a domain:

From the PeopleSoft Domain Administration menu, select Messaging Server Administration menu.
From the Messaging Server Administration menu, select Delete an existing messaging server.
From the list of defined servers, select the messaging server to delete.

After several status messages, the Messaging Server Administration menu reappears, displaying the remaining dedicated servers.

Configuring Messaging Servers

Once you create dedicated messaging servers, you must configure their dispatcher and handler processes so that they boot when you start the application server. You configure these processes using PSADMIN, as you do other server processes that run on the application server. Before you configure additional messaging server processes, familiarize yourself with the other server processes that run on the application server.

See Using PSADMIN Menus.

Two types of server processes comprise each messaging server: a dispatcher and a handler. Each process type requires that you set a different set of parameters. Most of the parameters are similar to other server processes, such as PSAPPSRV, but some parameters are specific to messaging servers.

Note. The following sections also apply to the _dflt messaging server processes. Only one parameter is different for a dedicated messaging server process and its _dflt counterpart—the Queues parameter. That parameter enables you to add message queues to the queue list. The _dflt server processes cannot be associated with a specific message queue.

Specifying Dispatcher Parameters

There are three generic process types that are the basis for all dispatcher processes:

PSBRKDSP, which is the publication broker dispatcher.
PSPUBDSP, which is the publication contractor dispatcher.
PSSUBDSP, which is the subscription contractor dispatcher.

The following parameters apply to all three process types.

Recycle Count	Specifies the number of times each dispatcher process is executed before being terminated (intentionally) by the system and then immediately restarted. You should intermittently recycled servers to clear buffer areas. The time required to recycle a server is negligible (a matter of milliseconds), however part of the server initialization process is to rebuild the in-memory queues. The time to complete this is dependent on the number of messages residing in the pub/sub database table. The Recycle Count parameter does not translate into a native BEA Tuxedo parameter in the PSAPPSRV.UBB file. Instead, the value is stored in memory and is managed by the system.
Allowed Consec Service Failures (allowed consecutive service failures)	This option enables dynamic server process restarts in the event of service failures. To set this option, enter a number greater than 0. To disable it, enter 0. The default value for this parameter is 2. The value that you enter is the number of consecutive service failures that cause a recycle of the server process. This is a catchall error handling routine that allows a dispatcher to terminate itself if it receives multiple, consecutive, fatal error messages from service routines. Such errors should not occur consecutively; however, if they do, it indicates that the server process needs to be recycled or cleansed. A retry message appears when the specified number of service failures occurs.
Dispatch List Multiplier	Limits the number of dispatched messages by the number you specify, multiplied by the number of associated handler(s). This parameter is useful for unordered queues when all messages could go out at once. The default value is 10.
Scan Interval	Specifies the number of seconds between scans of the work queue when idle. The default value is 15 seconds. The scan interval is necessary to detect the following types of messages: Messages published from an application server domain that is not the active pub/sub domain as selected on the Domain Status page in the Service Operations Monitor . Cases where the broker server does not receive a notice of the publication. When a message is in the queue, the broker server doesn’t receive a notice of the publication. A scan interval is required to make sure these types of messages are processed in a timely manner. The scan interval is analogous to the polling that PeopleSoft Process Scheduler performs on the Process Request table. In addition, the scan interval detects messages that have been resubmitted—for example, after an error. Decreasing the scan interval decreases latency for these types of publishes and error recovery. Note. The scan interval and ping rate (as a percentage) determines the actual interval for pinging any unavailable remote nodes. The algorithm used is: (attempts) x (ping rate) x (scan interval).
Ping Rate	Determines the number of seconds of inactivity before the server scans the database queues to restart any stalled or crashed items. The default value is 150 seconds. The ping rate is used in conjunction with the scan interval for pinging remote nodes. See the definition for Scan Interval in this section.
Maximum Ping Interval	Determines the maximum interval, in hours, between subsequent attempted pings of any unavailable remote nodes.
Dispatcher Queue Max Queue Size	Determines the maximum number of items per service operation queue that the dispatcher keeps in memory. The default value is 1000.
Memory Queue Refresh Rate	PeopleSoft Integration Broker maintains current asynchronous messaging queues in system memory for quick access. Occasionally, these cached queues can become corrupted. At that point, they must be refreshed from the PeopleSoft Integration Broker data tables. The likelihood and frequency of cache corruption depends on a combination of factors specific to the messaging system. If you need to periodically refresh the in-memory queues, you can use this parameter to tailor the frequency of the refresh to fit the situation. Each dispatcher on the system has its own queue. For each queue, you set the rate equal to the number of dispatch attempts that must occur before the queue is refreshed. The refresh occurs only when the specified number of dispatch attempts is reached for a given message queue. For example, with a memory queue refresh rate of 8, multiple queues could have up to seven dispatch attempts each without triggering any refresh. The following settings are also significant: A setting of 0 (the default) disables the refresh altogether. A setting of 1 triggers a refresh immediately after every dispatch attempt, effectively disabling memory caching.
Restart Period	Specifies the number of seconds between restart attempts on Started items in the work queue. An item which stays in Started state for more than a few seconds might be stalled—for example, the service request might have been lost, or the handler might have crashed. Decreasing the restart period reduces the latency for recovering stalled items with the status Started. However, under high load, items might stay in the Started state longer than normal for valid reasons. All handlers might be busy, and the handler service request for the item might be queued at the BEA Tuxedo level. Setting the restart period too low results in redundant restarts. The dispatcher dispatches the item again, even though the original request is still in the Tuxedo queue. A small number of extra restarts is benign; however, at higher volumes, the unnecessary restarts can fill up the queue and block real requests. The formula for a reasonable value for the restart period is: ((incoming requests per second) / (number of handlers)) × (average processing time per request) For example, if you have an incoming rate of 20 per second, and you have four handlers, each handler is busy processing one item and will have four others waiting in the queue. A new item must wait for the currently processing item—plus the four items in the queue—before it is processed. If each item takes 10 seconds to process, the new item will stay in Started status for approximately 50 seconds before the handler works on it. If it stays in Started status longer, it's likely that the request to the handler has been lost, and the item should be restarted. Note. Using a value greater than 3540 for the dispatcher restart period results in constant restarts.

Specifying Handler Parameters

There are three generic process types that are the basis for all handler processes:

PSBRKHND, which is the publication broker handler.
PSPUBHND, which is the publication contractor handler.
PSSUBHND, which is the subscription contractor handler.

The following parameters apply to all three process types.

Min Instances (minimum instances)	Specifies the number of handler server processes started at boot time.
Max Instances (maximum instances)	Specifies the maximum number of handler server processes that can be started or spawned.
Service Timeout	Specifies the number of seconds a handlers waits for a service request before timing out. Service timeouts are recorded in the TUXLOG and APPSRV.LOG. In the event of a timeout, the handler terminates itself and BEA Tuxedo automatically restarts the process.
Recycle Count	Specifies the number of times that the system executes each server before the PeopleSoft system intentionally terminates the process. Server processes must be intermittently recycled to clear buffer areas. The time required to recycle a server is negligible (a matter of milliseconds). The Recycle Count parameter does not translate into a native BEA Tuxedo parameter in the PSAPPSRV.UBB file. Instead the value is stored in memory and is managed by the PeopleSoft system.
Allowed Consec Service Failures (allowed consecutive service failures)	This option enables dynamic server process restarts in the event of service failures. To set this option, enter a number greater than 0. To disable it, enter 0. The default for this parameter is 2. The numerical value that you enter is the number of consecutive service failures that cause a recycle of the server process. This is a catchall error handling routine that allows a handler to terminate itself if it receives multiple, consecutive, fatal error messages from service routines. Such errors should not occur consecutively; however, if they do, it indicates that the server process needs to be recycled or cleansed. A retry message appears when the specified number of service failures occurs.
Max Retries (maximum retries)	Specifies the maximum number of times that the server attempts to restart a failed action. This parameter prevents a bad item from continuously crashing a handler process. The counter is incremented when the handler sets the status to Working but before it actually starts processing the item.

Setting the BEA Tuxedo Queue Size

The messaging system uses the Tuxedo queue size indicated in the application server domain section of PSADMIN to determine when the Tuxedo queue size has reached its maximum. The pub/sub system reads the actual queue size periodically, based on the Tuxedo Queue Status Check Count parameter. The system throttles itself so that it does not exceed this maximum, thereby preventing queue saturation and degraded performance.

Set the Tuxedo Queue Size parameter equal to that of the kernel parameter used by the machine running the pub/sub processes (msgsys:msgingo_msgmax).

To set the Tuxedo queue size for the messaging system:

In PSADMIN navigate to the Values for config section – PSAPPSRV part of the file. To do so:
1. Open PSADMIN.
2. Enter 1 for Application Server and press Enter.
3. Enter 1 for Administer a Domain and press Enter.
4. Choose a domain from the list and press Enter.
5. Choose 4 for Configure the Domain and press Enter.
6. Enter Y to shut down the domain.
7. Enter Y to change the configuration values.
8. Press Enter to scroll through the file and accept the current settings until you reach the following section:
```
Values for config section - PSAPPSRV
```
Enter Y and press Enter to change values in the section.
Navigate to the Tuxedeo Queue Size parameter. To do so, press Enter to scroll through the list and accept the current values. When you reach the Tuxedo Queue Size parameter enter a value.

A value of 0 (zero) disables Tuxedo queue threshold determination and usage.

Based on your environment, a value of -1 sets the queue size to the following default values:
- Windows: 65535.
- AIX: 4000000.
- Solaris: 65535.
- HP: 65535.
Press Enter to scroll through the remaining sections and accept the current settings.

PSADMIN will process the changes and then load the new configuration.
Boot the domain.

See Using the PSADMIN Utility, Using PSADMIN Menus.