Dashboard ArchitectureThis chapter discusses:
Dashboard architecture.
The messaging framework.
Dashboard ArchitectureThis section discusses:
Data transfer.
Data manipulation.
Integration setup.
Customization considerations.
Data TransferPeopleSoft dashboard applications are built on two complementary systems:
PeopleSoft transactional system.
This is your system of record for your operational data, and a dashboard system,
Oracle Business Activity Monitoring (Oracle BAM).
This system displays real-time interactive reports based on the data. The Active Data Cache (ADC) is the data storage component of Oracle BAM.
This diagram shows the movement of data from a PeopleSoft database to the dashboard:
High-level messaging architecture
Data moves from the PeopleSoft system to the dashboard system through the following process:
Relevant data changes within the PeopleSoft transactional system are registered with the messaging framework.
The messaging framework publishes a Java Messaging Service (JMS) message to which Enterprise Link subscribes.
Enterprise Link is an Extract, Transform, and Load (ETL) tool within Oracle BAM.
After Enterprise Link performs any necessary transformations, the data is saved to ADC data objects.
The dashboard viewer immediately updates its display to reflect data changes.
As the diagram shows, data also moves from the dashboard to the PeopleSoft transactional system. This occurs when a user submits an action from within the dashboard system.
Note. The data structure that is used in the ADC is known as a star schema. The Oracle BAM documentation discusses the ADC data structures.
Data ManipulationAs data is sent from PeopleSoft to the ADC, it requires these changes:
PeopleSoft relational databases use normalized table structures for efficient data storage. Oracle BAM, on the other hand, uses denormalized data structures that are optimized for the performance of the real-time reports. The application classes that fetch the data to be included in the message return the data in a format that is appropriate for the dashboard system. Enterprise Link performs certain additional data manipulations that are necessary before the data can be saved in the ADC tables.
The source system is identified.
Because dashboard applications support data from multiple sources, it is useful to tag data with a source system identifier. The dashboard system settings that you establish include a source system ID, and the Change Notification process adds this identifier to each record that is sent to the dashboard system.
A surrogate key is added to certain data objects.
Different source systems can use different key structure for similar objects. A surrogate key is a value that functions as a unique ID for each dashboard record, regardless of the data's native key structure in its source system. Oracle BAM Enterprise Link can generate surrogate keys at the time it saves a new record into the ADC. For performance reasons, PeopleSoft is selective about which data objects get surrogate keys.
The Enterprise Link plans that process incoming data include steps to convert native-key references to other records into the appropriate surrogate key. For example, in the Case data object, there is a field where the case status is identified by its surrogate key. If the case status table has a value of Open with a surrogate key of 123, then open cases need to have the value 123 in their case status field. When a message arrives with data for an open case, Enterprise Link looks up the surrogate key for the Open case status and ensures that the case data in the ADC is saved with the appropriate surrogate key.
Occasionally a native-key reference to another record cannot be resolved. For example, if you create a new case status in the PeopleSoft system, there could be a situation where a case with that status arrives in the dashboard system before the new case status record arrives. In this situation, Enterprise Link uses the surrogate key value of -1, which has a corresponding description of Unknown.
It is also possible to have records where there is no native-key reference available. In this situation, Enterprise Link uses the surrogate key value of -2, which has a corresponding description of None.
Integration SetupA JMS server must be available and configured for dashboard messaging, and a PeopleTools Integration Broker dashboard node must be configured with the appropriate URL for the JMS server’s Java Naming and Directory Interface (JNDI) service.
This PeopleBook does not describe the necessary JMS and Integration Broker setup. Instead, refer to the dashboard installation documentation.
See PeopleSoft Enterprise Customer Relationship Management 9 Supplemental Installation Guide
Customization ConsiderationsPeopleSoft dashboard applications are built on the Oracle BAM platform. Use these Oracle BAM tools to modify the delivered system:
Use Administrator to manage security.
Note. The dashboard integration framework enables you to manage security within the PeopleSoft system and then to send users and their access levels to Administrator.
Use Enterprise Link to perform the data manipulations that are necessary as data moves from the PeopleSoft system into the ADC.
Modifications to the delivered dashboard applications (other than security configuration) are considered customizations and are unsupported other than the support for Oracle BAM.
Refer to your Oracle BAM documentation for more information.
The Messaging Framework
This section discusses:
Messaging framework elements.
The message queues.
The Change Notification process.
Messaging Framework ElementsThe messaging framework is the mechanism by which data from PeopleSoft applications is sent to the dashboard system. The framework includes these elements for processing online changes:
Two message queues that store identifying information about data to be sent.
A daemon that you set up during installation to monitor the queues.
The Change Notification Application Engine process (PT_CDB_UPDATE) that the daemon runs to process queue contents.
PSCDBCHNGQ, which stores information that can be sent to the dashboard immediately.
Most dashboard data comes from this queue, including batch requests that are submitted from the Batch Data Load page.
PSCDBSCHEDQ, which stores future-dated information.
For example, in PeopleSoft CRM, contact methods, worker departments, and tasks can be future dated. Because the dashboard shows only current data, any future-dated changes to these objects is held in the PSCDBSCHEDQ queue until such time as the data becomes effective.
The framework also includes the Batch Data Load Application Engine process (PT_CDB_BATCH). This facilitates batch data loads by creating message queue entries for objects that you specify. CRM dashboard applications use a separate process, the CRM Initial Data Load Application Engine process (RBD_INIT_DRV), to optimize performance for the high-volume initial load of CRM data.
To maximize performance for the messaging framework, you can set up parallel processing of the messages for each dashboard application. Multiple Process Scheduler servers are required to support parallel processing of online changes. The CRM Initial Data Load process supports parallel processing with either multiple Process Scheduler servers or with one Process Scheduler server with multiple AESRV process instances.
See Also
PeopleSoft Enterprise Customer Relationship Management 9 Supplemental Installation Guide
The Message QueuesThere are two types of entries in the message queues:
Entries for a single instance of an object—for example, a single case.
Entries for a batch load of an object.
Message queue entries always include an object identifier (for example, case, lead, or order), an application ID (for example, Sales or Order Capture) and the entry type—single or batch.
When an online user updates a relevant transaction, the system populates the message queue with minimal identifying information about the object whose data is to be sent to the dashboard. (“Relevant” transactions are determined based on which dashboard applications have been installed.) Because the message queue requires only basic identifying information about an object, items can be added to the queue with minimal impact on the online user.
When the Change Notification process handles a single-instance message queue entry, it calls an object-specific application class to fetch the rest of the data that needs to be transmitted to the dashboard. The application classes that PeopleSoft delivers for this purpose are optimized to bring over only data that is considered relevant to a dashboard user—that is, data that is either used in a delivered report or that could be useful to an organization that is creating additional, custom, reports.
Message queue entries for batch data loads are typically the result of some other process. There is a general purpose Batch Data Load page where you can submit requests for batch data loads, and there is a CRM-specific page for performing initial data loads. PeopleSoft CRM also has additional processes that create message queue entries for batch loads. For example, there is a daily process that creates an entry for currency exchange rates, resulting in a daily refresh of all currency exchange rate information in the dashboard system.
Message queue entries for batch data loads do not include any data other than the object type and the fact that it is a batch data load. When the Change Notification process handles a batch load message queue entry, it calls the same object-specific application class that is uses to fetch data for single-instance entries. It also passes the application class the information that this is a batch load request. The object-specific application class includes a method that selects the appropriate instances of the object.
For some objects, such as the currency exchange rate load, a batch load processes all rows of data. Typically, batch loads of dimension tables will send all rows of data.
For other objects, the application class is more selective. In PeopleSoft CRM, there is a setup component where implementers enter criteria for the batch loading of certain objects, and the application class reads the settings in this component to determine which rows of data to return to the Change Notification process. Typically, batch loads of transaction tables are more selective. For example, an organization can choose not to load closed cases that are more than three months old.
After the object-specific application class returns the relevant data to the Change Notification process, the process packages the data into XML messages that it then sends to the dashboard system. A single message can contain data from more than one object, though all objects must be the same type. For example, one message can contain data from multiple cases, but not from a mix of cases and leads. This bundling improves performance.
When you define dashboard settings in the PeopleSoft system, you can set a maximum message size. However, data from a single object is never split into multiple messages. Although it is unlikely that a single object would generate data in excess of the maximum message size, if this were to happen, the message would exceed the maximum.
The system sends the XML messages to the JMS server for the dashboard system. Within Enterprise Link, an Enterprise Message Source (EMS) that you established at installation time, subscribes to the message. Enterprise Link plans, which perform the transformations to be applied to the data, process the data and save it to the ADC.
Message queue entries are not deleted until the PeopleSoft system receives an acknowledgement from the JMS server. If message receipt is not confirmed, PeopleSoft reprocesses the queue entries. This architecture ensures that data is not lost if the system goes down or the messaging connection is broken.
The Change Notification Process
The following diagram illustrates the Change Notification process:
Change notification process
These steps describe the process that the diagram illustrates:
An online change or a batch data load request creates message queue entries.
If the Change Notification process is not already running, the daemon that was set up at installation time reacts to the arrival of items in the message queue by starting the Change Notification process.
The Change Notification process marks the current set of entries in the change notification queue as being in process.
The Change Notification process queries the current set of marked entries to consolidate entries for the same object type.
For each object ID, the Change Notification process collects the data that is to be sent to the dashboard.
To do this, the Change Notification process invokes the application class that is appropriate for the object type.
The Change Notification process builds an XML message containing the object data for the changed objects.
A single message can include data for multiple queue entries for the same type of object. Bundling the data this way reduces traffic and individual message processing overhead.
The message is published to a JMS server.
JMS messaging is handled through the application server for online processes and through process scheduler for batch processes.
The JMS server acknowledges the receipt of the message and the Change Notification process removes the entries that were marked as “in process” from the change notification queue.
If there is an error, the Change Notification process will retry the operation. If neither an error nor an acknowledgement is received, the entries remain in process and are therefore sent again during the next pass of the Change Notification process.
The Enterprise Link EMS that subscribes to the message receives the message.
The dashboard system processes the message, updates the ADC, and updates any reports that are being displayed.
The Change Notification process checks for new entries in the queue:
If entries are found, the Change Notification process repeats the process with the new entries.
If no entries exist, the Change Notification idles for five seconds and then checks the queue for entries again.
A system setting determines the maximum number of times that the process checks the queue before going dormant. If the process goes dormant, the daemon that monitors the message queue starts the process again when there are new arrivals in the queue.
If errors occur during the Change Notification process, the error is recorded in the daemon log, the queue entry that was not processed is returned to the “Waiting for Processing” status, and the system adds one to a counter that tracks how many times the affected entry has been tried. A queue entry that fails three times is removed from the queue.
It is important for an administrator to review the daemon log periodically to see if queue entries have been removed without being processed. The daemon log is a standard feature of PeopleSoft Process Scheduler.
See Also
Enterprise PeopleTools PeopleBook: Process Scheduler