This chapter provides an overview of message publishing rules and discusses how to:
Assign full table publishing rules.
Assign batch publishing rules.
Set up message chunking.
This section discusses:
The Publish utility.
Terms used when defining rules.
The Publish utility automates the process of copying the contents of an entire table into a remote database or legacy system. Use the utility to synchronize data from an existing system when a new PeopleSoft system is installed. The data is chunked based on the MaxMessageSize parameter.
Control the size, number, and frequency of these data messages by using a series of data publishing rules and:
Whether to create header and trailer records.
Where the data comes from.
Whether to chunk the message.
Which related languages to publish.
The publishing rules include pages for full table publishing rules, batch publishing rules, record mapping, languages, and batch programs.
The MaxMessageSize (maximum message size) field in the PeopleSoft Option (PSOPTIONS) table limits the size of the message. Before processing each level zero record, the Publish utility compares the size of the message against the value in the MaxMessageSize field. When the message size exceeds the value in the MaxMessageSize field, the message publishes and a new message starts. You can also specify message chunking in the publish rules for the message, which enables the message to publish when the value of a chunk field changes.
When defining publish rules, you need to know the following terms:
Alternate Chunking Table |
This secondary chunk table is a separate view of an existing chunk table but with one or more field names customized. It enables you to reuse an existing chunk table. For example, if the record EO_BUSUNT_EOC has BUSINESS_UNIT as the chunking field, you can create a view of this table that has BUSINESS_UNIT_IN as the chunking field. |
Batch Publish |
This term describes jobs or processes that run independently from their initiating process. A batch process can also run at one or more predetermined times in the future from the initiating request. A batch process is appropriate for publishing incremental changes to data in a batch environment or when processing large volumes. |
Chunking |
Chunking refers to the automatic breaking of a message into several smaller messages based on values in fields in the level zero record. Chunking on business unit means that all transactions within the message are for the same business unit value. |
Chunking Fields |
Chunking fields are key fields in the level zero record that are used to break the message into parts. |
Chunking Rule |
A chunking rule points to the chunking table. Multiple chunking rules can point to the same chunking table. |
Chunking Table |
A chunking table is a derived or Structured Query Language (SQL) table that contains the fields by which the message is chunked. SQL chunk tables define the valid values of the chunking fields and the nodes to which the message is published. |
Message Header |
The header is data that precedes a message and triggers the subscription process to initialize tables before receiving the data messages. |
Message Trailer |
The trailer is data that follows a message and triggers the subscription process to indicate that all data messages have been received. |
Full Publish |
The full publish process seeds, or initially populates or repopulates, a copy of an entire table into a remote database or legacy system. The entire contents of the table are published to all systems that require a copy of the table. |
All PeopleSoft applications use common, centralized tables and pages to define how to publish full table messages. The Publish utility uses full table publish rules to process the data.
This section discusses how to:
Associate a rule to a message and characterize the rule.
Map a message record to another record.
Specify languages in which to publish messages.
Note. You can create multiple publish rules for the same message. The Publish utility treats each publish rule as a separate publishing cycle.
Page Name |
Object Name |
Navigation |
Usage |
Full Table Publish Rules |
EO_MSGPUBFULL |
Enterprise Components, Integration Definitions, Full Data Publish Rules |
Associate a rule to a message and characterize the rule. |
Record Mapping |
EO_MSGRECMAP |
Enterprise Components, Integration Definitions, Full Data Publish Rules, Record Mapping |
Map a message record to another record. |
Languages |
EO_MSGLANGUAGE2 |
Enterprise Components, Integration Definitions, Full Data Publish Rules, Languages |
Specify languages in which to publish a message. |
Access the Full Table Publish Rules page.
Publish Rule ID |
Enter an ID for the user-defined rule to associate with this message. Assign a logical name to make the rule easy to find.
|
Status |
Select Active to activate the publish definition. |
Create Message Header and Create Message Trailer |
Ensure that the subscribing process does not need the header or trailer process before you clear these check boxes for a Batch Publish message. Header messages trigger special logic (in a PeopleCode program) on the PeopleSoft full message subscription that deletes the existing application records. Also, some applications use the trailer message to indicate that all data messages have been received and to initiate the validation process. The documentation for the individual message should note whether headers and trailers are supported. |
Create Delay Records |
This check box appears only if the message name ends with FULLSYNC_EFF (such as MESSAGENAME_FULLSYNC_EFF). Select this check box to write all future-dated rows to the delay table. Also select this check box in conjunction with record mapping views that publish only the current effective-dated rows. |
Output Format
These options indicate the output of the Publish utility. Valid output formats are:
Message (default value).
Flat file.
Flat file with control record.
Select the flat file option when you set up publish rule definitions for the message. If there is a file layout object with the same name as the message object, you can modify the output format field on the Publish Rule Definition page. You can create multiple publish rules with different output formats for the same message. If you select flat file output, the header and trailer messages aren’t created, and a single output flat file is created even if you already specified the chunking rule. The file layout definition must have a record structure identical to that of the message; if the AUDIT_ACTN field does not exist in the record, you must add it to the file layout record definition.
The option becomes modifiable if there is a file layout object with the same name as the message definition. The directory location of the flat file is determined by the value in the OUTPUT parameter in PeopleSoft Configuration Manager. The flat file name is messageName_SequenceNumber.out, where messageName is the message name and SequenceNumber is the next available sequential number for files with that message name.
Access the Record Mapping page.
Specify the source data for a record in a message from:
A staging table.
A temporary table.
A view.
Regardless of which source table that you use, ensure that the source table field names are identical to the field names in the target message record. Key fields must also adhere to the parent and child relationship. (Keys of a parent record must exist in the child record, in the same sequence.) The Publish utility uses the Source/Order By Record Name field to select rows for publishing. You use key matching to find all child rows of a parent.
The chunk field should be the primary key field in all Source/Order By records. If the chunk field is not a key field of the level zero record, join the chunk field to all records in which the chunk field does not exist by using views in PeopleSoft Application Designer.
Three reasons to use the record mapping feature in a publish rule are:
The published data comes from staging tables or temporary tables.
Using staging tables is the only way to publish rows that have been deleted from application master tables.
The published data needs an order sequence that differs from that of the records in the message.
You must create a view that selects only current effective-dated rows in situations where the application table is effective-dated and the subscribing database cannot process future-dated transactions.
Enter only those message records with a different source or ordering record. If the message record name and the source or ordering record name are identical, do not insert a row for that record on the Record Mapping page.
Publish All Related Languages |
Select to indicate whether to publish the message in all of the related languages. If selected, the scroll area that you use to enter individual related languages is unavailable. This check box is clear by default. |
Publish Base Language |
Select to publish the message in the base language. This check box is selected by default. |
Language Code |
Select the related language in which to publish the message. |
The Publish utility creates the following related language messages:
One message in the base language of the publishing system.
One message for each language in the related language tables for the base tables.
The subscribing system receives the messages in the order in which they are published. For example, if the base language is English, with French and then German as related language tables, the Publish utility creates the messages in this order:
(Optional) Header message.
English message 1.
French message 1.
German message 1.
English message 2.
French message 2.
German message 2.
(Optional) Trailer message.
All applications can use common, centralized tables and pages to define how to publish incremental messages from an application program. The Publish utility uses batch publish rules to process the data from the application program.
This section discusses how to:
Associate a rule to a message and characterize the rule.
Map a message record to another record.
Assign an application program to a publishing rule.
Note. You can link application programs to multiple publishing rules for the same message or different messages. The Publish utility treats each publishing rule as a separate publishing cycle.
Page Name |
Object Name |
Navigation |
Usage |
Batch Publish Rules |
EO_MSGPUBATCH |
Enterprise Components, Integration Definitions, Batch Publish Rules |
Associate a rule to a message and characterize the rule. |
Record Mapping |
EO_MSGRECMAP |
Enterprise Components, Integration Definitions, Batch Publish Rules, Record Mapping |
Map a message record to another record. |
Batch Programs |
EO_MSGBATPGM |
Enterprise Components, Integration Definitions, Batch Publish Rules, Batch Programs |
Assign an application program (PROCESS_NAME) to the publish rule. |
Access the Batch Publish Rules page.
Publish Rule ID |
Enter the rule ID to associate with the publish rule that you define. |
Status |
Select Activeif the publish definition is valid. |
Chunking Rule ID |
Associate a chunking method with the publish rule, if needed. |
Access the Record Mapping page.
Message Record Name |
Enter the name of the record in the message that you want to map to another record. |
Source/Order By Record Name |
Enter the record name that the Publish utility uses to select data. |
This page specifies the source data for a record in a message. It works in the same manner and accomplishes the same purpose as the Record Mapping page for a full table publish.
Access the Batch Programs page.
Process Name |
Enter the name of the COBOL, Structured Query Report (SQR), or Application Engine program that is marking the records to be published. The Publish utility initially receives the process name from the batch parameter record that is created by the application program. The program then retrieves and processes each publish rule for the application process name. The process name can be any 12-character string, as long as it matches what the application program inserts into the batch parameter record. If you select a flat file format, the Publish utility does not create a header or trailer message, and the utility ignores any chunking rules. Instead, the utility creates a single flat file. |
This section provides an overview of message chunking and discusses how to:
Identify when to use chunking.
Select chunking fields.
Create chunking rules.
Define the chunking rule description.
Maintain chunking data for business units.
Maintain chunking data for setIDs.
Add nodes to existing chunking rules.
Assign business units or setIDs to a chunking rule.
Assign chunking rules to a business unit.
Assign chunking rules to a setID.
Create custom chunking tables.
Page Name |
Object Name |
Navigation |
Usage |
Chunking Rule |
EO_CHUNKRULE |
Enterprise Components, Map Chunking Rules, Define Chunking Rule |
Define a chunking rule description. |
BusUnit Mapping (business unit mapping) |
EO_CHUNKBU |
Enterprise Components, Map Chunking Rules, Business Units |
Maintain chunking data for business units. |
SetId Mapping |
EO_CHUNKSETID |
Enterprise Components, Map Chunking Rules, SetIds |
Maintain chunking data for setIDs. |
Add Nodes to Chunk Rule |
EO_ADNODECHUNK_PNL |
Enterprise Components, Map Chunking Rules, Node to ChunkRule |
Add nodes to existing chunking rules. |
Quick Map |
EO_ADDSIDNODE_PNL |
Click the Add button on the Add Notes to Chunk Rule page. This button is available after you have added nodes to the chunking rule. |
Assign business units to a chunking rule. |
Map Business Unit |
EO_ADDNODEBU_PNL |
Enterprise Components, Map Chunking Rules, ChunkRule/Node to BU |
Assign chunking rules to a business unit. |
Map Set IDs |
EO_ADDNODESID_PNL |
Enterprise Components, Map Chunking Rules, ChunkRule/Node to Setid |
Assign chunking rules to a setID. |
If you publish to multiple nodes, you might want the messages to be routed based on a specific field. Chunking rules direct the message. You can use the same chunking fields for breaking a large message into smaller messages, as well as for associating messages with a node (based on those same fields). You set up this kind of relationship between nodes and the fields used to break the message apart (break fields) by using chunking message pages.
You can, depending on some XML content-based logic, use message chunking to route and deliver groups of transactions to different third-party nodes.
For example, consider purchase orders. If you run the full batch publish, each third-party node receives an XML message containing all purchase orders that are dispatched, regardless of whether any purchase orders are intended for that particular customer. With chunking, however, you can set up a chunking rule to chunk a message by customer ID. This creates an XML message for each customer that contains only purchase orders intended for that particular customer.
Note. Do not confuse message chunking with channel partitioning. You use channel partitioning to partition a channel by a level zero key field. If a field exists on level zero of the record in the message by which you can uniquely distinguish and group transactions to be processed in parallel, partitioning the message by this field increases performance. Without partitioning, a PeopleSoft subscribing system must process each incoming message in the order in which the message is received according to the Publication ID (PUBID) field.
Use chunking when:
The message data is large, and the subscriber is consistently interested only in part of the data.
Subscribers can more efficiently process the message data by chunking messages.
You have trading-partner-specific content and legally do not want data to be shared among vendors.
You can chunk messages by:
Locations and inventory shipments by business unit.
Customers by setID.
Employees by department or company.
Sales order acknowledgements by setID and customer ID.
Purchase orders and purchase order changes by setID and vendor ID.
To maximize performance and prevent application developers from maintaining complicated views of the data, create staging or temporary tables that contain the chunking fields as the highest order key fields.
Chunking fields can affect performance and alter the options that are available to the Publish utility. The Publish utility creates SQL for each table that is defined in the message object. Tables that are defined in the message can be mapped to an alternative source table or viewed on the Record Mapping page under Publish Rule Definition page.
The source table (or view) serves two purposes:
It enables the data that must be published to come from a source other than the table that is defined in the message.
It enables the data to be ordered so that the Publish utility can process rows in the correct sequence.
Because the SQL is run only once and includes a subquery against the values in the chunking table, you must define all chunking fields in every table that is used to retrieve data for the message. The SQL order-by clause is set according to the key fields that are defined in the table. The result is that chunking fields must be defined as key fields for the Publish utility to work.
Example of Generated SQL for Chunking
This is an example of SQL that is generated for chunking:
Select * from PS_INV_ITEMS A where EXISTS (Select 'Y' from PS_EO_SETID_EOC B where B.CHUNK_RULE_ID = 'SETID' and B.EFFDT = '20000201' and A.SETID = B.SETID) order by A.SETID, INV_ITEM_ID, EFFDT
The field that you select to chunk on determines the view table that you must create:
Chunking Field Attribute |
Corresponding View Table |
Chunking field is a key field in level zero table. |
By rule, the chunking fields are also key fields in the child tables. The key fields of a parent table must be key fields in the child table and in the same order. If the chunking fields are not the highest order key fields, create a view that consists of all fields in the source table, with the chunking fields as the highest order key fields, followed by the remaining key fields from the source table. Then map this view table to the source table on the Record Mapping page under Publish Rule Definition page. |
Chunking field is not a key field. |
Create a view of the source data that consists of all fields in the source table, with the chunking fields as the highest order key fields, followed by the rest of the key fields from the source table. Then map this view table to the source table on the Record Mapping page under Publish Rule Definition page. |
Chunking field is not in a source table. |
Create a view table that joins the source table to an existing table that contains the chunking fields. This view must consist of all fields in the source table and the chunking fields from the joined table. The chunking fields are the highest order key fields, followed by the rest of the key fields from the source table. Then map the view table to the source table on the Record Mapping page under Publish Rule Definition page. |
Note. When running a batch publish rule, the Publish utility runs cleanup logic, which either updates fields or deletes rows in the source tables. If the source table is a view that contains a join, then the option to delete published rows fails.
Example
The following sample SQL code creates a view table that joins the PS_OMEC_CP_OPT_DET source table to an existing PS_OMEC_HDR_OUT table that contains the chunking fields. The B.SETID_CUSTOMER and the B.CUST_ID chunking fields are the highest order key fields from the joined table (PS_OMEC_HDR_OUT), followed by the rest of the key fields from the source table (PS_OMEC_CP_OPT_DET).
SELECT B.SETID_CUSTOMER , B.CUST_ID , A.BUSINESS_UNIT , A.ORDER_NO , A.ORDER_INT_LINE_NO , A.CP_MODE , A.CP_COMP_SEQ , A.OPTION_NAME , A.OPTION_VALUE , A.OPTION_DESC , A.VAR_TYPE , A.VAR_LENGTH , A.VAR_DECIMAL , A.PROCESS_INSTANCE , A.AUDIT_ACTN , A.IN_PROCESS_FLG FROM PS_OMEC_CP_OPT_DET A , PS_OMEC_HDR_OUT B WHERE A.BUSINESS_UNIT = B.BUSINESS_UNIT AND A.ORDER_NO = B.ORDER_NO
The chunking rule consists of four tables:
Level |
Table |
Description |
Level 0 |
EO_CHUNKRULE |
A system table delivered with live data. |
Level 1 |
EO_CHUNKEFFDT |
A system table delivered with live data. When a chunking rule is saved, a row is added to this table with the effective date (EFFDT) field automatically populated from the current date and the effective status set toActive. |
Level 2 |
EO_CHUNKNODE |
This is not a system table and is delivered empty. |
Level 3 |
NAME_EOC |
A user-defined chunking table. |
All user-defined chunking table names must end in _EOC. PeopleSoft provides two standard tables: EO_BUSUNIT_EOC for business unit values and EO_SETID_EOC for setID values. The different types of user-defined chunking tables are:
Table Type |
Description |
Derived Table |
Contains only the chunking fields. Can be used by the Publish utility to chunk the message whenever the value of the chunking field changes. In derived tables, there is no relationship between the value of the chunking fields and message node names that are used to route the message. OnRoute PeopleCode needs hard-coded routing logic or additional tables to route the message to the appropriate nodes. |
SQL Tables |
Contains the fields:
Limits the published data to the values of the chunking fields in the chunking table and contains the message node name that is used to route the message. |
Alternate Chunking Tables |
Enables reuse of existing chunking tables. Must end in _EOV. |
Access the Chunking Rule page.
Chunk field are the chunking fields that are defined in the Chunk Tablefield.
The chunking fields appear in this scroll area to verify that the correct chunking table was entered. PeopleSoft provides chunking rules for business unit and setID for all application databases. Adding a new chunking rule inserts a row into the EO_CHUNKEFFDT table, with a default effective date (EFFDT) of the current date.
Access the BusUnit Mapping page.
All four levels of the chunking rule tables appear.
PeopleSoft provides chunking tables for business unit and setID that are maintained by a series of components (such as components that are created for maintaining the business unit chunking table).
You can use each component to update the underlying relationship between the business unit and the subscribing nodes. You can maintain the data either by business unit or by node, individually or as a group, to reduce the amount of entry work.
Access the SetId Mapping page.
All four levels of the chunking rule tables appear.
PeopleSoft provides chunking tables for business unit and setID that are maintained by a series of components (such as components that are created for maintaining the business unit chunking table).
You can use each component to update the underlying relationship between the business unit and the subscribing nodes. You can maintain the data either by business unit or by node, individually or in a group, to reduce the amount of entry work.
Access the Add Nodes to Chunk Rule page.
To add nodes to an existing chunking rule:
Select the check box in the Add column of the nodes that you defined earlier.
Click the Save button to display the Add Chunk Values column.
Click the Add button in the Add Chunk Values column for the nodes that you want to add.
The Quick Map page appears.
Access the Quick Map page.
If you previously accessed business unit chunking rules, you can add business units to a chunking rule. If you previously accessed setIDs, you can add setIDs to a chunking rule.
Note. You cannot access the Quick Map page without first using either the BusUnit Mapping page or the SetId Mapping page to add an effective-dated node to the chunking rule ID.
Select All and Deselect All |
Click to add or remove all business units that are assigned to the node. Add check boxes are selected for business units that are assigned to the node. |
Access the Map Business Unit page.
Select All and Deselect All |
Click to add or remove all message nodes that are assigned to the business unit. Add check boxes are selected for message nodes that are assigned to the business units. |
Access the Map Set IDs page.
Select All and Deselect All |
Click to add or remove all message nodes that are assigned to the setID. Add check boxes are selected for message nodes that are assigned to the setIDs. |
This section discusses how to:
Create a custom chunking table.
Create a view for the component search record.
Create maintenance pages.
Create a component.
Create routing PeopleCode.
To create a custom chunking table:
Select File, Open in PeopleSoft Application Designer.
Select Record in the Definition drop-down menu.
Open the EO_BUSUNT_EOC record.
Save the record as YOUR_TABLE_EOC.
Remove the BUSINESS_UNIT field.
Insert the custom chunking fields at the bottom of the record.
Select File, Save.
Build the SQL table.
To create a view for the component search record:
Select File, Open in PeopleSoft Application Designer.
Select Record in the Definition drop-down menu.
Open the EO_CHUNKBU_VW record.
Save the record as YOUR_TABLE_VW.
Select the Record Type tab.
Open the SQL editor.
Modify the Where clause.
Change WHERE RECNAME_CHUNK=EO_BUSUNT_EOC to WHERE RECNAME_CHUNK=YOUR_TABLE_EOC.
Select File, Save.
Build the SQL view.
You can also create maintenance pages.
To create maintenance pages:
Select File, Open in PeopleSoft Application Designer.
Select Page in the Definition drop-down menu.
Open the EO_CHUNKBU page.
Save the page as YOUR_PAGE.
Select the Order tab.
In the Type column, double-click Grid to open the Grid Properties dialog box.
Change the value in the Main Record and Page Field Name fields to YOUR_TABLE_EOC.
Click OK.
Delete the business unit and description columns.
Add chunking fields from YOUR_TABLE_EOC.
Select File, Save.
To create a component:
Select File, New in PeopleSoft Application Designer.
Select Component in the New Definition box.
Select Insert, Page Into Component.
Enter YOUR_PAGE.
Click Close.
Select File, Definition Properties
Select the Use tab to edit component properties:
In the Search record field, enter YOUR_TABLE_VW.
Select the Update/Display, Update/Display All,and Correction check boxes.
Click OK.
Select File, Save As, and save the page group as YOUR_COMPONENT.
Add YOUR_PAGE_GROUP to YOUR_MENU that is used by your application.
OnRouteSend (and OnRouteReceive) are PeopleCode events that are tied to the message for routing, based on the message contents. If you want the contents of the message (such as a message chunking field value) to determine which subscribing nodes should receive the message, OnRouteReceive PeopleCode must contain the logic to examine the message and return a list of subscribing nodes.
PeopleCode functions provided by common components, GetNodes and RtnNodes, work with any message and chunking rule. For a given message, these nodes select the chunking rule for the publish rule that is assigned to the message.
The functions do the following:
Build SQL based on the chunking fields as defined in the chunking table.
Extract chunking field values from the message.
Run the associated SQL.
Compare the array of nodes returned to the application server against the list of nodes for the message channel.
Create a publish contract for nodes in both arrays.
You can override the publish rule from the message, specifying an optional parameter when calling the functions.
Return an array of nodes that is based on the nodes that are assigned to the message channel if the publish rule is invalid or does not contain a chunking rule.
Returning an array of nodes enables the functions to work regardless of whether chunking is set up for the publish rule.
To route any message that uses chunking, use generic PeopleCode functions.
These functions are called from routing PeopleCode:
GetNodes |
Returns an array of nodes to the application server. |
RtnNodes |
Returns an array of nodes to the calling PeopleCode. |
These functions are internal functions:
FndNodes |
Builds an array of nodes for the message. |
FndChannelNodes |
Builds an array of nodes for the channel. Used when there is no chunking rule for a publish rule or when the publish rule is not found. |
GetPubRule |
Selects the chunking rule for the publish rule. |
GetChunkInfo |
Selects the chunk table for the chunking rule. |
BuildSQL |
Builds SQL to select nodes from the chunking table for specific chunking field values from the message. |
GetValue |
Gets the chunking field values from the message. |
This is an additional function:
HasNodes |
Determines whether a chunking field is mapped to any nodes for a particular chunking rule. |
The following code example shows the logic that you can add to SavePostChange PeopleCode for the Customer_General component to verify if the setID can publish the message by calling the HasNodes() function:
Declare Function HasNodes PeopleCode FUNCLIB_EOEIP.PUBLISH_ROUTE_PC FieldFormula; Local Message &MSG; Local Rowset &RS0; Local string &PublishRule; &MSG = CreateMessage(MESSAGE.CUSTOMER_MSG); /* Check if message is active */ If &MSG.IsActive Then &RS0 = GetLevel0(); &PublishRule = "CUSTOMER_SYNC"; /* Call function passing publish rule and rowset, which returns true if this setID can publish the message */ If (HasNodes(&PublishRule, &RSO)) Then &RS0 = GetLevel0(); &MSG.CopyRowsetDelta(&RS0); &MSG.Publish(); End-If; End-If;
The following code example shows the PeopleCode that you can add to the OnRouteSend event for the message channel that is associated with any message where chunking is assigned to a publish rule:
Declare Function GETNODES PeopleCode FUNCLIB_EOEIP.PUBLISH_ROUTE_PC FieldFormula; Local string &PublishRule; /* Set PublishRule if you want to override the value in the message PSCAMA. */ /* &PublishRule = "CUSTOMER_SYNC"; */ /* Call function that looks at setID of first transaction in the message and returns a list of subscribing nodes to route the message */ GETNODES(&PublishRule);