Processing Staged Data

Page Name

Definition Name

Usage

Process FullSync Data Page

LM_RNCTL_FULL_LOAD

Run the Process FullSync Data (LM_LD_STGDAT ) Application Engine process to validate full sync data in the staging table, load validated data into Enterprise Learning Management, and delete data from the staging tables.

Process Asynchronous Data Page

LM_RNCTL_ASYNC

Run the Process Asynchronous Data (LM_LD_ASYNC) Application Engine process to validate asynchronous data in the staging tables, load validated data into Enterprise Learning Management, and delete data from the staging tables.

Certain types of data that you import into Enterprise Learning Management through EIPs goes into intermediary staging tables before it is loaded into the system. A control value assigned to each row of incoming data identifies whether the data is from a full sync or an incremental sync EIP.

After an EIP imports data into the staging tables, you run a validation process to ensure that there are no data errors. Valid data can then be quickly loaded without the risk of errors. You can delete data in the staging tables as often as needed. The system uses a truncate command on supported platforms, and uses a delete command otherwise.

Staging tables are used for data that undergoes frequent changes or for which there is a high volume in Enterprise Learning Management. Steps for running the validation and load processes depend on whether you are importing data from a full sync EIP message or an incremental sync EIP message:

Full Sync Processing

The typical sequence for processing data from a full sync EIP is as follows:

  1. Run the delete process to delete any existing full sync data from the staging tables.

  2. Run the full sync EIP data publish message from the external application, such as PeopleSoft HR, to import data into ELM staging tables.

  3. Ensure the full sync message has completed processing in PeopleSoft HCM and Enterprise Learning Management, and that no errors exist. If errors exist, resolve them before you continue.

  4. Use the Process FullSync Data component to run the validation process.

  5. View the log file for errors and ensure the data is corrected.

    If you are satisfied, continue to the next step. Otherwise, correct the errors and return to step one.

    Note: Make the corrections in the system of record. For example, if you are importing person data and information is missing, you may need to contact the HR administrator.

  6. Run the upload process to load the validated data into the application tables.

  7. Run the delete process to remove the full sync data from the staging tables.

Note: Rerunning a full sync EIP that uses a staging table does not delete data that resides only in Enterprise Learning Management, such as learner IDs, job codes, and learner department assignments.

Incremental Sync Processing

When incremental sync EIP messages import data into the staging tables in Enterprise Learning Management, a sequence log table captures the messages in sequence. When you initiate the validation process, data is validated and loaded in the order in which it was received.

When you process asynchronous data, the system automatically validates all incremental sync data in the staging table and loads the valid data. After the process is complete, you can check the message log files for errors. The system provides an option to automatically delete incremental sync data in the staging tables after the upload.

To ensure consistency between the data imported into Enterprise Learning Management and data in the source database, schedule the validation and load process for asynchronous data to run at a regular interval, such as every hour or every day. A minimum frequency of daily is recommended.

Run the full sync EIP for the data you want to import before running any processes for staged data.

Use the Process FullSync Data page (LM_RNCTL_FULL_LOAD) to run the Process FullSync Data (LM_LD_STGDAT ) Application Engine process to validate full sync data in the staging table, load validated data into Enterprise Learning Management, and delete data from the staging tables.

Navigation:

Set Up ELM > Process FullSync Data > Process FullSync Data

This example illustrates the fields and controls on the Process FullSync Data page.

 Process FullSync Data page

You can validate data, load validated data into Enterprise Learning Management, and delete data in the staging tables in separate processes or at the same time. The process runs in this order, for all selected check boxes: validate, upload, and delete.

Field or Control

Description

Validate Staging Tables

Select to validate the imported data. The process looks for data with missing required values and missing rows. For example, a person record with no name record or an employment record with no job record. The system stamps all valid data with a status of valid, and reports all errors in the process scheduler log.

Upload Data to ELM

Select to move validated data from the staging tables to the Enterprise Learning Management application tables.

Note: The system loads all valid data, including data that is future effective-dated. However, learners who are future effective-dated do not appear in the Internal Learners search page until their effective date is current.

Process

Select the type of data to validate, load, or delete. Your selection here may cause other data entry fields to display on this page.

SetID From and SetID To

These fields appear only when you select Organization Data in the Process field. Enter the lowest and highest SetID values that you want to process.

Employee ID From and Employee ID To

These fields appear only when you select Person Data in the Process field. Enter the lowest and highest employee ID values that you want to process.

Create unique employee ID ranges and run these ranges in separate processes, being mindful of the human resource database's collation sequence (for example, how it sorts data). The process is designed to run multithreaded, utilizing temporary tables to reduce contention. The groups of IDs must be unique have no overlap.

Note: Consider having a technical person use SQL queries to verify that the ID ranges are unique and encompass all employee IDs. To identify your system's employee ID range, use the following SQL statements: SELECT MIN(LM_HR_EMPLID), MAX(LM_HR_EMPLID) FROM PS_LM_STG_PRS —for person load, and SELECT MIN(LM_HR_EMPLID), MAX(LM_HR_EMPLID) FROM PS_LM_STG_PRS_ATT —for workforce load.

Delete Staging Tables

Select to delete the full sync data from the staging tables. This option uses a truncate command on supported platforms, and uses a delete command otherwise.

Note: For organizations with large database volumes of approximately 50,000 employees or more, and no truncate support, have a database administrator drop and recreate the staging tables. Dropping and recreating the tables is more efficient than using the delete command.

FullSync Messages

Each process that you can run using the Process FullSync Data page is associated with a specific FullSync message that is used to import data into ELM.

Process

Associated FullSync Message

Catalog Data

JPM_CAT_FULLSYNC

Cost Acknowledgement

COST_ACK_FULLSYNC

DIF Balance

DIF_BALANCE_FULLSYNC

HR Company Data

COMPANY_FULLSYNC

Instance Qualifier Data

JPM_JP_QUAL_FULLSYNC

Organization Data

DEPT_FULLSYNC

Person Data

PERSON_BASIC_FULLSYNC

Person of Interest Types

POI_TYPE_TBL_FULLSYNC

Persons of Interest Data

PERS_POI_FULLSYNC

Profile Data

JPM_JP_FULLSYNC

Profile Type

JPM_JP_TYPE_FULLSYNC

Training Hour Acknowledgment

TRN_HOUR_ACK_FULLSYNC

User Photo

USER_PHOTO_FULLSYNC

Workforce Data

WORKFORCE_FULLSYNC

Use the Process Asynchronous Data page (LM_RNCTL_ASYNC) to run the Process Asynchronous Data (LM_LD_ASYNC) Application Engine process to validate asynchronous data in the staging tables, load validated data into Enterprise Learning Management, and delete data from the staging tables.

Navigation:

Set Up ELM > Process Asynchronous Data > Process Asynchronous Data

This example illustrates the fields and controls on the Process Asynchronous Data page.

 Process Asynchronous Data page

When you process asynchronous data, the system automatically validates all data in the staging table and loads the validated data. To have the system delete the incremental sync data in the staging table after it loads the data, select the check box labeled Automatically delete completed process.

Note: If the Allow Auto Enrollment check box is selected on the Install Defaults - Enrollment page, the system initiates the LM_AENRL_AAF batch process. This process invokes the Active Analytics Framework (AAF) to evaluate learners against the defined AAF policies for HCM status or job changes and automatically enrolls learners into learning items accordingly.