Temporary tables are used exclusively with Application Engine programs
and are intended to provide parallel processing. The Application Engine
programs may be executed online via CallAppEngine or invoked through the
Process Scheduler.
Parallel or concurrent processing allows multiple instances of an
Application Engine program to execute against the same tables while
drastically reducing the risk for table contention.
Batch process performance can be improved by splitting the data to be
processed into groups and simultaneously running multiple instances of
the program to deal with different groups of data. For example, students
could be processed by last name by splitting the group into multiple
groups alphabetically.
If you have a program that uses a temporary table and it is invoked
multiple times, that single temporary table could be used concurrently
in multiple executions of the code.
This could create unpredictable results since the different instances of
the code would be issuing Deletes, Inserts, and/or Updates
unsynchronized with each other.
You could solve this problem by creating multiple temporary tables as a
pool of tables. Each invocation of your program would have to allocate
an unused temporary table, mark it as “in use” and release it back to
the pool when you are through with it.
By using the Temporary Table record type definition, you are able to
define a record and the PS Build process will build multiple uniquely
named copies of your Temporary Table as a pool.
Additionally, PS does Temporary Table management for your Application
Engine programs. You can code your program with supplied meta-SQL
(%Table) so each execution of your Application Engine program will be
given access to its own copy of the Temporary Table for its exclusive
use. When the program ends, the table will be returned to the pool of
Temporary Tables.
Temporary Table Pools
Online Pool
- Defined in PeopleTools Options by PS Admins. Our current setting is five (5).
- Used by Application Engine programs invoked by CallAppEngine.
Batch Pool
Defined in Application Engine program properties. Properties that must be set include:
Assign Temporary Tables to the Application Engine program (Temp Table tab)
Set the Instance Count (Temp Table tab). The number entered here will
dedicate the number of instances requested for the AE program.
Set the Runtime option (Temp Table tab). This is the action the AE
program will take if batch/dedicated tables cannot be allocated at
runtime.
Continue – the base table will be used instead (using Process Instance)
Abort – program execution terminates.
Set the Batch Only checkbox (Advanced tab). If the program will only be
run in batch mode and not executed from the CallAppEngine() PeopleCode
function, you should use this checkbox. Any dedicated temporary tables
used for Batch Only programs do not have online instances created.
Important Notes
Instance Count and the Batch Only checkbox should be set prior to building the record definition.
If the temporary table was originally built with online instances
(Batch Only checkbox is not checked) and then changed to “batch only”,
online tables must be dropped manually.
Program Meta-SQL
A critical step in implementing parallel processing is to make sure that
you’ve included all of the appropriate meta-SQL within the PeopleCode
that your Application Engine program executes.
To reference a temp table (Online or Batch), you need to use:
%Table(record)
You can reference any table with %Table, but only those records defined
as Temporary Tables get replaced by Application Engine with a numbered
instance of a Temporary Table from the Temporary Table pool.
For batch/dedicated Temporary Tables, when Application Engine resolves
any %Table, it checks an internal array to see if a Temporary Table
instance has already been chosen for the current record. If so, then
Application Engine substitutes the chosen table name. If there are no
more batch/dedicated instances available, then Application Engine uses
the base table instance by default (if the Runtime option Continue has
been chosen). Regardless of whether %Table is in PeopleCode SQL or in an
Application Engine SQL Action, the program uses the same physical SQL
table.
For synchronous calls to Application Engine, an available instance
number will be selected at random according to internal rules.
Synchronous refers to using the CallAppEngine PeopleCode function; all
other methods that you use to invoke Application Engine programs are
asynchronous which means the page is not “frozen” while the program runs
to completion.
Populate your Temporary Table Process Instance with the Process Instance
All temporary tables should be keyed by Process Instance as a general
rule. Also, if you have opted to use the “Continue” runtime option when
batch/dedicated tables can’t be assigned, Process Instance is required
as a key field. The current Process Instance is automatically put into
the State record, but when you Insert rows into your Temporary Tables
you must supply that Process Instance.
%ProcessInstance or %Bind(PROCESS_INSTANCE)
This meta-SQL returns the numeric (unquoted) Process Instance. The
%PROCESSINSTANCE meta-SQL is more efficient and faster than using the
%Bind(PROCESS_INSTANCE).
Note: The Process Instance value is always zero for programs initiated
with CallAppEngine. This is because the program called with
CallAppEngine runs “in process”, that is, it runs within the same unit
of work as the component with which it is associated.
Clear Temporary Tables (%TruncateTable)
You do not need to delete data from a Temporary Table manually. The
Temporary Tables are truncated automatically at the end of processing.
If the shared base table has been allocated because no batch/dedicated
instances were available, then Application Engine performs a delete of
rows by Process Instance instead of performing a truncate. In such a
case, the PROCESS_INSTANCE is required as a high-level key.
You can perform additional deletes of Temporary Table results during the
run, but you will need to include your own SQL Action that does a
%TruncateTable.
Implementing Parallel Processing
There is no simple switch or checkbox that enables you to turn parallel
processing on and off. To implement parallel processing, you need to
complete a set of tasks in the order that they appear in the following
list.
1. Define your Temporary Tables by defining and saving your Temporary Table records as type “Temporary Table”.
2. Set the Temporary Table Online pool. This will set the basic
Temporary Table Online pool based on the PeopleTools Options
specifications. (Note: This is done one time by the PS Admin group).
3. Assign Temporary Tables to your Application Engine program in its
Program Properties, setting the appropriate number of Instance Counts
and Runtime option.
4. Build/Rebuild your Temporary Table record. This will build the
necessary Batch temporary tables into that record’s Temporary Table pool
for use at execution time.
5. Code %Table meta-SQL as references to Temporary Tables in your
Application Engine program, so that Application Engine can resolve table
references to the assigned Temporary Table instance dynamically at
runtime.
