Ponder Pro Servie - Siebel Benchmark

Old Siebel eScript T (typeless) Engine vs. New ECMA Version 4 ST (Strong Type) Engine

Introduction
Oracle has recently introduced a new eScript engine known as the Strong Type or ST engine. This engine is available in versions 7.7, 7.8 and 8.0. The ST engine is the default engine in Siebel 8.0 and customers are encouraged not to disable it.

Oracle cites these benefits for the new ST engine.

+ Improved performance
+ Improved scalability
+ New features only available with the ST engine

In order to better advise our customers we undertook an independent benchmark comparing the T engine with the new ST engine. Our benchmark included these tests

+ T engine
+ ST engine
+ ST engine with deduce types enabled/checked
+ ST engine with all variables given types

We ran a series of two tests. The second test was added after we did not see the results we expected and the results Oracle cites in their material from Oracle Open World 2006. The tests were:

+ A transaction mix that included math, string and Siebel objects.
+ A transaction mix that included just math and string operations but excluded Siebel objects



Results Findings
Three major discoveries were found on this benchmark. The first was that for the first mix of transactions we did not see very much response time improvement. While CPU on the object manager improved with the ST engine the overall transaction time did not improve very much due to this transaction mix spending the majority of time doing SQL I/O.

The implication from this is that end users may not see much improvement in response time from the new ST engine although their CPU certainly will improve.

The second and third discoveries were identified on the second mix of transactions where we removed the Siebel object scripts and all I/O from the transaction mix. The second finding was when it comes to CPU intensive tasks like mathematical operations and string operations that the new ST engine is about twice as fast as the old T engine.

The third and very much unexpected finding was that using the ST engine with strongly typed variables resulted in horrible performance! With strongly typed variables the performance of the ST engine was twice as slow as the T engine. After opening a service request with Siebel Technical Support it was determined that two change requests exist for this issue and it is considered a product defect.

It was later determined that variables of type String, Number and Boolean are slower when typed on var statements. Other data types such as PropertySet, BusObject, BusComp and Date were as fast or even faster when typed on var statements.

Adding types to var statements presents other problems too. Here are examples of var statements with and without types.

var myString;

var myString : String;

One very undesirable aspect of typing variables is that object compares for String, Boolean, Date and Numeric data types no longer works the way it used to with the old T engine. If you type your var statements you must change your object compares to use the valueOf() operator.

For example if you type your strings this won’t work:

var myString : String;
var yourString: String;

if (myString == yourString) //This won’t work anymore

And instead you must code:

if (myString.valueOf() == yourString.valueOf())

Changing most of your if statements is painful even with our PPS Tools Helper product.

In addition to the above issue with adding types to your var statements you will find a service request that tells you not to type Arrays.

Update - 4/4/2007
After conferring with Oracle Product Marketing we investigated the use of the corresponding primitive data types of chars, float and bool instead of the object types of String, Number and Boolean.

We were delighted with the results. Performance was as good or better than non typed yet will still had the advantages of typing our variables. Plus with the primitive data types we did not have to use the quirky valueOf() function on comparisons.

Conclusion
We love the new ST engine and recommend for it’s features alone.

Given the finding above with strongly typing your var statements our current recommendation about the ST engine is:

Use the new ST engine but be sure to use primitive data types of chars, float and bool instead of String, Number and Boolean. Don't type Arrays until the stability issue that currently exists is fixed.

Customers with existing eScript code bases need not necessarily add types to their var statements. These customers should still see a big performance improvement.

Also be sure to carefully test your application with the new ST engine before deploying to production! An important product defect relating to passing function parameters by reference has only just been fixed with 7.8.2.5 and 8.0.