We’ve discussed in a general sense how the Scan Engine works. Let’s make this more specific.
Figure 91. SAFR Scan Engine Overview
Select Phase
Users define views in the Developer Workbench, which are stored in the View and Metadata repository. The first step in the SAFR Scan Engine is the VDP Builder. This program accepts parameters that tell it which views to select from the metadata repository.
The output is a file called the VDP, View Definition Parameters, which contains all the needed data from the View and Metadata repository for the Scan Engine to perform the requested process.
The following shows a view in a VDP through a small utility called the VDP Explorer which Al Sung built for debugging processes.
Figure 92. Copy Only View VDP
Logic Phase
The next phase of the process turns the metadata and views contained in the VDP into a logic table. A logic table is a set of detailed commands in a very efficient order to read records from an event file, perform record level filtering, and write selected records to the output file.
The following is a printout of the logic table.
Figure 93. Copy Only View Logic Table
The logic table has the following characteristics:
- The Logic Table rows are numbered consecutively from 1 to the last row.
- Each logic table begins with an HD or Header Function and ends with an EN or End Function. SAFR extract engine will perform all of the functions discussed below for each event file to be scanned.
- Each Event File to be read begins with an RENX or Read Next record function and ends with an ES or End of String function. SAFR extract engine performs all of the following functions for every record within an event file.
- Each View begins with an NV or New View function and ends with a WRxx Write function of one type or other. A copy view ends with an WRIN Write Input record function.
- The only real user specified function within this view is a CFEC function, a Compare Field, Event file field to Constant. The Event File field – the field to be tested – is at position 1, for a length of 9, of a format 3 which is zoned numeric.1 It is to be equal (a 001 comparison) to the constant value of “522349999” of a length of 9
- The CFEC function uses the GOTO ROWx columns to say what should be done based upon the test. If the CFEC comparison with the current event record proves true, execution continues at GOTO Row 5, writing the input record. If the comparison proves false, execution continues at GOTO Row 6 which loops to the RENX function to read the next event record.
- The CFEC row was created because the view contains the following general selection logic text: “SELECTIF({LEGAL_ENTITY}= 522349999)”
Reference Phase
Because the view contains no lookups, the reference phase performs no functions.
Extract Phase
I believe it was during the same month Doug called and asked me to make the DB2 table to test the new SAFR access methods that one day he said he had another feature for me to test out. He showed me how to turn it on, and explained the output. I realized what a powerful debugging tool it was. Doug told me that a few months before when we were debugging a problem, perhaps an OC7 and had to search the dump to find what record caused the problem, I had commented in passing that I would like to be able to see a printout of the logic table against each record of the event file. In that way, I would have known what record and what function I was doing in the view that caused the problem. This feature is known as the Logic Table Trace. We’ll use it to explain the Extract Engine.
Suppose our input file looks like the following:
Figure 94. Sample Event File
In this display, the hex versions of the displayed values are immediately below the characters. So a display 5 is composed of the hex values F5. This allows viewing the data in packed format – a format that compresses numbers to only use half a byte each – in the amount field.
The logic table trace output from running the extract engine against this input file is as follows:
Figure 95. Copy View Logic Table Trace
It shows that event record 1 from the DD name (a file handle) of “EVENT” was processed against NV, Logic Table Row number 3. The record then went against the CFEC function on logic table row 4. The Value 1 shows the value in the event record – the “E” in CFEC – and Value 2 shows the constant in the logic text. The two values are equal. So the next row to be executed is the true row, or the GOTO 5 row. So execution continues at logic table row 5, the WRIN row which copies the event record to the output file.
The SAFR Scan Engine has just completed the inner most loop of the logic table. It then performs an RENX to obtain the next event record. The RENX is not shown in the logic table trace, but the Event Record count is incremented, and we return to the NV row. This record is processed the same as the previous record.
On the third record, the CEFC comparison shows that the value in the event file record is not equal to the constant. Thus the GOTO 2 row or false row is executed. Since this is the last record in the event file, the program ends.
The output file then contains the following:
Figure 96. Copy View Output File
Format Phase
Because this view was a copy only view, the format phase is not required and performs no work.
Analysis
The function provided by this view is very simple. However, because we have generated machine code, it is even more efficient than is available in COBOL. Similar to what we have shown in Programming Tools, the CFEC function requires only a CLC Compare Logical Character, and a BC Branch on Condition assembler instruction. This is contrasted with the assembler instructions generated from the COBOL IF statement which uses a much more generalized pattern.
Doug explained one day that because SAFR is designed around an event file and an extract file, his programs dedicated something called base registers, a key register or set of “eyes” to the event file and the extract file. Because COBOL has to be able to solve problems that require opening, reading, and writing to scores of files simultaneously which requires more than the 16 eyes available on the computer, it does not do this. This fact alone can save millions – or even billions – of machine instructions when the event file contains 3 million, 300 million, or 3 billion records.
The copy view is perhaps the simplest type of SAFR process. In the next execution, let’s make it a bit more complex. This requires us to understand the extract record, as opposed to the event file.
Parent Topic: Part 5. The Programmer