The last step in my training class was a class project. I remember I was assigned the task of working with the database to produce the final outputs of the system. Although I didn’t know it, I was using a technology developed to solve both ends of the subsystem architecture, the data capture and reporting. Over the years IT recognized the limits of the traditional reporting model, as implemented in the subsystem architecture. There were a number of initiatives to overcome them. They go by names of data management, decisions support systems, executive information systems, business intelligence and data warehousing, and started with the advent of data bases; the same technology that spurred McCarthy’s initial paper.1
Operational versus Informational
In data warehousing, one of the latest approaches to overcoming reporting problems, the reports supported by the posting processes are called operational reports; the systems which do the posting are operational systems. Data warehousing says that reports which are needed beyond the posting attributes are informational. By some definitions, data warehousing is intended to support the informational reports, not the operational.
Bill Inmon, often referred to as the father of data warehousing, in his preface to his very influential book Building the Data Warehouse, noted:
“Databases and database theory have been around for a long time. Early renditions of databases centered around a single database serving every purpose known to the information processing community – from transaction to batch processing to analytical processing. In most cases, the primary focus of the early database systems was operational – usually transactional – processing. In recent years, a more sophisticated notion of database has emerged – one that serves operational needs and another that serves informational or analytical needs …”
“The split of operational and informational databases occurs for many reasons:
- The data serving operational needs is physically different data from that serving informational or analytical needs.
- The supporting technology for operational processing is fundamentally different from the technology used to support informational or analytical needs.
- The user community for operational data is different from the one served by informational or analytical data.
- The processing characteristics for the operational environment and the informational environment are fundamentally different.”2
I wouldn’t disagree with any of these points as they related to the state of system development at the time. However, I would argue the next evolution in reporting is recognizing that the informational systems must take on more operational characteristics. The finance systems provide the example of why that needs to be done, and how to accomplish it.
- The finance systems cannot completely separate the informational from the operational. The finance system is an informational system. In fact, it was the original enterprise data warehouse. The finance system has to deal with history, whereas the operational systems do not. Yet, with the required timing and critical nature of it, and because it does generate some original data, the finance system is an operational system. In fact, because it was the first to be automated, it is perhaps the original operational system. Thus it is both informational and operational. There is constant demand for higher integration of this finance data with additional attributes held in other originating operational systems.
- Because of the blended operational and informational need, the technology has to support both. The missing piece of technology is not the database. Data warehousing has driven innovations in the database that were needed to support better reporting. The missing technology is a high performance posting and report generation engine. We have not advanced the posting processes at the same rate as the database, exacerbating the distinction.
- The differentiation between types of users is not as distinct as it once was because operational reporting at times involves summaries and informational at times involves details; Demands for better information and tighter integration will cause this distinction to become less and less over time.
- Focusing on system performance – a near single minded focus on performance – can significantly reduce the differences in processing characteristics of the two environments and thus enable greater reporting, greater system flexibility, and lower IT maintenance cost by not having to support two different environments and processes.
To highlight the overlap further, consider certain finance processes which generate large amounts of data, and those which require access to detail business events.
Allocation Processes
A number of years later I was the architect for an allocation process for a large insurance company. An allocation process, as we noted in Management Accounting, assigns costs or overhead to individual production items, like the salary of the CEO to each bank loan. More often it is for costs such as the HR, finance, IT, and other support functions whose output is not measured by what the company sells.
The input to these processes could be composed of thousands to millions of business events, such as a listing of loans. The outputs could be millions more transactions as each one is assigned a portion of the cost. Are these types of processes operational or informational? I think it is difficult to say. Do these types of processes fit in a business event based model. I think they do.
Imagine if you will the desire of someone to know how much of the CEO’s salary is attributable to a single insurance policy. If there were a need to manage such a low level of detail, then the outputs from the allocation process would be the product of the number of rows to be allocated to (the count of the number of specifically produced items) times the count of the items to be allocated. That could be a very large number indeed.
However, that fact that no one manages these detailed events does not mean these types of processes have no value. A well formulated set of allocation rules can inform and motivate some important behaviors. And we shouldn’t doubt the fact that they are business events just because these processes produce millions of outputs. Business events are anything we want to plan, control, execute or evaluate. Each resulting row of data from an allocation process is an event and could be created in isolation from all others. We simply have millions of business events occurring in a relatively short period of time as the allocation process is executed. And what triggers the event is the fact that another event has happened; if not generated in batch from a time trigger, it might simply be based upon capturing the original transaction record.
Funds Transfer Pricing
A similar type of process is called Funds Transfer Pricing. This process assigns a cost to funds provided to various parts of a financial services company. For example, the commercial loan department might have an opportunity to invest for a short time in a particular high risk real estate fund. Other opportunities exist in other areas of the organization. The financial institution needs to assess what is the best use of its funds, something the market uses interest rates to decide. Funds transfer pricing is a type of internal interest rate calculation.
The outputs from this process are intended to motivate appropriate behaviors – to inform specific actions if you will – choosing the best use of funds available. If risk is not assessed correctly, the appropriate returns may not be obtained. The end result can be very damaging.
Determining interest rates requires understanding risk, which means dealing with the details of loans or deposits. The greater the number of parameters that can be used in the formulas, the greater the assessment of risk, and the better the motivation. Having the ability to do funding at the detail level increases its accuracy. Again, this is about generating business events, not reporting on them.
Balance Versus Transaction Based Processes
Other processes reflect the passage of time, often called temporal events. Accrual entries are a reflection of this; others are like allocations and funds transfer pricing, and revaluing balances in foreign currencies to reflect the impact of exchange rate fluctuations. These types of processes often depend upon balances as of a point in time.
For example, currency revaluation is the process of making journal entries to reflect a change in exchange rates. Suppose you live in the US and keep most of your finances in US dollars, but you have one account in the UK where you keep 100 pounds. If you want to produce a balance sheet, you can’t add a 100 pound row in the midst of the dollar rows on the report; the report would be meaningless. So you have to convert the 100 pounds into dollars using the exchange rate as of the date of the balance sheet. You need a journal entry affecting an income statement account that reflects the difference in the exchange rate multiplied by the 100 pounds.
The event that causes creation of this entry could be the change in exchange rates, but they can change minute by minute depending on which rate is chosen. Mathematically creating journal entries every second and then summarizing them will result in the same answer as waiting until the balance sheet date to create a single entry with the accumulated exchange rate changes. It is much more efficient to create the entries at some predetermined time – like closing the books. Thus processing the record containing the 100 pound balance during the close process can become the triggering event. This first can be used to perform processes against detailed balances covering history during the night, and then processing in the same way the remaining transactions that come later.
Note this point: Just because a process is a point-in-time process does not mean it must use a balance record as input. Because all balances are the result of accumulating individual movements, mathematically the results are the same if the amount is calculated from the movements or from the balance.
And remember, all these processes become more accurate or capable of informing more specific actions if they are informed by the details of preceding business events. I have often heard experts, with tools that struggle with volume, question why anyone would need all the details from these types of processes. Who has the time or need to go look at an individual product, be it a loan or deposit or policy or something else? Doesn’t leverage require managers to find ways of not having to do everything manually at the detailed level?
This argument moves from the subject of generating business events to reporting on them. It misses the point that the number of summaries required from these processes are quite expansive. The principles involved in reporting on the results of processes, including these newly generated business events, are exactly the same as for any other business event. Business events are anything the business wants to plan, execute, control or evaluate. By and large, reporting is the process of accumulating business events, and reporting from the detail provides more flexibility than any other approach.
In the end, the line where operational processes stop and informational processes start is at the back end of the finance processes, just before report production – when data creation stops and only summarizing to produce the report remains. The implications on reporting processes from any finance generated data are no different from business events which are sent to the finance system.
Let’s consider the reporting piece of IT architecture in more detail.