Hands-off
When I started working in software 25 years ago, the mantra was “Straight Through Processing” - i.e. process automation and limiting manual intervention to exceptional items only. This term is still around, but doesn’t seem to have escaped the post-trade processing world.
Data volumes and risk management drove the need for messaging systems to replace dual-keying of trades and other key information. Message feeds are great, but they’re not perfect – bugs, crashes, poor filters, redelivery, mapping errors etc. all can degrade the consistency intended between the systems.
Commonly in addition to a feed, a periodic reconciliation is done between source and destination systems in order to compensate for the degradation. This will highlight different quality aspects - missing and duplicate records and also mapping errors.
Trouble ahead
While the first two are generally reasonably straight forward (even though the technical reasons may not be) the mapping errors throw up a common quandary in feed quality control. If the two systems in question do not share (or support an export of) the same canonical format and hence a non-trivial amount of transformation (the T in ETL / ELT) is needed.
For regulatory reasons or other high-accuracy scenarios it can make sense to avoid cognitive overlap between the feed and reconciliation logic implementations to get close to a ‘clean room’ verification of feed logic. Use of a 3rd party can make sense here, especially when delivering data externally in standard formats.
Quandary
The quandary is that in order to verify the records fully, the reconciliation must replicate the same transformation logic. This duplication of coding is not ideal - it’s often twice the effort, may be in a different language, may slow down the rec and any breaks need to establish if the rec logic or the feed logic is in error – the problem surface simply grows.
Shifting goals
Using standardised message formats (or “schemas”, such as FpML) can mitigate this problem. In some sense it pushes the mapping inside each component, however this shifts the responsibility to a core capability and away from a feed dependency. A downside is that in order to facilitate the reconciliation a reverse transform from the component may be needed in addition (this may be needed for other communications also). Harmonizing external representations is a powerful route if time and complexity allow.
While there is extra effort to use a common data format, it reduces the overall effort in multi-way reconciliations (ie A vs B, A vs C, C vs D).
Standards support
We are currently in discussions about adding common rulesets and logic to aid customers and service providers in delivery of regulatory recs and managed services. CASS, MIFID and SFTR have all been raised as topics of interest. If you are looking for a solution in this space, now is a great time to reach out.
