Structured

Explicitly structured

The vast majority of data matching and reconciliation tends to focus on flat, csv-like data sets. However, much of modern data flow concerns richer data structures which are sent around via standard formats:

• XML (including industry specific schemas - FpML, XBRL, )
• JSON (including valid single object formats, and object-per-line - logs/MongDB etc)
• EDI
• ASN.1
• FIX
• HL7
• Protobufs / Parquet etc
• etc

We have built such solutions in the past using a number of approaches. While it’s always possible to transform a rich data structure into a flattened representation - such that csv-based tools can process them - this often both loses context and makes the system brittle to changes.

A better approach is to use a structured matching tool. We have such a tool on the roadmap leveraging previous enterprise knowledge hard won on client projects.

If interested, reach out and we can discuss prioritising your use case.

Implicitly structured

Depending on where you stand, all data matching/recs are structured. More than just keys and values, over time the data changes. This might be all new transactions, new sales, for example - this can be viewed as a larger version of the previous set of data.

Each data run / extraction / query can be viewed in isolation, if convenient.

However, you can also view this as monthly buckets of data in a single larger set with an extra column / sub-structure.

Which representation makes most sense, depends on the use-case and also how versioning interacts with the structure – see versioning

Examples

Array of objects:

{ "accounts" : [
  { "id": "000", "customer": "abc", "balance": 7040.50},
  { "id": "170", "customer": "def", "balance": 5444.00},
  { "id": "340", "customer": "ghi", "balance": 6212.00},
]}

Keyed sub objects:

{ "accounts" : {
  "000" : { "customer": "abc", "balance": 7040.50},
  "170" : { "customer": "def", "balance": 5444.00},
  "340" : { "customer": "ghi", "balance": 6212.00},
  }
}

Content keyed sub objects (commonly in IRS / financial swaps)

{ "trades" : [
  { "id": "000", "legs": [
    { "side":"recv", "type":"fixed", "rate":0.04, "freq":"6M"}, 
    { "side":"pay", "type":"float", "rate":-0.01, "index":"OIS", "freq":"3M" }
    ]
  },
  { "id": "170", "legs": [
    { "side":"pay", "type":"fixed", "rate":0.04, "freq":"6M"}, 
    { "side":"recv", "type":"float", "rate":-0.01, "index":"OIS", "freq":"3M" }
    ]
  },
  { "id": "340", "legs": [
    { "side":"recv", "type":"float", "rate":-0.01, "index":"OIS", "freq":"3M" }, 
    { "side":"recv", "type":"fixed", "rate":0.04, "freq":"6M"}
    ]
  }
]}