Capabilities of an Effective Reference Data Management Platform

What makes an effective Reference Data Management platform? Whether an organization buys an RDM platform or builds one, the required capabilities remain the same.

These capabilities work well as evaluation criteria, and I group them into four categories:

1. Usability for business users
2. Developer productivity
3. Data integrity
4. Enterprise integration

Usability for business users

Business users are responsible for the content of reference data, and they need tools that support their workflow without adding unnecessary friction.

• A central, accessible user interface. Business users need one easily accessible place to build and maintain their mappings, categories, and hierarchies. This space must be shared across a team to promote collaboration and shared ownership. It should apply all necessary validations and referential integrity directly at the point of entry. It must also support filtering and sorting of data, so users can easily find the specific records they need to maintain.

• Manual import and export. The solution should enable exports in CSV or Excel formats for analysts who prefer to work on large reclassifications offline. It must support importing bulk changes and facilitate the initial migration from spreadsheets. The import feature must be able to merge updated data instead of replacing it entirely, while maintaining an audit trail to track any changes. The import process should also map the schema from the spreadsheet to the schema of the target RDM table, even when the two differ.

• Approval workflows. The platform should support an approval workflow for changes to reference data. When an editor modifies a reference data record, a designated approver can accept or reject it with comments. The change is only published after approval. Since only some reference data changes require approval, the workflow should be configurable per table. The solution should also support notifications to keep approvers from becoming a bottleneck, and the approval record itself must be part of the audit trail.

Developer productivity

Developers and IT are responsible for the schema, the integration. The platform must make this efficient.

• Rapid definition of tables and domains. Setting up a new reference table, including its columns, data types, validation rules, and relationships, should take only a few minutes. Organize the RDM tables into domains that reflect data governance requirements, such as Finance, HR, and Operations. The ability to generate a table definition from a CSV file, an existing spreadsheet, or a SQL CREATE TABLE statement can significantly speed up development and adoption.

  • Testing, promotion and deployment. Test changes to both data and schema in lower environments before applying them to Production. Developers need to validate schema changes in a Development environment before promoting them higher. Business users also want to test significant data reorganizations in these lower environments. This means you need the ability to clone data from higher environments down, and, once changes are verified, to promote and merge data changes upward.

Data integrity

Data Integrity ensures the data is accurate, complete, and reliable before it reaches consuming systems.

• Validation. Apply integrity at the point of data entry using column-level rules. These rules should include valid value lists, data types and lengths, mandatory fields, and default values. It is also important to clearly distinguish between columns that end users can edit and those they cannot. In a mapping table, the synchronized base codes should be read-only, while the mapping column stays editable.

  • Completeness. The platform should enforce completeness of data. Every base code should be accounted for, and no codes should be orphaned. The solution should notify business users when a new unmapped source code appears in a mapping table or a target code is deleted from a target code list. These are precisely the kinds of defects that a spreadsheet cannot prevent and that a later reconciliation would reveal.

• Referential integrity. Reference data is inherently relational. A mapping references a target code set, a categorization references its list of defined categories, and a hierarchy assignment references a parent node. The platform must ensure that each reference relates to an existing and current record in the related table. When users map or categorize a code, they should select the target from a searchable list pulled dynamically from the related table.

• Audit trail. The platform should record who made changes, what was changed, and when those changes occurred. The degree of tracking should be selectable based on the table's needs. Three levels of detail are practical:

1. The first level records who last modified each record and when.
2. The second level maintains a complete change log, capturing every modification along with its previous and updated values.
3. The third level allows point-in-time reconstruction, so you can reproduce the entire table as it existed on any specific date.

Enterprise integration

Reference data must connect cleanly to the broader enterprise architecture.

• Data synchronization. Synchronize reference data between the reference data platform, your data platform, and applications. Inbound data should flow from source systems into the platform without manual entry. Outbound reference data should be published to the data warehouse, integration platform, or target application on a regular schedule. Any process that relies on a person remembering to export a file is prone to oversight.

• Integration. Beyond scheduled synchronization, the platform should expose its data through an API so consuming applications can read reference data directly. It should take part in real time orchestration on an integration platform, serving code translations as messages flow. Change notifications let downstream systems react when a mapping is updated rather than discovering it at the next batch run.

• Governance and security. The platform must implement the data governance framework agreed upon by the organization. This includes specifying which roles are allowed to modify the schema, which are responsible for maintaining the data within it, and which roles must approve changes before they are implemented. Administration should occur at the domain level. Permissions should inherit downwards but allow overrides for individual tables.

Next Steps

Evaluate TitanRDM for yourself - a dedicated Reference Data Management application. Set up your first governed reference table in 10 minutes. Free tier, no credit card.

Dive further into Reference Data Management - download the whitepaper - RDM: The Missing Capability in Enterprise Data Architecture.