At a recent FinTech North event, I had the opportunity to speak on a panel discussion and dive into one of my favorite topics: the value of smart data.
What is that exactly? Data isn’t inherently smart or not – but how you use it can be. It’s only once you build the right processes and pipelines around data that it becomes a strategic business asset.
Unlock smart data value with cleansing and standardization
The first part of any process involving data is to make sense of it. Cleansing raw data ensures consistency and accuracy, allowing for easier comparison across multiple sources to extract meaningful insights. Businesses then use this data for various reports, both for internal use and to meet external regulatory requirements.
When the right data quality processes are in place, the reconciliation and reporting process is more effective. Without them, teams will be very busy with time-consuming (and avoidable) downstream data work. Businesses should aim for the ability to standardize data from many different sources into a common language.
At Kani Payments, we get data from multiple sources, usually processors who work with issuers of Mastercard or Visa cards. Their transaction data is what we pull into our system. Typically, payment gateways provide data in their own format for use by their clients, who are the ones who come to us.
Data Science Lead at Kani Payments.
We take data from multiple files provided in various formats and import it into a database, which structures the data. An additional standardization layer allows us to use all available transaction data for reporting. Making it accessible via easy-to-use dashboards gives our clients a standardized view so they can query all of their data in one place.
Gaining this holistic view should be the main objective of your data strategy, especially if you’re working with multiple payment processors. One processor might give you a CSV file, another an XML file, and another a PDF or text file. In-house finance teams are generally not as technologically aware - they don't understand these different file formats, or how to open them and get the information they really want. The right solution does all of this for you.
How data standardization enhances business efficiency
A robust data governance framework means that finance teams don’t need to spend all day making sense of their data through manual, time-intensive data analysis tasks. A platform like Kani imports files in a few seconds, maps it into a single schema and produces actionable analysis. Teams don’t have to go through thousands of files before their data is usable and can instead concentrate on jobs that add real business value.
With smart processes, our clients can interrogate their data and underlying trends in cardholder behavior. Examples include looking to see what cardholders are spending, where they are spending and what method they are using (chip and pin, contactless, online purchases, etc.). Having standardized data provides much more flexibility when reporting on these metrics.
Once the data is easy to work with, you can generate custom reports for tailored business insights. One of the reports we developed looked at supermarket spending, where the client could see the effect of lockdowns as people shifted from smaller, more frequent shops to a more substantial weekly one. Building these custom reports would have been a laborious job using in-house resources.
Effective data standardization processes help to speed up investigations and rectify anomalies in reconciliations. With so many different files and data sets involved across multiple sources, it’s common for some data fields to be missing. Or perhaps a processor generates data columns named in a certain way (such as transaction date, currency and transaction amount), while another might use a random code for that same column. These differences create issues with reconciliations.
If clients tried to investigate those differences, they’d have to look through each processor’s specifications document and figure out each column in each data set to find what they were looking for. Advanced standardization practices resolve that by looking for certain transaction elements, such as dates, amounts or how the transaction was authorized.
Smart data can get even smarter
All of our work in the payments industry revolves around delivering solutions for our clients to gain the most value out of their data, and we’re excited that our work here is progressing into other areas. We have started to map clients’ data to wider economic data, like GDP figures or other macroeconomic data, to provide more intelligent insights and benchmark against the wider industry.
Internally, we do some really interesting trend analyses on countries and payment trends. We enable clients to see that from a visual level. If clients want to enter a new market, we can look at their data, analyse regional trends from other data sources, and establish what is best for them.
Looking into the future, we plan to apply different statistical models or machine learning models for more accurate forecasting. You might have a set of cardholder spending at a certain point in time, but how will that look six months from now? That’s the next step to gain more useful information out of your data and then use it to help drive management decisions.
I often get asked what the future of data science will look like. In one word – busy!
Data science will continue to attract more recruitment simply because there's so much potential to learn from doing it. It will only become increasingly mainstream as people become more aware of techniques such as machine learning, especially given the uptake of ChatGPT and other large language models. As more people become interested in these areas, that helps to build more research, development and interest in potential applications.
We feature the best credit card processing for small business.
This article was produced as part of TechRadarPro's Expert Insights channel where we feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro
