Data scientists have been called the rockstars of the corporate wold. Resident DWP rockstar and data scientist Charlie Boundy unlocks the potential and challenges of Big Data in the public sector.
How often when faced with a question do you immediately turn to the internet to get the answer? When you are choosing a film, music or a hotel do you look for reassurance from millions of anonymous customers that it’s a good choice? And have you noticed how more and more sites no longer ask where you are or what you’re interested in?
Big data has infiltrated our everyday lives without most of us noticing. In fact we expect to be able to get answers on-demand, anywhere and anytime on our phones, tablets and desktops.
In the corporate world there’s a huge buzz about ‘data’. It’s no longer an expensive by-product, thanks to falling database storage costs, or a heavy security burden, because of greater security awareness. Now companies like Google or Amazon have proved that data is a product in itself.
And there’s a wave of new technologies to help us manage and analyse these massive volumes of information, and make sense of them.
How DWP uses big data
Better use of data is core to the DWP’s transformation vision to create more efficient services. We're the UK's largest public sector organisation. So we can call on the experience of thousands of staff and 22 million claimants annually to understand what works well and what doesn’t. Then we can build and tailor services accordingly.
This affects how and where we might deal with our customers, for example to:
- look at what we know about a caller and their circumstances and find the best way to respond
- identify which groups are struggling to use our services and why
- predict which services are needed in which places
To do this we need to be able to turn data into insights that are usable, relevant and available when and where staff or citizens need them.
Data Science: revealing hidden patterns
The better we understand our customers’ actions, needs and motivations, the better services we will offer. Data Science is a way of working and set of skills which we use to make sense of large, complex, disparate data sources.
We can identify the most significant trends and behaviours by extracting context and meaning from data and using advanced analytics. The difference from other ad hoc analytics is not just the technology we use. It’s the fact that we can now embed the findings into other systems to tailor business processes, or the way we deal with customers.
At DWP we only see a part of the picture. So we work with other departments ( for example, HMRC, Cabinet Office) to share data. Access to that data helps us see the much bigger picture. It means we can do things like:
- make things easier for citizens - saving people from having to report and prove their circumstances to several different organisations
- reduce error and helping to prevent fraud
- help people find how others have dealt with certain situtations, like major life events
Data making predictions
Five years ago if you mentioned “data” to a technologist that immediately prompted thoughts of Data Warehouses, Business Intelligence (BI) or Management Information (MI) – the world of performance metrics.
These created volumes of spreadsheet cottage industries producing backward-looking stats focused on the performance of processes rather than users' needs. Technologies since then have rapidly matured and now there are open source machine learning techniques that use past behaviours to predict future ones.
We use the concept of ‘Next Best Action’ to do this. It’s a very human notion that makes decisions based on our own personal experience, and ability to learn from others. Obviously it’s far more powerful when we can tap into everyone’s collective experience.
Data lakes: the big challenge
The concept of data lakes - keeping relevant data until we know what to do with it - is a hot topic. We can’t each create festering pools with rapidly decaying data copies of every bit of data. So the big challenge ahead is creating a healthy stream of intelligent APIs. By that I mean interfaces that provide specific answers to relevant business questions, rather than wholesale raw data. Then we need data science to help us understand how to answer those questions, and make those APIs intelligent.
We need more technically innovative, creative thinkers with analytical minds that like a challenge. If this sounds like the new rock and roll to you then come and join us. Find out more about DWP jobs and apprenticeships, or sign up for regular updates from DWP Technology.