How Speaker Recognition significantly improves the security of Call Centres

Matt Smallman
Matt Smallman
7 mins read
Last updated
20 Apr 2024

Voice Biometrics is a powerful technology that uses the sound of someone’s voice to confirm that they are who they claim to be. When used as an authentication method in call centres, whether through automated or agent led processes, it delivers significant customer experience and efficiency improvements over traditional knowledge-based authentication. The key driver for many organisations, however, is the desire to increase the security of these interactions thereby reducing fraud, the risk of it and the relative vulnerability of the telephone channel.

There are a range of specific counter-fraud solutions on the market that employ Voice Biometrics to identify and target bad actors to reduce fraud in call centres that do have significant short term impact but as their effect tends to be rather narrow it doesn’t realise the full security opportunity. We believe the greatest long term security impact is achieved by maximising the coverage of Voice Biometrics as an identity verification process for your known customers and callers combined with the intelligent application of these tools.

Speaker verification using Voice Biometrics provides greater (but not complete) confidence almost automatically for enrolled individuals thereby allowing more targeted treatment of those calls that do not achieve the required standard. By using Voice Biometrics as the core component of a layered and intentional security approach and being intentional about the desired effect on bad actors these broader security objectives can be achieved.

Who are the bad guys?

How Speaker Recognition Significantly Improves The Security Of Call Centres

In previous posts we have discussed how to effectively incorporate the needs of different customers, agents and decision-makers when designing a Voice Biometric authentication service but in this post, we will look at the effects an organisation may wish to have on a final category of individual, it’s adversaries. Before considering these effects it’s important to bear in mind that there are a wide range of adversaries whose own objectives, capability and scale vary considerably. As with our discussion on customers and agents, it’s important to understand their objectives and behaviour before designing a service to meet (or in this case defeat) their needs. We generally categorise imposters into one of three groups but there may well be other categories in particular industries or geographies.

  • Professional – These imposters either alone or as part of groups make a living out of defrauding organisations. They have the knowledge and experience of how organisations internal processes work to rapidly scale any vulnerability they identify and usually represent the majority of fraud losses. One colleague even commented that they can be frustratingly loyal.
  • Opportunist – These imposters may have obtained customer information from any number of sources and may have been able to associate it with an organisation but lack the volume of data or detailed knowledge required to exploit it at scale. They can still, however, be a significant inconvenience to organisations and their customers.
  • Related Party – These imposters are in some way related to a legitimate customer and therefore generally target a single account but do so with almost perfect information and in the case of vulnerable customers may even be doing so with their knowledge. This category can often be the hardest to detect and repair.

What effect do we want to have on them?

Effects-based thinking is a common military mindset used by all branches and many nations to help commanders think about the full range of options available to them in order to achieve their objectives rather than concentrate on the most obvious. In our case, destruction of our enemy (the imposters detailed above) whilst potentially desirable is not essential to achieve our objective of reducing fraud. We’ve used some common effects based terms to reflect the full spectrum of impacts that Voice Biometrics can have on the security of call centres. Many of these are implicit in the implementation of Speaker Recognition but others require a more intentional application and may not be employed by every organisation.


“to prevent something from continuing in its usual way by causing problems”

The mere implementation of Speaker Recognition as a verification mechanism can and will disrupt the plans of imposters of all types. An imposter who thinks they have all the required information to pass knowledge-based authentication will initially be taken aback when they meet a Voice Biometric authentication mechanism for the first time. For some opportunists, this will be enough to make them go elsewhere but more determined attackers are unlikely to be dis-orientated for long particularly if only a few customers are using the service.

The organisation-wide impact, therefore, largely depends on the proportion of customers who have enrolled and are protected. During initial implementation, therefore maximising the proportion of callers eligible to enrol and the number that do enrol must be the number one priority for delivering every benefit category.


“to stop someone from doing something, by making them realize it will be difficult or have bad results”

As customer coverage increases, legacy security processes can be removed or strengthened imposters making it far more difficult for imposters to exploit the data they have access to. Additionally, as the vast majority of callers will engage with Voice Biometrics systems before they can even access these strengthened alternative security processes that they may still be able to compromise. The fact that this requires them to leave a significantly more irrefutable evidence trail is likely to deter a proportion of imposters regardless of whether an organisation is actively exploiting this data which will be opaque to outsiders.


“to notice or discover something, especially something that is not easy to see, hear etc”

As patterns of use are established and customer behaviour normalises suspicious behaviour can easily be identified by looking for unexpected similarities as well as irregularities across all voice biometric interactions and identifying cases for manual review.

Beyond Voice Biometrics itself, the mere fact that an organisation is more certain about the identity of the vast majority of callers that have been verified by their voice means that greater effort and scrutiny can be applied to ensuring that those interactions not secured in this way are legitimate.


“to stop something from happening, or stop someone from doing something”

How Speaker Recognition Significantly Improves The Security Of Call Centres

As confidence increases in identifying suspicious behaviour, these processes can be automated and used to trigger different agent or automated system action. Further, voice biometrics systems can also be used in an identification mode such that when they are unable to confirm the identity of a legitimate customer they can also check to see whether the speaker is more like a known imposter and trigger a different agent or automated system action as a result. Known imposters can be identified from analysis of voice biometric data as well as post-incident analysis of fraud cases.

As confidence in the detection, mechanisms improves they can be deployed further forward and even deny frequent imposters access to services. Whether an organisation chooses to go this far depends on their end game strategy.


“to charge someone with a crime and try to show that they are guilty of it in a court of law”

The final step which some organisations have already had some success with is the use of biometric data to support law enforcement in arresting and prosecuting fraudsters. Even after detecting suspicious behaviour and identifying known imposters, it’s historically been very difficult to piece together the needles in the haystack of millions of recorded hours of calls to evidence the full extent of their activity and irrefutably identify them.


Deploying Voice Biometrics is not on its own a silver bullet for call centre fraud but deployed intentionally as part of a layered security model can have a significant impact. Some organisations have reported greater than 50% year or year reductions. As the larger organisations, who have historically had the vast majority of professional fraudster attention, deploy this technology they are unlikely to go away but will instead turn their attention to the smaller players who are just as vulnerable as well as up their own technological game.

Please contact us if you want to understand more about how we can help maximise the impact of Voice Biometrics on your call centres’ security.

Popular Posts