Of course. Banks are now one of the most regulated economic sectors. Consumers have been given extensive rights to information. There are standardized information sheets, advisory requirements, documentation of consultations – if you want to know more, just ask a bank adviser how the job has changed in the last few years. However, this still does not seem to be enough to ensure that people really understand what is going on.
The second issue is covered by banking regulators. They monitor the use of analytical results, for example in the area of risk, and the analytical methods used have to meet certain criteria. The keyword here is “Model Risk Management”.
There is still some catching up to do on the third point, on adequate and equal communication. Using technical language does not make it easy for people to understand why exactly what happened. It is, of course, hard to explain how a machine learning algorithm or a neural network work! But it is important to try, because this will build trust.
Related: AI, Industry Disruptors, and Finding the Perfect Fit
That is the situation with banks. Do we need a regulator for all industries that use analytics?
I think we should at least think about it — and whether it needs to be a government agency. It would be helpful for everyone if there were trustworthy explanations. If, for example, product and service certification could show that the algorithms used operate to ethical standards, or that characteristics such as ethnic origin, gender or religious affiliation are not used in a discriminatory way.
Initiatives such as “Fairness, Accountability, and Transparency in Machine Learning” or Algorithm Watch go in this direction. Both these are about creating standards and opening the discussion about the fact that we would have a serious problem if we just have purely technological discussions about this. In our analytics economy, where so much already depends on algorithms, we cannot afford that.
What can we as software manufacturers do?
I think there are three main areas where we can action:
Interpretability
Our software has to work in such a way that results can be easily presented and understood by departments and decision makers, not just by specialists.
Traceabilility
We must ensure that the complex processes in data science — from data processing through modeling to deployment in production — can be documented in a way that is understood easily.
Communication
By promoting basic analytical knowledge and making our software attractive to different application groups, it is much easier to talk about it. Data scientists have a responsibility to actively discuss the opportunities and risks of their work.
This article has been republished with permission from SAS.