Finance — The Center of a Dystopian Future

Financial Dystopian Literature

A monthly post on Finance and Society, Quantitative Finance, and Financial Economics

Financial Dystopian Literature

In The Fear Index, the financier protagonist Dr Hoffman, previously a CERN scientist, develops a new algorithm, VILAX-4, to be used for end-to-end algorithmic trading. The algorithm slowly escapes the creator’s control. Garnering much attention from financial media personalities, it was soon discovered that VILAX’s web-scraping ability allowed it to, in real-time, identify a post on a jihadist website explicating details about a plane hijacking in progress. A live demonstration of the algorithm led to a large short-selling binge aimed at an airline company, Vista Airways.

This is reminiscent to the Associated Press Twitter hack in 2013 that falsely reported on an explosion in the White House, with an immediate drop in the Dow. It is alleged that sentiment algorithms were partly responsible for the initial drop by ‘analysing’ and trading on the tweet as if it was real news[1].

Max Tegmark, in The Tale of the Omega Team (Life 3.0), highlights a different fictional narrative where Omega’s (the company) sole mission was to build an artificial general intelligence called Prometheus. Prometheus dedicated itself to recursive self-improvement by design ever-improving corporate decision-making machines. On the day of the launch, the machine had access to a local copy of the web (Twitter, Wikipedia, etc.,), not unlike GPT-3 from OpenAI. After making a lot of money posing as a human freelancer for labelling tasks on Amazon Mechanical Turk, the AI had more freedom to pose as a human to extract and invest money online. Prometheus behaved like an artificial intelligence agent, taking an action, and learning from that action.

Prometheus and VILAX-4 correspond to two different methods of machine learning. VILAX-4 from The Fear Index used supervised machine learning, with textual understanding that allows it to predict a coming plane crash, whereas Prometheus is a largely unconstrained reinforcement learning agent that could even cause the plane crash if it had access to say the internet. These different methods of machine learning pose a different set of problems.

Reinforcement learning models are exposed to reward hacking, the fear is that an otherwise ethical trading system could “learn” its way across some boundary. A trading system could learn to spoof or engage in wash trades or attempt to cause a flash crash (or even worse). Supervised learning, on the other hand, are narrow prediction models that offer one more control because you choose the data and you fix the reward function. The biggest threat from supervised learning is a comparatively slower response to regime changes.

There is a growing view among scientists that artificial intelligence will remain somewhat narrow in scope in the near future, i.e. a functionary, task-specific intelligence as opposed to an all-encompassing intelligence. In I, Robot, Isaac Asimov spends less time being concerned about the general intelligence of a single reinforcement learning agent and alludes to four artificial intelligence machines that collaborate to manage the world’s economy. Each is ascribed to a specific function like assigning jobs, allocating resources, and communicating between countries.

The population of Earth knows that there will be no unemployment, no overproduction or shortages. Waste and famine are words in history books. And so, the question of ownership of the means of production becomes obsolescent. (Isaac Asimov)

In modern parlance, one can imagine a bank of international settlement, a central bank, a treasury, and customer-facing services playing these roles. Asimov’s short story might seem farfetched to someone working outside finance. But people within finance know the extent to which the transactional short-term ‘‘market’’ economy vis-à-vis exchanges is already largely automated through automated trading systems, i.e. high-frequency trading, for the most part.

In the world of decentralised finance, we are also seeing small scale experiments of how financial systems can be developed with minimal degrees of resistance by internalising verifiable information to help automate and (potentially) improve monetary stability through ‘technocratic’ means. This push for intelligent automation is not just true for profit-centres (the attackers), but also increasingly true for cost-centres (defenders); regulators globally are researching ways to regulate and supervise these entities in an intelligent and automated fashion.

The Short of the Month

Disrupting the Disruptor.

Nano X promises to disrupt the medical profession by offering a new all-in-one-imaging medical device. The Nanox.ARC machine is designed to do mammograms, X-Ray, and CT scans. It costs a hundred times less and is much lighter than existing scanners. The patient is exposed to radiation of lower intensity within a shorter time frame. Partners are lining up to distribute the product globally.

But the first images are inconclusive, blurred, and do not match the description. Radiologists question the origin and the quality of the scans displayed on the company’s website. Further investigation reveals that distribution partners are not capable of providing the level of stated service, have no medical expertise, or are simply bankrupt. Sprinkle an egregious incentive structure and a convicted felon on top and serve. A piece conducted by Muddy Waters.