Videos of Mark Zuckerberg acknowledging the stealing of data and Barack Obama abusing Donald Trump have been circulating the internet for quite some time now?
These videos are the outcome of a very advanced and futuristic AI technology named Deepfake.
Simply put, it is a photoshop alternative for video. On one side, it can revolutionize electronic media by eliminating the need for an actual person.
On the other hand, it severely threatens one’s identity as you can make anyone say anything on video.
Deepfakes use deep learning to create photos and videos of fake events, hence the name deepfake. It can not only swap faces on existing videos but also create new frames and videos from scratch.
The Origin of Deepfakes
Extensive academic research has pushed the boundaries of photo and video manipulation over the past few years. Deepfake is also the result of these academic researches.
The first case of video manipulation was reported in 1997. A video of a person was modified to speak the words contained in a different audio track. It was the first case of facial reanimation using machine learning techniques.
Further notable advancement was made in 2017 when a video of former US president Barack Obama was modified to speak different words matching a different audio track.
In 2018, researchers at the University of California, Berkeley, introduced an app that could create a fake dancing video using deep learning. This marked the expansion of deepfakes to the entire body as previous works were limited to faces.
How Deepfakes are Created?
Thanks to advancements in computing, you can now develop deepfakes relatively easily and at a low cost. Two main methods are used to generate deepfakes.
You will have to train a neural network on real video footage of the person. This will allow the neural network to understand the facial features of the subject at different angles and lighting conditions.
After that, you will process both the original face and the latent face through an AI algorithm called the encoder. It will find and learn the differences and similarities between the two faces and both faces are reduced to a compressed image sharing the common features.
Then comes the second AI algorithm called the decoder, which recovers faces from compressed images. Both faces are recovered by two different decoders.
To perform the face swap, you simply feed the encoded images into the other decoder.
For example, an encoder output of face A is fed into the decoder trained on face B which then reconstructs face B with the facial features of face A. You will have to do this on every frame of the video for a convincing output.
Another method to generate deepfakes is Generative Adversarial Network (GAN).
You will have to use two competing algorithms to generate deepfakes. The first one will use random noise to generate an image and hence it is called the generator. This synthetic image is fed to a stream of real images through a second algorithm called the discriminator.
The discriminator provides feedback to the generator which generates another image according to the feedback. In this way, both algorithms give improved results with each iteration. This process is repeated many times until the required level of accuracy is achieved.
GAN delivers utterly realistic results, but it is hard to work with and requires an enormous amount of training data and computing power. That’s why it is generally preferred for generating images rather than video clips.
Some Convincing examples of Deepfakes
There are some very convincing deepfakes revolving around the internet and most of them are of celebrities.
For example, there is a TikTok account solely dedicated to the deepfakes of Tom Cruise. Videos show Cruise golfing or demonstrating a magic trick.
Another highly complex deepfake was uploaded on YouTube with Tom Cruise, Robert Downey Jr., Jeff Goldblum, George Lucas and Ewan McGregor. It has some obvious flaws, but to process 3 to 4 deepfakes in a video simultaneously is a feat in itself.
Another example is a deepfake video of former President Barack Obama.
This one is stunningly convincing as it uses the voices and gestures of impersonators capable of mimicking the voices and gestures of the subject.
We are now seeing deepfakes in the modern mainstream entertainment industry.
It was used to shoot scenes of Paul Walker in Fast and Furious 7 after the unexpected death of the actor. The deepfake was used on his brother with remarkable accuracy.
What do Deepfakes bring to the table?
Deepfakes have proved to be a very reliable technology to bring revolution in media and entertainment.
Can you remember when Henry Cavill’s mustache was removed by CGI in “Man of Steel” and it was a disaster?
The same can now be done on a few thousand dollar computers with far more convincing results.
You can now meet your deceased ancestors and loved ones. You can even attend a Physics lecture from Albert Einstein himself.
Besides all this, deepfake has not been entirely used in the manner it was intended to be. Around 96% of the deepfakes on the internet are non-consensual pornography.
The high quantity of training data available for celebrities has resulted in them being the most targeted victims of deepfakes.
It has enabled us to put anyone in dangerous or compromisable scenarios and hence it poses a great risk to everyone.
Audio deepfakes have been reported to be used to scam corporations. In 2019, an impersonator used deep fake audio to instruct a CEO of UK-based firm to transfer €220,000 into a Hungarian bank by impersonating the firm’s parent company executive.
How to counter Malicious Deepfakes?
Normally, you can detect deepfake videos by keenly observing frame by frame and looking for artefacts and irregularities.
However, it’s a counter-intuitive process and many companies are working on algorithms and software to detect deepfakes.
Facebook recruited researchers from Berkeley, Oxford, and other institutions to build a deepfake detector. Similarly, YouTube announced that they will not be accepting deepfake videos related to the U.S. election, voting procedures, or the 2020 U.S. census.
You can also use programs like Reality Defender and Deeptrace to detect deepfakes.
Countries are also busy in lawmaking regarding the use of deepfakes in general. The U.S. has implemented several laws regarding deepfakes over the past year.
Deepfake is the living embodiment of the advancement of AI. It further blurs the boundary of the future, however, it is a potential threat to the credibility of video-graphic content on the internet.
There will be a time when people will start doubting every video on the internet and we will be pushed into an era of further uncertainty.