Turning unstructured data into usable insights is at the very heart of AI moviemaking. Indeed, as the title of this article suggests, it is as much an art as it is science. But how is this done and how is it possible for a machine to understand how humans interpret images and sounds?

In this article, we are going to examine how machine learning AI systems are able to find meaning in unstructured data sources and turn them into usable insights that film professionals can use to improve their movies.

Unstructured vs. Structured Data

The distinction between these two types of data sets revolves around how easy it is for computer systems or applications to interact with the data.

Structured data is highly organized and is in a format that the system can easily interact with it. Examples include anything from personal address data stored to users’ online purchase history. These are easy for the system to search and understand.

Unstructured data often has no predefined format or organization. As such it is much more complex for the system to understand without using high-level AI or machine learning, etc. Examples include images, video, soundtracks, music, etc.

What is Artificial Intelligence?

The term artificial intelligence is often used interchangeably with machine learning for the same systems. However, there are some fundamental differences between the two, most importantly that machine learning really only applies to the current forms of ‘AI systems’ we have today.

There are 7 types of steps of AI:

  1. Reactive Machines
  2. Limited Memory
  3. Theory of Mind
  4. Self-aware
  5. Artificial Narrow Intelligence (ANI)
  6. Artificial General Intelligence (AGI)
  7. Artificial Superintelligence (ASI)

If you wish to understand in detail the distinctions between these forms of AI then read this article since this topic requires an article in itself to comprehensively explain.

Machine learning is equivalent to Reactive machine and Limited memory AI. However, it stops short of being capable of Theory of Mind AI.

A better defination is that, “machine learning is based on the idea that machines should be able to learn and adapt through experience, AI refers to a broader idea where machines can execute tasks “smartly.” Artificial Intelligence applies machine learning, deep learning, and other techniques to solve actual problems.”

AI systems use neural networks to learn from data. Their structure or architecture has been likened to the neural networks we have in our brains. To create these neural networks, data scientists create a system that contains algorithms that guide the system by acting as rules with which they can learn from the data they are given.

In the case of facial recognition, for example, these algorithms help the system to develop a complex neural network that allows it to identify and categorize all the features of the human face. Data scientists act as guides to teach the program to improve until it is able to do it with a high degree of accuracy on its own. It is important to note that while the data scientists do guide the system’s development, the system itself is creating the neural network connections, i.e. doing the learning.

When a system creates undesirable connections, such as in the case of Microsoft’s chatbot, which learned to become racist from Twitter user data, the data scientists either have to delete these neural network points or classify them as undesirable within the system in order to correct the problem. This phenomenon is why all but the most basic and advanced AI systems will need constant human input throughout their development lifecycle.

Read Also

Creating Neural Networks that Understand Movies

An AI company starts by developing a basic neural network that understands the rules relating to various parts of a film’s construction.

However, this is just the beginning of a lengthy and difficult process that requires the data scientists to train the system to comprehensively understand all the aspects of a movie, and importantly, how audiences and critics perceived the film on a moment-by-moment basis.

The sheer complexity of this process explains why there are only a handful of AI-assisted moviemaking companies in the world today.

These companies have had to invest considerable time and money training their systems to understand a multitude of variables that include everything from dialogue, scene instructions, soundtracks including voice recognition, facial and image recognition, editing or shot cut pace, movie gross history, etc.

To understand how complex this process is just consider how complicated it is to train a system to understand what constitutes horror and what constitutes drama.

A set of rules must be developed that allow the system to understand what to look for in both a horror and in a drama.

Let’s consider the most basic examples of each.

Horror:

Image – Dark lighting long and close up shots

Dialogue – Infrequent or light

Voice – Screaming, anxiety, etc.

Soundtrack – Suspense music

Editing – Long shots often leading into rapid cuts

Drama:

Image – Bright lighting and close up shots

Dialogue – Frequent or heavy

Voice – Normal level (sometimes laughter sometimes anger)

Soundtrack – Light or emotion-evoking such as classic

Editing – Frequent cuts back and forth between characters

Now what the data scientists need to do is to begin training the system with huge pools of movie data , i.e. movies. The initial stage is the most intense as the AI system will have a very undeveloped neutral network so will be frequently wrong.

The scientists will continually tweak the amassed pile of computation results to produce a lower loss or error rate. The more inaccurate the system the larger the tweak required to rectify it.

In the example above, constant tweaks would be required to adjust the parameters that the system extrapolates in relation to each element, image, dialogue, voice, etc., since each film will exhibit different values. So, for example, the system will quickly learn that a film with no dark lighting and no screaming is extremely unlikely to be a horror. Likewise, a scene with no dialogue and dark lighting is unlikely to be a drama.

Constant training, which, in some cases might require the system to analyze the same film 5, even 10 times is vital if the system is to learn properly.

With continued training and a variety of training techniques including data augmentation, the AI system will learn to understand more and more deeply what makes a horror and what makes a drama film, and what are the differences between the two.

This process is never-ending. However, as the system’s accuracy improves, it requires less and less input from its data scientist teachers.

As these AI systems learn, they are also being trained in what humans like and dislike about the films they see. This means that they are able to offer valuable audience insights during a film’s production that can help make films more appealing to their target audience. The current systems are already able to offer accurate gross earning predictions that can help production companies manage budgets and avoid box office flops.

The top AI-assisted moviemaking companies have spent years developing and training their AI systems. The success of their endeavors can be seen in the growing accuracy of their results and the fact that major movie companies are now retaining their services to help them improve their production processes.

Finally, it is worth noting that even the training technology for AI is advancing rapidly. The example training methodology above is known as supervised learning. However, there are other forms of deep learning that are gaining traction. These include unsupervised learning, reinforcement learning, and generative modeling. These new training methods promise to speed up the AI development process and to provide a giant boost to AI development.