Artificial intelligence has come a long way in the past decade. Up until five years ago, no one could visualize and explain the power of neural networks, the mathematical models that enable smart machines to learn without being consciously thought about.

Now machine learning algorithms are being able to identify products they wouldn’t be able to perceive on their own, or improve the audio quality of vocal recordings without augmenting the data presented to them.

This piece of technology might not leave a mark on your life in the same way a human friend can; it’s too simplistic, but that doesn’t mean that it shouldn’t be developed. However, a team of researchers at the New York University, Columbia University, and Carnegie Mellon who worked on a new project called AVN10 is exactly doing that.

Their A.I. model can detect the shapes and colors used in videos of objects and process them into colorized images and sound. They call it a LUX analysis. It uses a wide range of parameters to determine how brightly objects are lit by light. The system depends on the presence of one or more of these parameters: different voxels can be assigned different parameters, depending on the amount of light brightness. To select one of the voxels in a video, a computer is presented with an open window where it has a range of voxels to choose from.

How it works

Visualizing voxels is relatively easy – it’s just like seeing it in real life. The issue lies in associating the various objects (human, dog, etc.) that are presented in a video to the objects that they move through.

One way to do this is by simulating how they move through space in real life. To use a framework, which is where the team’s A.I. model is made, consider all voxels between 13 and 17 centimeters tall as the characters in the video. In theory, the interface should be connected to the ones between each size. A deep neural network can model how a voxel will move across these two frames, and the difference in brightness between the two parts should be reflected in the final video output.

Using the team’s algorithms, they were able to identify objects in live videos in real time. The human team created a standard condition, while the A.I. was trained to recognize images in light.

The results

The real-time version of AVN10 was able to recognize the shape of objects in real time using algorithms. Using this data, the A.I. was able to colorize the images, which are basically blended images of two videos, such as cats moving through water. There are also videos which show objects moving and being reflected. A moving light reflected off a solid object will be able to be colorized based on the light brightness.

The more ways these computers are able to recognize images and assemble them into objects and visualizations, the more difficult it will become for the human brain to solve.

For example, once the world of visual interpretation has been drawn into the mathematical language of algebra, there will be no way for the human eye to understand a video – this is why the same sort of computing power that would be needed to build a human brain would be enough to analyze in real time.

The team says that future A.I. models are also capable of changing color over time. The researchers are working on building algorithms that would be able to assign this function to objects that are moving around a screen.

So if you’re wondering if we’re just going to put displays in living rooms and black out our TV sets, it seems like the odds are pretty high.

Read next: Visual search startup Lendlio raises $5M from eBay, explains its mission