Twitter’s Magic Pony
One of the most annoying things about clicking on an image or video online is finding out that it is extremely blurry or low-resolution. I never really knew how to deal with those low-resolution videos, other than waiting and hoping that my internet connection would somehow get better. But it seems that a more timely and trustworthy solution to that problem is underway, using the “magic” of AI and machine learning. This blog post will be looking at Magic Pony, an artificial intelligence company focused on visual processing, that was acquired by Twitter 2 years ago in order to make Twitter’s live videos (through Periscope) more professional, and to expand on Twitter’s AI capabilities [1].
What is Magic Pony?
Magic Pony is a company that uses artificial intelligence (deep learning, neural networks, all the buzzwords) for visual effects. Its algorithms have a wide variety of applications in terms of image and video processing. Magic Pony can be used to clean up pixelated and low-resolution images as well as improve video streaming in environments with low bandwidth and connectivity issues [1]. In addition, Magic Pony can also be used to automatically build landscapes for computer games. How cool!
Magic Pony’s tech in action!
How does Magic Pony it work?
Magic Pony’s algorithms work by training neural networks to process visual information. The network is first taught what a high-resolution image looks like when converted to a lower resolution image. The process is then reversed, taking a lower resolution image and turning it into a higher quality one.
What makes this algorithm unique is the fact that it does not need manually labelled examples. Instead, “it recognizes statistical patterns in high-resolution and low-resolution examples and then teaches itself what edges, textures, straight lines, and other feature should look like.” [3] Magic Pony can fine-tune blurry images and videos because its algorithm knows that certain patterns indicate letters, and other patterns indicate the edges of a face. After identifying those patterns, the pixels and images can be “contoured and sharpened as the system sees fit in order to make the image clearer.” [2]
Currently, deep learning is most commonly associated with recognizing objects such as faces in images and videos, and relies heavily on processing many labelled examples. Therefore, researchers have noted that this type of learning done by Magic Pony, which doesn’t require labelled training examples, will be very important to the future of AI. [3]
Magic Pony’s Potential
Magic Pony has a lot of potential, especially in the area of creating new images or expanding upon existing ones. According to Rob Bishop, one of the company’s cofounders, “possibilities for [Magic Pony] include generating miles of realistic-looking terrain from a sample of virtual-reality environment.” [4] Because Magic Pony is able to identify rules that govern the large, overall patterns, as well as the finer details, of images and videos, it will be able to make new images by simply following those rules. Bishop gives the example of a computer game, where textures and other image properties can be uniquely generated for each character, terrain, etc. The possibilities are endless!
After having a better look at Magic Pony, I can’t help but compare it to NVIDIA’s “image inpainting” process, which uses deep learning to reconstruct images that are missing pixels. [5] And I wonder how the technologies of these two companies will overlap and differ.
NVIDIA’s image inpainting filling in the blanks
In addition, in the future, will artificial intelligence be able to create works of art? Will it be able to identify the attributes of paintings that are associated with “good art” (personally, I don’t really know what makes a work of art a masterpiece or super expensive) and be able to churn out massive amounts of paintings? Will we even be able to tell the difference?
References
[1] http://adage.com/article/digital/twitter-buys-magic-pony-technology-boost-video-quality/304597/
[2] https://techcrunch.com/2016/04/14/magic-ponys-neural-network-dreams-up-new-imagery-to-expand-an-existing-picture/
[3] https://www.technologyreview.com/s/601258/artificial-intelligence-can-now-design-realistic-video-and-game-imagery/
[4] https://www.csmonitor.com/Technology/2016/0620/What-will-Magic-Pony-Technology-do-for-Twitter
[5] http://fortune.com/2018/04/24/nvidia-artificial-intelligence-images/
Users who have LIKED this post:
4 comments on “Twitter’s Magic Pony”
Comments are closed.
Very interesting topic! I believe there are many potential applications of this kind of image strengthening. I think it can be used to automatically establish not only the landscape of video games but also characters and backgrounds of animation. Many of animation movies current still need human labor to draw those extremely complicate images for the movies which takes tremendous amount of time. This technique could reduce the work of these painter and save tons of time and money.
Indeed a very interesting topic which I have not yet come across! I believe that Twitter has greatly benefited from the acquisition of Magic Pony and I perfectly see how the fundamentals of their technology will make its way into other machine learning applications. What I would be really happy to see if this technology gets increasingly pushed into mobile on-demand streaming. Until today, it is a great pain, for example, to stream any sports game on your mobile without a good quality network and if this issue could be fixed that could give the world of mobile streaming another major push! Let’s see where we get from here!
Wow, this really could make a difference in the way we deal with information. If this technology continues to develop, It might be more efficient to share all our pictures in low-resolution and later using Magic Pony’s algorithm locally to “convert” it back to a high-resolution picture. I am not entirely sure if this is in fact a more efficient method to share information, but it’s still crazy to think about it in that way. Maybe this algorithm could also be used for security; the intended receiver of information would have a version of the algorithm that “cleans up” the information to make it coherent, preventing other users that don’t have the algorithm from every understanding the information (that could literally be cryptography but it might be a different way of doing it).
Hi Sharon Chen,
Well-structured and interesting article! I had not known about Magic Pony before, and reading this article really expanded my view on how machine learning can be incorporated to visual processing. I believe this algorithm can be super useful in the future for many other applications. Moreover, the visual images on this article, and the contrast between Magic Pony and NVIDIA were a great addition to the article.