Augmented Reality Video Comes to Life with Banuba and the Agora Platform

This blog was written by Vadim Nekhai, Managing Director at Banuba. Vadim is a product guy who is passionate about turning ideas into products that consumers love.

Modern video calling has some significant limitations when it comes to the camera and the way it represents us. Hiding backgrounds for privacy, adding beautification to feel comfortable with your look, or just wearing funny face filters to warm up the conversation - these features, so seemingly simple yet essential, are now possible with Banuba.

Banuba, a computer vision European startup, is paving the way in the development of Face AR technologies providing SDKs to bring face filters, 3D animation, and avatars for any application or website. Now, Banuba technology can be integrated with the Agora platform to empower augmented reality video calls offering the next-generation video engagement experience.

Banuba Angel Background Filter

Face AR: How it All Began

Banuba was founded in 2016 with a shared mission of developing technologies that aim to help people self-express and communicate creatively using augmented reality. The company focused on the Face AR features developing face tracking and computer vision algorithms and optimizing them for low-end devices to make these features accessible to many people.

"Some years ago, augmented reality was just a gimmick, and we couldn't imagine anything bigger than Pokémon Go. Then suddenly, Snapchat lens flooded Instagram - everyone started using them. We realized, Face AR is the game-changer and the new way people can communicate. However, this technology was limited to Snapchat only. You couldn't use them in your apps or website. So, we decided to build our own Face AR tech and make it accessible for everyone - end-users, brands, and developers too. We wanted to empower them with a powerful Face AR platform that enables them to do their own thing. Developers can use our SDKs, Studio, and Filter Catalog to implement their ideas, on their terms, in their own products and apps.”— Vadim Nekhai, Managing Director at Banuba.

Today, Banuba provides cross-platform AR SDKs for face filters and beauty, the biggest catalog of AR effects, and Filter Editor & Viewer tools where designers can create custom effects. The startup focuses on two main categories: AR technology and AR commerce. It is empowering the new wave of video selfie apps and helping brands to bring face-wear products to their consumers using AR.

Banuba Broccoli Avatar Filter

The Need for Innovative Customer Experiences

As handheld devices become smarter with the introduction of apps and other robust features, the need for camera-based apps powered with AR and delivering high levels of user engagement is paramount. This is where Banuba’s Face AR technology shines improving video experiences between people, and why it is the perfect match with Agora’s SDKs.

In a time of massive disruption, people are becoming comfortable with using mobile and video applications to showcase their brand, product, self-express, influence or simply doing video chats. Whatever the reason, it’s increasingly important to provide tools and services that fit those needs. Banuba is here to help:

• Brands strive to reach and engage the younger generation of consumers, build an online presence and increase revenue with modern marketing approaches like virtual try-on apps, personalized and engaging Face AR ad campaigns on websites, or in-store.
• Developers create camera-based apps that empower end-users to be creative, record, and share stories with their networks. Face AR features help developers gain more organic traffic to their apps, increase user-generated content, and introduce monetization options.
• Consumers become their own creators by providing them with the same tools as professional video-makers or influencers and above all, the ability to show off their personality and create what they want.

Real-Time Video Engagement and AR Integration for Everyone

In our ever-changing reality, technology advancement in voice and video platforms has become an integral part of our daily lives, especially at a time like this when we all are more socially connected. When Banuba sought a strategic technology partner, it was important to go with a leading real-time engagement partner with a solid and stable development platform that delivers voice, video, and streaming to both developers and businesses globally. With this partnership with Agora, we have expanded our reach to delivering excellent and growing Face AR experiences to businesses; the Banuba SDK can be used by Agora customers to unlock a new level of AR immersion and experiences.

The following Banuba features can be incorporated with Agora’s SDK and API include:

• Face Filters that allow users to overlay AR objects and animation onto their faces during a video call on the web, mobile device, or desktop interface. Video face filters add real-time engagement and could serve as a monetization strategy for businesses looking to add this feature.
• Beauty AR enables real-time face enhancement during a video call or live streaming. It includes skin smoothing, teeth whitening, face morphing, and other beautification options to improve the quality of camera presenting and add more comfort to users.
• Virtual Backgrounds can be customized and changed in real-time during a video call. As people continue to work remotely, these backgrounds can allow them to have a sense of privacy or express themselves in a serious, fun, or casual mood.
• Avatars that allow users to see their digital selves and communicate privately yet still feel human.

The technology is cross-platform with Android, iOS, HTML5, and Unity game engine support.

Banuba Troll Grandma Face Filter

Other areas where it can be used:

• Games with video chats for users could stream and play together.
• Video conferencing software allowing users to change backgrounds, apply to touch up and makeup filters.
• Social apps where users can take professional photos, short videos, stories overlaying filters, animation, and style transfer effects.
• Virtual try-on apps allowing consumers to wear glasses, jewelry, hats, accessories, face wear via web and mobile.
• Makeup simulators and hairstyle apps for beauty brands with ‘magic-mirror’ product representation in colors, shades, and lighting and much more

Rich AR Libraries for Developers and Companies

Banuba’s technology provides brands and developers access to innovative libraries and content creation tools, letting them integrate AR experiences into products. From beautification to virtual backgrounds and more – all can be embedded within an existing web or desktop application.

Developer and Brand Benefits:

• Develop immersive cross-platform AR apps using lightweight and optimized SDKs and features for the camera.
• Engage users with face-changing filters.
• Reach younger audiences through real-time virtual try on and makeup simulators.
• Enhance video calls with touch up filters and add privacy with virtual backgrounds.
• Inspire users to create and share content and grow organically overlaying AR on videos.
• Retain users with regularly updated AR content and interactive features.

Banuba’s technology creates a responsive and precise user experience that works on high to low-end devices. Ensuring performance that attracts, widens the pool of potential users, and ensures anyone can take advantage of the experience.

What’s Next for Banuba and Agora

“While we are going with the trends of today, we are definitely looking at ways we can evolve as the AR technologies grow. Areas like augmented reality video calling, distance learning, and remote collaboration are seeing a big move, and we are already adapting our technology towards this relatively new frontier. New markets are emerging, and we want to be at the forefront of that as our partnerships strengthen with Agora.” — Vadim Nekhai, Banuba.

From AR face filters to e-commerce AR solutions, Banuba and Agora have forged a powerful partnership that provides a wide and growing range of services. Both Agora and Banuba will continue to find ways to expand that scope even more, with new SDKs to incorporate perfectly with the Agora platform.

Want to integrate Face AR features into your app or platform? Visit www.banuba.com

Discover more about Agora today. Sign up to start your next project that incorporates Face AR technology.