Building Scalable UI for Android Using Agora

One of the biggest challenges facing any developer is building applications that can scale. Making a video-conferencing application that can scale is especially challenging because video data is pretty heavy. And as the number of participants rises, it becomes increasingly difficult to make sure that your application can keep up.

In this tutorial, I use Agora to build an application that can scale to up to 17 users by optimizing the bandwidth usage of the video streams.

TLDR: You can get the source code for the demo on GitHub.

Prerequisites

1. Android Studio
2. An Agora developer account (see How To Get Started with Agora)
3. Basic knowledge of Kotlin and Android Development

Project Setup

1. Open Android Studio, create a project, and select the template as “Empty Activity”:

2. Give a suitable name for your application and click Finish.
3. Add the necessary dependencies to your app build.gradle file:

implementation 'io.agora.rtc:full-sdk:3.1.3'
implementation 'androidx.recyclerview:recyclerview:1.1.0'

4. Add the relevant permissions to your manifest:

<uses-permission android:name="android.permission.INTERNET"/>
<uses-permission android:name="android.permission.RECORD_AUDIO"/>
<uses-permission android:name="android.permission.CAMERA"/>

All done! It’s time to start building the application.

Build

Now we build the application. I’ll show you how to build the UI for the local video and then put all the remote views in a grid layout by using a RecyclerView.

The structure for the Kotlin files is pretty simple:

.
├── MainActivity.kt
└── ui
    └── RemoteViewAdapter.kt

I’ll discuss the RemoteViewAdapter more later on.

Setting Up the Main Activity

We’ll first do some basic setup of the main activity so that you can see how this activity is structured:

This is a basic skeleton of the project. The App ID should be familiar if you followed the guide I have linked to above. The Channel can be any string. The important bit is that if any client wants to talk to another client, they have to be on the same channel. For the token, you can get a temporary token by following this guide. For production use cases, you need to set up a token server.

The workflow of the application looks like this:

1. We check if the required permissions are already granted.
2. If the condition is true, we call the initializeApplication method, which will contain the bulk of our work.
3. If the condition is false, we request the permissions. Android then brings up the permission overlay.
4. After the user interacts with it, the onRequestPermissionsResult callback is called. We can then call initializeApplication if the permissions are granted.

Asking for the Relevant Permissions

First, we write the logic for checking if the required permissions are already granted:

We then write the logic for the method that is called when we want to request the permissions:

Finally, we write the logic for the callback that is executed when the user has interacted with the permission overlay:

If all the required permissions are granted, we call the initializeApplication method. However, if they are not granted, we call finish to destroy the activity.

Setting Up the Layout

Before we jump into implementing the final method of the application, let’s finish designing the UI. For the sake of simplicity, I’ve kept the UI minimal.

The FrameLayout here is the container into which the SurfaceView for the local video is injected. SurfaceView is where Agora usually renders its streams on Android. The RecyclerView will be used to display a grid of videos of the remote users. The Guideline is used here to divide the percentage of screen real estate used by each component.

Setting Up the RtcEngine

It’s time to start implementing the initializeApplication method. Here we have the core application logic:

We also need to make sure to add the following property to our activity:

private var mRtcEngine: RtcEngine? = null

So, the first thing we do is create an RtcEngine object by giving it our App ID (which we discussed above and an event handler (which we will discuss soon). We do some basic configuration such as enabling the video and setting parameters like resolution, FPS, bitrate, and orientation. We set up our local video and then join the channel. And we enable dual-stream.

Dual-stream is an Agora feature that allows a client to publish two streams at the same time. One stream is for a higher resolution and bitrate, and the other stream is for a lower resolution and bitrate. With this dual-stream setup, when a remote client subscribes to your stream, they can switch to a lower stream based on their bandwidth requirements.

Now let’s set up the remote videos.

Setting Up the Remote Videos

We will now make a grid layout of videos. We use RecyclerView to do this because we can’t load all the remote users into memory at the same time. Using RecyclerView allows us to lazy-load the required streams.

First, we create our adapter:

We are doing three main things:

1. Creating the ViewHolder in onCreateViewHolder method
2. Setting up the remote video in the onBindViewHolder method
3. Muting the remote video in the onViewRecycled method

Now that we have created our RecyclerView adapter, we can set up the RecyclerView in our initializeApplication method:

We use the GridLayoutManager to get our RecyclerView to use a grid layout. We set the span count to 2 so that we have a total of two rows in the grid. We set the orientation to horizontal.

Now we create an instance of the adapter we just wrote. Make sure to add the following properties to the activity:

private lateinit var remoteViewAdapter: RemoteViewAdapter
private var uidList = ArrayList()

We write the contents of the callbacks in the IRtcEngineEventHandler:

https://gist.github.com/samyak-jain/4ccc3da5cf07839978daa7098ec428f4

In the onUserJoined callback, we are doing the following:

• Muting the remote video stream.
• Adding the UID of the remote video to a list of UIDs that we use as the dataset of the RecyclerView. We need to make sure that the data is not accessed by multiple threads at the same time, so we use a mutex to maintain a lock.
• Checking if the number of remote users has grown to four to know whether to switch to using the lower stream in dual-stream mode.
• Notifying the RecyclerView adapter that a new item is added.

In the onUserOffline callback, we are doing the following:

• Getting the index of the UID to remove and then removing the UID from the list of UIDs that we are maintaining.
• Removing the remote video stream from being rendered into the SurfaceView.
• Switching back to the higher stream in the dual-stream mode if the number of remote users shrinks to three.
• Notifying our RecyclerView adapter to remove the item.

We need to add the following properties to our activity:

private val handler = Handler(Looper.getMainLooper())
private val lock = ReentrantLock()

Cleaning up

Finally, it is time to write the onDestroy method. We clean up all the resources relevant to Agora here:

There you go! You should now have everything necessary to run the application.

Testing

Before you build the app, make sure you did the following:

• You added your App ID to the APP_ID variable.
• You added your channel name to CHANNEL.
• If your project has App Certificate enabled, you used your token in the TOKEN variable.
• You followed all the instructions correctly, like adding the correct dependencies to the build.gradle, adding the permissions to your manifest, and so on.

After you build the app, you should see something like this:

Done!

Congratulations! You have learned how to make an application that has all the necessary optimizations to have scalable video-conferencing capabilities.

You can get the entire codebase from this repository.

If you want to test the app with a web client, you can use the following hosted demo, which has dual-stream enabled: demo

Other Resources

For more information about Agora applications, take a look at the Agora Video Call Quickstart Guide and Agora API Reference.

To learn more about dual-stream, you can take a look at our guide on stream fallback.

I also invite you to join the Agora Developer Slack community.