Try Agora for Free
Sign up and start building! You don’t pay until you scale.
When you’re making a video game, you want to squeeze every last drop out of performance out of your graphics, code, and any plugins you may use. Agora’s Unity SDK has a low footprint and performance cost, making it a great tool for any platform, from mobile to VR!
In this tutorial, I’m going to show you how to use Agora to create a real-time video party chat feature in a Unity MMO demo asset using the Agora SDK and Photon Unity Networking (PUN).
By the end of the demo, you should understand how to download the Agora plugin for Unity, join/leave another player’s channel, and display your player party in a scalable fashion.
For this tutorial, I’m using Unity 2018.4.18.
Since this is a networked demo, you have two strategies for testing:
Our “Charprefab” is the default Viking character we will be using. This object lives in Assets > DemoVikings > Resources.
It is already set up with Photon to join a networked lobby/room and send messages across the network.
Create a new script called AgoraVideoChat and add it to our CharPrefab.
In AgoraVideoChat let’s add this code:
using agora_gaming_rtc;
// *NOTE* Add your own appID from console.agora.io
[SerializeField]
private string appID = "";
[SerializeField]
private string channel = "unity3d";
private string originalChannel;
private IRtcEngine mRtcEngine;
private uint myUID = 0;
void Start()
{
if (!photonView.isMine)
return;
// Setup Agora Engine and Callbacks.
if(mRtcEngine != null)
{
IRtcEngine.Destroy();
}
originalChannel = channel;
mRtcEngine = IRtcEngine.GetEngine(appID);
mRtcEngine.OnJoinChannelSuccess = OnJoinChannelSuccessHandler;
mRtcEngine.OnUserJoined = OnUserJoinedHandler;
mRtcEngine.OnLeaveChannel = OnLeaveChannelHandler;
mRtcEngine.OnUserOffline = OnUserOfflineHandler;
mRtcEngine.EnableVideo();
mRtcEngine.EnableVideoObserver();
mRtcEngine.JoinChannel(channel, null, 0);
}
private void OnApplicationQuit()
{
if(mRtcEngine != null)
{
mRtcEngine.LeaveChannel();
mRtcEngine = null;
IRtcEngine.Destroy();
}
}
This is a basic Agora setup protocol, and very similar if not identical to the AgoraDemo featured in the Unity SDK download. Familiarize yourself with it to take your first step towards mastering the Agora platform! You’ll notice that photon.isMine is now angry at us, and that we need to implement some Agora callback methods.
We can include the proper Photon behavior by changing public class AgoraVideoChat : MonoBehaviour to public class AgoraVideoChat : Photon.MonoBehaviour
Agora has many callback methods that we can use which can be found here, however for this case we only need these:
// Local Client Joins Channel.
private void OnJoinChannelSuccessHandler(string channelName, uint uid, int elapsed)
{
if (!photonView.isMine)
return;
myUID = uid;
Debug.LogFormat("I: {0} joined channel: {1}.", uid.ToString(), channelName);
//CreateUserVideoSurface(uid, true);
}
// Remote Client Joins Channel.
private void OnUserJoinedHandler(uint uid, int elapsed)
{
if (!photonView.isMine)
return;
//CreateUserVideoSurface(uid, false);
}
// Local user leaves channel.
private void OnLeaveChannelHandler(RtcStats stats)
{
if (!photonView.isMine)
return;
}
// Remote User Leaves the Channel.
private void OnUserOfflineHandler(uint uid, USER_OFFLINE_REASON reason)
{
if (!photonView.isMine)
return;
}
Let’s play our “VikingScene” level now, and look at the log.
(You should see something in the log like: “[your UID] joined channel: unity3d”)
Huzzah!
We are in an Agora channel with the potential to video-chat with 16 other players or broadcast to around 1 million viewers!
But what gives? Where exactly are we?
Agora’s Unity SDK uses RawImage objects to render the video feed of webcams and mobile cameras, as well as cubes and other primitive shapes (see AgoraEngine > Demo > SceneHome for an example of this in action).
VideoSurface script to it (Component > Scripts > agora_gaming_rtc > VideoSurface).
// add this to your other variables
[Header("Player Video Panel Properties")]
[SerializeField]
private GameObject userVideoPrefab;
private int Offset = 100;
private void CreateUserVideoSurface(uint uid, bool isLocalUser)
{
// Create Gameobject holding video surface and update properties
GameObject newUserVideo = Instantiate(userVideoPrefab);
if (newUserVideo == null)
{
Debug.LogError("CreateUserVideoSurface() - newUserVideoIsNull");
return;
}
newUserVideo.name = uid.ToString();
GameObject canvas = GameObject.Find("Canvas");
if (canvas != null)
{
newUserVideo.transform.parent = canvas.transform;
}
// set up transform for new VideoSurface
newUserVideo.transform.Rotate(0f, 0.0f, 180.0f);
float xPos = Random.Range(Offset - Screen.width / 2f, Screen.width / 2f - Offset);
float yPos = Random.Range(Offset, Screen.height / 2f - Offset);
newUserVideo.transform.localPosition = new Vector3(xPos, yPos, 0f);
newUserVideo.transform.localScale = new Vector3(3f, 4f, 1f);
newUserVideo.transform.rotation = Quaternion.Euler(Vector3.right * -180);
// Update our VideoSurface to reflect new users
VideoSurface newVideoSurface = newUserVideo.GetComponent<VideoSurface>();
if (newVideoSurface == null)
{
Debug.LogError("CreateUserVideoSurface() - VideoSurface component is null on newly joined user");
}
if (isLocalUser == false)
{
newVideoSurface.SetForUser(uid);
}
newVideoSurface.SetGameFps(30);
}CreateUserVideoSurface() from our callback. methods.We now have our Agora module up and running, and now it’s time to create the functionality by connecting two networked players in Photon.
To join/invite/leave a party, we are going to create a simple UI.
Inside your CharPrefab, create a canvas, and 3 buttons named InviteButton, JoinButton, and LeaveButton, respectively.
Next we create a new script called PartyJoiner on our base CharPrefab object. Add this to the script:
using UnityEngine.UI;
[Header("Local Player Stats")]
[SerializeField]
private Button inviteButton;
[SerializeField]
private GameObject joinButton;
[SerializeField]
private GameObject leaveButton;
[Header("Remote Player Stats")]
[SerializeField]
private int remotePlayerViewID;
[SerializeField]
private string remoteInviteChannelName = null;
private AgoraVideoChat agoraVideo;
private void Awake()
{
agoraVideo = GetComponent<AgoraVideoChat>();
}
private void Start()
{
if(!photonView.isMine)
{
transform.GetChild(0).gameObject.SetActive(false);
}
inviteButton.interactable = false;
joinButton.SetActive(false);
leaveButton.SetActive(false);
}
private void OnTriggerEnter(Collider other)
{
if (!photonView.isMine || !other.CompareTag("Player"))
{
return;
}
// Used for calling RPC events on other players.
PhotonView otherPlayerPhotonView = other.GetComponent<PhotonView>();
if (otherPlayerPhotonView != null)
{
remotePlayerViewID = otherPlayerPhotonView.viewID;
inviteButton.interactable = true;
}
}
private void OnTriggerExit(Collider other)
{
if(!photonView.isMine || !other.CompareTag("Player"))
{
return;
}
remoteInviteChannelName = null;
inviteButton.interactable = false;
joinButton.SetActive(false);
}
public void OnInviteButtonPress()
{
//PhotonView.Find(remotePlayerViewID).RPC("InvitePlayerToPartyChannel", PhotonTargets.All, remotePlayerViewID, agoraVideo.GetCurrentChannel());
}
public void OnJoinButtonPress()
{
if (photonView.isMine && remoteInviteChannelName != null)
{
//agoraVideo.JoinRemoteChannel(remoteInviteChannelName);
joinButton.SetActive(false);
leaveButton.SetActive(true);
}
}
public void OnLeaveButtonPress()
{
if (!photonView.isMine)
return;
}
[PunRPC]
public void InvitePlayerToPartyChannel(int invitedID, string channelName)
{
if (photonView.isMine && invitedID == photonView.viewID)
{
joinButton.SetActive(true);
remoteInviteChannelName = channelName;
}
}Add the corresponding “OnButtonPress” functions into the Unity UI buttons you just created.
[Example: InviteButton -> “OnInviteButtonPress()”]
As you can see we need to implement two more methods in our AgoraVideoChat class. Before we do that, let’s cover some code we just copied over.
private void Start()
{
if (!photonView.isMine)
{
transform.GetChild(0).gameObject.SetActive(false);
}
inviteButton.interactable = false;
joinButton.SetActive(false);
leaveButton.SetActive(false);
}“If this photon view isn’t mine, set my first child to False“ - It’s important to remember that although this script is firing on the CharPrefab locally controlled by your machine/keyboard input - this script is also running on every other CharPrefab in the scene. Their canvases will display, and their print statements will show as well.
By setting the first child (my “Canvas” object) to false on all other CharPrefabs, I’m only displaying the local canvas to my screen - not every single player in the Photon “Room”.
Let’s build and run with two different clients to see what happens…
…Wait, we’re already in the same party?
If you remember, we set private string channel = "unity3d" and in our Start() method are calling mrtcEngine.JoinChannel(channel, null, 0);. We are creating and/or joining an Agora channel named “unity3d”, in every single client right at the start.
To avoid this, we have to set a new default channel name in each client, so they start off in separate Agora channels, and then can invite each other to their unique channel name.
Now let’s implement two more methods inside AgoraVideoChat:JoinRemoteChannel(string remoteChannelName) and GetCurrentChannel().
public void JoinRemoteChannel(string remoteChannelName)
{
if (!photonView.isMine)
return;
mRtcEngine.LeaveChannel();
mRtcEngine.JoinChannel(remoteChannelName, null, myUID);
mRtcEngine.EnableVideo();
mRtcEngine.EnableVideoObserver();
channel = remoteChannelName;
}public string GetCurrentChannel() => channel;This code allows us to receive events that are called across the Photon network, bouncing off of each player, and sticking when the invited Photon ID matches the local player ID.
When the Photon event hits the correct player, they have the option to “Join Remote Channel” of another player, and connect with them via video chat using the Agora network.
Test the build to watch our PartyJoiner in action!
You have now successfully used Agora to join a channel, and see the video feed of fellow players in your channel. Video containers will pop in across your screen as users join your channel.
However, it doesn’t look great, and you can’t technically leave the channel without quitting the game and rejoining. Let’s fix that!
Now we’ll create a ScrollView object, to hold and organize our buttons.
Inside of Charprefab > Canvas: make sure CanvasScaler UI Scale Mode is set to “Scale With Screen Size” (by default it’s at “Constant Pixel Size” which in my experience is less than ideal for most Unity UI situations).
In AgoraVideoChat add:
[SerializeField]
private RectTransform content;
[SerializeField]
private Transform spawnPoint;
[SerializeField]
private float spaceBetweenUserVideos = 150f;
private List<GameObject> playerVideoList;In Start() add playerVideoList = new List();
We’re going to completely replace our CreateUserVideoSurface method to:
// Create new image plane to display users in party
private void CreateUserVideoSurface(uint uid, bool isLocalUser)
{
// Avoid duplicating Local player video screen
for (int i = 0; i < playerVideoList.Count; i++)
{
if (playerVideoList[i].name == uid.ToString())
{
return;
}
}
// Get the next position for newly created VideoSurface
float spawnY = playerVideoList.Count * spaceBetweenUserVideos;
Vector3 spawnPosition = new Vector3(0, -spawnY, 0);
// Create Gameobject holding video surface and update properties
GameObject newUserVideo = Instantiate(userVideoPrefab, spawnPosition, spawnPoint.rotation);
if (newUserVideo == null)
{
Debug.LogError("CreateUserVideoSurface() - newUserVideoIsNull");
return;
}
newUserVideo.name = uid.ToString();
newUserVideo.transform.SetParent(spawnPoint, false);
newUserVideo.transform.rotation = Quaternion.Euler(Vector3.right * -180);
playerVideoList.Add(newUserVideo);
// Update our VideoSurface to reflect new users
VideoSurface newVideoSurface = newUserVideo.GetComponent<VideoSurface>();
if(newVideoSurface == null)
{
Debug.LogError("CreateUserVideoSurface() - VideoSurface component is null on newly joined user");
}
if (isLocalUser == false)
{
newVideoSurface.SetForUser(uid);
}
newVideoSurface.SetGameFps(30);
// Update our "Content" container that holds all the image planes
content.sizeDelta = new Vector2(0, playerVideoList.Count * spaceBetweenUserVideos + 140);
UpdatePlayerVideoPostions();
UpdateLeavePartyButtonState();
}
and add two new methods:
// organizes the position of the player video frames as they join/leave
private void UpdatePlayerVideoPostions()
{
for (int i = 0; i < playerVideoList.Count; i++)
{
playerVideoList[i].GetComponent<RectTransform>().anchoredPosition = Vector2.down * 150 * i;
}
}
// resets local players channel
public void JoinOriginalChannel()
{
if (!photonView.isMine)
return;
if(channel != originalChannel || channel == myUID.ToString())
{
channel = originalChannel;
}
else if(channel == originalChannel)
{
channel = myUID.ToString();
}
JoinRemoteChannel(channel);
}Comment out the UpdateLeavePartyButtonState() for now, and drag in your newly created ScrollView UI objects into the appropriate slots.
Now all we have to do is add the methods for “Leave Party” functionality in AgoraVideoChat:
public delegate void AgoraCustomEvent();
public static event AgoraCustomEvent PlayerChatIsEmpty;
public static event AgoraCustomEvent PlayerChatIsPopulated;
private void RemoveUserVideoSurface(uint deletedUID)
{
foreach (GameObject player in playerVideoList)
{
if (player.name == deletedUID.ToString())
{
// remove videoview from list
playerVideoList.Remove(player);
// delete it
Destroy(player.gameObject);
break;
}
}
// update positions of new players
UpdatePlayerVideoPostions();
Vector2 oldContent = content.sizeDelta;
content.sizeDelta = oldContent + Vector2.down * spaceBetweenUserVideos;
content.anchoredPosition = Vector2.zero;
UpdateLeavePartyButtonState();
}
private void UpdateLeavePartyButtonState()
{
if (playerVideoList.Count > 1)
{
PlayerChatIsPopulated();
}
else
{
PlayerChatIsEmpty();
}
}
and update our AgoraVideoChat callbacks:
// Local Client Joins Channel.
private void OnJoinChannelSuccessHandler(string channelName, uint uid, int elapsed)
{
if (!photonView.isMine)
return;
myUID = uid;
CreateUserVideoSurface(uid, true);
}
// Remote Client Joins Channel.
private void OnUserJoinedHandler(uint uid, int elapsed)
{
if (!photonView.isMine)
return;
CreateUserVideoSurface(uid, false);
}
// Local user leaves channel.
private void OnLeaveChannelHandler(RtcStats stats)
{
if (!photonView.isMine)
return;
foreach (GameObject player in playerVideoList)
{
Destroy(player.gameObject);
}
playerVideoList.Clear();
}
// Remote User Leaves the Channel.
private void OnUserOfflineHandler(uint uid, USER_OFFLINE_REASON reason)
{
if (!photonView.isMine)
return;
if (playerVideoList.Count <= 1)
{
PlayerChatIsEmpty();
}
RemoveUserVideoSurface(uid);
}
and in PartyJoiner:
private void OnEnable()
{
AgoraVideoChat.PlayerChatIsEmpty += DisableLeaveButton;
AgoraVideoChat.PlayerChatIsPopulated += EnableLeaveButton;
}
private void OnDisable()
{
AgoraVideoChat.PlayerChatIsEmpty -= DisableLeaveButton;
AgoraVideoChat.PlayerChatIsPopulated -= EnableLeaveButton;
}
public void OnLeaveButtonPress()
{
if(photonView.isMine)
{
agoraVideo.JoinOriginalChannel();
leaveButton.SetActive(false);
}
}
private void EnableLeaveButton()
{
if(photonView.isMine)
{
leaveButton.SetActive(true);
}
}
private void DisableLeaveButton()
{
if(photonView.isMine)
{
leaveButton.SetActive(false);
}
}Play this demo in two different editors and join a party! We start off by connecting to the same networked game lobby via the Photon network, and then connect our videochat party via Agora’s SD-RTN network!
If you have any questions or hit a snag in the course of building your own networked group video chat, please feel free to reach out directly or via the Agora Slack Channel!
Check out the link to the full github project here!
Sore eyes?
Go to the “misc” section of your settings and select night theme ❤️
