Try Agora for Free
Sign up and start building! You don’t pay until you scale.
This blog is part of a series comparing Zoom and Agora for different real-time video scenarios. Agora’s detailed testing demonstrated the performance advantages of its SDK in 1:1 PC calling scenarios. For a summary of all testing scenarios, check out this blog: Agora vs. Zoom: A Comprehensive Comparison of Video SDKs
As we continue dissecting the benchmark tests commissioned by Zoom, it’s important to remember that while adding real-time video to an app increases engagement and enhances business opportunities, not all SDKs are created equally.
When shopping for a video SDK, it is crucial to zoom out and look at the big picture, and that’s why we decided to look further into Zoom’s results and further expand on what is relevant for customers to consider when evaluating real-time engagement (RTE) solutions.
In this blog, we compare how the two video SDKs stack up against one another during multi-host video calls on PCs with 8, 16, and 32 participants under varying real-world network conditions.
To see the detailed results for each test case, check out blogs covering the other scenarios here:
For Agora, we tested using our Native SDK version 4.1.0, and for Zoom, we tested the recent Zoom Application version 5.12.3, to ensure Zoom’s best practices were properly implemented for comparison.
All results were observed and measured from the Receive Test PC.
A snapshot of the results:
In this test, we focused primarily on the effect of packet loss and limiting network conditions on the received frame rate.
Agora minimizes packet loss and other poor network conditions by leveraging our SD-RTN™ network as an overlay to the public internet, implementing technologies that optimize performance over last-mile connections, and device optimizations. Our SD-RTN™ routes traffic around impairments on the internet using AI algorithms and optimally shapes real-time traffic for the best performance. Agora also implements technologies to smooth out the effects of packet loss to optimize the end-user experience.
Under normal network conditions, we see the average received frame rate was between 27 and 28 Frames Per Second (FPS) for both Zoom and Agora in calls with 8, 16 and 32 participants.
With 25% uplink packet loss Agora maintained a slight 2 to 3 FPS advantage against Zoom on the received frame rate in calls with 8, 16 and 32 participants.
With 25% downlink packet loss Agora performed better than Zoom with a 6 to 7 FPS advantage against Zoom on the received frame rate in calls with 8, 16 and 32 participants
The test started with no bandwidth limitation, then the network was throttled to 1Mbps in the upstream direction and then in the downstream direction for a total of 30 seconds. This simulated a real-world scenario commonly seen on wireless networks.
8 Video Participants
When the 1Mbps limit was applied, the Receive Test PC measured a frame rate drop both for Zoom and Agora. Zoom dropped to 0 FPS for a few seconds and frozen video was experienced during that time. After the 1Mbps limit was removed, Agora rapidly recovered to achieve an average frame rate close to 25 FPS, while Zoom struggled to recover with significant fluctuations in FPS.
16 Video Participants
With 16 video participants, the results were very similar to what was observed in the case with 8 video participants above.
32 Video Participants
With 32 video participants, the results were very similar to what was observed in the cases with 8 and 16 video participants above. In this case, Agora took about 20 seconds to fully recover, but did so in a smooth and linear fashion. Again, Zoom struggled to recover with significant fluctuations in FPS.
The test started with no bandwidth limitation, then the network was throttled to 1Mbps in the upstream direction with 25% packet loss applied. Next the network was throttled to 1Mbps in the downstream direction with 25% packet loss applied. The total duration of these impairments was 30 seconds. This simulated a more challenging real-world scenario commonly seen on wireless networks.
8 Video Participants
When the impairments were applied, the Receive Test PC measured a frame rate drop both for Zoom and Agora. Zoom dropped to 0 FPS for over 15 seconds total and frozen video was experienced during that time. After the impairments were removed, Agora rapidly recovered to achieve an average frame rate close to 25 FPS, while Zoom struggled to recover with frozen video for more than 5 seconds, and then had significant fluctuations in FPS.
16 Video Participants
With 16 video participants, the results were very similar to what was observed in the case with 8 video participants above. In this case, after the impairments were removed, the received frame rate on Agora fluctuated more than in the 8-video participant case, however the average FPS remained higher than Zoom.
32 Video Participants
With 32 video participants, the results were similar to what was observed in the cases with 8 and 16 video participants above. In this case, Agora took about 20 seconds to fully recover, but then maintained a stable average FPS of around 27. Zoom initially recovered more quickly, however, a significant fluctuation in FPS was observed from 150-180 seconds.
Bitrate is also vital when communicating with others via a video call. To deliver the best possible experience and quality for end users, maintaining the highest possible video throughput for the given network conditions is essential.
The data below focused on the case with 32 video participants for simplicity. The observations for the 8 and 16 video participant cases were similar.
Under normal network conditions, Agora sent and received an average of 3000 Kbps vs. Zoom which sent at 2500 Kbps and received at 2300 Kbps.
With an uplink packet loss of 25%, Agora’s sent and received bitrate remained higher than Zoom.
With 25% downlink packet loss, Agora and Zoom had similar sent and received bitrate performance.
During the test scenarios, we monitored the CPU utilization on the Receive Test PC (HP ProBook 630 G8 Notebook PC with an Intel Core i7-1185G7 @ 3.00 GHz processor).
For simplicity, we summarized the results for the 32-participant video scenario, which is the most resource intensive.
CPU UTILIZATION
As you can see, Agora consumed fewer CPU resources vs. Zoom in all scenarios, with an average of 1.2% utilization vs. Zoom at an average of 1.4%.
During the test scenarios, we monitored the RAM utilization on the Receive Test PC (HP ProBook 630 G8 Notebook PC with an Intel Core i7-1185G7 @ 3.00 GHz processor).
For simplicity, we summarized the results for the 32-participant video scenario, which is the most resource intensive.
RAM UTILIZATION (MB)
Here Agora consumed more RAM than Zoom, with an average of 707MB vs. Zoom at an average of 449MB. For the PC use case, we would not expect a user experience impact for either Zoom or Agora, even with multiple applications running. A low-end PC typically comes standard with 8-16GB of RAM.
In this blog, we compared how Agora and Zoom stack up against one another’s video calls on PCs with 8, 16, and 32 participants under varying real-world network conditions.
The Agora SDK demonstrated considerable advantages under all network scenarios and was able to adapt efficiently to changing network conditions. Much like in Part 1 of this blog series, which featured Agora and Zoom testing for 1:1 calling scenarios between PCs, the Agora Video SDK performed more reliably and efficiently overall, especially under more challenging conditions where Zoom froze or had choppy video – Agora delivered fluent, natural audio and video.
Whether it’s online classrooms, virtual events, enterprise collaboration spaces, virtual meetings or use cases in the metaverse, stability and reliability can mean the difference between success and failure.
For a summary of all testing scenarios, check out this blog:
To see the detailed results for each test case, check out blogs covering the other scenarios here:
Ready to test Agora’s performance for yourself? Sign up for free
