Try Agora for Free
Sign up and start building! You don’t pay until you scale.
Summary: In Part 1, Part 2, and Part 3 of this blog series, Agora reviewed results from a recent Zoom study that suggested its Video SDK outperformed Agora’s Video SDK on several fronts and then detailed the results of our own comprehensive testing. Agora topped Zoom’s Video SDK throughout our testing in numerous vital scenarios.
This testing series highlights important considerations when assessing Video SDKs, from analyzing performance under constrained network conditions to maximizing throughput given available bandwidth and keeping central processing unit (CPU) and random-access memory (RAM) usage low, regardless of device type.
Each blog in the series focused on testing in predefined real-world scenarios applicable to Agora and Zoom.
A quick look back: in our testing and analysis, Agora demonstrated the performance advantages of its Video SDK in 1:1 PC calling scenarios, multi-host video calls (Desktop PCs), and during multi-party mobile video calls with up to 32 participants.
In each of those three scenarios, Agora’s Video SDK outperformed Zoom’s under varying real-world conditions.
This fourth and final installment of the series focuses on Agora and Zoom Video SDKs, examining how each performed during multi-party web video calls with 32 participants.
Read on to see the results!
Before we take a deep dive into the results, let’s review the test setup, configuration and scenarios covered in the testing.
For Agora, we tested using our Web SDK version: 4.17.0. For Zoom, we tested using https://pwa.zoom.us/wc/ [Version: 1.4.8.1250 (0322) to ensure Zoom’s best practices were properly implemented for comparison.
After nearly a decade of providing best-in-class video SDKs to thousands of customers and hundreds of use cases, the Agora team was not surprised to come out on top in every test case scenario against a relative newcomer to the Video SDK game.
The Agora Video SDK test results were robust even in poor network scenarios. A reliable Video SDK that can function while others struggle to deliver a high-quality user experience in adverse conditions can mean the difference between keeping customers or losing existing business.
In these tests, we again focused primarily on the effect of packet loss and limiting network conditions on the send and receive frame rates.
As you can see in the graph below, with the network functioning under normal conditions, the frame rate is steady at 23 FPS for Agora and 21 FPS for Zoom, for calls with 32 participants.
When introducing 25% uplink packet loss, Agora remains steady at 21 FPS while Zoom begins lagging at 17 FPS.
Much the same with 25% downlink packet loss, Agora is an even 20 FPS while Zoom shows 15 FPS.
A significant variation in latency can occur when sending traffic over an IP network. When simulating conditions of a highly congested network having uplink jitter of 600ms, Agora outperforms Zoom, holding steady at 18 FPS. In comparison, Zoom drops to 7 FPS, with a noticeable difference in perceived video quality by end users.
Nearly identical results occurred when conditions were simulated for downlink jitter of 600ms, with Agora holding at 12 FPS and Zoom bottoming out at 3 FPS. Under these conditions, participants in the Zoom call experienced choppy video.
This test starts without any bandwidth limitations. After 60 seconds, the network is throttled to 1Mbps in the upstream direction and then in the downstream direction for a total of 30 seconds. Under these challenging conditions, Agora performs significantly better than Zoom, rapidly recovering to 23 FPS once the limitations are removed. It takes Zoom 30 seconds to recover to 13 FPS.
This test starts without any bandwidth limitations or simulated packet loss. After 60 seconds, the network is throttled to 1Mbps in the upstream direction along with 25% simulated packet loss. These conditions are then applied in the downstream direction for a total of 30 seconds.
Once the bandwidth limitation and simulated packet loss conditions are removed, Agora rapidly recovers to a frame rate near 23 FPS, while Zoom struggles to recover to a much lower frame rate near 15 FPS. End users on the Zoom call experience frozen video for approximately 18 seconds while the limitations are present.
As we discussed in previous blogs, Agora has several ways to combat network disruptions and costly packet loss. Most notably, these include leveraging our SD-RTN™ network as an overlay to the public internet, implementing technologies that optimize performance over last-mile connections, and device optimization.
Our SD-RTN™ routes traffic around impairments on the internet using AI algorithms and optimally shapes real-time traffic for the best performance.
Agora also implements technologies to smooth out the effects of packet loss to optimize the end-user experience.
Table 3 – CPU Comparison
Processor: Intel Core i7-1185G7 @ 3.00 GHz
Agora consumed fewer CPU resources vs. Zoom in all scenarios, with an average of 7.48% utilization vs. Zoom at an average of 9.05%.
Table 4 – RAM Comparison
Throughout testing, Agora consumed less RAM than Zoom in every scenario, with an average of 243 KB to Zoom’s 648 KB.
Each of these four comprehensive blogs comparing Agora and Zoom’s video SDKs has taken you inside varying scenarios and conditions experienced when implementing. When it comes to providing real-time communication tools, delivering a high-quality and consistent end-user experience is critical.
At Agora, our customers—ranging from some of the largest social media companies in the world to bootstrapped startups—continue to leverage our SDKs to easily integrate video, voice, and live interactive streaming experiences into any app. We empower developers to deliver best-in-class real-time in-app experiences, with global delivery and scale.
We can help you too. Talk to us for a free consultation.
