Back to Blog

Testing Agora vs. Vonage for 1:1 Web Video Calls

Testing Agora vs. Vonage for 1:1 Web Video Calls

This blog summarizes the testing results of Agora versus Vonage for 1:1 Web video calls. The testing covers the most important considerations when assessing video SDKs, from analyzing performance under constrained network conditions to maximizing throughput given available bandwidth while keeping central processing unit (CPU) and random-access memory (RAM) usage as low as possible.

Test Configuration and Scenarios

Before we take a deep dive into the results, let’s review the test setup, configuration and scenarios covered in the testing.

Test Setup

In this test we used laptops with the following configuration for the sender and receiver:

  • Sender: MacBook Pro
    Processor 2.6 GHz 6-Core Intel Core i7
    Memory 32 GB 2400 MHz DDR4
  • Receiver: HP ProBook 630 G8 Notebook PC
    Processor Intel Core i7-1185G7 @ 3.00 GHz

All video feeds are set to 640×480 @ 30 FPS with the laptops connected to a stable Wi-Fi network.

For Agora, we tested using the Agora App Builder 2.0 and for Vonage we tested leveraging the most recent version of the Vonage WebRTC Demo, to ensure that Vonage best practices were properly implemented for comparison.

Test Case Scenarios

Test CaseDescription
UnlimitedNormal network conditions, with no bandwidth constraints, packet loss or jitter.
Limited500KThe test starts with no bandwidth limitations, then a low bandwidth of 500 Kbps is applied, first on the send side, then applied on the receiver side.
UL25%PLUplink network with impairment of 25% packet loss
DL25%PLDownlink network with impairment of 25% packet loss
Limited500K25%PLThe test starts with no bandwidth limitations, then a low bandwidth 500 Kbps with 25% packet loss is applied, first on the send side, then applied on the receiver side.
UL600msJitterUplink network with impairment of 600ms jitter
DL600msJitter Downlink network with impairment of 600ms jitter

The results are in – an overview:

  • Under normal network conditions: The frame rate (FPS) for each was steady at 28 FPS for Agora and 27 FPS for Vonage.
  • Packet loss and jitter: Agora consistently achieved a better frame rate than Vonage under numerous packet loss and jitter conditions, including when the network had uplink/downlink packet loss of 25%, as well as in cases where the uplink/downlink experienced jitter of 600ms.
  • Limited bandwidth: When throttling to 500 Kbps on the send side, and then on the receive side, Agora and Vonage performed in an analogous way.
  • 25% packet loss with 500 Kbps network bandwidth limitation: Vonage experienced choppy or frozen video while Agora recovered quickly to fluid video performance. After the limitation was removed, Agora recovered to an average of 28 FPS within ~2 seconds while Vonage struggled for ~50 seconds to get back to 26 FPS on average.
  • CPU and RAM: Agora and Vonage had comparable results for CPU and RAM consumption.

Frame Rate Testing

In these tests, we again focused primarily on the effect of packet loss and limiting network conditions on the send and receive frame rates.

Normal Network Conditions

As you can see in the graph below, with the network functioning under normal conditions, the frame rate was steady at 28 FPS for Agora and 27 FPS for Vonage. The overall experience for end users was similar under normal network conditions.

Figure 1: Normal Network Conditions
Figure 1: Normal Network Conditions

25% Uplink Packet Loss

When introducing 25% uplink packet loss, Agora remains steady at 15 FPS while Vonage begins lagging at 12 FPS.

Figure 2: 25% Uplink Packet Loss
Figure 2: 25% Uplink Packet Loss

25% Downlink Packet Loss

With 25% downlink packet loss, Agora continues to maintain 16 FPS while the performance of Vonage is impacted significantly, down to 1 FPS, resulting in choppy video.

Figure 3: 25% Downlink Packet Loss
Figure 3: 25% Downlink Packet Loss

Uplink 600ms Jitter

A significant variation in latency can occur when sending traffic over an IP network. When simulating conditions of a highly congested network having an uplink jitter of 600ms, Agora outperforms Vonage, holding steady at 9 FPS. In comparison Vonage drops to 6 FPS.

Figure 4: Uplink 600ms Jitter
Figure 4: Uplink 600ms Jitter

Downlink 600ms Jitter

When downlink jitter of 600ms was added, Agora held steady at 13 FPS with Vonage dropping to 5 FPS.

Figure 5: Downlink 600ms Jitter
Figure 5: Downlink 600ms Jitter

Frame Rate Recovery, 500 Kbps Limit

This test starts without bandwidth limitations, and after 60 seconds the network is throttled to 500 Kbps in the upstream direction, and then downstream for 30 seconds. Under these challenging conditions, Agora performed better than Vonage, rapidly recovering to an average of 27 FPS once the limitations were removed. It took Vonage 10 seconds to recover.

Figure 6: Frame Rate Recovery, 500 Kbps Limit
Figure 6: Frame Rate Recovery, 500 Kbps Limit

Frame Rate Recovery, 500 Kbps Limit and 25% Packet Loss

This test starts without any bandwidth limitations or simulated packet loss. After 60 seconds, the network is throttled to 500 Kbps in the upstream direction along with 25% simulated packet loss. These conditions are then applied downstream for 30 seconds. While the limitations were applied, Vonage users experienced consistent frozen video for up to 10 seconds at a time.

Once the bandwidth limitation and simulated packet loss conditions were removed, Agora rapidly recovered within about 2 seconds to 28 FPS on average while Vonage struggled for about 50 seconds before recovering to 26 FPS on average.

Figure 7: Frame Rate Recovery, 500 Kbps Limit and 25% Packet Loss
Figure 7: Frame Rate Recovery, 500 Kbps Limit and 25% Packet Loss

Agora has several ways to combat network disruptions and costly packet loss. Most notably, these include leveraging our SD-RTN (Software Defined Real Time Network)™ network as an overlay to the public internet, which routes traffic around impairments on the internet using AI algorithms and optimally shapes real-time traffic for the best performance.

Agora also implements technologies to smooth out the effects of packet loss to optimize the end-user experience.

CPU Utilization

Processor: Intel Core i7-1185G7 @ 3.00 GHz

Test Case ID AgoraVonage
Unlimited4.61%4.82%
Limited500K4.59%4.78%
UL25%PL4.72%4.98%
DL25%PL4.65%4.81%
Limited500K25%PL4.63%4.73%
UL600msJitter4.58%4.87%
DL600msJitter4.63%4.92%

Agora consumed slightly less CPU resources vs. Vonage in all scenarios, with an average of 4.63% utilization vs. Vonage at an average of 4.84%.

RAM Utilization

Test Case ID AgoraVonage
Unlimited149 MB129 MB
Limited500K156 MB134 MB
UL25%PL149 MB131 MB
DL25%PL146 MB125 MB
Limited500K25%PL163 MB131 MB
UL600msJitter145 MB123 MB
DL600msJitter152 MB125 MB

Vonage consumed slightly less RAM than Agora, with an average of 128 MB compared to Agora’s 151 MB.

Conclusion

This blog summarizes the test results covering various real-world scenarios and conditions experienced when implementing Video SDKs for point-to-point Web use cases.  When it comes to real-time communication, the ability to deliver a high-quality and consistent end user experience is critical.  Our testing shows that Agora consistently outperforms Vonage in common RTC scenarios involving jitter and packet loss. Agora’s advantage in these scenarios comes from our global SD-RTN (Software Defined Real Time Network)™ which uses AI to intelligently route traffic for optimal RTC performance.

At Agora, our customers—ranging from some of the largest social media companies in the world to bootstrapped startups—continue to leverage our SDKs to easily integrate video, voice, and interactive live streaming experiences into any app. We empower developers to deliver best-in-class real-time experiences, with global delivery and scale.

Sign up for free today to start building or contact us for a free consultation.

RTE Telehealth 2023
Join us for RTE Telehealth - a virtual webinar where we’ll explore how AI and AR/VR technologies are shaping the future of healthcare delivery.

Try Agora for Free

Sign up and start building! You don’t pay until you scale.
Try for Free
Get Started with Agora thumbnail