Network Distance and Adobe Connect Meeting Latency
The network distance between Adobe Connect clients and the Adobe Connect servers hosting a Meeting is usually the primary variable driving the amount of latency experienced by end users. Round-trip-time (RTT) and network distance between Adobe Connect servers and the majority of the clients attending Meetings or Seminars should be among the primary considerations in planning where to deploy an Adobe Connect cluster or clusters. Adobe Connect multi-tenancy hosted services offers clusters based in the US, Europe and Asia. Adobe Connect Managed Services (ACMS) offers hosting options in many of the AWS global cloud regions although some restrictions currently apply. Adobe Connect on-premise servers go wherever you install them.
A simple tracert is helpful in determining the network distance and routing to any Adobe Connect servers as is the hosted Connection test:
- To test connections to a California-based hosted multi-tenancy Adobe Connect cluster:
- https://arcps.adobeconnect.com/common/help/en/support/meeting_test.htm
- From a command prompt run: tracert arcps.adobeconnect.com
- To test connections to a Texas-based hosted multi-tenancy Adobe Connect cluster:
- https://na3cps.adobeconnect.com/common/help/en/support/meeting_test.htm
- From a command prompt run: tracert na3cps.adobeconnect.com
- To test connection to an Australia-based hosted multi-tenancy Adobe Connect cluster:
- https://apac1cps.adobeconnect.com/common/help/en/support/meeting_test.htm
- From a command prompt run: tracert apac1cps.adobeconnect.com
- To test connections to a UK-based hosted multi-tenancy Adobe Connect cluster:
- https://emea1cps.adobeconnect.com/common/help/en/support/meeting_test.htm
- From a command prompt run: tracert emea1cps.adobeconnect.com
Sometimes you may be able to identify a particularly long hop in tracert output and work with network providers to solve issues adding undue latency to a network route. In other cases the network distance may simply be able to be shortened by rerouting after contacting the carriers. Issues with labyrinthine network routing are more common in the APAC region than either EMEA or the US. With reference to a recent case with a client in Korea, Amazon Web Services (AWS) was able to reduce latency commensurate the RTT by more than half. Compare the trace on the left with the trace on the right and see the routing improvements made through an AWS support inquiry to the RTT between clients in Korea and Adobe Connect servers based in Singapore:
The hosted Adobe Connect Meeting test offers the following critical information:
Clicking on the “Details” button offers more granularity:
Above you see the Flash Player version as well as the rtmp connection to arfms1.adobeconnect.com.
Here you see bandwidth data as well as latency of 124 msec. The Adobe Connect Meeting addin is also checked.
Another tool that is also helpful in determining the amount of network distance driven latency is used by the ACMS team to assist with properly locating an Adobe Connect cluster for support of a regional audience:
The output from cloudping from each regional office in any enterprise can provide valuable information to help determine the optimal location for an Adobe Connect cluster deployment. Cloudping will sometimes reveal some surprises. For example, see the output from a test run from the Adobe office in Seoul; it reveals that an Adobe Connect cluster based in the US on the west coast would have a shorter round-trip time than either Sydney or Mumbai. Obviously best is Seoul itself followed by Tokyo, Singapore and Beijing.
Testing at different times can often reveal different results so a collection of samples is always prudent. Testing from outside a corporate infrastructure and comparing tests with those of external clients is also often telling.
Couldping is a standard test that is helpful in determining the latency for an HTTPS connection. Adobe Connect also runs the Real Time Messaging Protocol (RTMP) and most customers want it running securely – RTMPS. See a simple diagram of a two node Adobe Connect cluster in the cloud running RTMPS:
The HTTPS connection handles login and content while the RTMP(S) stream runs the Meeting room commands. About 80% of the traffic in any Adobe Connect Meeting is RTMP-based. Its ability to run unimpeded is critical to Meeting performance.
In the upper right corner of any Adobe Connect Meeting room there is a latency indicator which shows the RTMP(S)connection status. Click on the green bar stack beside the Help menu button:
The first latency number shows the server to server connection. Each Adobe Connect server is both an Adobe Media Server (AMS/FMS) and a Tomcat application server. The server to server connection is always (unless there are extraordinary circumstances) very low. In this case it is 1 msec. The second number is very important. In this case we see an excellent connection of 103 msec. This is very manageable and will allow the full suite of Meeting features to be delivered to the client.
Note: If a capital letter “T” appears beside the second latency number, it indicates that RTMP(S) is being blocked and is forced to tunnel encapsulated within HTTPS. This will drive up latency and have a negative impact on performance.
See the following article for information about RTMP tunneling and how to prevent it from happening:
The Up and Down numbers refer to the amount of bandwidth being utilized in both directions. For information on Adobe Connect and bandwidth utilization, see the following articles:
https://blogs.connectusers.com/connectsupport/adobe-connect-bandwidth-utilization/
Latency boarder estimates:
When planning to locate Adobe Connect cluster deployments, it is prudent to know what features within Connect are affected by latency and how much latency is tolerable for each. The numbers provided are meant as estimates; the best way to determine how features are affected is to run actual tests simulating what you will do in production and see firsthand what the Adobe Connect end-user experiences in a Meeting. If you wish to try tests in different location, Adobe can provide trial accounts in the various hosted locations to help facilitate. On premise Connect trials can also be conducted globally.
Boarder estimates follow by feature in Adobe Connect
- VoIP is first and most affected by high latency.
- Any interruption or glitch is easily identifiable by the listeners
- The boarder latency number for VoIP is the lowest: 200 msec is the max to avoid delays and choppiness.
- Camera
- This isn’t really that easy to identify but when coupled with audio (if the same person is speaking and on Video) this becomes an issue.
- Boarder latency number for the Camera pod is approximately 350 msec.
- Note that if the Camera is tied to VoIP then VoIP numbers bring this to 250 msec.
- Chat
- Since chat sent by any user becomes visible in their chat pod only after the full round trip, high latency may be disruptive (a user may press enter to send a chat but it will take some time to show up in the chat area)
- The high end of the boarder latency range for the Camera pod is approximately 300 – 400 msec.
- Screen Share
- This will be the least affected as there are no apparent synchronization signals available however it is a bandwidth sensitive feature.
- Best practice with latency of over 300 msec is to limit screen-sharing.
- As latency approaches and exceeds some of these estimates, it is prudent to:
- Use integrated telephony instead of VoIP (both can be used together allowing regions with higher latency to use the former while users with lower latency may use the latter in the same Meeting).
- Encode video on the side of lighter weight quality and limit its use
- Upload PPTX content to the room and let it mete out from there and not use screen-sharing
- Many of the best practices for large meeting and seminars apply:
Conclusion:
With best practices in place, an enterprise can run with subsets of clients experiencing high latency; it is manageable. Expansion of the Adobe Connect footprint in the enterprise by adding clusters in growing regions or by adding hosted accounts is the best way to mitigate latency caused by long network distance.
Note: This article has focused on latency as a key variable. Other variables, such as jitter and packet-loss are often related directly to high latency. Bandwidth, alluded to earlier in the article is best treated as separate topic although bandwidth restrictions can be felt by end users in ways similar to latency.