Cost and risk reduction through fully automated testing and qualification of network systems

Service providers are trying to achieve the best possible availability of their network infrastructure to ensure the smooth operation of their services. Any interruption, no matter how brief, leads to dissatisfaction, disruption of operations, loss of revenue, or even customers, and in extreme cases, breaches of agreed service level agreements (SLAs) can result in severe fines.

Therefore, service providers make significant technical and organisational arrangements to protect themselves against failures. However, rapid technological change is making it increasingly necessary to upgrade the components of the network infrastructure to the most current and most secure operating system versions. At the same time, new services and applications require more complex functions that these components need to provide.

This leads to ever shorter innovation cycles and correspondingly shorter test times. Even with the utmost care and use of ingenious test scenarios, errors in the software that negatively impact the quality and availability of network services cannot be completely eliminated.

A huge effort – testing and qualifying new OS versions

The testing and qualification of new operating system versions for network components require a correspondingly high technical and time effort on the part of the network operators. A high effort, which is also reflected in the operating expenses. Thus, operating system versions must be installed in the laboratory on the respective devices and tested by production-related test scenarios in detail. This requires technical know-how and a similar laboratory environment to correctly map the production network.

In this way, valuable internal resources are tied up and are not available for essential tasks, such as network expansion planning or ensuring network operation. New operating system versions can only be released to the production environment after successful completion of the tests and qualification. In this case, problems during the execution of the test, such as the occurrence of software bugs, can lead to an additional time delay and thus slow down the entire process. Also, if companies do not use modern automation techniques, more time is required for the manual execution and documentation of the tests.

The variety of network components (routers, switches, firewalls, gateways, etc.) in the signal chain also increases this effort and can mean that updates of operating systems are carried out rarely, usually once a year. This situation inhibits the innovation ability of the network operators and in many cases reduces the security of the components, since security updates must also first be extensively tested before bringing them into the production network.

One solution is test automation

With the help of test automation and load simulations, network operators can perform the vast majority of necessary tests entirely automatically. If service providers do have the required knowledge and the required test components, production scenarios can be easily integrated into an automation framework so that new operating system versions can be tested against these functions. This can reduce the risks associated with, for example, the introduction of a new operating system into production environments to an unavoidable minimum.

Xantaro develops customised, production-related test scenarios

Xantaro uses its many years of expertise in this environment and has developed a solution to perform such tests, without being limited to specific vendors and technologies. For the tests, Xantaro uses an open source test automation framework, which has been extended by its software components and linked to load generators from IXIA. The library developed by Xantaro enables us to create test cases in very easy-to-read commands, as you can see in the screenshot below.

Together with our customers, we define all protocols (e.g. OSPF, IS-IS, BGP, SNMP, QoS) and services (e.g. L3VPN, L2VPN, EVPN), which need to be tested on the respective components, and it is then modelled and integrated into the test framework. Subsequently, load patterns are specified that ensure a realistic load scenario that is as close to production as possible during the tests. In this way, we can develop a customer-specific test scenario that gives a production-related image of the network.

Fully automated tests on-site or in Xantaro’s XT³Lab

The developed test scenarios can be carried out fully automated, either at the customer’s site or in Xantaro’s laboratory – the XT³Lab. With the XT³Lab Xantaro operates a multi-vendor environment, which is unique in Germany. Equipped with the hardware components of our technology partners, we are able to cover many test scenarios without the hardware that needs to be tested having to be provided by the customer or vendor. If additional network devices are required, Xantaro’s experts support the planning and procurement of loan equipment.

Customers receive detailed reports on the default risk in the shortest possible time

During the tests, all the customer-defined functions are tested, and the results of these tests are documented immediately after completion and processed in the form of reports. The following graphic shows part of an exemplary PDF report of our test system. In addition to an overview, the PDF reports provide the details of each test result.

Depending on the complexity and scope, these tests can last from a few minutes to a few hours – a significant time saver compared to manually performing similar tests. In the event of errors that could lead to the failure or partial failure of certain services, the Xantaro experts subsequently assist in error analysis and communication with the vendor. Your operating team will be at ease and will be kept up to date.