Sustainable DC Infrastructure for Freenet Group

IP Fabric Architecture with EVPN Control Plane

The various brands of freenet Group makes it a digital lifestyle provider and the largest network-independent telecommunications provider in Germany. The company currently supplies around 13 million customers with digital products. In addition to the core business of mobile services, this also includes services in the TV and media segment as well as hosting services for business customers. 

The centrepiece for the provision of all Internet services, both for all subsidiaries and for external business customers, is located in Duesseldorf. Here, freenet Datenkommunikations GmbH, a wholly owned subsidiary, operates the central data centre (DC) of the group of companies – an infrastructure that has grown with the success of the group and has steadily increased in complexity over the years.  

Development of the internal service provider

As a maintenance services partner, the Xantaro Technical Assistance Centre (XTAC) has been providing support for many years. In 2017 a huge network failure resulted in the XTAC working intensively, alongside freenet team, to recover and stabilise the network operation. During this critical situation it became clear that there was a need for action. At short notice, the group decided to use the Xantaro Engineers‘ expertise to carry out a network audit to further evaluate the functionality of the multi-vendor Layer 2 network. The result: in order to ensure future growth and the reliable provision of new, innovative services, the previous data centre design needed to be modernised. 

“Our infrastructure was no longer suitable to deliver modern, sophisticated cloud applications,“ states Andreas Jürgensen, Head of Infrastructure at freenet AG / mobilcom-debitel, about the initial situation. “There have repeatedly been mutual influences which sometimes had DC-wide effects due to the Spanning Tree architecture and which could only be identified and recovered with great effort.“ At that time the network ran using VLAN trunks and the classic Spanning Tree protocol, which posed challenges to the stability of network operation from a certain scaling achieved by the infrastructure.

Confronted with the large-scale project, the freenet Group continued to rely on the integrator‘s know-how and commissioned Xantaro Deutschland GmbH to plan and implement the new data centre design. Since large parts of the data centre network of freenet Datenkommunikations GmbH are based on systems from Juniper Networks, whose systems are well known by the operations team, it was also clear that even the modernised infrastructure should be realised based on the technology of this vendor.

An innovative approach to high-performance infrastructures

Overall, the network modernisation should focus on reducing the error domains for faster fault identification and improving stability. But also, a higher level of automation, granular control of traffic flows and the ability to map Cloud connections or Hybrid Cloud services should be achieved in order to be prepared for the future.

Together, Xantaro and the technology partner Juniper Networks evaluated possible scenarios. Among them is an innovative approach based on a Layer 3 Clos Fabric with end-to-end EVPN overlay, which should meet all future infrastructure requirements and on which the decision was finally made. The resulting high-level design is based on modern Juniper Networks MX, QFX, and EX Series components with full redundancy to ensure consistent operation. In order to use the freenet Group‘s investment budget as efficiently as possible, existing components were reused where possible. 

Theory vs. practice: decision for stability

However, in a modern data centre, a variety of factors play important roles, and any change can impact operations. Especially the use of innovative technology requires a balancing act between innovation and reliability that needs to be considered carefully. Thus, in the course of the project, freenet, Xantaro and Juniper Networks discovered how essential testing under realistic conditions is.

In order to take sufficient account of the diverse requirements of the operation, the network of freenet was replicated in detail within Xantaro‘s XT³Lab in Frankfurt. This environment is fully equipped with the latest Juniper technologies, as well as with test systems, and enabled the engineers to accurately determine the interplay of systems and software releases. In addition, they benefitted from Xantaro‘s proprietary test automation solution – XFAST which automates numerous test runs, therefore, the team also gained knowledge from a variety of test setups what impacts for example, would be caused by changing configurations.

The results finally brought certainty. The initial innovative end-to-end design could not guarantee the required stability in freenet‘s network at the current time. Thus, the parties jointly decided to adapt the IP fabric architecture and implement a more conservative, segmented approach.

This solution provides a stand-alone routed Layer 3 substructure with Spine / Leaf architecture per data centre and a central MPLS substructure for data centre interconnection (DCI). The Control Plane uses EVPN in all three areas. Both, the overlay and the EVPN error domains are separated and restricted to the respective data centre or DCI area. Thus, problems or disturbances in a specific block can be isolated and do not affect adjacent areas or data centres. 

As an overlay, EVPN / VxLAN is used for the data centres and EVPN / MPLS for the DCI. The connection between each data centre and the DCI block is realised based on EVPN Active/Active Multi-homed Layer 2 transitions between DC Spine and DCI PE. This reduces the complexity of the DCI layer significantly compared to an end-to-end EVPN design without bringing any limitations to functionality. 

Layer 2 broadcast domains can now be set up selectively and per port based on EVPN Ethernet segments, both locally as well as throughout the DC. Pure point-to-point and many-to-many setups are also possible. 

Together to the goal 

“It was a trying time,“ says Stephan Feldhoff, Interim Manager Computer Centre of freenet Datenkommunikation GmbH. “The fact that Xantaro already knew the existing network and the requirement profiles of the many applications and services was a big advantage.“ Furthermore, the team of experts also provides multi-vendor know-how across different vendors‘ technologies interacting within a network and facilitates the necessary support throughout the partner ecosystem. 

In addition, the solution did not come by chance. The experience in the area of data centres and the combination of MX and QFX component technologies, such as EVPN and VxLAN, as well as innovative JUNOS software, which Xantaro was able to bring in from other customer projects, played an important role. 

Subsequently, Stephan Feldhoff is very satisfied after completing the project: “With the IP Fabric, the corporate data centre is now equipped with a highly available, easily scalable and future-proof network platform.“ Thus, new services such as the Hybrid Cloud are currently in progress. Following delivery of a successful project, the next step is already planned; the bandwidth within the IP fabric should be doubled together with Xantaro. 

For further information visit www.freenet-group.de  

Sustainable DC Infrastructure for Freenet Group – PDF

Get the full story