IP Fabric Architecture with EVPN Control Plane
The various brands of freenet Group makes it a digital lifestyle provider and the largest network-independent telecommunications provider in Germany. The company currently supplies around 13 million customers with digital products. In addition to the core business of mobile services, this also includes services in the TV and media segment as well as hosting services for business customers.
The centrepiece for the provision of all Internet services, both for all subsidiaries and for external business customers, is located in Duesseldorf. Here, freenet Datenkommunikations GmbH, a wholly owned subsidiary, operates the central data centre of the group of companies – an infrastructure that has grown with the success of the group and has steadily increased in complexity over the years.
Development of the internal service provider
As a maintenance services partner, the Xantaro Technical Assistance Center (XTAC) has been providing support for many years. In 2017 a huge network failure resulted in the XTAC working intensively, with the team of freenet, to recover and stabilise the network operation. During this critical situation it became clear that there was a need for action. At short notice, the group decided to use the Xantaro Engineers‘ expertise for a network audit to further evaluate the functionality of the multi-vendor Layer2 network. The result: In order to ensure future growth and the reliable provision of new, innovative services, the previous DC design needed to be modernised.
„Our infrastructure was no longer suitable to deliver modern, sophisticated cloud applications,“ states Andreas Jürgensen, Head of Infrastructure at freenet AG / mobilcom-debitel, about the initial situation. „There have repeatedly been mutual influences which sometimes had DC-wide effects due to the Spanning Tree architecture and which could only be identified and recovered with great effort.“ At that time the network ran using VLAN trunks and the classic Spanning Tree protocol, which posed challenges to the stability of network operation from a certain scaling achieved by the infrastructure.
Confronted with the large-scale project, which could not additionally be handled by its own team, the freenet Group continued to rely on the integrator‘s know-how and commissioned Xantaro Deutschland GmbH to plan and implement the new data centre design. Since large parts of the data centre network of freenet Datenkommunikations GmbH are based on systems from Juniper Networks, whose systems are well known by the operations team, it was also clear that even the modernised infrastructure should be realised based on the technology of this vendor.
An innovative approach to high-performance infrastructures
Overall, the network modernisation should focusses on reducing the error domains for faster fault identification and improving stability. But also a higher level of automation, granular control of traffic flows and the ability to map Cloud connections or Hybrid Cloud services should be achieved in order to be prepared for the future.
Together, Xantaro and the technology partner Juniper Networks evaluated possible scenarios. Among them is an innovative approach based on a Layer-3 Clos Fabric with end-to-end EVPN Overlay, which should meet all future infrastructure requirements and on which the decision was finally made. The resulting high-level design is based on modern Juniper Networks MX, QFX, and EX Series components with full redundancy to ensure consistent operation. In order to use the freenet Group‘s investment budget as efficiently as possible, existing components were reused where possible..
Theory vs. practice: decision for stability
However, in a modern data centre, a variety of factors play important roles and any change can impact operations. Especially the use of innovative technology requires a balancing act between innovation and reliability that needs to be considered carefully. Thus, in the course of the project, freenet, Xantaro and Juniper Networks discovered how essential testing under realistic conditions actually is.
In order to take sufficient account of the diverse requirements of the operation, the network of freenet was replicated in detail within Xantaro‘s XT³Lab in Frankfurt. This environment is fully equipped with the latest Juniper technologies, as well as with test systems, and enabled the engineers to accurately determine the interplay of systems and software releases. In addition, they benefitted from Xantaro‘s proprietary test automation solution – „XFAST“ which automates numerous test runs, so the team also learned by a variety of test setups what impacts e.g. would be caused by changing configurations.
The results finally brought certainty: The initial innovative end-to-end design could not guarantee the required stability in freenet‘s network at the current time. Thus, the parties jointly decided to adapt the IP fabric architecture and implement a more conservative, segmented approach.
This solution provides a stand-alone routed Layer 3 substructure with Spine / Leaf architecture per data centre and a central MPLS substructure for data centre interconnection (DCI). The Control Plane uses EVPN in all three areas. Both, the overlay and the EVPN error domains are separated and restricted to the respective data centre or DCI area. Thus, problems or disturbances in a specific block can be isolated and do not affect adjacent areas or data centres.
As an overlay, EVPN / VxLAN is used for the data centres and EVPN / MPLS for the DCI. The connection between each data centre and the DCI block is realised based on EVPN Active/Active Multi-homed Layer 2 transitions between DC Spine and DCI PE. This reduces the complexity of the DCI layer significantly compared to an end-to-end EVPN design without bringing any limitations to functionality.
Layer 2 broadcast domains can now be set up selectively and per port based on EVPN Ethernet segments, both locally as well as throughout the DC. Pure point-to-point and many-to-many setups are also possible..
Together to the goal
„It was a trying time,“ says Stephan Feldhoff, Interim Manager Computer Center of freenet Datenkommunikation GmbH. „The fact that Xantaro already knew the existing network and the requirement profiles of the many applications and services was a big advantage.“ Furthermore, the team of experts also provides multi-vendor know-how across different vendors‘ technologies interacting within a network and facilitates the necessary support throughout the partner ecosystem.
In addition, the solution did not come by chance. The experience in the area of Datacentres and the combination of MX and QFX component technologies, such as EVPN and VxLAN, as well as innovative JUNOS software, which Xantaro was able to bring in from other customer projects, played an important role.
Subsequently, Stephan Feldhoff is very satisfied after completing the project: „With the IP Fabric, the corporate data centre is now equipped with a highly available, easily scalable and future-proof network platform.“ Thus, new services such as the Hybrid Cloud are currently in progress. Following delivery of a successful project, the next step is already planned: The bandwidth within the IP fabric should be doubled together with Xantaro.
For further information visit www.freenet-group.de
Customer: freenet Group /
freenet Datenkommunikations GmbH
Market: Service-Provider, Datacenter
Vendor: Juniper Networks
Products: MX10003, QFX10002, QFX5110
- high failure risk due to full capacity utilisation and outdated design
- difficult troubleshooting due to large error domains
CONCEPT & REALISATION
- network audit
- planning of a future-proof DC infrastructure based on innovative, high-capacity Juniper Networks
MX routers and QFX switches
- intensive, realistic tests
- IP fabric architecture with EVPN / VxLAN overlay and EVPN / MPLS DC interconnect
- highest availability to ensure service delivery through modern technology and full redundancy
- reduction of error domains for faster identification in the case of failure
- scalability and sustainability, possibility to deliver new innovative services (e.g. hybrid cloud)