Implementing first large-scale process domain virtualisation in SA

November 27th, 2013, Published in Articles: EngineerIT

Exxaro’s Grootegeluk coal mine is one of the most efficient mining operations in the world and operates the world’s largest coal beneficiation complex. A sufficiently impressive pedigree that Exxaro saw fit to improve even more by virtualising their extensive process domain assets on a massive scale.

Background

Exxaro has five main strategic objectives and one of these is ensuring the company’s sustainability. One of the components of sustainability is to ensure that the correct infrastructure is in place so as to enable the company to realise its production and business goals. One way of achieving this is to increase the availability and reliability of the Grootegeluk process control environment while at the same time lowering the risk of asset failure in this domain.

It was in this light that Exxaro decided to implement a physical and logical separation between the process and the corporate environments in order to improve reliability. This would be followed by virtualising the Wonderware System Platform environment to improve maintainability and reliability while lowering operating costs.

The main project goals were:

  • To improve uptime by better distributing the load amongst servers

  • To reduce risk by being able to rebuild the environment ten times faster than when not running in the virtual environment

Implementation

One of the first items on the agenda was to separate the office and process domains. “Previously at Grootegeluk, there was the office domain which also included the process domain,” says Samantha Rabé, systems engineer at Control Systems Integration, the system integrator Exxaro selected for this project. “So any web-based threat that wasn’t stopped by the firewall affected both domains equally (Fig. 1).”

A demilitarised zone (DMZ) was created to isolate the office and process domains from one another. The DMZ is where such assets as reporting servers, automation objects servers, application servers, the ArchestrA Galaxy Repository and everything to do with the system platform can be found.

“The process domain is quite large,” says Rabé.  “It consists of 3 ArchestrA Galaxy repositories involving about 200 000 I/O, 2 Historians, 15 application servers, an MES server, a production reporting server, Wonderware Intelligence software, a terminal server, 45 InTouch View nodes and 51 PLCs. This is anything but a small plant.”

With the exception of the InTouch View nodes, the rest of these applications were installed on eight host servers running Microsoft Hyper-V (figure 2).  “The reason the InTouch nodes were not included was because they will be migrated to ThinManager in the next stage of this project,” says Rabé.

Although there was a completely new hardware implementation, there was no change in the overall system functionality. “A plant does not have to place everything in the cloud or virtualise all hardware assets. It just depends on where it makes sense to do so,” adds Rabé.

Prior to the project proper, a lot of work was carried out regarding the necessary infrastructure, including the routing of fibres. The project itself took 8 months and there was a dry run two weeks before the commissioning date to resolve issues. “Commissioning was done over a weekend and most of the time was taken up by installing an upgrade of application server on each of the InTouch View nodes – something which will not be necessary in future with the adoption of ThinManager and the centralised deployment of solutions,” says Rabé. “Otherwise, moving to the virtual environment would have taken just a couple of hours.”

Virtualisation and maintainability

Traditionally, purchasing and installing a computer is immediately followed by installing the operating system, application software and configuring the system. “Only after the system has been running for a while can we think of optimising it,” says Rabé. “But if the hardware fails or needs upgrading, it all has to be repeated without the guarantee that the new hardware will behave the same way as that it is replacing. What’s more, all previous optimisation is lost because everything that is done is hardware-dependent. In a virtual environment, a virtual server hosts the operating system, applications software, configuration and optimisation which are now all independent of hardware that can be replaced easily and quickly and without losing any past efforts. This means that optimisation initiatives build incrementally and are not lost because the hardware no longer dictates what can and can’t be done.”

Virtualisation

Because of virtualisation, systems are more stable and can be moved anywhere – even to the cloud. To improve things even further, Grootegeluk uses  virtualisation technology, an early warning system that runs 24/7 and warns of application and server events before they can threaten operations. This, together with a service level agreement with CSI, means that issues are addressed and resolved in good time leading to an environment of incremental and continuous improvement with the goal of reducing unwanted events to zero.

Differences between the process and corporate domains

According to Rabé, by being closer to web issues, the corporate environment is more concerned with security and can tolerate frequent patch roll-outs and restarts whereas the process environment is quite dissimilar and focuses on the stability and reliability of the infrastructure and applications that enable production. This requires a different skill set and a different approach towards management of the environment.

Benefits

  • Improved availability, reliability and sustainability of the process environment thanks to hardware independence

  • Improved security through better isolation from web-based malice

  • The process domain is separated completely from the corporate domain and runs as an isolated system. This enables the creation of a process domain security model with process domain groups that can include the ArchestrA Galaxies and provide the highest level of security.

  • Improved supportability from remote locations. Site visits are no longer required to bypass failed hardware as the functionality it supported can simply be switched over to another host remotely. The failed hardware can be replaced when convenient and without affecting operations

  • Streamlined disaster recovery process means that there is now hardly any downtime

  • Improved monitoring capability (less reactive maintenance and improved preventative maintenance)

  • With virtual servers, system integrators can replicate the production environment of their clients easily within their own offices for development and testing purposes

Conclusion

Virtualisation has given Grootegeluk’s process domain far more flexibility, security and reliability than it had previously. The new system has been running since October 2012 and has proved to be stable, maintainable and scalable. In future, with clustering, the moving of virtual functionality from failed hardware to a working host will happen automatically, thereby improving reliability while reducing maintenance urgency.

Future developments will include moving to the Microsoft Cloud platform, installing ThinManager for all the InTouch nodes, reducing the event count to as close to zero as possible, providing full project development capability on site and rolling this solution out to all of Exxaro’s production sites that show long-term productive potential.

Contact Jaco Markwat, Invensys Wonderware, Tel 011 607-8100, jaco.markwat@invensys.co.za

Related Articles

  • Now Media acquires EngineerIT and Energize from EE Publishers
  • Printed electronics: The defining trends in 2019
  • Charlie and the (fully-automated) Chocolate Factory
  • SANSA app calculates best HF communication channel
  • New app supports predictive services for drive systems