IT organizations are used to running older servers, storage, and networking gear. The typical mantra is to run them as long as possible; then even longer in an attempt to save money, reduce IT project churn, and stabilize the IT infrastructure. We’ll discuss a few of the problems that are created by this policy mindset, and the risks that are perpetuated either knowingly or unknowingly.
In this article, there are two core viewpoints of the typical company thought processes that are utilized. Richard Tengdin will consider the IT management component, and Jerry Craft will convey the standard cybersecurity mindset. Both mindsets are often seen in our day-to-day work with companies throughout the region. These thoughts are not entirely new, but we believe they are exhibited in many enterprises throughout the nation. As will be demonstrated, common business decisions and outcomes sometimes result in unexpected risk, and breach opportunities.
Hardware and software vendors are required to support their products for 5 years after End of Sale, but that support will be limited. Once a server platform is retired, the vendor will stop certifying new software (OS, applications) on it which locks companies into an aging footprint. As an example, HPE ProLiant Gen8 servers will end support in October 2021. Companies tend to keep these IT hardware footprints in existence for longer than the end of support date to extend the value of the capital expense purchase for as long as possible. This is for internal depreciation incentives, taxes, and building purchasing confidence from a cost-conscious Chief Financial Officer and Executive Board. Executives and Board Members lead the way on this risk by using hardware well beyond its years and security lifespan.
Operating systems have support lives as well. VMware ESXi v6.5 and 6.7 General Support will end in October 2022, and ESXi 7.0 is not supported on Gen8 server hardware. So as companies and executives choose to hold onto their aged systems, they box themselves into older software with older functionality and older security risks. Many companies forget that as software ages and more vulnerabilities are discovered, older software platforms become a hotbed for breach risk.
Legacy systems lead to legacy software, and demands keeping legacy hardware on giant corporate networks. This circular acceptable risk tolerance decision by business units becomes a chicken-and-egg scenario that keeps business units running, but also introduces long-term unrecognized acceptable breach risk as the hardware/ software combination lives beyond its support lifespan.
We have seen this with Nth Generation’s Pentesting service. This year, our team identified another 2003 Windows XP system with the unpatched MS08-067 (Conficker) vulnerability on older hardware supporting a global enterprise company. We have also tested against older AIX systems with similar results. These systems are still on these networks to support some grand business function. The justifying reasons for keeping these systems are complex. One was a simple time clock function that could easily be migrated to the cloud. Another was kept because old records still needed to be researched on databases with “lifetime” data retention policies, and yet others are kept because they are simply still functioning.
Another issue is hardware with hard-coded security information like unalterable encryption certificates. Wireless access points from a major vendor have embedded digital certificates valid for 10 years after assembly. When the certificate expires, the AP cannot be securely connected to the wireless LAN controller (WLC), and new APs cannot connect to a legacy WLC.
Regular hardware and software maintenance and patching are invaluable to prevent outages and data loss. Core hardware can be maintained, but look at your expanding footprint to see if those devices have updatable firmware. Without up-to-date firmware, these legacy systems tend to pigeonhole a company in older technology which also drives companies to consider grey market systems to expand and maintain business processes. This further complicates security risk by adding unsupportable aged technology to a critical business function. It does extend the life of the business solution, but also gives the executive team and board a false sense of security and success. These grey market systems add unseen risk, expand breach potential, and go unreported in many companies.
Some firmware updates are critical. For example, there is widespread SSD firmware bug that disables the drive after 32,768 hours of operation. The drive fails when a counter exceeds its maximum size of a software set integer value. Again these software vulnerabilities lead to outages and risk but also stops business, leading executives to wonder why there was no forewarning and questioning the IT team about sustainability lifecycles.
One last example is when a widely used but grossly insecure technology is retired, breaking every product which relied upon it. Yes, Adobe Flash, we’re talking about you. Many management applications used Flash to support user interactions, and old hardware running old firmware doesn’t support UI replacements like HTML 5. Companies may then hold onto these legacy systems to keep business running. All the while, new Flash exploits are being developed, and exploited on the laggers that fail to migrate.
In summary, IT hardware should be refreshed on a four-to-five-year cycle to manage security vulnerabilities and prevent catastrophic downtime when hardware simply stops working.
What we see are various business decisions that are pushing executives to make unrealized risk and breach outcomes. Our suggested steps below will help IT teams manage their security and operational risks.
Four-to-five-year hardware cycles need to be used to manage security and hardware. At the five-year mark, IT teams can use their partners to add value to their migration concerns, and predict future hardware migration plans and potential pitfalls for security.
Business “crown-jewel” reviews. IT teams need to understand what systems and software support the critical business functions of the enterprise. In that same effort, a four-to-five-year business plan should be in place to understand if those systems will support business in that time period, and if the technology will support those long-term business goals. Armed with both the technology and business needs, an IT team can equip the CIO to present a five-year technology strategy that supports the business goals, but also educates the executives on a future business spending plan that needs to be in place five years from now to migrate to newer technology.
Change management. Our security team sees change management providing both the ongoing support of the hardware and software as a critical function to maintaining these systems in a state that will reduce risk and prevent breaches.
The IT team needs to be business leaders. Working with each business unit and building relationships with them will allow the IT team to understand how the unit is growing and changing over the next five years. Knowing where the business leaders need technology will help in planning that change strategy in the future.
Only through communication, change management and planning will a company be resilient to attacks and reduce risk.