By Gregory Hale
Whether it is a major oil refinery in the heart of the oil patch or a water utility in the desert, secure modernized networks allow for greater visibility for organizations to protect assets and achieve greater productivity.

The water facility at the Albuquerque Bernalillo County Water Utility Authority understands that issue all too well.

“For our environment, the equipment that we had in place was extremely antiquated and needed to be replaced,” said Kristen Sanders, chief information security officer at the Albuquerque Bernalillo County Water Utility Authority. “We were able to install industrial specific switches that have the ability to pretty much send the net flows so we can actually have visibility into exactly what is going across the network, and see what assets are connected.


“We can have alerting when a new device connects things that are very specific to the OT environment. In IT, it’s pretty common to have new devices connect and talk to different servers and even out to the Internet, but in an OT environment, there shouldn’t have a whole lot of change. It should be pretty consistent on what is connecting and what is talking to what. We are able to see that and get a baseline of what is normal and then use that to use anomaly detection to let us know if something has suddenly shown up on the network that has never been there before, or we start having communications going across some new protocol that we have never seen before, and that will alert us to that immediately,” Sanders said.

Heightened Level of Security
The Albuquerque facility understands all about the heightened level of security that is falling upon the water industry after the Oldsmar, Florida, incident where a hacker used the plant’s remote access capabilities and broke in and increased the amount of sodium hydroxide, or lye, by a factor of more than 100, from 100 parts per million to 11,100 parts per million.

The water industry is so important and key to everyday life, but yet, it ends up being overlooked which means security is often weak at best. Like at Oldsmar, which serves served 15,000 residents, costs are always a major factor when it comes to technology at a water company, and Albuquerque was no different.

“Working with (our partner) we were able find grants to update the infrastructure,” Sanders said. There is money out there, you just have to go look for it. That is big thing to try and upgrade the infrastructure. Within the critical infrastructure, a lot of these networks have been neglected for years, so it is out there.

“Another big thing is our upper management has been very supportive of security and has really embraced new technology. They will help us find the money if that is what is needed,” she said.

Another aspect when it comes to cost is regulatory mandates.

Water Infrastructure Act
“There is a new America’s Water Infrastructure Act that requires every water utility that serves more than 3,300 people to come up with a resiliency plan and risk assessment which passed in 2018,” said Sielen Namdar, Cisco’s global water business lead. “For the first time, it included cybersecurity.”

One other issue water companies have is a bare minimum staff. Oldsmar had that issue, and Albuquerque is no different as they have three people focused on security for the facility that serves over 650,000 residents.

“The big thing we have done is we have tried to automate as much as possible and we do have some managed services. We are just trying to use our time as best as possible because we are a very small group,” Sanders said. “We are working with the NIST Cybersecurity Framework and finding that low hanging fruit that you can easily knock out. It is understanding what is in your environment, what normal communications are, and being able to validate if there was even a threat actor in the environment. Inventory asset management, understanding what’s in the environment and what is normal; once we started doing these upgrades and bringing in (an asset inventory and threat detection tool), we were able to actually see what is in the environment, what it is talking to, and what’s normal traffic. We were able to move to there from going through and trying to manually figure all of that out on our own, which would take so many man hours. Luckily, the tool was able to bring all that data in just by looking at the net flow traffic that was already going across the network.”

Incidents like Oldsmar really highlight the need to harden system to only have the absolute minimum services running that need to be on it.

Zero-Trust Framework
“There needs to be a zero-trust framework with segmentation and having proper controls in place with your firewall. Having multiple layers of protection and multiple firewalls in place,” Sanders said. “Identification of the asset that is connecting and the user and ensuring that really is who is connecting is important. We are logging everything, even if you think you will never need it. You can’t go back in time to get them. We have the geo location turned on so we know exactly who is connecting from where. We can be alerted if someone is connecting from someplace that shouldn’t be.”

“Unsophisticated attacks, like what appears to have taken place in Oldsmar, are easily prevented by following industry standards and best practices such as ISA/IEC 62443 or NIST 800-82,” said John Cusimano, vice president of industrial cybersecurity at aeSolutions during the Oldmar incident. “We always recommend starting with a vulnerability and risk assessment to understand the vulnerabilities that present the highest operational risk to the organization and then follow that by preparing a mitigation plan that is prioritized by risk.”