Have you ever experienced an Operational Technology (OT) security breach that turned into a digital transformation effort?

We had a close call when somebody plugged in a USB drive—with the wireless on—that had a Trojan on it. So we took a two-way approach at the time: we had AV installed on all the machines and we were using Symantec back then to harden the devices. We put out a device policy that no USBs are active. And then we added these metal USB locks, which I wasn’t familiar with until one of the guys on the team suggested them. If you're really determined, you can pry them open, but as soon as people see that, it raises their mental awareness that somebody did this intentionally.

What we were not able to accomplish was to completely lockdown. We were monitoring the traffic that was going out of those two networks, but it was not foolproof. Along with that, we had Darktrace looking at all the industrial controls that were there already by the time I started. It did highlight some of the standard pattern matching where a machine wakes up and it makes too many calls or displays other unusual behaviors but that's no match for complete micro-segmentation. We went as far as realigning the like size or like type of Rockwells, the manufacturing machines and our core critical machine. We had a huge robot that goes from mixing to filling in one place, so that one had its own dedicated network.

Just when we thought we had everything locked down, the technician from the robot company came to update the machine’s firmware and inserted their USB drive. Of course, their USB had viruses all over it, so it infected that machine and spread to others as well. I almost got fired twice because our lab was down for a week or two weeks while we were cleaning up the mess. And that's not good when you're running a diagnostics test in a lab that’s Clinical Laboratory Improvement Amendments (CLIA) certified.

After that happened once, we put a rule in place that technicians can only use our encrypted USB drives and we would take whatever the firmware update was, transfer it to the clean USB drive, scan it for viruses and everything, and then we would allow them to update it. We had to put a process around it so that the lab managers knew not to let anybody in our lab with USB drives or anything else. Nobody can touch a computer without being escorted by somebody in IT.