By Chris Rogers, Winona Post
This was not what Bobby Falcon had in mind for his third week on the job as Lewiston city administrator. He came in on the morning of July 22 to find his staff locked out of the city’s network, the only place where some of its most important files were stored. It was crippling.
“That essentially closed us without any ability to operate for the foreseeable future because all of our accounting software, utility billing software ran off of that server,” Falcon explained.
Not so many years ago, this was the sort of thing that didn’t happen in a small town like Lewiston. But cyber attacks are a growing threat to organizations large and small across the country, and local governments and businesses need to be prepared, experts say.
“These are targeting all local entities: local governments, counties, schools,” Minnesota Chief Information Security Officer Rohit Tandon said. Businesses, nonprofits – every industry is a potential target for cyber attacks, he stated. “These actors are opportunistic. It’s less about the victim’s size or mission … It’s more about finding a target they can compromise,” he added.
In Lewiston, the ransomers didn’t exactly leave a note with letters cut out from magazines. It took the city a day to figure out exactly what had happened. Falcon and his staff immediately contacted the city’s insurance company, the League of Minnesota Cities Insurance Trust, and at their advice, brought in a law firm and forensic IT team that specialize in responding to cybercrime. “The initial IT company that we had already retained knew that we were having some sort of attack… That forensic IT team was in place the very next day and quickly identified it as ransomware.”
All of the files on the city’s server and one city computer were locked. When technicians tried peering at the files’ contents, they found an email address that is synonymous with a particular hacking group, Falcon said. “There was a particular note left on the server about files being locked and instructions on who to contact,” he explained.
Fortunately, the hackers did not steal sensitive information; they just held the city’s files captive, Falcon said.
Lewiston city leaders ultimately agreed to pay a $60,000 ransom – in the form of 1.5 Bitcoin – to the attackers. “The forensic team was the party in contact with what they termed the threat actor, and in the process of negotiating a ransom to be paid, ultimately that ransom was agreed to and was paid by the city,” Falcon said. The ransom was covered by insurance, so it did not cost the city directly, he added.
Tandon said Minnesota IT Services discourages organizations from paying ransoms because, in the larger picture, it may encourage more attacks. However, sometimes organizations face an urgent need to restore service, he noted, throwing out the example of a website with information about the hours for COVID vaccination, testing, or treatment being shut down by an attack. “In some cases, the fastest recovery path they determine could be rebuilding the data or paying the ransom,” he said.
Lewiston’s ransom didn’t work. The attackers took the city’s money, then demanded another $120,000 to release the data. “We knew there was a chance of re-extortion. We had been warned of that by the team that we hired,” Falcon said. “However, we felt that because we paid the full ransom that was requested … we sort of felt like it would be sufficient.” He continued, “I guess the reaction was disappointment that we were back to square one and having to contemplate paying the ransom all over again.”
This time the city refused. While waiting for the attackers to respond to the first ransom payment, city officials began piecing together the lost information they needed to run the accounting, payroll, and utility billing systems.
“We began rebuilding those, and in the process of rebuilding those we discovered a backup that was in an unintended location in our system,” Falcon said. The city had not purposefully backed up the files, but by a stroke of luck, it turned out that a copy of some of the locked files did exist on one of the city’s computers. “The final outcome was that we were re-extorted and chose not to pay a second ransom and restored our systems with backups we were able to find available, but all of the other data on our system was lost,” Falcon explained.
“It was a relief, but it was ultimately three files out of thousands,” Falcon said. “It was a relief in that the three most critical files were found on a desktop rather than a server.” However, he explained, “Part of the issue is you don’t really know for sure what all you lost. There are things that come up in day-to-day operations that you realize, ‘Oh yeah, that would have been on the server.’ … There are historical records that may have been stored on the server that we may not know. Time will tell what needs to be recreated or what may be lost as a historical record.”
Lewiston is just the latest example of a growing trend across the country. Fortunately, there are steps organizations can take to stay safe.
In Lewiston’s case, Falcon said the attackers simply cracked the city’s security from the outside. “The entry point from the analysis that was conducted points to a brute force attack, meaning that the attackers identified an open port on the firewall and used a script to basically try a number of passwords,” he explained.
Tandon recommended training staff to be constantly alert for phishing emails, possibly even running phishing drills. This is something the city of Winona employs. Its IT staff send practice phishing emails to various employees, testing to see whether they’ll open a potentially hazardous link.
For keeping passwords and credentials safe, Tandon recommended using multi-factor authentication whenever possible. In addition to requiring a username and password to log-in, multi-factor authentication will also send an email or text to the user to verify it’s really them.
“Cyber crime has definitely become a widespread issue across all sectors, not just local government,” League of Minnesota Cities Chief Information Officer Melissa Reeder said. “But what’s interesting is we’re seeing an uptick in attacks on local governments in ransomware and payment fraud.” She added, “Almost 80% come in as some sort of a phishing email; so it’s email-driven.”
A common payment fraud attack that targets local governments focuses on bank wire transfers to contractors. Because government projects are often awarded through a public bidding process, attackers may use that information to send a phony payment request, Reeder explained. “Now we’re seeing more bots scrolling those meeting minutes, seeing what vendors are new, and the perpetrators will pretend to be a vendor requesting to set up an automatic payment. It may be as detailed as including what the first payment was listed in the meeting minutes,” she said. To deal with payment fraud, Reeder recommended calling or emailing the contractor directly to confirm the request.
The main three ways cyber attacks happen is through phishing emails, compromised credentials and stolen passwords, or un-patched systems, Tandon said. “Knowing these three common pathways are used by ransomware, that’s where you want to start defending,” he said.
Phishing emails try to trick the recipients into letting hackers in, often by opening a link or attachment that may contain malware or by entering their password into a counterfeit log-in portal. If users accidentally share a password or use the same password across many accounts, the password may be compromised and used by hackers to gain entry, Tandon explained. Software companies are constantly updating and patching software to close loopholes that could let attackers in. If systems are updated and secured, that can be another vulnerability, Tandon said.
As for patching systems, that is a continuous responsibility for IT professionals, Tandon said. For ordinary users, the National Cybersecurity Alliance’s StaySafeOnline.org recommends setting all software to automatically update.
Backing up files is a key step to defend against ransomware. Reeder recommended that cities keep backups of important files with “air gaps” — meaning that the files are stored in a location that is not connected to the internet, so that it cannot be hacked.
“A lot of our cities are really good at backups,” Reeder said. “They’re good at that process, but if they have data that is private information, you’re still faced with paying a ransom because of the threat of exposing it online.” Encryption can protect such data, but cities operating with small budgets may not be able to afford it, she said.
“Both small, local governments and small businesses need to take it seriously,” Falcon said of cybersecurity. “Even though you are small, it can happen to you. Budgets need to be increased. For business, money needs to be allocated toward protecting your systems. In our case, we did make a change to a different IT vendor, and also the strategies we are using to protect our data is probably a 180-degree swing.” He added, “If it would happen today, we would not even contemplate paying a ransom. We wouldn’t need to.”
More information about best practices to defend against cyberattacks is available at StaySafeOnline.org.
*Published with permission of Winona Post.
Patrick Butler says
For any government entity not to be paying more attention to online security is inexcusable. Securing your systems has been preached for decades. Not backing up their data? Does anyone there have a clue?
It takes 0.08 seconds to search on online security tips and a few minutes to put them into action.