Internet outage exposes frayed equipment, relationships in Greenfield

---

This article was written in collaboration between The Montague Reporter and The Shoestring.

GREENFIELD — Earlier this month, after a power fluctuation caused a hardware failure that severed the connection between the city’s municipally-owned internet service provider, Greenfield Community Energy and Technology (GCET), and the rest of the internet, customers took to social media to express their frustration.

“I’m sick of watching DVDs,” one commented on Facebook. Others wondered how the outage, which would ultimately drag on for 76 hours, was affecting local businesses and remote workers. 

For Greenfield Mayor Virginia “Ginny” Desorgher, the outage was a matter of public safety. While city hall and the fire and police departments were able to switch over to redundant services, the lack of connectivity presented a risk for some customers. 

Desorgher said she had talked with one resident, a new mother of an eight-day-old, prematurely born baby, whose critical health-monitoring instrumentation relied on an internet connection. Likewise for the CPAP machine used by one city official’s spouse. 

While no serious harm has been reported due to the outage, for Desorgher, a former emergency room nurse, this sort of disruption was unacceptable. She worried about the approximately 200 customers who used GCET’s voice-over-IP (VoIP) telephone service, who might not be able to call 911 if they did not also own a cell phone. 

The mayor shared these concerns with GCET General Manager John Lunt in a meeting late in the evening on Friday, Feb. 13, in the aftermath of the three-day outage, which had affected every customer of the municipally owned ISP. Desorgher had asked for a list of the names and addresses of the VoIP customers, who might not have a means to communicate during the outage. 

Lunt could not tell her the names and addresses of the VoIP customers — his hands were tied. “My understanding is that giving you this information is breaking the law,” said Lunt, according to draft minutes of the meeting reviewed by the Montague Reporter and The Shoestring. 

“I would never ask someone to break the law,” Desorgher reportedly replied, “but I want to find a way to be able to protect the city as well.” 

Although GCET is municipally owned, this doesn’t mean local officials — even the mayor — may access customer data. Federal privacy law forbids telecom providers from disclosing identifying information about telephone subscribers, and GCET could potentially be fined millions of dollars for doing so. 

The privacy dispute was a symptom of a broader power struggle between GCET and the mayor’s office. Desorgher can call the public works director, for example, and have snow cleared from the sidewalks.  But GCET is a separate entity from the city, and she cannot direct its staff to do anything.

GCET is a municipal light plant, a type of quasi-independent utility authorized under Massachusetts law. It was formed following a 2015 vote of Greenfield residents for municipal broadband, at a time when about 40% of residents weren’t served by any high-speed internet provider. The goal was to extend coverage to 90% of the city. 

The early years were turbulent. Though owned by the city, GCET is funded by subscription revenue rather than tax dollars. The plan approved by voters in 2015 created a five-member board of commissioners to oversee the entity, but under Lunt’s predecessor, interim director Dan Kelly, GCET was expected to take orders from the mayor.

Kelly clashed with city officials over accounting practices, and was fired in September 2017 after refusing to turn over financial documents. That December, the state Legislature approved GCET’s board of commissioners. Kelly sued under the state whistleblower statute, but the state Appeals Court sided with the city in 2022. 

Lunt and other staff report to GCET’s board, rather than the mayor, but this independence has a limit: under state law, GCET cannot borrow money without a vote of the City Council. And according to Lunt, the council has not approved a revenue bond for GCET in years, despite its repeated requests.

State financial filings show GCET’s enterprise fund has held less than $42,000 in retained earnings at the end of each of the last three fiscal years. For comparison, the piece of hardware that failed this month retails at between $40,000 and $100,000.

A Cascade of Issues

On Monday, Feb. 9, at approximately 1:16 p.m., GCET staff noticed that one of the utility’s two edge routers – the gateway connecting the city’s network to the internet – had lost several critical interfaces, according to Lunt. Every customer on the network went dark.

Troubleshooting with Juniper Networks, Inc., the router’s manufacturer, led GCET “very quickly to believe it was a hardware fault,” Lunt said in a telephone interview this week. But though the terms of the service contract should have required Juniper to immediately overnight ship a replacement, the company insisted on performing its own extensive diagnostic tests. A return merchandise authorization was not approved until nearly 8 p.m., too late for the equipment to arrive the next day. 

The replacement router arrived in Greenfield at around 10:30 a.m. Wednesday. By then, Lunt’s team had managed to restore service to roughly half of GCET’s customers by rerouting traffic through the network’s second router. But as technicians began installing the new hardware, Juniper stopped them: the new router’s firmware had to be upgraded first.

Lunt objected — GCET had been running the failed router on a specific firmware version, and he anticipated that moving to a different software version would cause a cascade of new technical issues downstream — but Juniper’s engineer insisted on a newer release. Hours of upgrades ensued. At 9 p.m. there were still bugs; GCET staff worked through the night to troubleshoot them. 

At some point, according to the draft minutes of Lunt’s meeting with Desorgher, the utility’s customer service supervisor went home with his phone, but fell asleep before transferring the line to overnight service, and missed about 50 calls. 

“It wasn’t until around 3:15 or 4 a.m. that we were able to fix all the bugs,” Lunt said in the telephone interview. “Networks are about balance, getting things tuned correctly. When systems go offline you have to confirm data is flowing from the correct IP address, and this takes some time.” 

Finally, on Thursday afternoon, after roughly another 12 hours of configuration and troubleshooting, service was fully restored — for most customers, at least; some issues lingered related to users’ improper “unplug it and plug it back in” operations. Lunt said that 11 people canceled their service following the outage. 

Lunt said GCET was prepared to deal with an outage, but had not expected the loss of an edge router: “Everything is about how you can mitigate risk, and how much it costs to mitigate it.” 

As an example, regional ISPs such as GCET must rely on larger entities, known as Tier 1 providers, to connect with the rest of the internet, and GCET pays $60,000 a year simply to keep a second Tier 1 provider available as a fallback. “That is something that does go down for minutes, to sometimes hours, at a time,” he said. “We always have a backup, because that’s a failure that we know is going to happen.”

And GCET had a plan for the router’s failure, but not for its replacement being delayed. “Ten years ago, the decision was made to take out the insurance instead of buying a spare router,” Lunt said. To his predecessor, this had seemed preferable to shelling out money up front — the suggested retail price in 2016 was $47,500 — for a machine that would almost certainly sit unused for years. 

Under the Umbrella

Still, some are questioning why the critical equipment was not replaced sooner. Erin Anhalt, the mayor’s chief of staff, pointed last week to the proposal GCET submitted in 2021 to use $1.28 million of Greenfield’s federal American Rescue Plan Act (ARPA) funding to expand access to customers in the northern area of the city. The cost breakdown included $150,000 for “core equipment and control software,” including an “upgrade” to the very router which failed this month. 

In an interview, Anhalt said that GCET “clearly” knew in 2021 that the router was approaching its end of life, but did not replace it despite being awarded the funds for the project. “That’s one of the big problems,” she said. “It really shows some negligence.”

“Negligent is a really tough word,” added Desorgher, who said she would instead call it “concerning.” 

“We’ve said consistently that the money has to be spent on the North Build,” Lunt said when asked for comment. “To say now that ‘you need to go back two years in time, and you should have replaced the routers then,’ shows a misunderstanding of the original ARPA proposal. It somehow became, in people’s minds, that the money was supposed to be spent on replacing the router — that’s not what we ever said or did.” 

Instead, he said, GCET provided the increased capacity required by the buildout by installing new networking cards in the existing core routers. 

This is not the first time Desorgher and Lunt have butted heads over the ARPA funding. In 2024, her first year in office, the mayor tried to claw back about $300,000 of the funding, citing GCET’s lack of progress on the expansion since 2021. A month later, the City Council’s Ways and Means Committee unanimously voted to restore the funding. 

“My job is oversight,” Desorgher said in an interview with the Greenfield Recorder at the time. “I want [GCET] to succeed and I want this to happen, but if I gave somebody $1 million to pave the street in ’21 and now it’s ’24 and the street wasn’t paved, I’d have to ask what happened to the million.” 

A significant cause of the delay, according to GCET staff, has been difficulty in coordinating with the owners of utility poles on “make-ready work” necessary to install cables and transmission equipment — particularly with Verizon, a direct competitor in providing internet services to Greenfield customers. Lunt said the make-ready work has now been completed, and new customers will be added during the next six months.

The funding dispute extends beyond the ARPA-funded project. Though GCET is governed independently and receives no local tax money, it remains under the city’s financial umbrella; like the water and sewer departments, the utility’s money flows through a separate account that tracks its revenues and expenses. For GCET to borrow money for capital expenses, the city must issue bonds. 

Lunt said GCET has submitted borrowing requests in five of the last six fiscal years, but none have been approved since 2021, including requests made under the city’s previous mayor, Roxanne Wedegartner. In light of this, he told the Montague Reporter and The Shoestring this week, GCET is exploring ways to restructure under Chapter 164 of state law that might give its board of commissioners fuller control of its finances by allowing borrowing, removing City Council from the equation.

“The previous administration asked us to pursue grants instead [of borrowing],” Lunt said. “We were very successful, and got several million in grants. We will need capital — at some point we’ll say, ‘This has to go through,’ because we have things we need to upgrade or replace.”

As it turns out, this includes the replacement router the utility installed this month. The manufacturer’s end of service life for the pair of edge routers is March 2028. Lunt expects their replacement will cost nearly a quarter million dollars.

---