With eight states and Washington D.C. already allowing recreational cannabis possession, and more than half the country offering some form of medical marijuana access, it may seem like only a matter of years before state and federal legislators realize the error of their ways and end prohibition altogether. But while TV, movies and the country’s coasts treat weed’s acceptance as a given, legislators across the country are still too scared, or lack motivation to make the moves necessary to pass legalization in their own states.

According to a new report from the Pew Charitable Trust, opponents of legalization are feeling just as successful in their ability to stop legal weed as cannabis advocates have been in their ability to end prohibition, whether it be in the form of a plea to staunch republican constituents and the president they serve, or simply because they’re not feeling enough pressure to act on the controversial law shift.

“The question I always ask is, ‘Do [voters] care about it?’ ” G. Terry Madonna, a public affairs professor who coordinates polling for Franklin and Marshall College, said. “What’s the level of concern that they have. And right now, for the vast majority of Americans, it’s not a cutting-edge issue.”

In the past year, more than 20 states have considered legalizing weed, but most bills and proposals have fallen by the wayside, and progressive lawmakers are far outweighed by their peers persuaded by fear-mongering myths and active opposition groups. When state law enforcement officials put their money with anti-cannabis action groups, most lawmakers get in line.

“Nobody wants to see the uniforms [supporting] their opponent,” Mark Kleiman, a public policy professor at New York University, said.

And while so far, ballot measures and voter-backed initiatives have turned the page on prohibition, the next domino to fall will be state legislatures passing legalization on their own. In Vermont, legalization passed both state bodies before being shot down by Republican Governor Phil Scott. If legalization is really going to spread to the South, Midwest and further on the East Coast, heads of state like Scott will need to bare some of the responsibility.

“You don’t get out of a hundred years of hysteria easily,” Jim Borghesani, communications director for the campaign that legalized recreational marijuana in Massachusetts last fall, said. “A lot of public officials are just afraid of this issue, even though the public just keeps passing these ballots initiatives.”

Even if voters can pass a legalization initiative, state lawmakers are often too worried to even implement the legislation, bickering over regulations until months, or even years have passed. In Massachusetts, retail sales will be delayed at least until July of 2018, more than a year and a half after voters approved legal weed in the Bay State.

If cannabis advocates are going to succeed in legalizing cannabis nationwide, they’ll need to prove they’re here to stay, and demand that lawmakers stop ignoring the civil rights infractions that prohibition invites. Legalization activists certainly aren’t going anywhere, but if they’re going to succeed, expect them to get a whole lot louder before our country’s leaders start listening.