The fate of BNSF Railway Co.’s new terminal near the Port of Los Angeles is now in jeopardy even though it has been in development for 10 years at a cost of more than $50 million and would provide badly needed rail capacity.

In March, a judge halted the railroad’s plans, ruling its environmental review wasn’t properly done. Opponents of the project, which include nearby businesses, neighbors, air-quality regulators and the City of Long Beach, had dug in their heels over the expected increase in truck traffic, pollution and noise.

BNSF is now deciding whether to appeal, negotiate, or abandon the effort.

The project was already costly, and “it now appears as though it has become even more expensive,” said Stevan Bobb, the railroad’s chief marketing officer. With a price tag of $500 million, “the real spending was yet to come.”

Railroad expansion plans nationwide are running afoul of communities and environmental activists with increasing frequency. Permitting disputes are becoming prevalent, given the high stakes at some of the nation’s biggest ports where changes within the past year in industries such as crude-by-rail and world-wide shipping are increasing the need for rail capacity.

Federal regulators have flagged one potential route for an 11-mile rail extension at Port Canaveral, Fla., as potentially “not reasonable or feasible.” In Maryland, officials last year put on hold an air-permit application for a crude-by-rail terminal in Baltimore pending more information from the developer. In Oakland, a new coal-shipping facility under development is now facing opposition after nearby residents balked at its use for transporting the commodity to Asia.

Within the next five years, U.S. seaports and their private-sector partners estimate spending $155 billion on expansions and improvements—including on-dock rail infrastructure—to accommodate large ships.

In the last three years, as the world’s biggest ocean-shipping companies formed alliances to operate more efficiently they also began pooling their container cargo, sending it on bigger Post-Panamax ships, which will fit through the newly expanded Panama Canal.

Now when a ship arrives in Los Angeles, for instance, it is carrying larger loads, often double that of a smaller ship. From the docks, much of that cargo is trucked to a rail terminal.

As larger loads arrive, many in the industry say rail capacity needs to be increased to keep up with the inflow of containers. A lack of capacity can grind port operations to a halt as containers stack up and are unable to leave. Rail capacity is already tight at the Ports of Los Angeles and Long Beach, with container volumes projected to grow 25% by 2020, and more than double by 2035. Railroads say they must get ahead of the growth.

BNSF and Union Pacific Corp. are the only two major railroads serving the Southern California ports. But both have run into trouble with their expansion plans. Currently, the closest BNSF terminal to the ports of Los Angeles and Long Beach is roughly 25 miles away. Its new terminal would be 4 miles away from the ports. Meanwhile, Union Pacific has been locked in what it calls “environmental review purgatory” at the ports for nearly a decade as it tries to double capacity to handle 1.5 million containers annually.

Without rail, ports won’t be able to keep up and ships will go elsewhere, which “would have serious social and political as well as economic consequences for the region,” says global-trade economist Jock O’Connell.

The Panama Canal expansion, expected to bring bigger ships and more business for U.S. East Coast ports that are ready for them, is scheduled for completion this month. The Port of Baltimore is one of only a handful of East Coast ports with cranes and a deep-enough harbor to unload Post-Panamax ships carrying around 14,000 containers, says James J. White, executive director of the Maryland Port Authority.

But Baltimore doesn’t have enough rail capacity to handle the cargo volume those ships can bring, despite two decades of effort. “We’re probably the only major port from Florida to Maine that doesn’t have double-stack capability,” Mr. White said, referring to the ability to stack one container on top of another on rail.

CSX Corp. tried to build a $90 million rail terminal near the Port of Baltimore, a process that ended in 2014 after residents balked at expected noise and traffic. Now, the railroad and port are planning a $450 million project to lower the floor of an existing tunnel to accommodate stacked rail, the railroad said.

Louis Renjel, CSX vice president of strategic infrastructure, said public feedback is important and can help shape plans, but “it’s not uncommon for it to take twice as long to get the permit as it takes to actually construct a major infrastructure project.”