Veeck attempted to move the Browns back to Milwaukee (where he had owned the Brewers of the American Association in the 1940s), but the move was blocked by the other American League owners, seemingly for reasons that were more personal than business-related. Undaunted, Veeck got in touch with Baltimore Mayor Tommy D’Alesandro and attorney Clarence Miles, who were leading an effort to bring the major leagues back to Baltimore after a half-century hiatus. However, he was rebuffed by the owners, still seething by the publicity stunts he pulled at the Browns home games.

Although there was never any official word that the 1953 season would be the Browns’ last in St. Louis, there were enough unofficial signs that attendance flatlined to only 3,860 fans per game. Under the circumstances, the Browns made a wretched showing, finishing 54-100, 46 games out of first. This was partly because late in the season, the Browns were running so low on baseballs that they were forced to ration them during batting practice. When what would be the Browns’ last game in St. Louis—a 2-1 loss to the White Sox—went into extra innings, the Browns had so few baseballs on hand that the umpires were forced to recycle the least damaged ones that had previously been used. Reportedly, the last ball used was gashed from seam to seam.[6]

After the season, Veeck cut a deal with Miles which would see the Browns move to Baltimore. Under the plan, Veeck would have remained as principal owner, but he would have sold half of his 80% stake to a group of Baltimore investors headed by Miles. Despite being assured by American League president Will Harridge that there would be no problem getting approval, only four owners voted aye – two short of passage. Reportedly, this was due to Yankees co-owner Del Webb drumming up support to move the Browns to Los Angeles.

Veeck, Miles, and D’Alesandro realized that the owners were simply looking for a way to push Veeck out. Over the next 48 hours, Miles lined up enough support from his group of investors to buy out Veeck’s entire stake for $2.5 million. Veeck had little choice but to agree. He was facing threats of having his franchise canceled, and he’d given up his only leverage by selling Sportsman’s Park to the Cardinals. The other owners duly approved the sale. While Baltimore brewer Jerold Hoffberger became the largest shareholder, it was Miles who was named president and chairman of the board. His first act was to request permission to move the team to Baltimore, which was swiftly approved.