The Colts finished 0–8–1 in the strike-shortened 1982 season, thereby earning the right to select Stanford quarterback John Elway with the first overall pick. Elway, however, refused to play for Baltimore, and using leverage as a draftee of the New York Yankees baseball club, forced a trade to Denver. Behind an improved defense the team finished 7–9 in 1983, but that would be their last season in Baltimore.

The city of Indianapolis, Indiana, made an offer for the Colts franchise to move there. Baltimore was unsuccessful at persuading them to stay, so the city government attempted to get the state legislature to condemn the Colts franchise and give ownership to another group that would promise to keep the Colts in Baltimore. Oakland, California had just had some success in court trying the same tactic with the Oakland Raiders. Under the threat of eminent domain from the city of Baltimore, the franchise relocated to Indianapolis in the middle of the night on March 29, 1984. The city of Baltimore did not give up and sued to condemn the franchise anyway and seize ownership. Baltimore did not prevail in court, but eventually acquired a new NFL team in 1996 with the establishment of the Baltimore Ravens following the Cleveland Browns relocation controversy.