After a planned move to Newark, New Jersey fell through the team opted to stay at the Long Island Arena for the second year, and changed its name to the New York Nets. The name “Nets” was used because it rhymes with the names of two other professional sports teams that played in the New York metropolitan area at the time: Major League Baseball’s New York Mets and the American Football League’s New York Jets, and because it relates to basketball in general, as it is part of the basket.

The team finished last in its first New York season and drew a paltry 1,108 a game, about half of what it had drawn a year earlier. They posted a 17–61 record, and shuffled 23 different players on and off the roster. Brown sold the team to clothing manufacturer Roy Boe after that season. Boe started off his ownership with big hopes for the offseason. Desperate for a star, the team pursued UCLA star Lew Alcindor, and won rights to him in a secret ABA draft. Alcindor was reportedly interested in playing in his native New York, but after contemplating his options for a month, he instead opted to sign with the NBA’s Milwaukee Bucks. Nonetheless the Nets moved on with their offseason plans to relocate to the Island Garden in West Hempstead. Led by an ABA top-3 scoring guard Levern Tart, the Nets finished in fourth place and made the playoffs for the first time in franchise history in the 1969–70 season, and attendance went up threefold to 3,504.

During the 1970 offseason, the team finally managed to acquire a star in Rick Barry after trading their No. 1 pick and cash to the Virginia Squires. After another playoff season at the Island Garden, the team moved to the brand new Nassau Veterans Memorial Coliseum in Uniondale for the end of the 1971 – 1972 season.

In 1972, two years after the acquisition of Barry, the Nets advanced to their first ABA finals. However, they could not overcome the Indiana Pacers and lost the series four games to two. Barry left after that postseason, sending the Nets into rebuilding mode. The 1972 – 1973 season was one of disappointment, as the Nets only managed to win 30 games.

The 1973 – 1974 season saw the Nets finally put all the pieces together. The key event of the season though would come in the 1973 offseason, as the Nets acquired Julius Erving from the Virginia Squires. With Erving, who was affectionately known as “Dr. J”, the Nets ended the season with a franchise record 55 victories. After Erving was voted the ABA’s MVP, the Nets advanced in the playoffs and won their first title, defeating the Utah Stars in the 1974 ABA Finals.

The summer of 1976 saw the ABA–NBA merger finally take place. As part of the merger agreement, four teams from the ABA—the Nets, Denver Nuggets, Indiana Pacers, and San Antonio Spurs – joined the NBA. The Nets and Nuggets had actually applied to join the NBA in 1975, but were forced to play a lame-duck season in the ABA by court order. Prior to their first NBA season, the Nets traded two draft picks to the Kansas City Kings for guard Nate Archibald. The Nets appeared to be poised to pick up where they left off in the ABA.