Rollins was born in New York City, to parents who were born in the United States Virgin Islands. Rollins received his first saxophone at age 13. He attended Benjamin Franklin High School in East Harlem. He said that a concert by Frank Sinatra there, accompanied by a plea for racial harmony, changed his life.
Rollins started as a pianist, changed to alto saxophone, and finally switched to tenor in 1946. During his high-school years, he played in a band with other future jazz legends Jackie McLean, Kenny Drew and Art Taylor. He was first recorded in 1949 with Babs Gonzales (J. J. Johnson was the arranger of the group). In his recordings through 1954, he played with performers such as Miles Davis, Charlie Parker and Thelonious Monk.
In 1950, Rollins was arrested for armed robbery and given a sentence of three years. He spent 10 months in Rikers Island jail before he was released on parole. In 1952 he was arrested for violating the terms of his parole by using heroin. Rollins was assigned to the Federal Medical Center, Lexington, at the time the only assistance in the U.S. for drug addicts. While there he was a volunteer for then-experimental methadone therapy and was able to break his heroin habit. Rollins himself initially feared sobriety would impair his musicianship, but then went on to greater success.
As a saxophonist he had initially been attracted to the jump and R&B sounds of performers like Louis Jordan, but soon became drawn into the mainstream tenor saxophone tradition. Joachim Berendt has described this tradition as sitting between the two poles of the strong sonority of Coleman Hawkins and the light flexible phrasing of Lester Young, which did so much to inspire the fleet improvisation of bebop in the 1950s.
Rollins began to make a name for himself in 1949 as he recorded with J. J. Johnson and Bud Powell what would later be called "hard bop", with Miles Davis in 1951, with the Modern Jazz Quartet and with Thelonious Monk in 1953, but the breakthrough arrived in 1954 when he recorded his famous compositions "Oleo", "Airegin" and "Doxy" with a quintet led by Davis. Rollins then joined the Miles Davis Quintet in the summer of 1955, but left after a short stay to deal with his drug problems. Rollins was invited later in 1955 to join the Clifford Brown–Max Roach quintet; studio recordings documenting his time in the band are the albums Clifford Brown and Max Roach at Basin Street and Sonny Rollins Plus 4. After Brown's death in 1956 Rollins began his subsequent career as a leader, his first long-playing albums released on Prestige Records. He also recorded during the 1950s for Blue Note, Riverside, and the Los Angeles label Contemporary.