After World War II a number of German rocket scientists defected to the USA, continuing their work. Early testing was done at White Sands New Mexico. With the successful launch of these V-2 rockets, they began experimenting with a 2nd stage rocket that sat atop of the V-2. These became known as a bumper rocket. The second stage fired after the V-2 had run out of fuel. In the upper atmosphere, velocities of rockets are essentially additive, and the bumper rocket was able to reach a record height of over 250 miles and a velocity of 5100 miles per hour.
The Bumper V-2 was the first missile launched at Cape Canaveral on July 24, 1950.
Why Cape Canaveral?
Cape Canaveral was chosen for rocket launches to take advantage of the Earth's rotation. The linear velocity of the Earth's surface is greatest towards the equator; the relatively southerly location of the cape allows rockets to take advantage of this by launching eastward, in the same direction as the Earth's rotation. It is also highly desirable to have the downrange area sparsely populated, in case of accidents; an ocean is ideal for this.