No when it comes to music and the arts, but the general point holds when it comes to sports. I grew up in areas you would likely call 'posh', and we were never as tough as the kids from the rougher neighborhoods, no matter how hard we tried. We were smarter, and sometimes that made the difference, but when it came to competing at the absolute highest levels it was never quite enough.
That said, Kobe Bryant came out of similar areas to mine, whatever he tried to claim over the years. Lower Merion is far from the straight up 'hood. IIRC, Tim Duncan was the child of two doctors in the Virgin Islands. Then there's Nicklaus, Jack and Woods, Tiger.
Growing up rough is not a prerequisite. It does tend to confer an advantage in sports. There are also ways to compensate, if you're talented enough.