It depends on what part of the world you're in, but here in the U.S. you'd be fighting an uphill battle against some already significant odds trying to go it alone without a degree. It's true that some great designers have succeeded without degrees, but that was then. Today, for someone just starting out it's a different ballgame, and it's a game in which employers are increasingly insisting on design degrees — especially corporate employers.
It's very difficult to succeed in this field, and most who start out don't make it, even with a degree. Trying to do it without the degree, greatly reduces your chances. It's still possible, but the odds are stacked against it.
Where I work, for example, a job application that doesn't list a relevant college or university degree would immediately be discarded by our human resources department. I, as the creative director, would never even know you applied, let alone get to see your portfolio.
And the degree isn't just about getting that magical, door-opening piece of paper. The degree serves as evidence that you've taken the classes, passed the exams, studied in a structured environment and undergone the rigorous critiques and evaluations needed for learning and improvement.
In other words, even if portfolio evaluations were all that mattered when seeking a job, your portfolio of work is likely to be much better having come from a structured, mentored learning environment that it would be if you'd just done your own thing and proceeded down your own paths without the guidance of others with more insight and experience.