I left college at 20 and then went back at 26. Both are on my "top 5 best decisions in life" list.
Having some time to get older and know myself (or, perhaps, reflect on the nature of selfness) led me to make great use of the incredible resources and space a university offers.
Everyone's path is different, so yours might vary, but you never know! I don't know if you still carry any fear or shame about the college experience (I did), but you're clearly driven and capable of great success. Don't let any of that other stuff get in the way should you ever find yourself with a passion and the desire for university as a space to focus and invest in yourself. In any case, congrats on success!