I would not recommend this setup for a number of reasons: major lobing issues in your midrange and midbass, dispersion pattern will be very narrow, with a 24 and 12 on the bottom your tweeter will be no where near ear level, and you'll need 10 channels of amplification (assuming the 6.5 and 4's are wired together) just for a stereo pair. There are many proven Hifi and HT tower designs out there that can be built active or passive that would yield much better results for much less time, effort, and cost.
If you're just after building something that looks crazy and gets super loud than build away, but if you want something with decent sound quality I'd look elsewhere.