lol, I just came here to post the exact same thing. I just use 3-4 firm hits with a ball pein hammer. And just so people realize, the cup stays on due to the magnet, so don't keep hammering away expecting it to pop off.
Also, I've had a lot more success getting the magnet to pop by having the speaker on an approx 60 degree angle, and the chisle verticle. I also have someone else hold the speaker while I hammer the chisle.
And wear safety glasses. The magnets like to sheer, and due to the magnetic forces the shrapnel flies off in funky directions. I had a friend holding the 8" and half the magnet sheered and flew UP into his forehead. It's very sharp, and gave him a pretty good cut.
Edit:
Be sure to support the driver by the back plate. If the lip on the frame, or the magnet, is taking the load you have a much higher chance of shifting the assembly.