I just gutted my interior while installing the new stereo system, so I figured I'd document the door panel removal in case anyone else gets stumped while trying to install new speakers.
The procedure is the same for passenger and driver side panels. Most of these photos are of the driver's side.
1) GENTLY but firmly, pull straight up on the elbow cushion, starting at the outside corner of it . There are about 6 white plastic tabs holding it in place, but they should pop loose as you pull. You may need to use a flat screwdriver to pop the one nearest the door handle. You can see it from under the cushion after the others let go.
2) Using a flat-tip screwdriver (wrap the tip in a rag to avoid scratches), gently pry the top half of the door latch trim ring loose as shown. It's held in place with two plastic tabs that will "pop" when they let go.
3) Repeat on the bottom half of the trim ring.
4) Use your screwdriver to gently pop the door latch cup loose.
5) In order to release the forearm cushion with the window/door switches in it, you need to insert a flat screwdriver through the newly accessible door latch cavity and pop the metal spring clip loose. Cover the latch handle with a rag to prevent scratches, and press the tip of the screwdriver straight towards the forearm cushion while pulling up on the cushion. Disconnect the wiring harness from the cushion.
6) You now have access to the three bolts holding the door panel in place. Remove them with a 10mm socket.
7) With the three bolts removed, the only things holding the panel to the door are 8 white plastic fittings mounted around the perimeter of the door panel. Simply grab the bottom corner of the panel and give it a tug, pulling the panel away from the door. Each of these connectors will "pop" when they let go. When the panel "lets go", pull the bottom edge away from the door and list the whole panel straight up to pull the top lip of the panel up and out of the window well.
Here's what the door skin looks like without the interior panel (passenger side).
Here's what mine looks like with the new Polk/MOMO 6.5" midbass driver and crossover mounted.