There are 3 screw at the bottom of the panel then you can use a door panel remover which the local auto stores sell them to change the handle you will have to gut the door to locate the handle.
its under access panel located in the trunk
passenger side kick panel. remove it, and you will see the tcm
there is a small clip between the handle and door panel that u have to remove it is easy if u have a small pick to hook it with then to install just place clip on handle and push on the window regulator shaft.
http://forums.mazdaworld.org/index.php?showtopic=432
driver door panel........near bottom i know awkward place.
The fuel filter is located through the trunk of the car. There is an access panel on the passenger side.
On my 96 and 98 I removed plastic by handle. Then screws at very bottom of panel now lift up on it Install just the opposite
Take your car to the closest Mazda dealer and get it fixed there. They charge only 1 hour ($60) for the labor. I did it last week, by the way, what it is expensive is the handle !!!. Otherwise, remove the two screws (handle and pocket) and unplug the panel starting from the bottom. The problem is not removing the side panel, the problem is replacing the handle itself.
Remove inner door panel for access
The access panel for the fuel pump is under the carpet in the trunk. Remove the carpet then the access panel.Disconnect the hoses from the pump then remove from the tank.
You can replace the inside drivers door handle by removing the door panel. With the door panel removed the door handle mechanism will be visible. Remove the linkage from the door handle. Remove the door handle retaining bolts. Reverse the process to install the new door handle.
Unless there is an access panel in the trunk then yes it must be removed.