A number of shopping carts (including osCommerce) can handle "digital download" products. If you have just one product, a shopping cart (even though it's free) may be a bit of overkill. You may want to put your book on a commercial sales service (the names escape me at the moment; I think one is "Payloadz" and another is something like "Hulu") and just point to it from your website. Of course, they collect a commission, but they handle all the payment stuff for you. If you host it yourself, the biggest issue is protecting the downloadable file from being accessed by unauthorized people (i.e., thieves). Processing payments is also a bit of a hassle, and costs you a fee.
.mp3 is a popular audio format, even though it's not that good for music (makes it sound shrill and harsh). Nevertheless, it may be quite adequate for spoken words, and its compression is quite good. I would suggest looking around at major online book purveyors and seeing what they use.