Perhaps it's because I heard the soundtrack before I read the book or saw the movie, but I simply cannot imagine any other music being used for the Harry Potter world. While many film scores are well-composed and seem to complement their films well, this score seems to grow naturally out of the story itself, by way of the great John Williams.
One thing to keep in mind about the music, though, however great it is, is that it was composed to complement the movie and not necessarily as an independent listening experience. That said, the soundtrack does, in fact, make a great listening experience, though a few of the tracks do seem to go on for a while without a lot of direction. Williams seems to be big on providing a constant musical wallpaper--or perhaps Columbus requested this, and that means there is simply a LOT of music that went with this film, and much of it is simply atmospheric.
That said, the entire sountrack is worth buying for the trailer music, opening theme, and end titles alone. These pieces bring the main themes of the entire album together (including the excellent Hedwig theme) and integrate them into coherant stand-alone pieces. Also, even the "atmospheric" tracks provide an excellent ambience for exercising or jogging or just working around the house.