One guess found on the band's web page: From p.78 of Hemingway's A Movable Feast: "Anything was better than Ezra learning to play the bassoon..." Another guess is that the unnamed band had entered a contest against another group known as "Ezra". They used this idea and chose the name "Better Than Ezra".