What's the story with Babylon?

I sing in a Sacred Harp group. Last Sunday, we sang a song that I find pretty confusing.

The words of the song include:

Hail the day so long expected,
...
Zion's walls are now erected
...
Babylon is fallen to rise no more

Although I went to Sunday School until I was twelve, I never heard the story of Babylon. Maybe the sex stuff was too much for the teachers.

All I knew about Babylon was that it had hanging gardens, which sound pretty cool to me.

So off I went to find out why we should be happy that Babylon is fallen (to rise no more!).

This page by Joseph McCabe gives an account of an account by Herodotus of a visit to Babylon. McCabe says that Herodotus said that women had to prostitute themselves in a temple, and that marriage was a once-a-year mass auction. McCabe thinks this wasn't true, but that Christianity has taken this account as the truth, and so Babylon has become a symbol or corruption and immorality.

Wikipedia describes the story of the Tower of Babel, built by a cooperative bunch of people who wanted to reach heaven. God destroyed the tower and made the previously uniform language into many, to prevent further attempts to reach heaven that way.

Wow. I think I need a rest from this. I might look into it further another time.