Firefox 4 will automatically reload the tabs and windows from when you last quit if the "When Firefox starts" option is set to "Show my windows and tabs from last time". Unfortunately, this means that it will always show your last session instead of a blank browser window.
In prior versions, it would only show the windows and tabs from last time if you quit and told it to save. With Firefox 4, the previous session's state can be restored via History->Restore Previous Session. Alternatively, the browser.showQuitWarning setting can be set to true in about:config to restore the Firefox 3.x behavior.