1. In Safari > Preferences > General, change ‘New windows open with’ and ‘New tabs open with’ to either ‘Homepage’ or ‘Empty page’ (as you prefer).
2. In Safari > Preferences > Bookmarks, uncheck ‘Include Top Sites’.
Now you also need to get rid of the caches, and to stop Safari from continually storing images of your web page history (Tip: Safari will still track your History in the normal way, but here we are going to prevent it from downloading the image files that are used in Top Sites), so:
3. In Safari > Reset Safari…, check ‘Reset Top Sites’ and ‘Remove all webpage preview images’.
4. Go to your home folder Library (~/Library) by clicking on the Folder icon in the dock, pressing ‘shift-command-g’, and typing ~/Library in the box.
Navigate to Caches > com.apple.Safari.
5. Click once on the Cache.db file. Hit ‘command-i’ on the keyboard. In the Get Info panel that opens, check the ‘Locked’ box. Close the panel.
6. Click on the Webpage Previews folder in com.apple.Safari and press ‘command-i’. Check the ‘Locked’ box. Close the panel.
7. Navigate back to Caches > Metadata > Safari> Bookmarks. Go into the Bookmarks folder, hit ‘command-a’ and then ‘command-delete’ to send all the selected files to the Trash.
8. With the Bookmarks folder selected in Finder, press ‘command-i’ and check the ‘Locked’ box. Close the panel.
And what about later versions of Safari? There’s no way to remove Top Sites in Safari 7 that I know of (if you know different, please leave a comment below). However, there’s no reason to suffer in silence! Let Apple’s Safari dev team know how much you dislike it: