Although some users have reported success, most installations I've tried of MySQL 5.1 on Vista have failed, even on fresh Vista installs. The first problem appears at the end of the service instance configuration. All appears to be well, however the server refuses to start with Could not start MySQL service or Could not start the service MySQL. Error: 0.
The trick is to start MySQL from the console so that you are able to see the error message (you can access the command console by typing cmd into the Run dialog):
cd "C:\Program Files\MySQL\MySQL Server 5.1\bin"
mysqld -nt --defaults-file="C:\Program Files\MySQL\MySQL Server 5.1\my.ini" --standalone --console
In my case, MySQL always returned the same error message:
Plugin 'InnoDB' init function returned error.
Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
Unknown/unsupported table type: INNODB
Forcing shutdown of 1 plugins
This message is a symptom of the log file size problem (just google InnoDB: Error: log file .\ib_logfile0 is of different size for more information). All you need to do is to clear the following files from the folder C:\ProgramData\MySQL\MySQL Server 5.1\data:
Restart the MySQL server and all should be well. Note that the C:\ProgramData\ folder is hidden, so unless you have enabled hidden folders from the Folder Options dialog, you will need to copy/paste that folder path directly into the address bar in order to access the folder.