These cheap chinese ipod clones are incapable of playing mp4 files and mtv conversion just creates such large files that are really a waste of space.
Although your firmware is modifyable, there isn't an upgrade that will allow you to play mp4.
My recommendation would be to just leave it as it is with it's original firmware intact. There are a number of different versions of AMV Converter in the download section. One of those should give you 'watchable' movies. Perhaps version 3.57 or 3.70
One of the key things about using VirtualDub is you need patience to experiment with a variety of settings until you get the result you desire.
...Or you could always get yourself a new player