It's been a while since I've fiddled with one but I had no problem getting the motor going with some AAs, not even that many (three, I think, but my memory is notoriously bad). It's just a bog standard DC-motor, a small one at that, so it doesn't take much to move.
Are you powering the motors directly or through another stage like a micro-controller or some logic? If it's the latter then you'll need a current amplification stage of some sort. For example, according to these
schematics, the motor is usually driven from a 6v line through a H-bridge transistor arrangement.
Good luck!