Apple has a really cool sample project on their Developer website for playing videos back in the Terminal on your Mac:

You can download this yourself at the link above, and it actually ships with a binary-version that you can install on your Mac. — The system requirements are just that you have at least Quicktime 6 or 7 and Mac OS X 10.3 or later.
Installation and playback instructions:
- Open a new Finder window and press “Command (⌘) + Shift + G”, then use the dialog that appears to navigate to: /usr/bin
- Drag the “ASCIIMoviePlayer” binary file from the download into that folder. (The file will be the only one with a plain, black “Console”-esque icon.)
- To play a video file, for example, just type: ASCIIMoviePlayer "~/Desktop/nameofyourmovie.mov"
Awesome. 😎
UPDATE: There's actually a similar, color version available elsewhere as well. — I don't think it's as good, but it's not bad! (Thanks for the tip, iMeowBot.)