4.2 Video
The GBA and DS are capable of playing video and as such several games use full motion video of various forms in their games. On the DS at least a company (now owned by Nintendo) called Mobiclip made a format used for a lot of the games.
Reverse engineering video formats is in many ways one of the hardest things you can do in game hacking. Fortunately on the handhelds you are quite lucky as they are not usually powerful enough to allow for some of the complex methods that make up a modern video format like H264, or indeed that much in the way of a true legacy format like MPEG1. Mind you the DS homebrew media player known as moonshell uses MPEG1 for the video as part of the DPG format and there were ports of MPEG4 ASP aka xvid/divx to the DS as well. The GBA also saw a codec from the same people that made Caimans. If you want some history then this post has a bit more.
The traditional thing at this point is to point at MPEG1 coding methods and say MPEG1 had a final draft over 20 years ago at time of writing (late 1992). Now if you recall back to simple 2d graphics and how just changing a single tile width could drastically alter things consider trying to work backwards via analysis methods from there to getting images and then building a compatible encoder; some DS stuff is somewhat simpler than this but not by a lot.
As most hacking work on handhelds does not allow for video encoding or editing there have to be workarounds to do things. If just ripping the video is your desire most emulators have recording options and you can augment things here by changing the video files so if a game has an ending cutscene or something you can repoint the intro sequence to play that instead and rip it from there or inject it into a more suitable game for ripping and by the same logic if you are “undubbing”/“delocalising” a game you can often just drop the equivalent video in and call it a day. If you do need to add something to the video the traditional method used in a handful of DS hacks works off the fact that video is just 2d images in the end so you can add images, subtitles and such as sprites or overlay something; this is quite an involved hack and will probably require some knowledge of assembly (it is part of the reason hardware was discussed back in graphics editing) unless the game itself already has images placed over the video that you can subvert.
As just as reverse engineering a video format is a hard task the act of creating a new one is equally or even troublesome (and that is before you get into the likes of software patents that trouble just about anyone wanting to break into the video encoding world) game developers/companies will tend to buy one off the shelf for use in a game.
Do note that video seen on the DS and games in general frequently does not to have audio built into it so you might have to find another method by which to rip the audio or account for this if you do a basic relink/repoint hack or undubbing the game (especially if the video length changes).
4.2.1 General video theory
Following on from the graphics and audio is that video can work by tricking senses with the general idea behind video is you play back enough frames fast enough and you can create the illusion of movement; the magic number seems to be somewhere around the 17 frames per second mark although the low to mid 20s are where it gets better although lower can work depending upon what you are doing. Updating full images to a screen 25 odd times a second places a serious demand on system resources (storage space and bandwidth mainly) and when you think about it most video does not really change much frame to frame so there are things that can be done. More so than other areas moving video is still very much the domain of the lossy encoder (several great lossless options exist but they are mainly for storage, editing and capture purposes as opposed to playback) and in many ways the DS is no exception. The two principle methods/assumptions are
- One frame tends not to change from one to the next.
- One pixel probably does not change much from the one next to it.
Lossless codecs take advantage of this and lossy codecs go one step further and choose to lose some information based on those principles. There are a very wide selection of methods and levels of use of those methods which only get more complex as time goes on but videos are typically broken up into squares (quantisation - if you have seen what is usually a high action scene break down into squares where the action should be this is the reason) or treated as a waveform (wavelet encoding used in encoders like Dirac and Snow but there was a DS homebrew program called DSVideo that used it). On top of this although one frame does not change much from one to the next (on average) inter frame encoding is not mandated and glorified slideshows are quite common on lower powered systems (motion JPEG is typically given as an example) but some have been seen on the DS as well.
4.2.2 Mods/VX/act imagine by Mobiclip.
First it should be stated Mobiclip (also known by the former name of Act Imagine) did make a codec/standard for mobile phones and web which did enjoy a measure of success there but it is nothing really to do with the console side of things (certainly if you find the codec for it nothing much will come of it).
Although now a Nintendo subsidiary before that happened they sold their video encoding software for use on the gameboy advance and it became part of the Nitro SDK so as such became the standard video encoder for DS games, use of it is not as widespread as the likes of the SDAT audio format for not every game has video and not every game that uses video uses this.
Some tentative reverse engineering work has been conducted (GBAtemp thread) and it seems to be a fairly simple format without many of the trappings of higher end formats. multimedia.cx has some more and notes it used Block Truncation Coding.
There are also a few versions of the standard floating about in different games (VX became mods) so that will need to be accounted for when time comes.
4.2.3 RAD/Bink
Seen usually with the extension .bik it is probably the most well known computer game video format and it is used everywhere on the PC and home consoles as well as the likes of the PSP; you can even download a simple playback codec from the developer’s website. The DS has support for it but it has not been seen in many games presumably as Mobiclip was tied to the SDK quite closely and those games that have used it have largely been multiplatform titles. For this document a handful of games from the list were tested against standard decoders and they worked.
Looking at the various output methods and sales information it appears as those they support a fairly raw decoding of YV12 encoded colourspaces; YV12 is a subset of the YUV colour encoding method which rather than representing all three component colours as has been common up until now instead chose to split the resulting colours into luminance and chromiance which allows some greyscale compression.
Output of “attract.bik” (a full video taken from a camera as opposed to an animation rendered and stored as a video file) from the fairly early game “Greg Hastings Tournament Paintball Maxd” has info of
Video: YV12 512x192 12.00fps \[Video\]
Audio: PCM 22050Hz stereo 705kbps \[Audio\]
It also clocks around 8.36 megabytes and other videos have been seen to use lower bitrate audio.
Upon upscaling significant blocking can be seen suggesting a measure of quantisation and as the audio output says there is audio in the file although it is unknown if it is interleaved or left as blocks (in some cases easier on a system but non contiguous reading/seeking is not something the DS is good at). Header appears to start with “BIKi” in ASCII and be followed immediately by a 32 bit value stating the length minus 32 bits which presumably is the the length of the stamp and size combined (remember many values will ignore the header or parts thereof).
?20 appears to indicate sound presence (credits.bik lacks it in file where others have it) and there may be a mono/stereo flag.
100 and C0 appear in the format and seem to indicate 256 and 192 dimensions (the decoder reports double height for all videos including those from other consoles).
Bik was also seen in Impossible mission with a few different framerates (20fps for logo.bik, 10 for winning.bik and 12 for losing.bik)
Of special interest is likely to be beep.bik which has following output as the information about it and was largely blank.
Video: YV12 512x32 1.00fps \[Video\] Audio: PCM 16384Hz mono 262kbps \[Audio\]