1.7 Tools of the trade continued
The following simply lists some example tools as there are usually several options for a program to tackle a task, some very basic usage and some screenshots where necessary. In the case of large tools or area specific tools the proper usage will be covered in other parts of this document where there may also be other little tools to help out things. Basic usage of programs to parse relevant filesystems was already covered and will not be covered here. With the exception of the paid hex editors, a couple of audio programs, the debugging/reverse engineering tool known as IDA (a big exception) and some of the translation memory/CAT tools all the programs covered are freeware, often even open source, and everything can be done with the open source/freeware programs.
1.7.1 Hex editor
As part of the underlying project this document is part of several hex editors were tested and rated for their usefulness so as to come up with a shortlist. Most ROM hackers will have several at their disposal with each aimed at different tasks. However if you can pick a paid one, get the four freeware suggestions and get a ROM hacking specific one most things can be done with relative ease. In short though a hex editor is a hex editor and everything else is so much extra that could one day make your life slightly easier. The GUI for most of these is extensively customisable but for the most part the images are those of the stock editor with a few menus displayed as appropriate.
The features that make life easier for ROM hackers are
- Bitwise operations
- Boolean logic/operations (sometimes lumped in with bitwise operations)
- Byte flipping
- Search and search all
- Search and replace
- Hex distribution
- Operations on selected areas (some editors will only operate across the whole file)
- Scripting
- Format/structure listing support
- Variable width windows
- Undo/Redo
- Insert
- Hash values including custom options
- String dumps (occasionally you might just want every string in a game even if inserting it back in will be a pain without the surrounding info)
- Custom character encoding support
- Compare files (including size differences/inserted section support)
Raw disk editing, program/memory editing, X86 disassembly, base64 decode and similar things are other features that an editor might be sold as having are great but they have been of limited use to ROM hackers in the past, even program/memory editing is usually taken care of by specialist programs as you will see when cheats are discussed.
Paid The freeware editors in many ways provide a more than adequate replacement for the commercial tools but many still like the commercial offerings. The two best ones are quite pricey and are roughly equal in ability.
Hex workshop Hex Workshop homepage
Probably the most popular paid editor among ROM hackers.
PIC
010 editor 010 editor homepage
Another paid editor on a par with hex workshop
PIC
Freeware The freeware offerings here, unlike some other areas of computing, are not on a par but with a slightly different GUI. However when the suggestions in the freeware category are combined it makes for all the functionality of the commercial offerings.
ICY Hexplorer Sourceforge page
Almost at the level where you could drop it in as a replacement for the commercial offerings (save for the lack of ability to have multiple files open at once). Needs some setup to get the GUI functioning well but once done it is suitable for use as a day to day editor.
PIC
XVI32 Homepage
Still being actively developed and it is mainly here as it features a powerful scripting language which can accomplish most tasks the paid and function heavy freeware editors sport with a bit more after that owing to it being a true scripting language.
PIC
Tiny Hexer Filetrip download
A discontinued editor but has some very impressive features the equal of, and sometimes even better than, the commercial offerings.
PIC
HxD Homepage
Probably the simplest editor on this list but the go to freeware editor for a lot of people.
PIC
ROM hacking specific As wonderful as the editors, commercial or otherwise, above are they lack things like high grade table support (most of the above will support a measure of custom characters but nothing truly custom like that which is seen in hacking) which is fairly essential for text hacking purposes.
Crystaltile2 Filetrip download
Supports many character sets out of the box and more importantly supports table files.
Lacks boolean manipulations along with the standard hex operations and seemingly fixed to 16 bytes per line.
Has a very good relative search (perhaps not quite as friendly as monkey moore but it works and goes right to up 4 byte/32 bit search as well as many other text grade features covered later)
Has a compression search (mainly type 10 LZ and lesser support for type 11 LZ and huffman).
CRC 16 and 32 are available and can be focused on a selection.
DS filesystem support and header viewing, top flight tile editor/viewer, full ARM9 and ARM7 as seen on the DS disassembler
Support for a fair few SDK and common formats (NARC, SDAT, NFTR, DS 2d formats, some general archive formats)
PIC
Windhex32 Not to be confused with the popular disk forensics grade hex editor “winhex” which is not in the paid list owing to a lack of bitwise features and similar things (it is very good at disk forensics though).
Great table and text support (including multitable support you can switch between), some SNES specific memory mappings and SNES/NES tile editor. Mainly just a very nice text capable hex editor with table support and some tools to complement that. It lacks undo support and some GUI choices are a bit odd which prevents it from being a drop in replacement for HxD.
PIC
PIC
Goldfinger Romhacking.net page Not to be confused with the GBA assembler Goldroad, the common translation of the Chinese term for cheats or the the common translation of the Chinese term for cart pins.
Support for 9 tables at once, it does not come with ASCII readout as standard so you will have to find/make one. It does feature some table editing abilities.
Although not quite suited to full text display it is unlike most other editors in that it is not necessarily bound by the end of the line. This makes it a nice choice for text editing without having to make a custom tool or dump the text and attempt to get something done in a more conventional text editor.
PIC
Translhextion Romhacking.net page
New fork/version Romhacking.net forum thread
For many the standard ROM hacking hex editor for a long time now (although crystaltile2 is edging it out a bit).
Adjustment of hex window size possible via editor but not grouping.
Jump including relative jump support available and can manipulate bits
Can search using tables and relative search support is available.
No undo support but a nice read only option by pressing tab.
PIC
1.7.2 Tile editor
Although you can edit anything with a hex editor it gets very complex to do anything other than the most basic editing using one and the first thing to move to a higher level tool is 2d graphics which get a tile editor. There are several available although only a handful will be focused on here. Various homebrew development kits have some nice programs as well aimed at conversion from common formats to the somewhat odd formats used by the handhelds and other consoles.
Crystaltile2 Filetrip download
Features one of the best tile editors out there (support even for the odd custom hardware display formats and a tile editor capable of being set to arbitrary widths) and has support for various DS image formats on top of the DS file system itself. Exporting and importing images is also possible.
PIC
TileEd2002 Homepage
A GBA vintage editor but as the GBA and DS hardware are largely the same it can get far. It can do basic sized tiles in the two most common hardware formats and has a nice palette fiddling option (one colour at a time if you want), something which some of the others lack and thus is useful when trying to figure out what amount of padding a palette format uses. Lacks support for highly custom tile sizes (it will crash if you try on GBA format imagery) although it does support loading of savestates to get palettes directly from those. Note also the use of imagery to display text as opposed to a text rendering engine; such a thing is very common in smaller puzzle games where there is not much need for actual text, for use in stylised text and in menus in general.
PIC
Also the palette as held in the GBA.
PIC
TileGGD Github page
Although the above two should do for most editing purposes this program has hugely customisable support meaning most conceivable hardware formats should be covered (from 1 to 32bpp with big and little endian support) and in some ways has a slightly nicer user interface than crystaltile2. Unlike the other two there is no editing capability built into the program but there is export and the information can be used to direct an editor of another program.
PIC
1.7.3 Spreadsheet and command line
The following is a few basic tools that can be used to help out when ROM hacking when existing tools fall short and before/instead of jumping to programming a game/format specific tool.
Libreoffice usage Office suite homepage
Calc is the libreoffice spreadsheet program and it supports hexadecimal after a fashion. It is certainly no substitute for a fully realised programming language but it has proven quite valuable when making quick and dirty scripts or reverse engineering formats.
There are seven main operations that get done beyond the basic addition, subtraction and multiplication.
Pasting At least one of your hex editors should have a text export option that when you have set the appropriate amount of columns can export a text list of the hexadecimal (effectively making an array) and equally a search option should be able to export the results. Either way you will need to paste this into the spreadsheet which for the most part is fairly intuitive and automated but you will occasionally have to import as a fixed width or as a delimited set of text (usually a space or tab doing the delimiting). Merging cells (say for a 32 bit value spread across 2 columns where you do not want to change your editor’s behaviour) can be done but the quick and easy way is to paste the columns into a text editor and search and replace for the delimiting value.
If you must though you are better off abusing a maths function and multiplying by the appropriate hexadecimal value (65025 and 255 to shift the hex equivalent by 2 and 1 bytes respectively) and the reverse using mod, floor and other functions.
Bitwise, boolean and flipping operations are best done in a hex editor and given the option you will also want to import as text (all the functions will still work) as numbers have a habit of being parsed to something.
Fill A basic command/option but not one everybody knows about. In the bottom right corner of a given cell when selected there will be a small square which you can click and hold before dragging down or up (or across) and the cells have the contents replicated in the cells covered by the drag range. If you have a pattern it will tend to be continued and if you have a formula that will tend to be continued but the cell contents aligned to the same thing (if the original was c4 - c3 the next will likely be c5-c4), it is not foolproof and some of the more advanced things you want want to do can be tricky to pull off but it has worked far more often than it has not.
Dec2hex and hex2dec Although calc does support hexadecimal and you can combine items into one function it is usually easier to have the initial hexadecimal values, the decimal equivalents and the conversion back again.
In calc the commands are dec2hex to convert from decimal to hexadecimal and hex2dec to do the opposite.
Differences Granted this is more of a technique than an actual function but it is the most used concept that actually changes/generate data. If you have a field of pointers (covered later but the general idea is a value that contains the location of another value) and the results of a search for something that indicates the start of a value you might need them to line up but it might not be readily apparent. Most of the time with pointers values change between them (if the data is a fixed distance apart there is no need to incur the time penalty for looking up the pointer and maybe doing some operations upon it) and this can give things away. To do this simply take the next pointer value and subtract the current one. The result will be the difference and if you do it for an unknown pointer set you can quite easily match things up and determine if they are “out” by a given amount. You can do a similar thing in reverse to generate new file lengths to save calculating and changing an entire pointer table by hand but by this point it is probably better to build a proper program.
Rounding function As mentioned data tends to like to be found at 8,16 or 32 bits or some other interval (several file formats on the DS have been observed to align to an address that is a multiple of 100h). CEILING is the main function here although remember it takes decimal input for the number to round to. MROUND can also be used in a pinch but remember it can also round down which would be bad so best to add an amount if you are going to use that.
Sort function Not quite so useful in ROM hacking as it is in day to day use is the ability to sort by a value (either letter order or number order)
True/false queries and parsed data Humans are not so good at recognising and interpreting numbers at pace but nice coloured squares are a different matter and quite possible in various spreadsheets. Still if you must use numbers 1 and 0 are easier to account for than lengthy values and spreadsheets can then help with this. The basic method uses the IF command and is typically formed “IF(some value/cell, equals/is less than/is greater than, then FALSE/TRUE)” but deleting as appropriate.
Filecutter crackerscrap.com (click on downloads)
Usage: filecutter file.in length file.out <-s start>
As windows lacks the ability to slice up files from the command line you have this program. Once you have your list of addresses you can use this to generate a batch file with the addresses as the arguments and although it will be specific to that incarnation of the file (not such a problem if you just need everybody to slice up the file as it comes from the ROM) you have just built an archive unpacker. If you need to couple it with a decompression tool you can do that as well in just a few extra steps back in the batch file stage.
Input is in decimal by default but hex can be used if you stick 0x in front of the relevant numbers.
Getmyhex Romhacking.net download
A simple tool to get the hexadecimal readout of various short sections of text.
Radare(2) Project homepage
Now taking the place of the Romulan program featured in earlier editions. It is a scripting language but not quite, it is aimed at reverse engineering purposes though the focus is more on the PC and related platforms.
1.7.4 Compression
Compression was once the bane of ROM hackers but it got a lot easier to handle on the DS and is not so bad for the GBA either. At this point it might even have reduced to making for a simple extra step using a known tool when extracting something from a ROM or putting it back in but not much else.
DSdecmp Github page
Supports compression and decompression of LZSS formats seen on the GBA and DS (type 10, 11, 40 and binary/BLZ), RLE and Huffman.
Cue’s GBA DS compressors GBAtemp thread
Also supports compression and decompression of LZSS formats seen on the GBA and DS (type 10, 11, 40 and binary/BLZ), RLE and Huffman as well as LZE (used in Luminous arc titles).
Crystaltile2 Filetrip download
Has a measure of compression support built into the file manager (type 10, type 11, binary decompression support some RLE and maybe Huffman) and support for some compression searching options. Somewhat buggy but you can learn them and play to them well enough.
GBA specific BIOS and to a lesser extent general LZ compression can be searched for as it makes fairly distinctive changes to the hex. There are also a few tools geared towards being able to deal with GBA ROM images directly and work around issues stemming from a lack of a filesystem.
GBA Multi DeCompressor romhacking.net download
Can be directed and fed VBA SWI logs (SWI being the name for the BIOS calls and as mentioned the BIOS in the GBA and DS feature decompression functions).
NLZ-GBA Advance romhacking.net download
Ostensibly a graphics editor but one with compression support and compression searching.
unLZ-GBA romhacking.net download
A slightly older tool but one of the few ones capable of compression searching.
Lz77restructor2 Filetrip download
A newer tool with abilities in graphics and text extraction and insertion/edition on top of the ability to search for compression and restrict those searches.
GBADecmp romhacking.net download
A simple tool to decompress and recompress data from/to a known location.
Crystaltile2 Filetrip link
Supports type 10 LZ which is the same as the GBA BIOS LZ compression. Also supports compression searching.
GBACrusher Filetrip link
A tool to compress files using GBA BIOS compatible compression methods like the 8 and 4 bit Huffman compressions, Differential, Run length encoding, LZ (type 10) for VRAM and for WRAM. Command line version included.
PIC
1.7.5 Music
Format and console specific tools will be covered in the relevant sections. However a few high level tools are useful to have.
Wave editing - Audacity Audacity Sourceforge page
Imports most wave, PCM and ADPCM variations and features editing, some mixing ability and filters.
Tracker format - Open MPT Open MPT homepage
A fairly advanced program with support for playing, editing and exporting various tracker formats. Should have a measure of DLS support although it can be problematic. Formerly known as ModPlug Tracker which is what some tutorials written before the rename will refer to it as.
Midi specific - Anvil Studio Anvil studio homepage
A freeware program that several of those editing audio for the GBA like to use.
General editing - Awave studio Awave studio homepage
A largely paid piece of software that can help convert files and deal with less than perfect implementations of some audio formats various game specific tools might output. Midi and DLS support is available.
1.7.6 ASM/Assembly
Usage is often as extensive as assembly itself but some tools none the less
Emulators (debugging/hacking grade) The following is a list of emulators that possess debug functions of a grade that is useful to ROM hacking without going to abstract methods of debugging.
DS there are a handful of emulators available but only three have any real support for commercial ROM images and debugging.
Desmume
The developer and regular versions feature memory viewers, disassemblers, VRAM, OAM and other such viewers. Also features support for GDB and LUA type debugging as seen in high grade hacking focused emulators like FCEUX. Its cheat making options are fairly well developed nowadays as well.
no$gba
The gaming version of no$gba features very few debugging features (although there are some memory editors that interface with it) but there is a debugging version, which is now free to download, available with extensive debugging abilities. Note that ROM images may well need to have their secure area encrypted to run but eNDryptS Advanced should be able to handle that.
iDeaS
Though slightly less developed than Desmume on the commercial ROM front it does however support something closer to breakpoints as seen in no\(gba and the GBA emulators as standard. Function logs and run to selection command are more prominent in the debugging section though and it is not quite a full replacement for no\)gba.
GBA The GBA has a somewhat larger and more featured collection of debugging grade emulators.
VBA-SDL-h
Version of the popular GBA emulator reworked to add debugging support like the ability to set breakpoints.
VBA-h
VBA-sdl-h above is geared towards assembly hacking and lacks much in the way of a GUI where VBA-h is geared towards memory viewing and cheat making.
no$gba
Along with the DS the GBA is well supported in the debugging editions of no$gba.
BoycottAdvance
Some prefer this to VBA-SDL-h and it certainly is a bit more GUI happy. It can take a bit more to get some ROM images working and some of the features are not as extensive but it does have breakpoints which counts for a lot.
PIC
Disassemblers Disassemblers are tools that can be directed to turn machine code and related information back into assembly code. They are pretty dumb for the most part and their output will tend not even to be able to be reassembled without some modification by a human, however take the time to set one up properly for the task you want and they are invaluable.
GBA and DS Emulators will usually provide some disassembly and as they know what mode the processor is running in at the time and have various viewers for memory (video, normal or otherwise) they can be even more useful but standalone disassembly tools do exist. Note that the DS does feature a custom, albeit widely supported, compression format that its binaries can and do use.
- Crystaltile2Filetrip download. Has a basic disassembler for ARM9 and ARM7 built into the program and the ability to interface with other programs.
- NDSDIS2 NDSDSI2 homepage Filetrip download. A basic standalone disassembler aimed at the DS.
- arm-eabi-objdump Part of devkitpro/GNU toolchains. Not so useful on ROM images/for single files only and does not support compression but should work well if you can get it something it can sink its teeth into. If dealing with newer systems then looking at these sorts of toolchains will get you something.
- IDA IDA homepage Paid software, the freeware edition is quite locked down though (basically X86 or nothing). This is the go to general purpose disassembler/debugging tool and one all new disassemblers and debugging plugins/tools for various platforms tend to be written for.
Assemblers The processors in the GBA and DS are quite similar so you can usually go from one to the other. Developer no$gba and crystaltile2 feature single instruction editing and IDA has some abilities in this arena too. Also tending to be 16 or 32 bits in length you can often edit instructions by hand. This will focus more on hacker grade assemblers, mainly as programming grade assemblers have great features like the ability to create variables/human readable references and similar things by default where hacker grade ones tend to require more raw input (although armips does have a lot of niceties here).
The GBA ARM7 and DS ARM9 are very similar and the added instructions for the DS ARM9 (all of three which are not all that commonly used) you can live dangerously and switch between them. Since the earlier versions of this document ARM have risen up even further in the world (they basically own all of mobile phones and tablets) so if you are using newer tools, or ones not suggested here, then make sure you are using the right modes.
Again assembly will be covered later (including some links to the official specifications) but in the meantime imrannazar.com ARM Opcode Map has a full listing in a more readable form.
armips
A relative newcomer in the ROM hacking assembler world (the first release was back in September 2009). Geared towards GBA and DS ROM hacking (also MIPS R3000 for the PS1) it has the option to use macros, labels (global and local), can load tables so as to be able to load custom strings and something closer to C/C++ family maths than the average assembler. Owing to the way it works it has pretty good support for overlays as well.
ARMeabitoolchain
ARMeabi deals with the underlying assembler for the GNU development toolchains (although for the GBA/DS specific stuff you will want to be looking at devkitpro).
As part of an earlier hacking project a kit was made to assemble small file fragments into things that could be dropped into the ROM. Two main methods aimed at hacking here
cracker’s ARM ASM kit
crackerscrap.com (click on projects)
gbatemp download (older version)
Garmy
For the GBA you may also like this script from Dwedit.
goldroad
For quite a while the main assembler available for the GBA as far as ROM hacking was concerned. It is not the cleanest tool out there but can get things done and for some armips replaced it but the above tools are now the preferred method.
armish
Written in lisp and aimed more at homebrew programming it is another assembler for the ARM processor family.
arm sdt
More of a programming assembler and features some very nice functions to help with program development. Many of the GBA homebrew emulators, and some versions of moonshell on the DS, were coded using this in preference to the GNU toolchains, something which made maintenance, forking and third party contributions more difficult in some cases.
FASMARM
An ARM plugin for the X86 and x86-64 assembler FASM. You can find FASMARM here.