Our story starts years ago, when I was at Gamestop with a subset of my family. We were browsing the Wii and Wii U games, looking for any that stood out. One of us noticed the game NES Remix Pack. This game allows you to play small sections of various NES games, often with special challenges and constraints. Upon getting this game, I played it a little, then forgot about it. Over the years, I gained an interest in older Nintendo games. I got the SNES Classic console and bought many old games on the Virtual Console (back when that was still a thing). I could play many of the games I wanted to. However, one was missing: Super Mario Bros: The Lost Levels. I had wanted to play this for a while, but had basically given up. The virtual console was no more, and there was no way I could get it legally. However, when looking through my Games cabinet, I noticed NES Remix Pack. A quick search revealed that Super Mario Bros 2 (the Japanese version), which is the Japanese release that became The Lost Levels is one of the Games that can be played in NES Remix pack. Knowing this, I had an idea.
Over the last couple years, I played around a little with modding the Super Mario Galaxy Games. This gave me an introduction to dumping and extracting Wii games and working with Nintendo's file formats. With this knowledge under my belt, I decided to look around in the NES Remix Pack files to see if I could find anything interesting.
Note: I completed this process close to a year ago, and some of the specifics are lost to time. I have done my best to reconstruct my process, but there are likely inaccuracies.
Wii U games contain three directories: code, content, and meta. Code contains the executable file, meta contains some image and metadata files, and content contains all the assets. After looking around in the content folder, I found the roms folder. This contained 47 .bin files with names like 'WUP-FA8E.bin' and 'FESJ.bin'. This seemed promising, but none of these files would open in an emulator. During this process, I found an article on The Cutting Room Floor about the game. This is a site that catalogs unused and removed content in video games. The page can be found Here. This contains a list of all the games, as well as their title. However, in some cases, the ROM name had a .fds at the end and no corresponding titles. After some digging around, I determined that these are Famicom Disk System Files. As SMB2J, my target, was released for the Famicom Disk System, I determined that one of these was my target.
Since these files wouldn't run in an emulator directly, some processing obviously needed to be done. I looked into the file formats of NES games. For some games, like Super Mario Bros 3, I had a legally-obtained, playable ROM already, and used this as a comparison point. By comparing the modified version with the original, I could see what specific changes were made. I found that the modified ROM had a 0x30 byte header and a 0x20 byte footer added on. Once these were removed, there was a second header present, a modified iNES header. This is a common and publicly-documented format, so decoding it was simple. I noticed that the byte at position 0x4 needed to be changed to 0x1A, but once these changes were applied, the games ran fine on fceux, the emulator I used.
This process was simple enough, but it was annoying to do manually for each file. To rectify this, I created a tool called Romfix to automate the fixing process. With romfix in hand, fixing the NES roms was simple. However, the FDS games proved a much greater challenge.
I found some online documentation of the FDS format, and determined that the roms I extracted mostly matched the format. However, when I had removed the header and footer, the games still wouldn't run on emulator. (Note: To run these games in an emulator, you first need to acquire the FDS bios rom. I won't link a source here, as this bios is copyrighted, but you could find one if you're determined to play these games.) In order to find the problem, I needed to compare this rom to a playable one, so I sourced one for research purposes only. To assist solving, I created a tool called bindiff. Bindiff is fairly limited, but can find simple differences between two near-identical binary files.
Using bindiff, I determined that the extracted one had the two bytes 0x0000 added in in several places. Consulting the format documentation, it appeared that these extra bytes were a placeholder for CRC-16 codes placed after each section. To make these roms playable, I would need to excise all these CRC placeholders.
My first naive attempt at solving this was to look through a ROM, find each CRC placeholder, and hardcode romfix to remove each. I quickly ran into a problem, though. These blocks
were not always easy to identify, and sometimes blended in with the surrounding data. Furthermore, I soon realized that the CRC codes didn't appear in the same spot in every ROM.
I needed to find a better approach. After digging through documentation of the format, I found that the first two CRC spots were always in the same spot, and the second block contained
the number of files that followed.
(A quick note: the FDS format contains a header, two data blocks, and a number of files. Each file contains a block 3 and a block 4. Block 3
is the file's header, and has some details on the file. Block 4 containes the file data. Block 2 specifies the number of files.)
For each file, the tool removes the CRC from block 3, then determines the size of block 4 and removes the CRC for block 4. Once this process is complete for all files, the end of the
file needs to be modified. The data should end at a multiple of 0xffdc, so any excess data must be trimmed.
Once the file is trimmed, it's fully converted. Now, the data just needs to be written out to the new file, and it's ready for use.
This was a fun project, yet required some thought and effort. I'm glad I did it.
I've provided the tools I wrote, in case anyone wants to use them for this or a similar project.
Once I had verified that romfix produced playable FDS roms, I deleted the fixed file I sourced for research. I ran romfix on all the files, and verified that they played fine, and could now play a bunch of classic NES games I previously had no way to play legally.
Tools downloads can be found on the Downloads Page