I have edited this post from the original version. Revision log follows.
- (2 July 2009) Added Makefile section with code to run xkas to transform .s files into .smc files.
- (5 June 2009) Show output from newer snes-rom.rb that includes version; link to wiki page about SNES header.
- (3 June 2009) Add find-runs.rb to find free space, or runs of repeated bytes.
- (31 May 2009) Added a public domain dedication; added Subversion notice; added ruby-ips-2.rb and snes-rom.rb to the post; removed the chat about async I/O from the top of the post; replaced the bottom of the post with a short note below Ruby example 2.
- (4 May 2009) Posted the original version.
Dedication to the public domain: I intend to allow the general public to use my contributions to this SMW-tool-related code library. I place the source code, that I write and embed into this post, into the public domain. Therefore, you do not need a copyright license from me to use this source code.
Subversion: My SVN repository
http://opensvn.csie.org/kernigh/trunk/examples/ruby-snes/ holds some files that I link from this post. I may replace these files with newer revisions.
------------Ruby 1.9---------------
Ruby language is not popular like C#, but I use Ruby with SMW. Kaizo and Ersanio have three pieces in C#, I can redo them in Ruby 1.9.
1. Getting ROM data in to a byte array
This is the code:
Coderombuffer = open(path, "rb") { |f| f.read }
This is an example to use my code at the interactive Ruby prompt:
Codeirb(main):001:0> path = "smw-clean.smc"
=> "smw-clean.smc"
irb(main):002:0> rombuffer = open(path, "rb") { |f| f.read }; 0
=> 0
irb(main):003:0> rombuffer.length
=> 524800
irb(main):004:0> rombuffer.index "SUPER"
=> 33216
irb(main):005:0> rombuffer[33216, 21]
=> "SUPER MARIOWORLD "
2. Writing data to a ROM file
This is the code:
Codeopen(path, "wb") { |f| f.write(rombuffer) }
This continues my example:
Codeirb(main):006:0> rombuffer[33216, 21] = "Super Ruby 1.9 World "
=> "Super Ruby 1.9 World "
irb(main):007:0> path = "smw-ruby.smc"
=> "smw-ruby.smc"
irb(main):008:0> open(path, "wb") { |f| f.write(rombuffer) }
=> 524800
Now if emulator plays smw-ruby.smc, then the title of ROM is "Super Ruby 1.9 World " (and the checksum is bad).
(31 May) This example was slow and wasteful because I used code that reads and writes the entire file, but I only needed to seek and write 21 bytes. HyperHacker (in the post before this post) explained, "Reading/writing the entire file at once is not a good method for most programs."
3. Patching an IPS patch (ruby-ips-2.rb)
Old version:
ruby-ips.rb
(31 May 2009) New version:
ruby-ips-2.rb (svn)
Around March 2009, I wrote
ruby-ips.rb and began to use it as my normal IPS patcher before I play SMW hack. This program has two known bugs: the error messages at line 115 does not work, and Emacs does not like "-*- encoding: US-ASCII -*-" at line 2.
Around C3 of May 2009, I wrote
ruby-ips-2.rb and began to use it as my normal IPS patcher. This second version fixes the known bugs and also adds
-o option to copy clean ROM before apply patch. Now I do not need to copy my clean SMW ROM before I start patcher.
From command line, to apply patch.ips to rom.sfc, modifying rom.sfc:
Code$ ./ruby-ips-2.rb -f rom.sfc patch.ips
To copy smw-clean.smc to smtko-demo1.smc then apply smtko_demo_1.ips:
Code$ ./ruby-ips-2.rb -o smtko-demo1.smc -f smw-clean.smc smtko_demo_1.ips
This is the main loop to process the patch, where
patch is the IO for the patch,
file is the IO for the ROM,
pread.call(3) is a lambda to read 3 bytes from
patch but raise an error if there are less than 3 bytes,
unpack2 and
unpack3 are methods to convert bytes to numbers,
hex is method to convert number to ASCII.
Code # process each record
while true
offset = pread.call(3)
# "EOF" marks the end of an IPS patch, but check if there are
# bytes in this patch after the "EOF".
#
if offset == "EOF"
truncation = patch.read(4)
if truncation == nil
puts "\tend of patch" if list_records
elsif truncation.length == 3
# truncate the file, as does Lunar IPS
truncation = unpack3(truncation)
if list_records
puts "\tend of patch, truncation at #{hex(truncation)}"
end
file.truncate(unpack3(truncation)) if filename
else
raise "unexpected data after \"EOF\" in patch\n" +
"problem with offset \"EOF\" #{hex(0x454f46)}?"
end
# break from 'while true' loop
break
end
offset = unpack3(offset)
length = unpack2(pread.call(2))
case length
when 0
# this patch record uses run-length encoding (RLE)
length = unpack2(pread.call(2))
byte = pread.call(1)
if list_records
puts "\toffset #{hex(offset)}, length #{hex(length)}, " +
"RLE #{hex(byte.ord)}"
end
if filename
file.pos = offset
file.write(byte * length)
end
else
# this is a normal patch record
if list_records
puts "\toffset #{hex(offset)}, length #{hex(length)}"
end
if filename
file.pos = offset
file.write pread.call(length)
else
patch.pos += length
end
end # case length
end # while true
4. ROM expansion with clean ROM check
I have some code to expand a ROM and compute the checksum, but I cut and derived the code from a longer Ruby program that I wrote for my SMW hack, so some variable names might be strange.
This code makes an expanded copy of a ROM and also checks that the original ROM is clean Super Mario World. The clean ROM must be headerless or have a valid SMC header.
CAVEAT: the expanded ROM will be headerless. (I am not sure how to make the header, because I noticed that SMW hacks have invalid SMC headers. I need to learn more about SMC headers.) Also, the expanded ROM will have a bad checksum.
This code might
not be ready for the code library, because I would need to give SMC header to expanded ROM, and to separate the ROM expansion and the clean ROM check into two pieces of code.
Change
clean_rom to path to clean ROM. Change
expanded_rom to path to expanded ROM. Change
Expanded_banks to 32 for 1 MB, 64 for 2 MB, or 128 for 4 MB. Other numbers from 17 through 127 will play in snes9x, but the ROM size in the Super NES header will wrong, because it will be the next power of 2.
Coderequire 'digest/sha1'
clean_rom = "smw-ruby.smc"
expanded_rom = "expanded.sfc"
# each lorom bank has 0x8000 bytes
SMW_banks = 16
Expanded_banks = 128
# SHA1 of the clean SMW ROM (all 16 banks, without SMC header)
SMW_clean = "6b47bb75d16514b6a476aa0c73a683a2a4c18765"
smw = nil
rom = nil
begin
# open SMW ROM image for reading; seek past any SMC header
smw = open(clean_rom, "rb")
size = smw.stat.size
goal = SMW_banks * 0x8000
if size == goal
# headerless ROM
elsif size >= (goal + 512)
# check SMC header
size = smw.read(2).unpack("v")[0] * 0x2000
if size == goal
# seek past header
smw.pos = 512
else
raise "#{clean_rom}: wrong size in SMC header"
end
else
raise "#{clean_rom}: wrong size"
end
rom = open(expanded_rom, "wb")
# copy banks
check = Digest::SHA1.new
SMW_banks.times do
bank = smw.read(0x8000)
check.update(bank)
rom.write(bank)
end
smw.close
unless check.hexdigest == SMW_clean
raise "${clean_rom}: not a clean SMW ROM"
end
# expand ROM
bank = "\xff" * 0x8000
(Expanded_banks - SMW_banks).times do
rom.write(bank)
end
# Write ROM size to Super NES header. If the number of
# banks is not a power of 2, then round upward.
#
byte = (Math.log2(Expanded_banks) + 5).ceil.to_i
rom.pos = 0x7fd7
rom.write([byte].pack "C")
rom.close
rescue Exception => e
smw.close if smw and not smw.closed?
rom.close if rom and not rom.closed?
File.delete expanded_rom # delete bad file
raise e
end
5. Recompute checksum in Super NES header
This is a good programming exercise. This code recomputes the checksum, but my version only works with LoROM (like Super Mario World) and only if the ROM contains a whole number of banks. If the number of banks is not a power of 2, then my code will mirror the banks, so that the checksum equals what snes9x would compute.
(Lunar Magic seems to adjust some bytes so that the ROM has the same checksum as before, instead of recomputing the checksum.)
Change
target_rom to path to ROM.
Codetarget_rom = "expanded.sfc"
io = nil
begin
io = open(target_rom, "rb+")
size = io.stat.size
bank_count = size / 0x8000
case size % 0x8000
when 0
base = 0 # no SMC header
when 512
base = 512 # SMC header
else
raise "#{rom}: wrong size"
end
# Compute checksum in Super NES header. Use the formula
# from wlalink/compute.c of WLA DX, which is the same
# as that from memmap.c of Snes9x ("from NSRT").
#
io.pos = base + 0x7fdc
io.write([0xffff, 0x0000].pack "vv")
# compute checksum of each bank
banksum = []
io.pos = base
bank_count.times do |bank|
banksum[bank] = io.read(0x8000).sum(16)
end
# compute checksum of all banks
checksum = 0
(2 ** Math.log2(bank_count).ceil).times do |bank|
# handle the mirror banks
bank >>= 1 while bank >= banksum.length
checksum += banksum[bank]
end
checksum &= 0xffff
# write checksum
io.pos = base + 0x7fdc
io.write([checksum ^ 0xffff, checksum].pack "vv")
ensure
io.close
end
6. Examine the Super NES header (snes-rom.rb)
(31 May 2009) Full source code:
snes-rom.rb (svn)
My
snes-rom.rb is a program to print the information from the SNES header.
Code$ ./snes-rom.rb
Usage: snes-rom [options] rom...
-l List all candidate headers
The Super NES header consists of 64 bytes inside the ROM at SNES address 0x00ffc0. The SNES header contains information like the checksum and the interrupt vectors.
(5 June 2009) For more information, see
a wiki page about the SNES header.
My
snes-rom.rb tries to pick the best SNES header from four possible locations, and print the information from this header. (My scoring system to pick a header seems to work for my ROM images, but is more primitive than the scoring system in bsnes, mess or snes9x.)
Here is the info from when I run
snes-rom.rb with my Super Mario World ROM.
Code$ ./snes-rom.rb smw-clean.smc
smw-clean.smc:
SMC header: yes
SNES header: offset 0x81c0, score: 9/9
name: "SUPER MARIOWORLD " (21 bytes)
status: 0x20 (slow LoROM)
cartridge type: 0x02 (ROM and save-RAM)
ROM size: 0x09 (512 kilobytes)
RAM size: 0x01 (2 kilobytes)
country: 0x01 (NTSC)
licensee: 0x01 (Nintendo)
version: 0x00
complement/checksum: 0x5f25/0xa0da
interrupt handlers (native mode):
COP BRK ABORT NMI UNUSED IRQ
0x82c3 0xffff 0x82c3 0x816a ------ 0x8374
interrupt handlers (emulation mode):
COP UNUSED ABORT NMI RESET IRQBRK
0x82c3 ------ 0x82c3 0x82c3 0x8000 0x82c3
Here is the part of the code that unpacks the header.
Codeclass SnesHeader
attr_reader :name, :status, :cartridge_type, :rom_size, :ram_size
attr_reader :country, :licensee, :version, :ckcom, :cksum
attr_reader :nvector, :evector
class Vector
attr_reader :cop, :brk, :abort, :nmi, :reset, :irq
alias_method :irqbrk, :irq
def initialize(array)
@cop, @brk, @abort, @nmi, @reset, @irq = array
end
end
def initialize(string)
a = string.unpack "a21CCCCCCCvva4vvvvvva4vvvvvv"
@name = a[0]
@status = a[1]
@cartridge_type = a[2]
@rom_size = a[3]
@ram_size = a[4]
@country = a[5]
@licensee = a[6]
@version = a[7]
@ckcom = a[8]
@cksum = a[9]
@nvector = Vector.new a[11..16]
@evector = Vector.new a[18..23]
end
end
Here is an example after I pasted the above code into the interactive Ruby prompt:
Codeirb(main):032:0> s64 = open("smw-clean.smc") { |f| f.pos = 0x81c0; f.read(64) }
=> "SUPER MARIOWORLD \x02\t\x01\x01\x01\x00%_\xDA\xA0\xFF\xFF\xFF\xFF\xC3\x
82\xFF\xFF\xC3\x82j\x81\x00\x80t\x83\xFF\xFF\xFF\xFF\xC3\x82\xC3\x82\xC3\x82\xC3
\x82\x00\x80\xC3\x82"
irb(main):033:0> h = SnesHeader.new s64
=> #<SnesHeader:0x2b9f5270 @name="SUPER MARIOWORLD ", @status=32, @cartridge
_type=2, @rom_size=9, @ram_size=1, @country=1, @licensee=1, @version=0, @ckcom=2
4357, @cksum=41178, @nvector=#<SnesHeader::Vector:0x2b9f51b0 @cop=33475, @brk=65
535, @abort=33475, @nmi=33130, @reset=32768, @irq=33652>, @evector=#<SnesHeader:
:Vector:0x2b9f5180 @cop=33475, @brk=33475, @abort=33475, @nmi=33475, @reset=3276
8, @irq=33475>>
irb(main):034:0> printf "complement/checksum: 0x%x/0x%x\n", h.ckcom, h.cksum
complement/checksum: 0x5f25/0xa0da
=> nil
irb(main):035:0> printf "reset handler: 0x%06x\n", h.evector.reset
reset handler: 0x008000
=> nil
In the above example, I looked at the SNES header of Super Mario World. I found that
h.ckcom is 0x5f25 and
h.cksum is 0xa0da, which matches what I see in the emulator. The reset handler is at SNES address 0x008000.
7. Find free space, or runs of repeated bytes (find-runs.rb)
(3 June 2009) Full source code:
find-runs.rb (svn)
To find free space in Super Mario World, one must look for runs of repeated bytes 0xff. There might be other tools to look for free space, but I wrote a new tool.
Code$ ./find-runs.rb
Usage: find-runs [options] file...
-b HEX_BYTE Look for HEX_BYTE (default ff)
-s Sort by length of run
-t THRESHOLD Require THRESHOLD bytes per run
For a clean headered SMW ROM, here are areas of free space of at least 500 bytes, sorted by length:
Code$ find-runs -st 500 smw-clean.smc
smw-clean.smc:
offset 0x7f190: run of 4208 bytes
offset 0x772f0: run of 3856 bytes
offset 0x37738: run of 2760 bytes
offset 0x3e96e: run of 2194 bytes
offset 0x34b63: run of 1693 bytes
offset 0x1bc02: run of 1534 bytes
offset 0x3a378: run of 1160 bytes
offset 0x2de46: run of 954 bytes
offset 0x1e25c: run of 932 bytes
offset 0x3fe90: run of 880 bytes
offset 0x2713e: run of 834 bytes
offset 0x5ff0c: run of 756 bytes
offset 0x6f28a: run of 630 bytes
offset 0x223b6: run of 586 bytes
offset 0x1ffe0: run of 544 bytes
My code might be too slow, because it requires 2 to 3 seconds of my computer time to look through a file of only 512 kilobytes (like Super Mario World). The first version required 8 to 10 seconds, before I changed
file.read(1) to
file.getc. This change caused Ruby to fill a buffer, instead of doing one system call per byte.
Here is the code to find the runs, where
file is opened to read in binary mode,
free_byte is the byte to look for,
threshold is the minimum length to report.
Code while char = file.getc
offset += 1
if char.ord == free_byte
# save the offset of this first byte
first_offset = offset
run_length = 1
# look for more bytes
catch(:end_of_run) do
while char = file.getc
offset += 1
if char.ord == free_byte
run_length += 1
else
throw :end_of_run
end
end
end
if run_length >= threshold
# ... code to report this run ...
end
end
end
------------Makefile---------------
This section is for the
make(1) tool from BSD, GNU/Linux and Unix systems.
1001. Run xkas to transform .s files into .smc files
I have some xkas patches with the .s suffix. I want to test each patch with a clean ROM of Super Mario World. I want a different ROM for each patch. I also want to use the save-RAM with "all exits (Star 96)" from
SMWC's Official SRAM/Save State Archive, so that I can test any level.
This Makefile copies the clean ROM, runs xkas to apply to patch, and copies the save-RAM. Here is the entire Makefile.
CodeROM=../roms/smw-clean.smc
SRM=../distfiles/clean_smw.SRM
XKAS=xkas
all:
@for i in *.s; do \
if test -e "$${i}"; then \
j="$${i%.s}.smc"; \
make "$${j}"; \
fi; \
done
.SUFFIXES: .smc .s
.s.smc:
@rm -f "$@"
@cat < "${ROM}" > "$@"
xkas "$@" "$<"
@i="$@"; j="$${i%.smc}.srm"; cat < "${SRM}" > "$${j}"
clean:
@for i in *.smc; do \
if test -e "$${i}"; then \
j="$${i%.smc}.srm"; \
echo rm -f \""$${i}"\" \""$${j}"\"; \
rm -f "$${i}" "$${j}"; \
fi; \
done
Attention! I run xkas v0.12, so I need to use ''xkas "$@" "$<"''. Some other SMW hackers run xkas v0.06, which reverses the order of the arguments, so they would need to use ''xkas "$<" "$@"''.
If I run
make force-balloon-up.smc, then this Makefile copies the clean ROM to
force-balloon-up.smc, applies the patch
force-balloon-up.s to the new ROM, and also copies the "all exits (Star 96)" save-RAM to
force-balloon-up.srm.
If I edit
force-balloon-up.s and run again
make force-balloon-up.smc, then this Makefile discards the ROM and the save-RAM, starts again from the clean ROM, and creates again the ROM and the save-RAM.
If I run
make clean, then this Makefile deletes every .smc file and every matching .srm file in this directory. (I must put the clean ROM in a different directory.) If I run
make, then this Makefile creates the ROM and the save-RAM for every .s patch in this directory.
Hacking Super Mario World since 28 February 2009