Language…
17 users online: Anas, AnunnakiGirl, CourtlyHades296, CroNo, DaveStateGaming, Green, Heitor Porfirio, Hiro-sofT, Mohamad20ZX, Papangu, SAMYR DUTRA ARAUJO, signature_steve, SomeGuy712x, StarWolf3000, TheBubbster123, TrashCity, yoshisisland - Guests: 288 - Bots: 689
Users: 64,795 (2,368 active)
Latest user: mathew

Split Disassembly-based hacking

  • Pages:
  • 1
  • 2
Thanks to Amphobius pointing me towards the Sonic hacking community and how they hack Sonic, I discovered split disassembly-based hacking, and I'm thinking of applying this technique to SMAS as well.

What is a split disassembly?


It's a disassembled ROM (which can be reassembled), but it's split into various meaningful parts. Instead of having one big .asm file (such as all.log for SMW), you'd have the data divided into smaller parts like at least banks. Each bank then would have their tables extracted into binary files. These tables would be meaningful tables such as graphics, level data, tilemap data, and so on. All references are also labeled in the ASM file so that you won't have any hardcoded addresses left, allowing you to insert or remove code without breaking the game.

These files then would be divided into various directories. A directory for graphics, a directory for levels, and so on. Seeing how SMAS is programmed, it's possible to divide all of these under certain games as well. Eventually, by running a simple command such as "asar smas.asm" it would put all of these files back together into a playable ROM.

How do you hack a split disassembly?


There is no ASM knowledge required for casual hacking. You download a split disassembly with all necessary ASM hacks installed already (to allow more GFX or levels for example, which the vanilla SMAS ROM can't do). Theoretically speaking, one can write a level editor which loads all these binary files and puts them together so the level is viewable and you can edit it. Then the level editor will save data to the level binary files. It can save palette, map16 edits, etc too. Finally, the hacker runs the assembler to create his ROM. (After that, you could run the ROM through an IPS patcher or something when releasing)

If you do know ASM and would like to edit the ROM on an ASM level, this can be done easily as you're working with the disassembly directly anyway.

Why split disassembly?


It'll allow ASM hackers more control over their ROM while allowing 'casual hackers' to freely hack SMAS without bothering with ASM at all. Furthermore, by putting defines on top of the ASM file which the hacker can edit, you can put a bunch of 'options' which get taken care of at assembly-time. Imagine the following code:
Code
!EnableSMB1 = true
!EnableSMB2 = false
!EnableSMB3 = false
!EnableSMBTLL = false

Reset:
SEI
CLC
...

With these definitions you tell the assembler that you only need SMB1 code in the ROM and that it can discard the other games, thus enabling more free space for your SMAS SMB1 ROM hack. (Although in my opinion it's better if you do this manually if you want to avoid depracated code cluttering your ASM files)

How would one create tools and patches for this method?


Frankly, I have no idea. It's something I'll need to deeply think about. Unless someone has suggestions then feel free to post. The only 'solution' I have is just hack the SMAS ROM as extensively as possible before releasing the split disassembly, but it still doesn't solve the issue of easily allowing '3rd party tools'.

Bonus: Why not apply this for SMW too?


SMW has progressed too far for this technique to be meaningful anymore. If SMW switched to split disassembly-based hacking, it would mean remaking all resources and integrating them into the disassembly. Not to mention that FuSoYa would need to make all the Lunar Magic ASM hacks open source. This goes for other closed-source tools as well. Too much effort for too little. As for SMAS, because it doesn't have any resources yet, it's the perfect time to apply the split disassembly hacking technique.

Will SWR take notice of this?


I told him the idea. He says "it can be done right now" which is pretty promising.

And my question to you (the hackers who are interested (if any))...

Would you like split disassembly-based SMAS hacking? Why (not)?


My blog. I could post stuff now and then

My Assembly for the SNES tutorial (it's actually finished now!)
That actually doesn't sound too bad. I'd support it.

It would be better for ASM nerds to have everything nicely organized already instead of finding the code in an uncommented mess of a single file.

...on the other hand though, it could easily mean that hacking SMAS will ultimately never happen. A split disassembly sounds like enough work that you'd have to start from scratch.
Wasn't MushROMs already supposed to do something similar? Or at least, I thought it was set up to where the editor would output assemblable ASM files that contained the new contents of the ROM. I don't know whether it was a split disassembly or not, though.

Anyway, it sounds like a good idea to me, as long as we're not forced to move things around. I like knowing exactly where everything is in the ROM (except added stuff that wasn't in the original game), I like being able to call routines like $01A7DC and $03B72B by their address rather than needing a define, and it seems to me you'd have a heck of a time making any custom patches if code could move around. In any case, though, a split disassembly could keep things more organized and let you more easily track exactly what is being done to the ROM and when.

(On a side note, I already have several ideas for stuff to add to the SMAS ROM, as I think you know from conversations on IRC....)

----------------

I'm working on a hack! Check it out here. Progress: 64/95 levels.
If I understand you correctly, the idea is simply to go away from a assembled rom to its source code, and base hacking around the source code. If I seem to misunderstand you, ignore the following :)

Patches should be easy then, just normal source code patches should be sufficent, it would allow especially for crazy things as collaborative hacking efforts via git/svn/etc.

Things which should be modifiable by tools (such as level data) could be external of the source code in binary files and .incbin'ed. Tools would simply modify these files. Another approach would be reading the assembly files and rewrite them, which sounds kind of dangerous (given that asar does not provide a real grammar).
Your layout has been removed.
Originally posted by imamelia
Wasn't MushROMs already supposed to do something similar? Or at least, I thought it was set up to where the editor would output assemblable ASM files that contained the new contents of the ROM. I don't know whether it was a split disassembly or not, though.

It's similar. It would generate ASM patches to modify a clean SMAS ROM, but it wasn't the entire SMAS source code itself. It was like an ASM patch generator which also modified binary files such as levels.

Quote
Anyway, it sounds like a good idea to me, as long as we're not forced to move things around. I like knowing exactly where everything is in the ROM (except added stuff that wasn't in the original game), I like being able to call routines like $01A7DC and $03B72B by their address rather than needing a define, and it seems to me you'd have a heck of a time making any custom patches if code could move around.

With the introduction of a split disassembly, there's almost no point in the ROM map because things WILL be moved around when we start adding in our own ASM hacks before the release. We're most-likely going to end up compressing the graphics which means the ROM's going to reduce in size and anything beyond the first GFX in the ROM will be moved around. Also, imagine removing blocks of code to make more free space (such as removing everything BUT SMB3). In fact, maybe making different versions of a split disassembly would be cool as well. A version with SMB1 only, SMB2 only, etc. But it'd be a pain to maintain them all.

Quote
In any case, though, a split disassembly could keep things more organized and let you more easily track exactly what is being done to the ROM and when.

Yup. And at the same time, it's still beginners-friendly for casual hackers. They don't have to worry about the ASM at all.

Quote
(On a side note, I already have several ideas for stuff to add to the SMAS ROM, as I think you know from conversations on IRC....)

I know you have ideas but I forgot which #ab{>_>} maybe a new thread is in order. a todo list of very necessary SMAS ASM hacks (like levelASM for example) or a thread for ideas.

Originally posted by Horrowind
Patches should be easy then, just normal source code patches should be sufficent, it would allow especially for crazy things as collaborative hacking efforts via git/svn/etc.

What I somewhat imagine right now is someone writing a patch but not being able to place an ORG $hijack in the patch, thus requiring the hacker to manually figure out where to put the patch. The author could write something like "find LDA #$something JSL $something around line 48372 in bank18.asm and replace them with JSL newcode" but that isn't exactly a... friendly approach.

Quote
Things which should be modifiable by tools (such as level data) could be external of the source code in binary files and .incbin'ed. Tools would simply modify these files. Another approach would be reading the assembly files and rewrite them, which sounds kind of dangerous (given that asar does not provide a real grammar).

That's exactly what I had in mind with this split disassembly approach. The tables will be extracted into binary files, but will be incbin'd by the assembler.
My blog. I could post stuff now and then

My Assembly for the SNES tutorial (it's actually finished now!)
I simply meant source diffs (like the ones you get if you use the unix tool "diff"). This should allow for editing the disassembly as well as adding new files.
Your layout has been removed.
But then every time you want to test a tiny change in your ROM (like moving a sprite 2 blocks down to check if a jump is possible now) will take aeons since you have to assemble the ROM again, right?
Your layout has been removed.
Nah, assembling the entire ROM takes like 10 seconds max I think. Not sure what you mean by "aeons". A few seconds? A few minutes? Its literal meaning?

I mean, the level editor could have a command to assemble the ROM automatically if you want to run it in an emulator. It'd be the same as Lunar Magic's F4 key except it'd take a few seconds longer to start up I think.

The sticky thread with the disassembled ROM itself has an assemble-able version of the ROM (to be used with asar). You could try assembling it on your PC to see how slow/fast it is for you. Admittedly it doesn't have a lot of incbins/incsrcs it's supposed to have (it's just a wall of assembly and tables), but it shouldn't make that much of a difference.

If anything, one could rewrite asar just so it has even more performance. I dunno.
My blog. I could post stuff now and then

My Assembly for the SNES tutorial (it's actually finished now!)
10 seconds is a pretty long time to wait every time you do any change...
Considering how often I/people F4 to test out a jump or obstacle or change in a sprite's asm, that'd stack up pretty quickly, I was already extremely sick of Sprite Tool taking 10 seconds every time when I had like 40 sprites in the sprite list (before being smart and making a second list with only the sprite I'm currently writing and all related ones).
Your layout has been removed.
Originally posted by Ersanio
With the introduction of a split disassembly, there's almost no point in the ROM map because things WILL be moved around when we start adding in our own ASM hacks before the release. We're most-likely going to end up compressing the graphics which means the ROM's going to reduce in size and anything beyond the first GFX in the ROM will be moved around. Also, imagine removing blocks of code to make more free space (such as removing everything BUT SMB3). In fact, maybe making different versions of a split disassembly would be cool as well. A version with SMB1 only, SMB2 only, etc. But it'd be a pain to maintain them all.

Eeeeh...yeah, as you said, that really isn't a friendly approach for patches. But I suppose if anyone were patient enough to edit the game using only the ASM files and not the editor, they could still put everything wherever they wanted. Now, it would be all right to have things in a different location from where they were in the original SMAS but in a fixed location after hacks are inserted. For instance, if you moved the game code from banks 03-05, 0D-0F, 11-15, and 20-2A respectively to 01-03, 04-06, 07-0B, and 0C-16, leaving banks 17-3F open and moving the data in banks 01, 02, 06-0C, 10, 16-1F, and 2B-3F into the expanded portion of the ROM (presumably 40-68). Then perhaps most of the new ASM hacks could be inserted into bank 17. Just as long as we know where everything is, except the data that constantly changes (graphics, level data, music, and whatnot), and even that would still have pointers to it.

----------------

I'm working on a hack! Check it out here. Progress: 64/95 levels.
Originally posted by leod
10 seconds is a pretty long time to wait every time you do any change...
Considering how often I/people F4 to test out a jump or obstacle or change in a sprite's asm, that'd stack up pretty quickly, I was already extremely sick of Sprite Tool taking 10 seconds every time when I had like 40 sprites in the sprite list (before being smart and making a second list with only the sprite I'm currently writing and all related ones).


His all.log has too many labels. Remove the extra labels and you get a 10x performance boost. DKC2 a 4MB ROM, takes me 1.4 seconds to assemble, prior to superfluous labels being removed it took near 1 minute to assemble.
If anything, a potential project could be rewriting an existing assembler (either asar or xkas) so it gains an incredible performance boost.

But yeah. The assembleable project has way too many excess labels at the moment. Once I'm done fixing the ROM access instructions and turn them into labels (...this is gonna take ages, I could probably use help. Turning stuff like LDA $8000 to LDA DATA_018000), I'll run a script which filters out unused labels. After that, we'll see how long it'll take to assemble it. But considering it's half the size of the DKC2 ROM, it'll probably take at most 1 second to assemble.

It's gonna take ages because I'll have to split countless amount of tables. Need to work on a script for that. I already have the base for it thanks to p4, but I haven't had the time to tweak it.
My blog. I could post stuff now and then

My Assembly for the SNES tutorial (it's actually finished now!)
Originally posted by p4plus2
His all.log has too many labels. Remove the extra labels and you get a 10x performance boost.


You mean like this?

Also, I am working on a specialized assembler which has great performance, but between that, MushROMs, and my very busy life, there's not much time for it. I should git it.
----------

Interested in MushROMs? View its progress, source code, and make contributions here.

Originally posted by spel werdz rite
Originally posted by p4plus2
His all.log has too many labels. Remove the extra labels and you get a 10x performance boost.


You mean like this?

Also, I am working on a specialized assembler which has great performance, but between that, MushROMs, and my very busy life, there's not much time for it. I should git it.

Correct. The new problem you will hit is that of hard drive io from large numbers of files. The true solution would be an assemble capable of object code. Then most of the code odd cached in binary form. I bet it would be possible to do this using asar as a dll and then caching labels. The first build would probably be significantly slower but future builds significantly faster.

The question is what methods can be used to detect non relative code. The first is assemble twice with different offsets, slow but lets you easily look for bytes that changed, since branches wouldn't change.
The other method we use macros to wrap exportable labels aka all nonrelative labels, faster but ugly code. The third method we hook the opcode assemble in asar and make it dump the labels that matter.

Typing this from my phone so sorry for typos.
This'd probably be a more optimal hacking method in the long run, since split disassemblies usually are easier to navigate than just one big file for all the source code. Plus it reflects on object-oriented programming, which is what 97% of programs these days use.
I really need a layout. :<
I'm definitely down for this. This is what I was trying to do with the smw source code, although it seems nobody got on board. Anyway, I've disassembled the smw portion of all-stars, maybe sometime I'll do the rest if there is enough interest?
Super Mario World Source Code
Ah, unfortunately (for you?) I'm working on the SMW-less version of SMAS, because I saw little point in doing +W when we have SMW hacking this advanced already.
My blog. I could post stuff now and then

My Assembly for the SNES tutorial (it's actually finished now!)
I wasn't suggesting using SMAW strictly, but I like the improvements they added in that version (Luigi graphics, 4 saves). Either way I don't have much interest in smb1/2, I mainly want to see smb3 sprites/various effects disassembled so they can be ported to smw. Why write an editor all over again when we can just use the good stuff from that game, raw code style :)
Super Mario World Source Code
I'm a Sonic hacker so...

You don't need to create patches at all. Basically you put all your code into a post and tell people where to put it for example:

How to put the spindash into Sonic 1

Patches are redundent because of this.
Team CornersoftYoutube
That is actually more work for the average user, applying a smw-style patch is much faster than copy-pasting all that.
The only one that's easier for is the one writing the patch...
Your layout has been removed.
  • Pages:
  • 1
  • 2