Language…
16 users online:  AmperSam, asterkafton, ClaireChan, DanMario24YT, Everest, Gamet2004, h4shcat, Hayashi Neru, LightAligns, MorrieTheMagpie, PaceTheAce, Rhubarb44230,  Segment1Zone2, signature_steve, simon.caio,  zuccha - Guests: 257 - Bots: 353
Users: 64,795 (2,378 active)
Latest user: mathew

The SM64 Console Compatibility thread.

So I had a little talk with Alcaro and we've decided to recreate this thread as getting SM64 to work on consoles can broaden our understanding of the game and it's clear that understanding hardware incompatibilities can have other benefits such as keeping hacks future proof with newer emulators.

All posts not directly about specific hardware incompatibilities will be removed, if this thread clashes with your personal believes then you can just not join in the discussion.

We will be keeping a close eye on this thread to make sure all posts are on topic.
Kaze talks a bit about running on an actual n64 in some of the comments here if that's useful at all.
New layout by RanAS!
While I was learning what the ROM Extender and Level Importer did, I took some notes on what ran on the N64 system and what didn't. Below are my cleaned up notes. These are the steps that the Level Importer takes to extend a stock ROM and make it ready for the importer. Steps 2-5 assume 64MB Extended ROM with 16-byte aligned MIO0 blocks and restored boot code and checksum corrected.

1. Extend to 64MB and decompress MIO0 blocks
Does it run?
Not by default. The title screen shows, but when you press start to enter the main menu, all you see is a white screen. This breaks due to unaligned MIO0 blocks that need to be accessed with DMA.

To allow it to run:
I don't know if they need to be on a 4 or 16 byte boundary, but forcing them to a 16-byte boundary allows it to run. You can do this with sm64extend by passing it the "-a 16" command line option:

Code
sm64extend -a 16 "Super Mario 64 (U).z64"


2. Patch with obj_import18S
What the patch does:
This patches some trivial things like reduce Mario's shadow size and relocated Whomp's star, but also creates instrument set table in the extended ROM and updates references to it and allocates places for level data.

Does it run?
Yes, applying this to the 64MB extended ROM with MIO0 data aligned to 16-bytes works.

3. Collision improvements fix
What the fix does:
Simple instruction patch that stores 0x7FFF instead of 0x4000 out to 0x8038EEA0

Does it run?
Yes, this seems to run just as well as 2.

4. Music volume bug fix
What the fix does: 00b40046 -> 00960064 @012a788

Does it run?
Yes, this seems to run just as well as 3.

5. Additional music settings and sequences
What this step does:
Moves music to 0x3e00000, updates pointers, overrides soundAlloc() call
Explicitly, the fix makes these ASM changes:
Code
0d4714  lui  a0,0x2f0   -> lui  a0,0x3e0
--
0d4768  lui  a0,0x2f0   -> lui  a0,0x3e0
--
0d4784  lui  a1,0x2f0   -> lui  a1,0x3e0
--
0d48b4  jal  800317040  -> lui  v0,0x803d
0d48b8  li   a1,256     -> ori  v0,v0,0x0
--
0dc0b8  swc1 ft1,32(a3) -> nop


Does it run?
Yes, with a fix. Title screen shows, but the audio sounds very wrong. After pressing 'start' to try to get to the main menu, screen glitches and system locks up. Can't even reset, need to power cycle.

Edit: Turns out 0x803D0000 is not a good choice for sound allocation. Using 0x807B0000 instead allows it to run on the console. This is done by changing one byte at ROM offset 0xd48b7 from 0x3D to 0x7B. See This post for more details

6. Importing levels
Does it run?
Not really. Level importing works, but the imported textures don't always show. Note that this must be done before any of the alignment fixes are applied, otherwise all the MIO0 data pointers get set wrong. I'm still looking into this.
So, if you write a code in sm64 that the rom can be compatible with a real n64, it would really work correct? Maybe show a video showing that N64 roms.(.z64) can be played with a real n64 console using Ultra 64. Hopefully that when you're playing in the console with the code in the game, it won't lag or crash or even corrupt you data. Please correct me if I said something wrong.
Originally posted by Kazeshin
copy function is used in extended/expanded roms at 101B84. clear cache should be JAL $322f40, try fitting that in somewhere around there and im positive it will run.


80322f40/0DDF40 is "osWriteBackDCacheAll". I think this needs to be performed before the DMA. Then, after the DMA, the following two procedures: "osInvalCache" (80324610/0DF610) and "osInvalDCache" (803243B0/0DF3B0). This forces data that hasn't been written back to main memory from cache before DMA and then invalidates the cache so CPU accesses to DMAd data will be pulled from main memory.

For example, the procedure "FixedCopy" (802786F0/0336F0) has this near the middle of it:

Code
  lw    $a0, 0x24($sp)
  jal   bzero # 80324570/0DF570
  lw    $a1, 0x1c($sp)
  jal   osWriteBackDCacheAll # 80325D20/0E0D20
  nop
  lw    $a0, 0x24($sp)
  lw    $a1, 0x2c($sp)
  jal   DmaCopy              # 80278504/033504
  lw    $a2, 0x30($sp)
  lw    $a0, 0x24($sp)
  jal   osInvalCache         # 80324610/0DF610
  lw    $a1, 0x1c($sp)
  lw    $a0, 0x24($sp)
  jal   osInvalDCache        # 803243B0/0DF3B0
  lw    $a1, 0x1c($sp)


Maybe we can try to replicate this.

Originally posted by Kazeshin
i suggest fitting the "clearcache" function somewhere into the "jal $278504" function, since that is a perfectly working function on emu, which i used for some of my patches. (so even those hacks that use the MOPs will work)


Fortunately, DmaCopy 80278504/033504 is one of those inefficient functions that has an unnecessary "branch two instructions ahead" in it. We can easily replace that "b LDmaCopy_FC" with a JAL and do whatever cache invalidating there.

Code
LDmaCopy_F4: # 802785F8
  b     LDmaCopy_FC
  nop
LDmaCopy_FC: # 80278600
  lw    $ra, 0x24($sp)
  addiu $sp, $sp, 0x30
  jr    $ra
  nop
Hey guys,

Back in December/early January, I was screwing around with XVI32, trying to get more hacks working. One of the issues I encountered was with the "Additional music settings and sequences", but I think I've sort of managed to identify what part caused the problem.

My test subject was the Year of the Plumber C3 Demo.

When you run the pre-patched hack on console, this is what you get:
https://www.youtube.com/watch?v=rLjxKzkMB5c

I put together a fairly lengthy xsc script (to use with XVI32) which did a few things:
1 - Fixes MIO0 data (replicates what Kyle describes here, which itself follows Vertrex's notes)
2 - Skip the CRC Check (as described here)
3 - Fixes the VI (as I describe here, last post)

If you take the pre-patched YotP demo, apply this script to it, then save it out as a new file and run it on console, this is what you get:
https://www.youtube.com/watch?v=ud7FNIbYLk4

This is, I'm pretty sure, what queueRAM was describing here:
Quote
Does it run?
No. Title screen shows, but the audio sounds very wrong. After pressing 'start' to try to get to the main menu, screen glitches and system locks up. Can't even reset, need to power cycle. As I find time, I'll individually apply some of these changes to help narrow down which is the problem one, but they are mostly dependent on each other. If anyone has better ideas, please help me here.


Back in December/January, I basically went through the same process as queueRAM, and also identified the "Additional music settings and sequences" modification as causing issues. Then I compared a ROM at stage 4 (Music volume bug fix) which worked, and a ROM at stage 5 (Additional music settings and sequences) which didn't work, and I noted the differences.

When comparing the differences, it looks like there's huge amount of changes, but it really all broke down into a few "blocks" of varying sizes:
1 - 3 small clusters at $D4716, $D476A and $D4786
2 - A little bit at $D48B4
3 - About 20 lines of code at $7F0000
4 - About 100 lines starting at $7F1000
5 - Several thousand lines, starting at $3E00000

A bit of trial and error later, it seemed that only blocks 1 and 2 needed to be changed back in order for the ROM at stage 5 to run. Reversing these two blocks in the YotP Demo also allowed it to run. Here's the script:

Code
REM /////////////////////////////////////////////////////////////////////////////////////////
REM Undo the Music Changes

REM Fixes 3 sets of 2. 000D 4710, 000D 4760, 000D 4780
ADR $D4716
OVERWRITE 02 F0
ADR $D476A
OVERWRITE 02 F0
ADR $D4786
OVERWRITE 02 F0
REM ^^ Just a black screen when booting, if this is missing

REM Fixes 7 pairs on line 000D 48B0
ADR $D48B4
OVERWRITE 0C 0C 5C 10 24 05 01
REM Ah ha, this is the culprit! ^^


Now, when you run the YotP Demo after running both scripts (I've only posted the second one here, the first is rather lengthy), this is the result:
https://www.youtube.com/watch?v=kuVpaYIgwAg

The game more or less works now, though there's still the usual missing texture issues in one stage. The audio "works", but most of the instruments are wrong (also note how chain chomp sounds like a monkey, and vice versa).

As you can probably tell, I'm pretty much in over my head here, and I don't have an understanding of why these changes allow the hack to run, only that they do.


I hope what I've posted here will be helpful to you, or will at least narrow things down a bit. :)
Originally posted by macN64

3 - Fixes the VI (as I describe here, last post)

Thanks. I had linked to your post on krikzz in a different thread on SMWC, but that post has since been deleted. The VI change is also something we need to consider when fixing ROMs already out in the wild. For now though, I'm just going to continue taking baby steps starting from a working stock ROM going through all the steps to importing levels and other data.

Originally posted by macN64

A bit of trial and error later, it seemed that only blocks 1 and 2 needed to be changed back in order for the ROM at stage 5 to run. Reversing these two blocks in the YotP Demo also allowed it to run. Here's the script:
...
The game more or less works now, though there's still the usual missing texture issues in one stage. The audio "works", but most of the instruments are wrong (also note how chain chomp sounds like a monkey, and vice versa).

Thanks for these notes. Reverting the data at D4716/D476A/D4786 is reverting the asm references from the new music table at 0x3e00000 back to the old music table at 0x2f00000. From my notes on 5. Additional music settings and sequences above:
Originally posted by queueRAM

Code
0d4714  lui  a0,0x2f0   -> lui  a0,0x3e0
0d4768  lui  a0,0x2f0   -> lui  a0,0x3e0
0d4784  lui  a1,0x2f0   -> lui  a1,0x3e0


You're referencing the old tables, so things starting working again but don't sound right. I still think Kaze's idea on the cache issue with DMA is what is causing the problems with the new table, but I haven't had time to put together proper tests.
I tried to implement the cache writeback, DMA, cache invalidate routines as follows, but it made no difference on the console. I still get to the title screen, press start, and screen glitch and console locks up. While the code below didn't help with console compatibility, it might be better to use than the obj_import18S.ppf version since it uses the much more efficient version of bzero to clear 4MB of RAM. You can also view the annotated GNU as version that I used to generate this.

Code
80384e04 <Behavior_12>:
80384e04:	27bdffe0 	addiu	sp,sp,-32
80384e08:	afbf0014 	sw	ra,20(sp)
80384e0c:	3c048040 	lui	a0,0x8040
80384e10:	3c050040 	lui	a1,0x40
80384e14:	afa40018 	sw	a0,24(sp)
80384e18:	0c0c915c 	jal	80324570 <bzero>
80384e1c:	afa5001c 	sw	a1,28(sp)
80384e20:	0c0c9748 	jal	80325d20 <osWriteBackDCacheAll>
80384e24:	00000000 	nop
80384e28:	8fa40018 	lw	a0,24(sp)
80384e2c:	3c050120 	lui	a1,0x120
80384e30:	0c09e141 	jal	80278504 <DmaCopy>
80384e34:	3c060130 	lui	a2,0x130
80384e38:	8fa40018 	lw	a0,24(sp)
80384e3c:	0c0c9184 	jal	80324610 <osInvalCache>
80384e40:	8fa5001c 	lw	a1,28(sp)
80384e44:	8fa40018 	lw	a0,24(sp)
80384e48:	0c0c90ec 	jal	803243b0 <osInvalDCache>
80384e4c:	8fa5001c 	lw	a1,28(sp)
80384e50:	8fbf0014 	lw	ra,20(sp)
80384e54:	03e00008 	jr	ra
80384e58:	27bd0020 	addiu	sp,sp,32


I'm going to try focusing on only the "Additional music settings and sequences" changes and see if I can make better sense of data copying there.
I went through all of the changes in "Additional music settings and sequences" there are no issues with the new music tables or storing them at 0x3e00000. The problem is with using 0x803D0000 instead of an address from soundAlloc at d48b4. I think soundAlloc can't be used since the new music data requires more space than can be allocated. Looking at the memory in Nemu08, Project64, or mupen64plus, it it appears that 0x803D0000 is unused, which is probably why it was chosen. However, those emulators don't emulate the hardware correctly because something is using this area. It might be some other hardware or some dynamic allocation, but this RAM conflict causes the console to lock up. To prove this, I ran a test on the stock SM64 ROM, hijacking RenderHud_MarioLives (802E3744/09E744) with the following code to print the contents of two locations within 0x803D0000 on the screen:
Code
RenderHud_MarioLives:
  addiu $sp, $sp, -0x18
  sw    $ra, 0x14($sp)
  la    $a2, 0x80338388 # lui $a2, 0x8034/addiu $a2, $a2, -0x7c78
  lhu   $a3, 0x803d0000 # lui $a3, 0x8034/lh $a3, -0x4da0($a3)
  addiu $a0, $zero, 0x16
  jal   PrintInt
  addiu $a1, $zero, 0x20
  la    $a2, 0x80338388 # lui $a2, 0x8034/addiu $a2, $a2, -0x7c78
  lhu   $a3, 0x803d0120 # lui $a3, 0x8034/lh $a3, -0x4da0($a3)
  addiu $a0, $zero, 0x56
  jal   PrintInt
  addiu $a1, $zero, 0x20
  nop
  nop
  nop
  b     LRenderHud_MarioLives_54
  nop
LRenderHud_MarioLives_54: # 802E3798
  lw    $ra, 0x14($sp)
  addiu $sp, $sp, 0x18
  jr    $ra
  nop


On the above emulators, they all print '0' '0' as one would hope with unused memory.


However, on the console and in cen64, they print non-zero values that change, usually based on user input or other events. This means that 0x803D0000 isn't as unused as we thought, so we must use something else!


Edit: Base on input from Kaze, I am instead using 0x807b0000. This is accomplished by changing one byte at ROM offset 0xd48b7 from 0x3D to 0x7B. I also have a tool that I've put together to align all the MIO0 data to 16-byte boundaries, update the pointers in all the levels, make this 3D-to-7B change if needed, and updates the header checksum. It's a hacked from my sm64compress tool that's still a work in progress. I've tested it working to get Kaizo, Green Stars, Year of the Plumber not crashing on the console, but behaving very badly. See notes below on level importing/textures.

sm64compress-v0.1.1a.zip
Source Code

Now I am on to step "6. Importing levels". I made a simple level and replaced BOB with it and it runs on the N64 after applying all the above change, however, it has the weird texture glitch that macN64 was seeing in his conversions where it flashes from black to white instead of rendering the level imported textures. Much more work to do.

Edit2: This is what a simple level looks like on the console. Depending on viewing angle, the textures are either all black or all white.
Originally posted by Kazeshin
ah, so 803D0000 cant be used? that might be what tricked me into thinking that DMAcopy wouldn't work. I've tried to fix the DMAcopy for chaos edition, which loads code to 803D0000, but when trying to print a string from there, i got weird randomly changing strings.

i suggest to move the data to 807B0000, which is a huge unused RAMarea.


Yeah, 803D0000 is definitely being used by something on the console, still not sure by what. Thanks for the info on 807B0000 - it appears to be unused, so I switched to using that and updated my notes and tool above.

Originally posted by Kazeshin
for the texture thing, i would guess there is something wrong with the display list.. have you tried swapping the 0xB7 command at the beginning and the 0xB6 command at the end? they are kind of in the wrong order. maybe moving the 0xF5 command infront of the 0xFD command instead of after it would also help.


I don't see a 0xB6 command in the display list at all - did you mean 0xB8? Doesn't 0xB8 mark the end of the display list?

Edit: I tried moving the 0xF5 commands in front of the 0xFD ones, but that didn't seem to have any impact on the console.

I think I need to study these display lists more. Do you have a recommended display list in the stock ROM that I could take a look at as a good example?
I think I've finally solved the mystery of the texture glitching. It seems to be caused by the absence of fog. See for yourself: http://www.youtube.com/watch?v=FmgC_EANaHk

The level model is completely mangled, but you can still see which levels are textured and which are not. Here's my notes of what changes were made with each import:
Originally posted by Level Import Notes
1. Bob-omb Battlefield
Fog: None
Music: Bob-omb Battlefield
Background: Bob-omb Battlefield
Textured?: No

2. Whomp's Fortress
Fog: None
Level Music: Bob-omb Battlefield
Background: Rainbow Ride
Textured?: No

3. Jolly Roger Bay
Fog: None
Level Music: Dire Dire Docks
Background: Jolly Roger Bay
Textured?: No

4. Cool, Cool Mountain
Fog: Intense White (FF, FF, FF)
Level Music: Snow
Background: Cool Cool Mountain
Textured?: Yes!

5. Big Boo's Haunt
Fog: Intense Black (0, 0, 0)
Level Music: Haunted House
Background: Haunted House
Textured?: Yes!

6. Hazy Maze Cave
Fog: Subtle 1 Black (0, 0, 0)
Level Music: Hazy Maze
Background: Bowser First Battle
Textured?: Yes!

7. Lethal Lava Land
Fog: Subtle 1 White (FF, FF, FF)
Level Music: Lethal Lava Land
Background: Lethal Lava Land
Textured?: Yes!

8. Shifting Sand Land
Fog: Subtle 2 Yellow (EE, FF, 00)
Level Music: Lethal Lava Land
Background: Shifting Sand Land
Textured?: Yes!

9. Dire, Dire Docks
Fog: Moderate 1 Blue (00, 00, FF)
Level Music: Dire Dire Docks
Background: Jolly Roger Bay
Textured?: Yes!

10. Snowman's Land
Fog: Moderate 2 White (FF, FF, FF)
Level Music: Snowman's Land
Background: Cool Cool Mountain
Textured?: Yes!

11. Wet-Dry World
Fog: Moderate 3 Purple (8C, 00, 8A)
Level Music: Hazy Maze
Background: Wet-Dry World
Textured?: Yes!

12. Tall, Tall Mountain
Fog: Moderate 4 Green (00, FF, 00)
Level Music: Bob-omb Battlefield
Background: Bob-omb Battlefield
Textured?: Yes!

13. Tiny-Huge Island (Big Painting)
Fog: Intense Yellow (FF, FF, 00)
Level Music: Bob-omb Battlefield
Background: Bob-omb Battlefield
Textured?: Yes!

14. Tick Tock Clock
Fog: Very Intense Black (00, 00, 00)
Level Music: Slide
Background: Shifting Sand Land
Textured?: Yes!

15. Rainbow Ride
Fog: Hardcore Pink (FD, 99, FB)
Level Music: Slide
Background: Rainbow Ride
Textured?: Yes!



Originally posted by queueRAM
I also have a tool that I've put together to align all the MIO0 data to 16-byte boundaries, update the pointers in all the levels, make this 3D-to-7B change if needed, and updates the header checksum. It's a hacked from my sm64compress tool that's still a work in progress.

It works like a charm. Here's another video of Year of Plumber which I've used that version of sm64compress on, then fixed the VI and (re)fixed the checksum: http://www.youtube.com/watch?v=m0SLFmiS4SY

Music sounds much better now (chain chomp and monkey voices are still reversed, not sure if that's connected?).



Getting there!! #tb{^V^}
A couple more videos for you.

The first is a better version of this one, showing the effect of fog on textures: https://www.youtube.com/watch?v=mV7D8_WPtIk


All my imports thus far had been made using either Importer 1.9s or v16 (old). I thought I'd try making similar imports with the most recent release, and, well, this happened:
https://www.youtube.com/watch?v=SoHqFRqnwGw

Originally posted by readme.txt
Changes in 1.9.2S:
*Increased 3D draw distance, no clipping in large levels
I'm thinking this might be the culprit?


Issues with 1.9.3S aside, do you think it would it be possible to add fog effects to levels that have already been imported into the rom? If so, it may then be possible to fix a lot of hacks that have already been released.
I'm not sure if it's just me experiencing this, but the Skip Mario Screen tweak seems to be causing issues for me. The save file used here has some stars (you can hear that I make it into the Cool Cool Mountain painting).

I tried again to double check, making minimal changes to the file.

Here, I took a vanilla “Super Mario 64 (U) [!].z64” and made two copies – one I called “Vanilla.z64” and the other I called “Vanilla+Skip.z64”.

I then took Vanilla+Skip.z64, and changed the following lines:
269F4C: 01 10 00 14 00 26 9E A0 00 26 A3 A0 14 00 02 0C
269FD8: 01 10 00 14 00 26 9E A0 00 26 A3 A0 14 00 02 0C

I then compared Vanilla.z64 and Vanilla+Skip.z64 using VBinDiff. This image shows the only differences found between the two files (in red). I haven't touched anything else (I even left the checksum parts alone).

Vanilla+Skip.z64 had no previous save file when I tried it (you can hear Peach and Lakitu).

Does anyone else experience this?


I'd also like to bring up Star Road here. Star Road skips the Mario Screen using a different method (I assume), as 269F4C looks unchanged from a normal, unaltered Rom, and 269FD8 looks like it contains a variation of what is normally there in an unaltered Rom*. For me, the skipped Mario screen doesn't cause any issues on console. However, when I get a game over, the problem returns.
*Star Road 269FD8: 1D 04 00 00 01 10 00 14 02 FB 00 00 02 FB 04 90

Well, this is what I experience anyway.
Thanks for the help Kaze. I hope I put this in correctly (sorry if I goofed). It now looks like this:
269F4C: 1D 04 00 00 05 08 00 00 14 00 02 0C 20 04 00 00
269FD8: 1D 04 00 00 05 08 00 00 14 00 02 0C 20 04 00 00

Unfortunately, I get the same result as before - the game continues, but nothing appears on screen.
Thanks again for your guidance Kaze. After some trial and error, I think I've got it working.

269F68: 05 08 00 00 14 00 02 0C
269FF4: 05 08 00 00 14 00 02 0C

Looks ok to me. :)
bump
Your layout has been removed.
Originally posted by queueRAM

The game more or less works now, though there's still the usual missing texture issues in one stage. The audio "works", but most of the instruments are wrong (also note how chain chomp sounds like a monkey, and vice versa).


Originally posted by queueRAM

Music sounds much better now (chain chomp and monkey voices are still reversed, not sure if that's connected?).


Oh I'm sure that was intentional #tb{^V^}

MeltFire's Youtube Channel
MeltFire's Sound Cloud Account

I don't know if you have read this thread, but there is a lot of talk about how to get SM64 hacks to work on real hardware here.

http://krikzz.com/forum/index.php?topic=1576.0

[edit]

nevermind i saw a link to this thread already in another thread.
Originally posted by aterraformer
bump

If you don't have anything to add to the thread, then please do not make a post. It just clutters the thread making it less useful to anyone reading through it who can help.

Originally posted by Final Theory
I don't know if you have read this thread, but there is a lot of talk about how to get SM64 hacks to work on real hardware here.

http://krikzz.com/forum/index.php?topic=1576.0

[edit]

nevermind i saw a link to this thread already in another thread.

You can delete your own posts if you later realise that they don't help. Just click the 'Delete' button above your post.
Since this is kinda.. uh resting, would anyone be willing to get Zelda's Birthday running on console? Like, importing the maps onto a 1.0 or 1.2 rom or something? Does that technology even exist? I dunno, worth a try to throw that out there.
Your layout has been removed.