My newer OAM writing for uberasm tool (works best for level and doesn't work on gamemode).
Make sure this doesn't execute every frame during pause since JSL $7F8000 isn't executed during it (it will start filling up the slots each frame).
Also note that you must use
this patch in order to prevent weirdness from happening.
This one handles the “extended bits” such as the x position bit 8 (there are actually 9 bits; bits 0-8 instead of normal bits 0-7) and the tile size ($0420 table), therefore it handles displaying sprites tiles that overlap the left edge of the screen without wrapping to the right edge and the size of the tiles.
It also have “smart” off-screen check, simply put, if the tile's body is entirely offscreen (X or Y position is $FFF8 (for 8x8 tile) or $FFF0 (for 16x16)), it sets the Y position to #$F0 as a free slot.
The routine is also “stackable”, as in when you need call it again, you don't have to set the Y value for the OAM index again, as it will pick up right where it left off, without potential unnecessary already-checked slots.
Code;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;Make sure you have the index register be 16-bit (REP #$10)
;and A be 8-bit (SEP #$20) before calling this.
;
;Input:
;OAM writer for uberasm tool
;Make sure you have the index processor flag be 16-bit (REP #$10).
;
; Y (16-bit): the starting index to search for OAM.
; Y = 508 ($1FC) would search all 128 slots.
; $00-$01: OAM X position
; $02-$03: OAM Y position
; $04: OAM tile number
; $05: OAM tile properties
; $06: Tile size (0 = 8x8, 1 = 16x16)
;
;Output:
; Y (16-bit): Index that represent the previous slot that
; is free.
; if Y = $FFFC, then all slots are taken.
; Carry: Set if offscreen and clear if onscreen.
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
WriteOAM:
.FindFreeSlot
..Loop
LDA $0201|!addr,y ;>Y position
CMP #$F0 ;\Free slot found.
BEQ .SlotFound ;/
...Next
DEY #4 ;\Next group of 4 bytes
BPL ..Loop ;/
SEP #$20
RTL
.SlotFound
..YPosHandle
REP #$20
LDA $06 ;\Position #$FFF8 (X and Y) is invisible for 8x8 tiles and
ASL ;|Position #$FFF0 is invisible for 16x16 tiles.
AND #$00FF ;/
TAX
LDA $02 ;\Don't take up a slot of offscreen vertically.
CMP #$00E0 ;|
BCC .HandleXPos ;|
CMP.l TopLeftScreenBoundaryPos,x ;|
BCS .HandleXPos ;/
.OffScreen
SEP #$20
LDA #$F0 ;\Be a free slot and not write.
STA $0201|!addr,y ;/
SEC
RTL
.HandleXPos
SEP #$20
LDA $02 ;\Y position
STA $0201|!addr,y ;/
..XPositionHandle
REP #$20
LDA $00 ;\Determine if on-screen horizontally
CMP #$0100 ;|
BCC ..OnScreenHoriz ;|
CMP.l TopLeftScreenBoundaryPos,x ;|
BCS ..OnScreenHoriz ;/
BRA .OffScreen ;>If not skip the whole thing.
..OnScreenHoriz
SEP #$20
LDA $00 ;\X position bits 0-7
STA $0200|!addr,y ;/
REP #$20
TYA ;>Take index
LSR #2 ;>Divide by 4 to obtain slot number
SEP #$20 ;\Round down to the nearest 4th value (Value = floor(SlotNumb/4)*4)
AND.b #%11111100 ;/
STA $02 ;>Store $0420 indexer to $02
REP #$20
TYA ;>Transfer IndexNumb to A
LSR #2 ;>Convert IndexNumb -> SlotNumb
SEP #$20
AND.b #%00000011 ;>MOD 4
STA $03 ;>Store pair bits position to $03.
PHY
REP #$20
LDA $02 ;\Because indexes must be 16-bit, I had to clear their high bytes.
AND #$00FF ;|
TAY ;/
LDA $03
AND #$00FF
TAX
SEP #$20
LDA $01 ;\If X position negative, set bit to enable tile
BNE ...SetXPosBit8 ;/exceeding the left edge of screen without wrapping.
...ClearXPosBit8
LDA BitTableXHigh,x
EOR #$FF
AND $0420|!addr,y
STA $0420|!addr,y
BRA ...TileSize
...SetXPosBit8
LDA $0420|!addr,y
ORA BitTableXHigh,x
STA $0420|!addr,y
...TileSize
LDA $06
BNE ...SixteenBySixteenTile
...EightByEightTile
LDA BitTableTileSize,x
EOR #$FF
AND $0420|!addr,y
STA $0420|!addr,y
BRA .HandleTiles
...SixteenBySixteenTile
LDA $0420|!addr,y
ORA BitTableTileSize,x
STA $0420|!addr,y
.HandleTiles
PLY
LDA $04 ;\Tile number
STA $0202|!addr,y ;/
LDA $05 ;\Tile properties
STA $0203|!addr,y ;/
.Done
DEY #4
CLC
RTL
TopLeftScreenBoundaryPos:
dw $FFF9, $FFF1
BitTableXHigh:
db %00000001
db %00000100
db %00010000
db %01000000
BitTableTileSize:
db %00000010
db %00001000
db %00100000
db %10000000
Sample usage:
Code!Freeram_OAMXPos = $60
!Freeram_OAMYPos = $62
main:
LDA $15
BIT.b #%00000010
BNE .Left
BIT.b #%00000001
BNE .Right
BRA +
.Left
REP #$20
DEC !Freeram_OAMXPos
SEP #$20
BRA +
.Right
REP #$20
INC !Freeram_OAMXPos
SEP #$20
+
LDA $15
BIT.b #%00001000
BNE .Up
BIT.b #%00000100
BNE .Down
BRA +
.Up
REP #$20
DEC !Freeram_OAMYPos
SEP #$20
BRA +
.Down
REP #$20
INC !Freeram_OAMYPos
SEP #$20
+
REP #$30
LDA !Freeram_OAMXPos
STA $00
LDA !Freeram_OAMYPos
STA $02
SEP #$20
LDA #$40
STA $04
LDA.b #%00110000
STA $05
LDA #$01
STA $06
LDY #$01FC
JSL LibraryOAMWrite_WriteOAM
;Second OAM slot test
REP #$20
LDA !Freeram_OAMXPos
CLC
ADC #$0010
STA $00
LDA !Freeram_OAMYPos
STA $02
SEP #$20
LDA #$40
STA $04
LDA.b #%00110000
STA $05
LDA #$01
STA $06
JSL LibraryOAMWrite_WriteOAM
SEP #$30
RTL
The problem is that SMW's sprites uses sprite OAM allocation instead of manually checking if the OAM slot is free, therefore overwriting sprites that were allocated to use slots used by this code.
Give thanks to RPG hacker for working on Asar.