MW ULTRA - work in progress

15th August 2019:

Enemy work for MW ULTRA is going on, along with bugfixes, minor AI adjustments and optimizations.

When one starts to work on the enemies seriously, AI routines get severely stresstested, and some odd behaviors stand out. For example, the player would occasionally get noticed even when in cover, if an enemy was approaching from the left and the player was behind cover, also facing left. This was a simple logic mistake, easily fixed once detected, but also easily missed if only using straightforward frontal attacks on enemies when testing.

How enemy spawning works got implemented. Spawn points are to be placed into the levels with the world editor. These are item actors with an illegal type, that will not be put onscreen as actual items. Instead, they're processed on each area change, and create an enemy at their position, using randomized enemy types and weapons from lists, similar to practically all Covert Bitops games so far. The horizontal position can be changed slightly to produce variation. Spawn points can also be turned on and off by the game's story progression boolean flags. The spawned enemies are removed when leaving the area.

But this is not yet the complete picture. MW ULTRA contains horizontally long areas, and the game world might seem empty if you were to walk to the other end of an area and defeat all the enemies along the way, then walk back, and find it still empty, until you actually cross to the next area. Therefore the spawn points within that same area will also respawn enemies after enough time has passed. This requires some care: the spawn point must be far enough from the player, and the spawn point's previous enemy must actually have been defeated. This is to ensure that the player is never overwhelmed. Staying in one place will never produce a continuous stream of respawning enemies (as can happen in Hessian), so the player can proceed at their own pace.

Along with enemy spawning, item drop logic & rules were also needed. The straightforward answer is for an enemy to drop their weapon, but there are also other pickups like coins, first aid kits and armor. Similar to Hessian, it's about thinking of what the player is likely to need most at the moment, along with some randomness:

- If player doesn't yet have the enemy's weapon, it should definitely be dropped.
- If player is low on health, first aid drop probability should increase.
- If player is already carrying maximum of something or has the same melee weapon, it shouldn't be dropped.
- If the enemy can drop coins, it's a good fallback choice.

Something that both Metal Warrior 4 and Hessian got occasional criticism for was showing only the amount of spare magazines instead of spare rounds, similar to the original Deus Ex game. This is perhaps more realistic, but could create ambiguity. So now MW ULTRA displays rounds in magazine and amount of spare rounds, like most recent shooter games. This required removing the expanded borders in the score panel (40 column mode instead of 38), but was well worth it, as now the selected weapon is also displayed completely thanks to more available space, and there's two chars more available for each line in conversations.

Since the last WIP update, the scrolling also got one more improvement. Rastertime would get very tight on an unaccelerated NTSC C64, leading to occasional visual glitches at the midpoint of the screen when redrawing the screen. The solution is to split the NTSC screen update to 3 parts: top, middle and bottom, instead of just top and bottom. No more glitches!

This means the REU scrolling code path could even be removed in case of severe compatibility trouble or lacking disk space, however it's preferable to keep it in, as it leaves REU-enabled machines much more free CPU time for all non-scrolling code, preventing potential slowdowns.

Now, the enemy work shall continue! Here is a work-in-progress video of the first enemies that were implemented, and fighting against them:

15th July 2019:

The initial pass through the whole of MW ULTRA's world map is complete now! It's about 750 screens split over 21 tilesets and 30 level loads.

Though there are no enemies yet, creating the world meant also designing all movement related puzzles, making sure the movement and level pacing is OK, as well as placing lockers & containers for weapon / item pickups. When enemies and story are added, these might need further adjustments, and it might be nice to add some more secret areas, but the majority of the work should already be done.

But as it has turned out already, development of MW ULTRA is never straightforward, so world map editing is never just world map editing, but also making engine tweaks and fixing bugs.

Since the last update, MW ULTRA changed the data compressor again, to Doynamite by Johan "Doynax" Forslöf. Its advantages are faster decompression, smaller code footprint, and less runtime variable space needed, at the disadvantage of slightly larger compressed data compared to Exomizer 3. Using Doynamite, the custom tile map storage format (written about in the last update) could be replaced with compressing each room / area individually, making the engine code yet smaller as the custom decompressor code could be removed. Doynamite also responds well to recompressing the collection of area datas to yield the final compressed level file, making it usually yet smaller on disk.

Pathfinding also needed a second rewrite. The previous algorithm stored bounding boxes of platforms, stairs and ladders for each room, and the AI code would head toward the closest one in the direction of the target. However, it needed quite a bit of storage space, and could make a wrong turn in some cases. Therefore, the levels now store "junctions" for each horizontal platform, to aid the AI in making decisions to turn left or right. The junctions include stair and ladder endpoints, dead ends, mantleable spots, and too high drops. Fingers crossed for not needing a third rewrite!

When creating the game world progressed, it became apparent that the game will need disk side changes during gameplay - boot / play sides like in Metal Warrior 4 aren't enough to fit all data in. The plan is to have a disk turn at about halfway through the game, with the common "city hub" level data shared on both disk sides to eliminate repeated turns.

Now that the game world is done, it's easier to reflect on the total amount of data and memory needed, and to make a sort of mid-project postmortem on the world data-related decisions.

- Overall, the redraw-based scrolling works well and keeps CPU use lower than traditional scrolling, however there are some "problem areas" when some levels have so much color changes, that the color-RAM scrolling optimizations aren't of much use. On NTSC machines time can be tight in these spots, and so the engine got a REU optimization path for scrolling in addition to C128 & SuperCPU turbo modes. In the REU scrolling mode, the original scrolling routines get patched over by code that writes each room / area to the REU memory on room changes, and as the screen scrolls, the REU DMA transfer is fired for each row to quickly rewrite the screen and color RAM.

- Since the scrolling needs chars arranged in a certain way to match the char code and color code (done automatically by the editor), making charset animations becomes errorprone as the char order changes when the tileset gets edited. It's best to write the charset animation code only after the tileset is final, however "final" is often relative.

- The physics / collision code requires using special tiles to mark the ends of slopes and slope crossings to the map. This speeds up collision checks, but is manual and errorprone, and should not be used again in further Covert Bitops games!

- The "realistic" setting places some constraints on the world design, practically each area should make sense as a place people (even if villains) could actually exist and work in! In practice this turned not a problem, or was even liberating, since it reduces the amount of design possibilities, similarly as the C64's limitations force the scope of the game smaller and manageable.

- As no further world interactions or movement modes will be added (unless it turns out to be something absolutely vital) it's easy to take inventory of the ideas that will remain unused. It looks like there will for example be no lift platforms as in Steel Ranger. One planned idea was to have movable objects like boxes for reaching some hard-to-reach areas, but that looks not to be required either. MW ULTRA is still a traditional disk-based game that needs to keep all the engine code in RAM, and one has to be mindful of its total size. Some more advanced ideas were a hacking minigame (as in the latest Deus Ex games) or even taking control of enemy units, but these will remain just ideas as well. Therefore, the core of the gameplay will remain traditional: movement / exploration, combat (close / ranged / stealth), and advancing the story through character interactions.

That last point prompted the thought that making a C64 game cart-only could be the "next step up", almost becoming another platform, since code could be executed out of the cart ROM directly, and compromises could almost always be made to favor code speed. For example, speedcode-based scrolling. The amount of player character movements or interactions would also be much less limited, with just the cart total size as the limit.

Naturally, MW ULTRA will be finished as planned, being able to run both from disk and cart.

The next step after the world map will be the enemies. In addition to just enemy sprite pixeling and behavior coding, the algorithm for spawning enemies needs to be decided on. There will certainly be enemies in fixed positions for each playthrough, but in addition it would be good to add some unpredictability, or enemies reppearing to places that were cleared already, for example the city streets. Care must just be taken to not overwhelm the player with constant respawning.

Until the enemy work gets well underway, there won't be more videos, but in the meanwhile here are the game world screenshots from along the way so far:

City upper level

City lower level


Rehearsal space

Utility tunnels

Construction site

Outside the city

Condemned building


12th February 2019:

Time flies! But MW ULTRA has also progressed significantly (if sometimes unpredictably) in the meanwhile.

The world pixeling is well underway, with the game's city streets (AKA the main "hub" area) complete, and also about half of the city building insides done. The game contains two bars / nightclubs you can visit, but something was missing - they should have their distinct musical themes! So, two more songs were composed.

The game's loading sequence is also very likely final now, with the loading picture fading in / out, and after loading there is a menu for reading the game's manual, or proceeding into the actual game. For disk, the disk side change happens at this point and is the only one needed (similar to Metal Warrior 4), unless the game grows so much to actually need game content on both disk sides.

December - January also saw the need to consider the best game world storage data format. It should preferably be small on disk, small when loaded into memory, and fast to decompress into the actual scrollable data used by the game. The initial format was just compressing each room or area individually with Exomizer, but that wasn't yet optimal. The results of the investigation were somewhat surprising, and in the end a new custom lightweight compression format was devised. It's described in the new "rant" published a little while ago - Tile map data compression - and the idea is that this intermediate format responds well to further Exomizer compression.

Speaking of Exomizer, the game now uses Exomizer 3 for better loading performance - the new decompressor is significantly more efficient in the bitstream manipulation.

In between world pixeling and some bugfixing, it also occurred that the actor / world collisions on sloped ground weren't 100% solid yet. This was particularly noticeable when throwing grenades against stairs. Both Hessian and Steel Ranger employ somewhat of a hack - when the actor is traveling downward toward a slope and the "landing" is to be checked, they add the absolute horizontal velocity to the vertical velocity for the check. This prevents glitching through the slopes, but can cause too early landing. Initially MW ULTRA inherited this code.

But a better approach is to move the actor first horizontally, checking for a slope crossing, then vertically, again checking for crossing. If the crossing is horizontal, the physics / collision routine can report a "wall hit" just like when hitting an actual wall, and for example a grenade would then bounce back. Writing the code this way was a success, and the main physics routine also got a little bit faster due to eliminating redundant checks for world collision data.

However - it grew in size.

This prompted a few somewhat mad days of code size optimization, with the aim of at first trying to get the code back to the original size, of course without losing functionality. However, I didn't stop there, and in the end, over 200 bytes were shaved off from the engine code, which I honestly didn't think to be possible at this point. Some observations:

- If there are separate pieces of code doing almost the same thing, try to combine, if just performance considerations permit. For example, the sprite multiplexer has a loop which is executed for every sprite past the first 8. It copies the sprite variables to sorted arrays for the multiplexer IRQ, and also builds a list of IRQs that need to be happen in vertical order. This was executing in two separate sub-loops: "find the first sprite of IRQ" and "find the rest of the sprites for the IRQ." Combining these activities yielded a nice code reduction, with a minimal performance loss of having to check a zeropage flag using a BIT instruction.

- Self-modifying code isn't always the shortest, due to the absolute writes that take 3 bytes per instruction. Several dynamic jumps were converted from selfmodifying to using a zeropage indirect jump instead. However, a JSR cannot be converted this way.

- Sometimes reorganizing data or the meaning of variables allows shorter code. For example, Ian can do a high punch (FIRE+UP) or a low punch (just FIRE.) Originally the "attack state numbers" for these were high punch = 6 and low punch = 7. But reversing these so that high punch is 7 instead allows to LSR the joystick control bits, so that UP ends up in the carry flag, and then do an ADC instruction to determine the new attack state without any branching.

- In a similar vein, the code for spawning bullets or other actors was expecting a 16-bit offset in world coordinates for both X and Y directions. This offset is produced by running the actor draw code in a "fake" manner without producing actual sprites for display. It operates on 8-bit pixel coordinates, which are expanded to 16-bit after the draw code has finished executing. But turns out it's more efficient to keep the spawn offset in 8-bit pixel space until the very end, when the new actor's final world coordinates are calculated and set. This practically reduced code size in every case where actor spawning was being handled.

- Sometimes subroutines just need to be used more efficiently. Case in point, drawing health bars, which could be modified to (mis)use scorepanel text printing routines. The original unoptimized code does its own drawing to the screen & color RAM instead:

        ; Draw a three-segment health bar
        ; Parameters: A value, X position, Y color
        ; Returns:
        ; Modifies: A,zpSrcLo

DrawHealthBar:  sta zpSrcLo
                jsr DHB_Segment
                jsr DHB_Segment
DHB_Segment:    tya
PanelRowAccess3:sta colors+SCROLLROWS*40,x
PanelRowAccess4:sta colors+SCROLLROWS*40+1,x
PanelRowAccess5:sta colors+SCROLLROWS*40+2,x
PanelRowAccess6:sta colors+SCROLLROWS*40+3,x
                lda zpSrcLo
                cmp #9
                bcc DHB_FirstCharOK
                lda #8
DHB_FirstCharOK:sta panelScreen+SCROLLROWS*40,x
                lda zpSrcLo
                sbc #7
                jsr DHB_MiddleChar
                sbc #15
                jsr DHB_MiddleChar
                sbc #22
                bcs DHB_LastCharOK
                lda #0
DHB_LastCharOK: adc #19-1
                cmp #28
                bcc DHB_LastCharOK2
                lda #27
DHB_LastCharOK2:sta panelScreen+SCROLLROWS*40,x
                lda zpSrcLo                       ;Decrement total value for next segment
                sbc #30
                bcs DHB_ValueUpdateOK
                lda #0
                sta zpSrcLo
DHB_MiddleChar: bcs DHB_MiddleCharOK
                lda #0
                bcc DHB_MiddleCharOK2
                adc #9-1
                cmp #19
                bcc DHB_MiddleCharOK2
                lda #18
                sta panelScreen+SCROLLROWS*40,x
                lda zpSrcLo
The optimized routine modifies the char color first, does the drawing, then restores the color to the default white needed by all status texts. There is increased subroutine use, resulting in more CPU cycles, but drawing the bars is infrequent in any case (only when player's health or experience points change):
        ; Draw a three-segment health bar
        ; Parameters: A value, X position, Y color
        ; Returns:
        ; Modifies: A,zpSrcLo

DrawHealthBar:  sty PrintPanelColorValue+1
                sta zpSrcLo
                ldy #3
DHB_SegmentLoop:jsr DHB_Segment
                bne DHB_SegmentLoop
                sty PrintPanelColorValue+1  ;Restore white color for text printing
DHB_Segment:    lda zpSrcLo
                cmp #9
                bcc DHB_FirstCharOK
                lda #8
DHB_FirstCharOK:jsr DHB_Char
                sbc #7
                jsr DHB_MiddleChar
                sbc #15
                jsr DHB_MiddleChar
                sbc #22
                bcs DHB_LastCharOK
                lda #0
DHB_LastCharOK: adc #19-1
                cmp #28
                bcc DHB_LastCharOK2
                lda #27
DHB_LastCharOK2:jsr DHB_Char
                sbc #30
                bcs DHB_ValueUpdateOK
                lda #0
                sta zpSrcLo
DHB_MiddleChar: bcs DHB_MiddleCharOK
                lda #0
                bcc DHB_MiddleCharOK2
                adc #9-1
                cmp #19
                bcc DHB_MiddleCharOK2
                lda #18
DHB_Char:       jsr PrintPanelChar
                lda zpSrcLo
With the game engine optimized to good shape, now nothing should prevent just concentrating on the world pixeling. Until the next unexpected thing comes up :)

Here are two new videos, showing the grenade bounce after getting the new physics / collision code in, and an early visit into the Hades Club, familiar from the original Metal Warrior games. Its music uses dual filtered channels for a raw distorted sound, particularly on a 6581 SID.

3rd December 2018:

The engine code for MW ULTRA has been open sourced on github as the c64gameframework project! This doesn't include any of MW ULTRA's content or game-specific code, but just the basics: scrolling, sprites, resource loading, sound playback, and helper routines including movement and collision.

The project includes an intentionally simple sidescroller game example, with a player character, shooting, powerups to pick up, enemies, moving between areas, and in-memory checkpoint save & restore. Sprite assets & music are taken from Steel Ranger.

As MW ULTRA's release is still a long time away, it's good to contribute the engine code + editors already, in the hope they will be useful or inspiring. Compared to earlier Covert Bitops games, the main differentiating features are the concept of logical sprites (+ graphical collision bounds editing), and the flexible screen redraw-based scrolling, which saves CPU time for screen areas where the char colors don't change.

After the last WIP update, the engine gained one more feature: mastering also to GMod2 cartridge format. This gives the option of potentially also publishing MW ULTRA on cartridge. C64gameframework's example also demonstrates mastering to all of the supported formats: .d64 image, EasyFlash .crt and GMod2 .crt.

This practically means that four variants of the game's loader code must exist:

- Fastloader, using custom drivecode for 1541, 1571 (almost same as 1541), 1581 and CMD FD/HD
- Fallback disk loader using Kernal routines, used for IDE64, devices like SD2IEC, and "fast drive emulation"
- EasyFlash cart ROM loader. Also includes flash save support using the EasyAPI
- GMod2 cart ROM loader. Implements save support using the 2KB EEPROM

For disk support, the two first variants of the loader code are both included in the "LO" (loader) file that is read immediately after boot, but only one is left in the memory after initialization. The EasyFlash & GMod2 cartridge ROM loading code is rather similar, only the data layout on the cart differs due to different banking ($8000-$9fff and $a000-$bfff for EasyFlash, $8000-$9fff only for GMod2). Cartridge initialization after reset and saving differ much, though.

The game main part relies on the loader routine jump addresses always being the same, and the loader always occupying the same memory space. In practice the loader API is extremely simple, as there are only three routines actually to do with file I/O:

- Open a file
- Read a byte from opened file
- Save a file

The disk loader was written first, so it sets the amount of available space for all of the loader variants. It would be possible to have the memory layout different in the cart builds, but having it the same speeds up the build process, as the main part and loadable code (which depends on the memory layout of the main part) only have to be compiled once. It also makes testing more reliable, as the amount of free memory in the same ingame situation will be the same across the disk / cart builds.

In practice, the cart loading code is extremely simple: just choose the correct bank, and copy bytes to the "sector buffer" in RAM at $0200-$02ff, similar to how the fastloader operates. Cart saving code is more involved, though.

On EasyFlash, cart space used for saving must be erased in 64 KB sectors. This sets all the bits in the sector to "1," and they can be changed to 0's while writing, but not back without erasing again. To reduce wear on the flash memory and speed up saving, it's a good idea to use up the whole 64KB sector before starting over and erasing it again.

The EasyFlash save file system uses a 512 byte "directory" in the beginning of the save sector to mark allocations (256 bytes for file numbers per page, then 256 bytes for number of bytes in the file's last page). When a save file is overwritten, the old file must be marked as obsolete. This is based on an idea demonstrated in Per Olofsson's Ultima 4 Remastered EasyFlash I/O code: because writing a $00 byte is always legal without having to erase, it's used to mark obsolete file entries in the save directory. Similarly, $ff (the initial value of bytes in a freshly erased save sector) marks a free page.

When it's time to erase the save sector, the latest versions of all save files currently on the cart must be backed up to RAM, under the Basic ROM region ($a000-$bfff.) This presents an additional logic problem: there necessarily isn't enough free RAM for this when the game is running! Therefore the save routine must call into the game main part and ask it to deallocate (purge) loaded resource files, until there is enough free memory. The EasyAPI code that does the actual flash ROM interfacing also occupies a fixed RAM location, adjacent to where the music is located, so there is an additional rule that while saving, the currently loaded music must not take too much memory! In MW ULTRA this is not a problem, as saving always happens in the title screen.

For GMod2 the save support implementation is thankfully much simpler, as the EEPROM does not need to be erased, but can be written to word-by-word (not byte-by-byte!) Therefore the save directory format can also be different: just 3 bytes per file entry (file number, size lowbyte, size highbyte.) However, there's still the trouble of memory in the loader area running a little short, so the stack and the sector buffer are reused creatively during the save. But the GMod2 save routine does not need to deallocate game resource files.

The fastloader save and the cart saves operate on the same principle, that a specific save file must always be the same size. In MW ULTRA (same as in Hessian & Steel Ranger) this is not a problem, as there is only the configuration file, a "savegame list" which contains summary information of all saves, and finally the save files themselves, and their sizes never change. On disk, space is preallocated so that the fastloader doesn't need to handle allocation. However, the game must be prepared for the fact that on cart initial save files will not exist, so for example the configuration must be able to use valid defaults.

Writing the cart images is done using two separate utilities, one for EasyFlash and one for GMod2. You can take a look at both cart loaders, and the cart writing utilities in the c64gameframework source tree:

Boot code & loaders:
efboot.s gmod2boot.s

Cart writing utilities:
makeef.cpp makegmod2.cpp

Also, if you're interested in cartridge coding yourself, here are the references used during development:

EasyFlash Programmer's Guide
EasyFlash examples repository by Luigi Di Fraia
Ultima IV EasyFlash Loader by Per Olofsson
GMod2 programming information

Chester Kollschen from Knights of Bytes also provided some necessary consulting on GMod2 save; this was needed because of a failure to RTFM diligently enough.Thanks!

Now, as the NaNoWriMo 2018 month is over, and the engine code is released, it's time to get back into serious MW ULTRA development, starting with game world pixeling at last! As a summary, here's what's finished at this point:

- Game engine code
- Loading sequence & title screen
- Player animations
- Weapons, combat & stealth mechanics
- Standard enemy & pathfinding AI
- Music & 1st pass of sound effects
- Story (excluding detailed dialogue)

Finally, here's a video of the c64gameframework example game in action:

19th October 2018:

Some time has passed since last update, and the game has also progressed much in the meanwhile, bringing the game's always-resident engine code to completion for now.

After getting the weapons in, it was time to look at how the enemies and NPCs do navigation / pathfinding in the game world. The early Metal Warrior games did not have much movement AI to speak of, the enemies either made random decisions at junctions, or just tried to home in at the player. Metal Warrior 4 and Hessian both use a kind of "ad-hoc" pathfinding, where the character looks for routes of opportunity at their current location. For example if the target is below, and there is a ladder leading down at the character's feet, it makes sense to climb down. This works well most of the time, but trouble arises when the character is far from a junction, and doesn't know whether to go left or right first. This could lead to them trying the wrong direction first, appearing less intelligent.

This kind of pathfinding was also initially in use for MW ULTRA. But as the old problems reappeared, it was time to see if something better could be done. The ideal solution would be a full precalculated navmesh of all walkable / climbable sections in each area, with information on how the sections are connected, but the memory use would easily be too much for the C64. Therefore a simpler "halfway" solution was developed. The world editor now automatically detects the walkable platforms, stairs and ladders in each area, and stores their coordinates / sizes, but doesn't store connection information. Instead, the character AI finds out the next adjacent section during runtime. It also knows the section the target character (typically the player) is on, and tries to choose the next route step which allows to get closest to the target. Finally, any large drops and platforms the AI shouldn't attempt navigating to, can be tagged in the editor. This is to avoid getting stuck in a loop when the player character for example uses the grapple gun to get above, but the AI cannot follow.

With this system in place, enemies now appear to react intelligently to the player's position, and re-evaluate their route as necessary. For example if the player opts to drop from a large height (and likely take fall damage), the enemy will instead turn around and head for a ladder down if available, to reacquire the player on the platform below. Each navigable section takes 5 bytes of memory to define (type, coordinates, size) and there can be a total of 48 in a single scrolling area. The exact amount can be adjusted later as necessary.

After pathfinding, the next step was to work on "indicators" that appear on top of an enemy's head to let the player know their status. An exclamation mark appears when they see the player. Lethal and non-lethal defeat of an enemy also have their own icons.

Then it was time for another larger feature which had been postponed for a while, the stealth gameplay elements. The enemies already could detect line-of-sight to the target, so it was possible to hide from sight above or below a solid platform, or behind walls. But open areas allowed the enemies to easily detect and fire on the player, so something more was needed. With a few good suggestions from the Lemon64 forum, the game now implements "cover objects" like pillars or closets, that the player can go hiding next to. This is activated by pressing up on the joystick; the player character appears to go a little further in the depth direction. Enemies' sight is now blocked from the side of the object. If an enemy will then walk by, not yet alerted, the player can perform an attack with a surprise damage bonus.

Stealth mechanics also needed the reimplementation of the "make noise" feature from Metal Warrior 4. Footsteps, gunshots and all other sounds define a radius in which they make the enemies suspicious. This is indicated by a question mark indicator. The loudest sounds always alert every enemy on the screen no matter the distance, and like in Metal Warrior 4, for a time all new enemies coming into the view will be automatically suspicious.

The final piece of the stealth gameplay rules was to make it so that a suspicious enemy, or an enemy that has seen the player once, will also automatically look above and below non-solid platforms, while a completely unalerted enemy doesn't.

A "rock" weapon was added, also after Lemon64 discussions, that the player has an infinite supply of. These can be used to deliberately make small noises and distract enemies, but owing to the rule above, that can actually turn out as a very bad idea!

Next up was a small detour back to the soundtrack. The level and bossfight songs were done, yet something was missing. Hessian used to load a short gameover tune whenever you died, which meant a loading break (not good!) Therefore MW ULTRA turns back to what Metal Warrior 4 already did: every level has also its unique gameover theme. There ended up being 12 of them, lasting only a few seconds each. Several use the same chord progression as the "death theme" from the first Metal Warrior game: Am - A# - Am. One or two should never be heard, as it should be impossible to die in some of the levels, but for the sake of completeness they're still there...

For a long time during MW ULTRA development, dying meant reloading. But at last the game gained the pause / gameover menus similar to Hessian and Steel Ranger, and restarting from last entered area became possible. The final game will need some designing in this matter, it might be useful to have enemy-free "save rooms" which checkpoint the player, followed by longer stretches of enemy-filled areas that don't checkpoint. This would avoid respawning in the middle of trouble, with health down to the last recharging segment.

MW ULTRA will have light RPG elements similar to Metal Warrior 3: defeating enemies or completing other tasks will fill the experience bar on the bottom right of the screen. Once the bar is full, a "skill point" for upgrading the player character's skills is awarded. The skills will give damage bonuses, speed up weapon reloading, improve jumping and climbing abilities, allow for more carried weapons / ammunition, and improve damage resistance. Implementing both the XP gain and skill effects was straightforward enough, while the combined skill upgrade and stats screen (shown below) was more work.

Skill upgrade menu

After the skill menu, the final thing to work on before writing this update was to prototype the NPC conversation system. In Steel Ranger this was done in the project's final stages, and it led to ugly spaghetti code and even bugs. Doing it earlier this time allowed for a more controlled approach. The plan is to show the speaking character's portrait in the bottom left of the screen, in addition to the familiar speech bubble icon:

NPC dialogue test

Showing the portrait wasn't completely straightforward. The portrait is a "logical sprite" that consists of several sprites, and rendering it in a fixed position on the screen is easy enough. But since the game assumes that all sprites are world objects that move along with the scrolling, a separate bit was needed for the sprite multiplexer to indicate that this is an overlay sprite, don't scroll it! This was actually stuffed into the sprite color byte, where there was still one bit free (color byte also controls sprite flickering, invisibility and X-expansion.)

A thing that recurred with all of the pause menu, skill upgrade screen and the conversation system was the need for a firebutton press to return to the game, but not yet initiate an attack by mistake. The previous games solved this by some ugly hacking of the joystick reading code, but utilizing the status bar / menu state machine, which already has states like "inventory," "pause menu" and "conversation" allowed a better solution this time: define an extra state which we call "resume", whose only purpose is to wait for releasing the fire button, before normal player character control (and attacks) resume.

Next up: some more fun with code size-optimization. Substantial gains are likely not possible at this point though... Then it's off to NaNoWriMo 2018, which might be partially used for working on the game's detailed script, and possibly, also starting the game world pixeling work at last!

In the meanwhile, here are the YouTube videos from the last two months, showing stealth and close combat:

2nd August 2018:

Now also the shorter bossfight and "danger" songs of the soundtrack are done. These 7 additional tunes were expertly composed by Kamil Wolnikowski (Jammer), so now the soundtrack totals 23 subtunes.

The initial set of weapons for the game are complete as well, adding up to 15 of them with the grapple gun counted in.

The disheartening part of implementing weapons, or any new features that go into the always-resident main part of the game, is seeing the code & data size grow. For weapons, there's much data to include: the weapon definition data including bullet speed, duration, damage, which sound effects to play for shooting and for reloading, the actor definitions for the bullets, the bullets' movement code (if special behavior is needed, such as the bouncing of grenades), the weapon sound effect data itself, and also the name texts of the weapons.

Therefore, after adding a bunch of new stuff, it's often beneficial to stand back and see whether the resident code & data could be reduced in size to compensate. In the best case some code doesn't need to be executed at all, leading to both size and CPU time reduction. Other times it's a tradeoff between memory use and execution speed. For example I know that by writing unnecessarily to the 40th (invisible) screen column, the screen redraw code could be reduced by almost half a kilobyte, at the expense of being slower due to the unnecessary writes. This might yet become in handy in a memory emergency.

One of the first things to optimize this time was the sound effect format. The Miniplayer follows the same idea as the Prince of Persia C64 sound routine, so that each sound effect frame starts with a bitmask byte, telling what further data to read & put into the SID registers this frame. However, in practice changing the waveform and frequency occurs most often. Ideally these should be optimized to not require a separate bitmask byte at all.

Since MW ULTRA's sound effects are single-channel, they cannot use sync & ringmod bits in the waveform, so those bits could be used instead to tell if a frequency change or frequency slide are also going to occur on the same frame.

Frequency-only frames could likewise be optimized for a limited range of frequencies. We just need to be able to distinguish between waveform changes, frequency changes, ordinary bitmask frames, and "delay / wait" steps. To simplify code, the Miniplayer feature of being able to change pulse only would be dropped as unnecessary. Therefore the final encoding of sound effect frame start bytes becomes:

$00     End of sound effect
$01-$8f Waveform change, bit 2 = frequency follows, bit 4 = frequency slide follows
$90-$9f No waveform change. Bit 1 = init ADSR/pulse follows, bit 2 = frequency follows, bit 4 = freq. slide follows
$a0-$df Set frequency only, possible range $00-$3f (high byte)
$e0-$ff Delay for x frames

Here's the data of one sound effect before and after:


Optimized, it becomes:


In total, over 100 bytes were saved by the sound effect format change. More bytes could be eliminated by changing some logic in the code. Due to working on the weapons last, I focused on the weapon & attack code now as well (an earlier size-optimization pass had already removed about 300 bytes from all over the code.)

One example: Ian can do a flying kick move while jumping, but he shouldn't be able to kick twice during a single jump. The unoptimized code transitioned from the "flying kick" state back to the ordinary "jumping / falling / walking" state after the kick animation, and a variable needed to be set & checked to prevent the second kick. This logic could be changed to stay in the flying kick state until landing instead, in which case a not-insignificant amount of code could be eliminated.

The following piece in the flying kick state code (check for animation end and do state transition)

ldy actF1,x
lda flyingKickDelayTbl-FR_FLYINGKICK,y
jsr OneShotAnimation
bcc MH_FlyingKickAnimDone
lda #FR_JUMP+2
jmp MH_SetStateAndFrameCustom

would become (note that the oneshot animation routine stays safely in the end frame even if called after finishing)

ldy actF1,x
lda flyingKickDelayTbl-FR_FLYINGKICK,y
jmp OneShotAnimation

And now the check for the second kick in the attack triggering code (shown below) could just go away:

lda currFireCtrl
and #JOY_UP
beq AH_Done
lda actFd2,x
bne AH_Done ;Do not allow second kick
inc actFd2,x ;Mark kick already performed
jsr MH_SetStateAndFrame

Additionally, the check for the "up" direction (least significant bit) can be changed to an lsr instruction, so the final code would become:

lda currFireCtrl
bcc AH_Done
jsr MH_SetStateAndFrame

These optimizations do not save *that* many bytes at once, but the idea is to keep going through the code with a creative mindset, until the bytes saved begin to add up. Another trick to remember is reordering data where possible so that extra instructions accessing it can be kept to a minimum.

After this size-optimization pass it's finally time to start refining the combat AI, and begin implementing the proper stealth mechanics in the game, which might include hiding behind objects. Until next time!

22nd July 2018:

The "long" songs (title, level themes, ending) of the MW ULTRA soundtrack are all done!

After this, before tackling more of the gameplay things (enemy behavior, stealth mechanics), I took an opportunity to test something that had never featured in Covert Bitops games before: a grapple gun in the style of Batman the Movie and Bionic Commando.

The gun is just a weapon that fires a special kind of bullet: the grappling hook. When the hook's code is active, it does some "magic" to the player attack state: firing again (or switching weapons) is prevented, and the player character remains in the aiming pose (arm extended) as long as the hook is flying through the air.

This is really the easy part, but the rope itself presents two math-heavy challenges for the C64's limited instruction set: the arc-like motion when the player is swinging on it, plus drawing it on the screen. Since MW ULTRA doesn't use char bullets, just sprites, it's logical for the rope to also be all sprites, which are fed into the sprite multiplexer just like any actor sprites.

My hypothesis was that 12 predefined sprite frames with different angles would be enough to get a reasonably accurate rope "line", and it turned out OK. These sprites are placed at 21 pixel intervals vertically, until the whole rope length is drawn. For more accuracy, one can alternate between two frames when the angle is "inbetween." Finally there are another 12 frames with only half of the vertical space covered for a shorter final piece. As these are stored compressed, the total memory use of the hook + rope sprite frames is reasonable, about 500 bytes.

Initially the frame to use would be chosen from a 16 x 16 lookup table, indexed by horizontal and vertical distance (scaled by bit-shifting right until both are below 16) but this was later replaced with real-time math: multiply X distance by 11, then divide by Y distance, and you have the sprite frame between 0 - 11. If there's a remainder equal or greater to half of the Y distance, then alternate between two frames. Since the divide routine I use only operates on 8 bit values, the X * 11 needs to be pre-shifted right (along with the divisor) until it fits into 8 bits.

To avoid overflows or the need for 16-bit math the distance from hook to player is calculated at two pixel accuracy both horizontally and vertically; it turns out to not look too bad.

Without the "each actor has its own draw routine" mechanism drawing the rope would have required support in the resident game engine code, but now it doesn't, practically the engine does not need to even know of its existence at all. The rope is drawn as part of the grappling hook's draw routine, which means that if the hook is too far outside the screen, the rope doesn't draw at all! Therefore the maximum length needs to be limited.

This covers the rendering, but what about the movement?

It turns out there's no need for trigonometry tables, or using an angle representation for the swing motion, instead applying the Pythagorean theorem (x^2 + y^2 = l^2, where l is the total rope length) is enough. It's needed in two cases:

- When the grappling hook attachs, the rope length gets calculated from the current X & Y distance between the player and the hook.

- When the player position is updated during the swing, the Y-coordinate is calculated from the rope length and player's current X position (ie. y^2 = l^2 - x^2.)

Solving l or y requires a square root routine. Thanks go to Graham for the "fast square root" routine on Codebase64.

The acceleration to make the player swing back and forth comes simply from the rope frame: the acceleration is strongest on the extreme sides and zero in the middle. Angles greater than 45 degrees cannot be drawn with the 12 rope sprite frames, so the maximum swing speed needs to be limited proportionally to the current rope length to prevent a too extreme angle happening in the first place.

The control mechanics are the same as in Batman the Movie; while swinging it's possible to either spool the rope in or out to ascend and descend, or apply a small swing force to the left or right. Pressing fire detaches the hook.

Also similar to Batman the Movie, colliding with a wall during the swing will cause the hook to detach automatically and for the player to start falling. This simply reuses the ordinary jump collision detection.

When the hook detaches, it starts to spool in automatically; it simply "homes in" toward player at a speed proportional to the distance. If the player is falling, the hook could end up "chasing" the player for a longer time than looks good, but this can be fixed by adding the player's Y-velocity to the hook (this fix is not yet in use in the video below.) Upon detaching, the player's final swing velocity is applied as the new jump velocity, so that the transition to jumping or falling is smooth.

Another thing that contributes to the smoothness is making sure that launching the hook can be done at any time: standing, walking or jumping, and it doesn't restrict player motion before the hook attaches. This is in contrast to Batman the Movie, where Batman stops while the rope spools out.

Being so math-heavy, the hook + rope routines do consume some CPU time, and a maximum length rope also stresses the sprite multiplexer fairly much. Using it with many enemies around could result in a momentary overload, either slowing down the game or resulting in missing sprites. However, this should be balanced by survival instinct: as the player character can't attack while on the rope, using it in combat is risky and likely best avoided :)

Here's the end result captured on video:

1st July 2018:

Among other work (mastering also to EasyFlash cartridge format, plus the option for using a second firebutton to jump, both requested on Lemon64), creating the soundtrack for MW ULTRA is now underway, with 6 songs complete so far.

The soundtrack is shaping up to be a little more atmospheric and less drum-heavy compared to Hessian & Steel Ranger, and could even be considered as a complementary method of storytelling. MW ULTRA will not have a silent protagonist like the two previous games, but yet we can count on Ian not speaking that much, and certainly not actually thinking aloud, so the soundtrack aims to (hopefully) provide additional insight into his headspace.

The basic workflow for composing is still the same as before: songs are initially composed in GoatTracker 2, taking into account the player limitations, then converted. The conversion is now easier, as the Miniplayer converter program produces the output directly in source code format ready for the build process, instead of having to manually run NinjaTracker 2 and use its packer / relocator function to save the final song data.

For testing on real C64 hardware, and to also check how the sound effects play on top of each song, the game's build process also creates a sound test program, in which the tunes & sounds can be triggered with keys. When music is playing, sounds are played on the 1st channel only, that is typically reserved for the lowest volume instruments, such as arpeggios or secondary melodies.

The Miniplayer uses a one-frame pseudo hardrestart somewhat similar to DMC 4. One frame before the next note's start, the gatebit is switched off and ADSR release is set to maximum (15.) Combined with a silent first frame (typically testbit + gate), this ensures even note starts, but doesn't give time for the sound to automatically start from zero volume, like a proper two-frame hardrestart would do. Therefore some care is needed in composing, to set the notes to manually release early enough. Fortunately GoatTracker 2 can be made to mimic this kind of operation with the hardrestart ADSR parameter, and setting "gateoff timer" to 1 in all instruments.

Because notes don't always start strictly from zero volume, they blend to each other more naturally, giving the music a more organic feel. All previous Metal Warrior games that used the earlier music playback routines had 1-frame hardrestart as well, so it's also keeping the tradition :)

The CPU and memory use gains from switching from NinjaTracker 2 to Miniplayer are minimal to tell the truth, but after using the same routine for so long (NinjaTracker 2 was written in 2006) it was good to try something else for a change. The Miniplayer is based on some unfinished player concepts written at the same time or earlier, the main unique point is having the pulse and filter modulation compare to a destination value, instead of counting frames remaining in the modulation step. This reduces CPU use slightly, but makes it harder to use step-modulation loops that don't loop exactly (ie. down for 10 frames, up for 20, getting further away from the start point each time.) That's mostly not a big deal, since both pulse and filter will sound bad if the pulsewidth or cutoff overflows and "wraps" during a long note.

Again, using the SID filter presents a challenge. Each 6581 SID has its unique cutoff frequency curve, and filtering a voice also lowers its volume a little. Emulation usually doesn't account for this, so there's a risk of making some sounds too loud for emulation users, or too silent for real hardware. Yet using the filter is an essential part of C64 music, so this time either, leaving it out would not be a real option.

Continuing the idea from Steel Ranger, the game detects the SID model (6581 or 8580) on startup, and will adjust the filter cutoff for 8580 to mimic a typical 6581 (as the songs are composed for 6581.) This time the cutoff adjustment is no longer just a constant subtraction, but is strongest in the middle of the cutoff range, using a linear function resembling an inverted triangle wave. It takes only a few assembly instructions, yet is surprisingly effective.

To not spoil "significant" themes like the title screen theme yet, here's a preview of the new "inside buildings" theme. In the original Metal Warrior it was just a repeating melody, less than one minute total:

11th June 2018:

This update goes more in-depth into the sprite handling and scrolling in MW ULTRA.

Previous Covert Bitops games since Metal Warrior 4 have operated on the principle of each sprite frame having hotspot and "connect-spot" coordinates, and multi-sprite actors being constructed by chaining these sprites one after one. The hotspot defines the sprite's origin, and the "connect-spot" tells where to place the origin of the next sprite. For example humanoid actors' chain is: lower body, upper body, weapon.

That particular chain works well, but designing multi-sprite boss enemies quickly gets tedious, as the sprites typically have to be joined from their corners to form the complete shape. For example the 4-sprite large spherical droid that is the first boss in Steel Ranger. Furthermore, when pixeling the sprites individually it's harder to get the shape right.

Therefore MW ULTRA improves this system by introducing separate "logical sprites" in addition to physical (hardware) sprites.

Each logical sprite has the hotspot and connect-spot, as before. But it can consist of multiple physical sprites as necessary. To use the Steel Ranger boss as an example, since it has no separately moving parts, it could be one logical sprite with four physical sprites.

The handling of human actors doesn't change much; since the lower and upper body can animate individually, they still need to be separate logical sprites.

The logical sprites also include collision box information, so collision bounds can now be edited graphically instead of typing them into source code. Physical sprites within a single logical sprite can have different colors, as well as X-expansion on or off. Each physical sprite is a separate "object" within its resource file, and duplicate shapes can be detected, so for example a large car with front and rear wheels the same would save memory.

The previous games used a few hardcoded routines for splitting actors into sprites. Typically these were an optimized routine for one sprite actors, a generic routine for n sprites long sprite chain, and a separate piece of code for the humanoids.

In MW ULTRA, each actor type has its own "draw" routine in addition to the tick update routine, and it will be called to output the actor's logical sprites. This allows greater freedom and extra logic during the draw if necessary. Furthermore for NPCs, enemies and bosses both the draw and update routines can be loadable code instead of residing in the main game engine, so the engine's resident code size can be reduced.

The logical sprites are in turn split into physical sprites. In practice, optimization mandated that there's a separate fast path for logical sprites that have only one physical sprite and max. one collision box. In this case the logical sprite resource object also contains the actual sprite data, which saves resource file memory overhead. Furthermore, one resource file has a limit of max. 128 resource objects, so that allows placing more sprite frames into the same resource file.

The screenshot below illustrates the new sprite editor in action, editing a large enemy:

Multi-sprite editor + large enemy being edited

Then the second topic, the scrolling.

Arriving at the somewhat unorthodox "whole screen redraw" scrolling in MW ULTRA required some prototyping and experimenting.

Both the first and fourth Metal Warrior games (with Ian as the playable character) opted for color-per-tile graphics and 4x4 char tiles, making it impossible to get colorful details to the graphics. At the time of writing Metal Warrior 4, it seemed absolutely impossible to get both color-per-char graphics and any sophisticated AI routines in. Hessian solved the problem by having 8-way scrolling with limited speed, dividing the screen and color RAM update on 3 frames total. Steel Ranger had multidirectional scrolling, but a smaller scrolling window and much simpler enemy AI code. Plus neither player or enemies had separate weapon sprites, so the total load on the actor and sprite system was kept lower.

Reusing Hessian's 8-way scrolling was not really an option for MW ULTRA, as it will lag behind in fast falling motion, and generally introduce a jagged feeling to the scrolling, which would fit badly with the varied scrolling speed (running vs. crouchwalking) and shifting the scrolling center depending on the player character's facing direction (ie. show more of the world in forward direction.)

Therefore the initial options appeared to be:
- Traditional color-per-char, multidirectional scrolling, but with a limited-size scrolling window.
- Color-per-tile, with 2x2 tiles. The saved CPU time would allow a max.size scrolling window (22 rows) while keeping one row for the status display.

Before starting Steel Ranger's development, I had briefly experimented with redraw-based scrolling, using the tilemap data directly, but it seemed to have too much CPU overhead and therefore not be promising. But in contrast to traditional scrolling, redrawing could allow to handle different tiles differently for optimization, e.g. some tiles could use a single color RAM value, or even a single screen RAM value, or skip color RAM writes altogether.

This time Ian would hopefully get the colorful background graphics he deserved, so the first option of the two looked more promising. It would also cut down on scrolling code size, since only three different color RAM copy loops (up, down, horizontal) would be required. In contrast, an optimized 2x2 color-per-tile scrolling would practically need separate color shift routines for each of the 8 directions.

However, early tests proved that scrolling that way could consume CPU time so much that slowing down could result in situations where the screen scrolls fast, so that practically every frame is a scrolling frame, the player fires a full-auto weapon, and there are also enemy actors on the screen. And this was without any AI or enemy firing so far!

Therefore I returned to examining redraw-based scrolling methods, and found a way to optimize the overhead of the tilemap access to acceptable. The key is indexing both the screen and the tilemap data with the same CPU index register, leaving the other index register for the current tile's number for tile data access. With 2x2 blocks, the screen indices would skip every second value, ie. 0, 2, 4... so also tilemap data would need to have the same tile row's tiles spaced to every second byte.

This proved not to be a severe problem, as two tilemap rows can be interleaved. Therefore, no wasted memory space.

Each of the visible tile map rows is handled in its own piece of code, so before the redraw, tile map addresses for each row need to be calculated. The redraw code is modified directly for that, so that absolute address access can be used.

The redraw writes to both the screen RAM and the color RAM at the same time, from the top to the bottom, so a doublebuffered screen is not required, but the redraw takes almost a whole frame (on NTSC it gets very tight!) At first this seems counterintuitive, but the redraw happens in an IRQ that starts from the middle of the screen, and meanwhile the main program is already calculating the next frame ahead as much as it can, without any busy-waiting. Given this, it appears that it's better to have a large amount of CPU time taken away, followed by a frame that's completely free for game logic processing, instead of taking half of every frame for alternatively shifting either the screen or the color RAM, as Steel Ranger does.

The final thing to decide was: which kind of tiles would be possible? Initially, the redraw supported only two: either color-per-char (charcode & colorcode the same), or a "default color" tile which skips the color RAM write completely. The default color would only be initialized when entering an area, not during scrolling. And other colors could not be allowed to "leak" onto the default color areas. This scheme required only one branch per tile, but proved too limiting in practice.

Therefore it was necessary to add also actual color-per-tile tiles. This slows down the redraw slightly, as one more branch per tile is possible. However otherwise it would likely be impossible to produce the background graphics the game needs.

Doing scrolling by redraw also gives the possibility to easily make arbitrarily large background animations (e.g. opening doors), as the CPU cost is just the same for scrolling, and there doesn't need to be separate redraw code for the animations, like in the previous Covert Bitops games.

After the redraw code was in place, the world editor (based on Steel Ranger's world editor) needed to be modified to support the tile types, and the char allocation scheme that matches the charcode to the char's color. The editor needs to analyze the map data and check for which tiles the color RAM writes can be optimized away. When necessary, two copies are produced from a single tile: one with optimization, the one without. Color-per-tile tiles are actually slowest to draw due to needing to access the tile color separately, so they are avoided if the right charcodes for color-per-char operation (max. 16 chars with the same color) can still be allocated.

The editor can also show a debug visualization, to see how the tiles will be drawn, which is shown in the screenshot below: (C is "color-per-char" and O is "optimize" which skips color RAM writes)

World editor & tile type debug visualization

6th June 2018:

MW ULTRA is the new Covert Bitops C64 game in development!

It's essentially a reinterpretation or reboot of the first Metal Warrior game, but with substantial changes planned to both the story and gameplay. The genre is still sidescrolling action-adventure, but the aim is to blend shooting, melee fighting and stealth in a semi-realistic (or "semi-cinematic") way, with the melee fighting taken further by making the characters capable of multiple moves, including different punches and kicks.

Plans for revisiting the first MW game were on the backburner for a long time, including the possibility to use a more powerful platform. During Steel Ranger's development, ideas how to do it on the C64 materialized instead, along with a very solid rework & expansion of the original's story. Like in the original, you take the role of Ian, a troubled musician in a not-so-bright near future, but apart from that, expect surprises...

Technically, the game borrows the 2x2 tiles from Steel Ranger, but with a larger scrolling window that is redrawn completely from the tilemap data for each scrolling update, instead of shifting the screen. Tiles can have their color RAM data updated differently to optimize the CPU use of scrolling, either color-per-char with char & color code the same (completely inspired by Quod Init Exit II), color-per-tile, or skipping color RAM update where there is a larger patch of the same color.

The resource management & code loading / relocation from Steel Ranger likewise makes a comeback. But Steel Ranger contained a shortcoming where loadable or "story script" code couldn't load further resources from the disk, in the fear that it would get relocated in memory due to old resources getting purged, and the code execution returning to an illegal address afterward, crashing the game. This time the engine should be capable of handling stack return address relocation to fix this!

The music routine is no longer NinjaTracker 2, but the "miniplayer" which was released earlier in 2018 as open source.

The engine work is going along well, with almost all the "whole time resident" code written, but naturally there is still much work to be done. The game is planned to be finished in 2019, 20 years after the original's release!

Here's the first WIP video, showing player character movement & initial city background graphics. Note: despite being absent in this video, there's certainly going to be all-directional scrolling, and ingame music.