Watch me now....

I've been fighting with .NET list views for the watch window and have finally managed to get a watch class being mapped and displayed via the .NET data mapper. The Data mapper is very cool, and lets you map a class directly into a control. It also lets you add new items and will allocate/edit/delete new entries without me lifting a finger! Very neat.

I'll start on the actual watch processing tomorrow, and with any luck that'll be all that I'll need to do to implement basic watches! After that I'll need to get breakpoints working and I suspect the first version will closely follow afterwards.

Coming together....

I've been making steady progress and have now got the dissassembly window scrolling around under user control (without tracing). So you can now use the cursor keys to scroll the window, and the page-up and page-down keys to move quicker. It's going pretty well but I'm not happy with the ricj text box control. It feels slow, and there are a few keys you can't seem to override. This means although it gets rewritten every tick, you can upset things by pressing enter at the wrong time. I think I'm going to have to draw it myself somehow - perhaps even on a bitmap or something....

However that aside, its making good progress, and once I have user breakpoints in I can start looking at memory and watch windows. But the drawing of the display remains a concern....

Single stepping.

After a long discussion with Russell on Friday about the best way to single step, I decided to try out the new method. It basically involves having the STUB execute a single command inside the STUB itself. Currently I simply place a breakpoint one instruction later, and then run the application; the program runs, hits a BRK then stops again. This new way is friendlier to machines like the ZX Spectrum that has issues (and bugs in the ROM), but its tricky.

Imagine you've stopped the execution of a program, saved all the registers, flags etc. and now want to single step. What you would have to do is copy the instruction you want to trace into a 3 byte slot (thats prefilled with NOP's), restore all registers and flags (also stack) and execute the command. You then resave everything and return back to the debug comms loop. It sounds fairly simple but its not as easy as it appears.

;******************************************************************************
;
; Name:  SingleStep
; Function: Given an address and a byte count, execute a sequence of bytes
;
;******************************************************************************
ExecuteCommand:
             jsr   GetByte             ; Get destination address
             sta   Dest
             jsr   GetByte
             sta   Dest+1
 
             jsr   GetByte             ; get opcode size
             tay
             dey
 
             ; Clear out exec buffer
             lda   #$ea                ; Clear the instruction space 
             sta   ExecBuffer+1        ; in case the instruction isn't 3 bytes long!
             sta   ExecBuffer+2
 
             ; copy 
!lp1         lda   (Dest),y            ; Copy the instruction to execute
             sta   ExecBuffer,y
             dey
             bpl   !lp1
 
             tsx                       ; Save current stack
             stx   StackStore
             ldx   RegSP               ; Restore application stack
             txs
 
             lda   RegF                ; Get the flags ready
             pha
             and   #$4                 ; We must keep interrupts OFF!
             sta   RegF
             pla
             ora   #4
             pha
 
             lda   RegA                ; restore registers
             ldx   RegX
             ldy   RegY
             plp                       ; restore flags
ExecBuffer:
             nop                       ; Command goes here.
             nop
             nop
 
             php                       ; Save flags
             sta   RegA                ; Save registers
             stx   RegX
             sty   RegY
             pla                       ; get the flags and store
             and   #%11111011
             ora   RegF
             sta   RegF
             tsx
             stx   RegSP               ; Store the application stack
 
             ldx   StackStore          ; get the debugger stack back
             txs                       ; and restore it.
             rts
StackStore   db    0

Now you'll also notice you have to do some jiggery-pokery with the flags so you dont reenable interrupts by accident (since we're actually still IN an interrupt), but aside from that theres a lot of stuff involved. The bit I really dont like is that emulators simply wouldn't do this. They would simply set a break point and run. I really don't want to have 2 ways to step through code, I'd like it if the CPU module simply didn't care if it was a real machine or an emulator. Now, the spectrum can still do it with breakpoints, but since the spectrum ROM has bugs in it, its limited in exactly what it CAN do.

We currently think you would need to do breakpoints with CALL's, as the RST instruction needs ROM support and its the part thats bugged. NOW calls take up a few bytes which means you have limits as to where you can place a breakpoint. For example, if you were to branch over an XOR A but wanted to put a breakpoint on the XOR, then the breakpoint would overrun onto the next instruction, and it might crash.

Now for single stepping, that shouldn't happen; particually if the coder is aware of the limits. On other Z80 machines, it should be possible to use the RST instruction to do the breakpoints so they would be fine. I've put breakpoint control on the STUB for a few reasons, but because of this it means each STUB can decide how it wants to do them.

So... after playing around a lot with it (and actually getting it to work), I've decided not to use it, but to go back to the breakpoint idea. I think it has the widest compatibility, and if your really stuck you could still implement you own single step if you modify a CPU module and package it together with a dedicated COMMS module.

I've also started to think about the TCP/IP module for using with emulators (and I'll do a UDP one for the RR-NET later), and I'm starting to get excited about the possibility of it being adopted by other emulator guys. If all goes well, then no one should ever have to write a built in debugger again!

Lookin' good!

I've finally done enough to make some visible progress. With the addition of the expression evaluation, symbol table lookup and command line processing means I can finally display symbols in the dissassembly view. It's now starting to look like a proper debugger. I need to get left side labels in next (where the addresses are displayed) so you can see where your branching to, then we'll be mostly done in that view.

I still have to add control of the view but thats next. The biggest unknown is the registers, I've no idea yet how to display a generic number of registers or how to lay them out. I'm tempted to let the CPU module display them as well as handle them, but I don't know yet.

Good progress!

I've gotten on pretty well tonight! I've managed to finish BOTH the lexical analysis class and the expression evaluation system. Not bad for a nights work. I can now evaluate things like...

((2<<(4*4)+8*4)&$ffff-MySymbol)*4

This means watches can enter calculations to be evaluated, which you can then apply the [...] modifier to so that its a memory lookup on the target. I still have a way to go as theres currently no way to get the evaluation system to look at the targets memory OR registers. However that shouldn't be hard and then things will start to get very cool. Of course the inital reason for this is so theres an interface for the CPU module to lookup symbols for the dissassembly, but through the same interface you'll be able to execute actual calculations.

I also have the ability to add commands to this, like Lo() and Hi(). This means you could do calculations like...

Lo( 2<<(4*4)+8*4 )<<2

This is pretty neat, but it also opens up for more complex functions like ...err... SQRT() or something, which is pretty cool too. The whole expression and lexical stuff is just 2 small C# files, so its pretty portable if I want to do anything else, so this could be handy!

Expressions and Lexical analysis

So I started to look around after yesterdays revamp of the plugins, and realised that to finish the intialisation system I need to pass in an expression evaluation interface. Oh well.... more code to port..... I started to port over SNasms expression stuff but it soon became apparent that I needed a lexical processor first. Now the one in SNasm it pretty fully featured, but a bit much for a command interface, then I suddenly realised that I dont need very much at all. I need to be able to pick out symbols (like *,."$&[]() etc.) and symbols. This means I only need a very simple one as at text I dont understand will be a label (or I can do a quick dictionary lookup to got basic commands).

So I've almost got this basic one going, and its really very simple but will do the job at hand. So I'll finish this tomorrow, and then port the expression stuff the night after and finally I'll be able to get the intialisation stuff done. It also means I can start to load symbol files and get a proper symbolic dissassembly!

Slow going, but we're headed in the right direction.

Slow going....

I've not been doing much on the debugger as I've been busy with real life, but I did get some time today to mess around. I've been updating the plugin system to make it more flexible and so that I can do some work on the command line arguments system. Currently I can specify which modules I want to use on the command line (along with symbol table loader and symbol file), and the next step is to spawn a monitor window using the selected modules.

I've rearranged all code so that multiple monitor windows are possible, in theory this will allow debugging of secondard CPU modules as long as the debugging stub or interface supports it - although thats a long way off yet.

Future plans....

There was a question about using the RR-NET instead of the serial port, so I thought I'd answer in depth here.....

Currently I'm debugging via the user port on a Commodore Plus/4 and the parallel port on the PC, but the plan is to then do two C64 versions using both the userport AND the RR-NET (since the RR-NET is far easier to get a hold of). However the user port version would allow you to develop more hardware as you can't always use RR-NET if your trying to develop for a bit of hardware where theres no passthrough, or it's simply not available (The current cart disables the pass through when in use.)

That said, for normal apps and games, I hope the MMC64/RR-NET (and the MMC Replay) will be invaluable for getting things working on real hardware. Theres nothing quite like debugging on a real machine, particually if you are using the MMC64 file loader stuff I did in the framework, as thats currently impossible to develop with via an emulator. Of course someone could also implement a SilverSurfer stub as well - nothing to stop them!

In between these versions (probably at the same time as the RR-NET version), I'll write my first emulator plugin and get Minus4 working using it. This will then be given away for everyone to use. I then hope that new modules will start popping up as all the hard work should be done, and emulators should just be able to plug it in. Once we have some simple CPU types (Z80, 6502 etc.) then all an emulator should have to do is implement the STUB and it'll all magically work.

The idea is to give away the source to the 6502 module, the COMMS modules, all the STUBS that I end up doing, and the Symbol table module. This should allow you to write whatever system you want, without having to implement all the crap around it that makes up the debugger - in theory. It also means you can easily adapt modules for your own use; so the 6502 one could be changed to be a 65816 one, or a 65c02 etc. allowing for new machines like the SNES or PC Engine to be added easily.

Lastly.... if theres ever any real problems with getting a particular machine to work using the interface provided, then all you need to do is put both modules in the same DLL (comms and cpu) then you can add extra interfaces to expand or alter the behaviour. Lets say you couldn't implement single stepping in the way thats assumed you would, then you can add a new comms command that only YOUR cpu module knows about and call it instead.

Thats the theory......

Bingo!!

I've now fixed the tracing system so I can trace without a problem. I managed to single step through quite a bit of XeO3 which is a good sign. It appears to be pretty solid, I can stop/start and trace without any issue now - which is great. That means I have the core functionality now needed to progress. The current CPU and COMMS interface does have an issue however.... Single stepping on the spectrum (a 48k one at anyrate) isn't easy, and in order to make it better you really need to get the target to execute the single step so it can do it best for itself. However, getting the stub to do that takes quite a bit of memory and I dont want that, so I need to think about it a little longer....

Almost there!

I've now got things almost working, single stepping is rock solid and I now have the RTS instruction being simulated fine. There appears to be a slight glitch where on a specific instruction in XeO3 it doesn't step properly but jumps off to $FFFF for some reason - no idea why as yet. While I'm trying to figure this out I'll start adding breakpoints to the interface and then get the symbol table loaded.

I suddenly realised that I shouldn't be passing the an interface top the symbol table to the CPU model, but an interface to an expression evaluation module. This means the CPU code could evaluate expressions if it needs to, and that leaves me to deal with symbol table lookups inside the expression evaluation without having to worry about it. You'll still be able to write your own symbol table so that you can load funny symbol file formats, and you'll still be able to query for symbols but this time it'll go through the evaluation system instead.

I'll pinch the expression stuff from SNasm and put an interface around it, this means it'll handle anything SNasm could (which was a lot!). It also means that watches (with their complex expressions) should be a breeze to process, all I need to do is add the new [..] context to get memory from the target. Should be fun when its all working! I'm still not sure how fast its all gonna run, as getting lots of memory over a parallel cable isn't quick, still emulators should be very fast and at least you'll be using a common interface then!

I also realised that each module (ICPU, ISymbolTable and IComms) needs to be able to change some options, so I'll add a new interface that means it can pop up an options box and allow you to change things. For example, the CPU module might allow you to enable undocumented opcodes which aren't on by default.

So there you go! Its going pretty well so far! Once I get it to a basic level, I'll need to see if I can get some other coders to try it out and give me some feedback. of course, having only SNasm labels might hinder things a little, but then again it would be a good opportunity to have someone else write a module and tell me how easy or hard it was!