The rubber hit the road

I've written elsewhere about me acquiring a PCW8256 machine. This was in support of this project: if I wanted to test my T80 based UnIC design in real HW, well, I need real HW don't I. I have a Timex Sinclair 1000 (a.k.a ZX81), but that's such a difficult and peculiar machine that I didn't feel like starting on it. I wanted something that can run CP/M too if for no other reason, to be able to run ZEXALL for real.

So, with the machine at hand, I started getting ready for the exercise.

I got these 40-pin (for lack of a better word) things from Vetco:

They were cheap, and I suspect, they were designed to attach to ribbon-cables on one side and to a DIP40 socket on the other. That's not what I was going to do with them thought. Instead, I got some ZIFF sockets from Amazon, then soldered one on top of these things. I also added pin headers on the left and right side for easy probing.

Being done with this, I removed the Z80 processor from it's socket, and put the ZIFF contraption in between. Booted, tested all was fine.

Next, I created a converter that took my UnIC devboards' 40-pin header and turned it into a 40-pin DIP of sorts:

Plug this whole train together (UnIC board - adapter board - ZIFF socket - motherboard; oh and a JTAG programmer for good measures) and the hard part of the work is done. Right?

Well, not so fast. I needed to create a variant of my core that didn't have any of the extra bells and whistles I had added for testing: no internal SRAM, no UART, nothing. Just a plain vanilla T80 core properly pinned out to the external world. Also, no internal clock; let's start using the true clock pin. After all this, I was finally at the point where I could:

1. Program the UnIC board with the right core through JTAG
2. Plug the UnIC board into the PCW
3. Flip the power switch and...

See absolutely nothing.

What gives?

Obviously some debugging is needed. And that starts with visibility into what's going on. In the case of digital electronics, the primary tool for that is a logic analyzer.

I happen to have one, but hooking it up is a pain, so is there something better? Most FPGA vendors give you cores that create a logic analyzer inside the FPGA. This not only has the benefit of being free, integrated and easy to set up, but also that it has access to the internal signals of the logic inside the FPGA. Intel (Altera) has one, so does AMD (Xilinx). GoWin, the manufacturer of my choosing does too.

Setting this core up was relatively simple, though the documentation could be better. Especially around trigger conditions, there are some rough edges. Making their SW work though was much more difficult.

As it turns out, one has to give write-access to /etc/modules.d to the process so that it can blacklist the FTDI driver that doesn't get loaded on modern Linux systems anyway. And it's not even that they gracefully fail if they can't edit the file or check whether they need to do it. Grrr...

Anyway, logic analyzer at hand, the chase can begin!

Reset

Well, almost. To get repeatable results, I need to be able to reset the target. For that, I need a GPIO I can control. Luckily, the FTDI part I'm using for JTAG is a dual-channel device, so the second serial port should be usable for this purpose, right? Right? No. As it turns out - probably due to the previously mentioned juggling of the device drivers - I can't talk to the second port while the logic analyzer is running, at least not without disturbing its piece.

I ended up requisitioning a second FTDI part just for this purpose. This of course has the side-effect that now there's some ambiguity in the system as to which FTDI part is the one with the JTAG interface and which one is the one pulling the reset.

But at least it worked, and I managed to get on to capturing signals

First boot

This is my first-ever capture of signals from the logic analyzer (and here, I hooked up probes to the Z80 pins):

Not much to see, except that the execution is very slow, due to the "N_WAIT" signal being asserted for a long long long long time. At least, the wait-state detection logic works! Moving on, it appears that the first few bytes of the boot code did get read. Due to the extremely stretched out execution though there's not much else visible.

I've written up about the boot process of the PCW elsewhere, so I won't repeat myself. Suffice to say, that there's a reason for all these wait-states. The boot process from a SW perspective is documented here: http://www.chiark.greenend.org.uk/~jacobn/cpm/pcwboot.html.

Capturing some signals later in the boot process, I started to see something strange:

The M1 cycle doesn't end with MREQ_N going inactive and flows straight into a refresh cycle. This probably confuses the hell out of the memory controller. Certainly not cool. According to the datasheet MREQ_N should go high for half a clock-cycle between the M1 and the refresh cycle:

Could I have missed that in previous validations in the simulator? Or in the FPGA? I've fired up the simulator and re-run some tests:

Nope, it's there. Even with forced external wait-states, similar to what the real system experiences. So, what on Earth?

Some digging surfaced the following line to generate 'MREQ':

MREQ_rw <= MREQ and (Req_Inhibit or MReq_Inhibit);

Tracing these signals confirmed that one of the signals is stuck high:

That explains why the half-cycle de-assertion is missing, but not why the signal is stuck high. Especially because both 'Req_Inhibit' and 'MReq_Inhibit' is generated exactly the same way, except on different clock-edges:

-- Generate Req_Inhibit
   process (Reset_s,CLK_n)
   begin
      if Reset_s = '0' then
         Req_Inhibit <= '1';  -- Charlie Ingley 30/10/19 - changed Req_Inhibit polarity
      elsif CLK_n'event and CLK_n = '1' then
         if MCycle = "001" and TState = "010" and WAIT_n = '1' then  -- by Fabio Belavenuto - fix behavior of Wait_n
            Req_Inhibit <= '0';
         else
            Req_Inhibit <= '1';
         end if;
      end if;
   end process;

-- Generate MReq_Inhibit
   process (Reset_s, CLK_n)
   begin
      if Reset_s = '0' then
         MReq_Inhibit <= '1'; -- Charlie Ingley 30/10/19 - changed Req_Inhibit polarity
      elsif CLK_n'event and CLK_n = '0' then
         if MCycle = "001" and TState = "010" and WAIT_n = '1' then  -- by Fabio Belavenuto - fix behavior of Wait_n
            MReq_Inhibit <= '0';
         else
            MReq_Inhibit <= '1';
         end if;
      end if;
   end process;

And why it would work in simulation, but not in real life? Anyway... Since they are generated the same way, they could be generated in a simpler manner: one could be a half-cycle delayed version of the other. Not only would that work (fingers crossed) but would be ever so slightly less resource-hungry. I made the change, re-compiled, and...

The notch is back! Not only that, but suddenly the CRT turned on and started glowing green. So, progress? Maybe, but there are at least two conundrums here:

1. Why didn't the original code work?
2. What else doesn't work due to similar problems?

I have to remind you here that I had FPGA validation running in the same device (in fact, same board!) with built-in SRAM and ROM emulation. I had some confidence in the design.

Repeatedly resetting the board resulted in some strange beeps and some corrupted display being shown. Sometimes. Sometimes, nothing. Again, progress? But why the irreproducibility?

Chasing Ghosts

At this point, I decided to whip out the ‘big gun', that is, the HP1660 logic analyzer I have. The one problem with it was that I never got around getting its Ethernet port hack finished.

So I did that, put it back together, futzed around a lot to get X11 working (I did), then hooking up all 40 pins to probes in it, then turning everything back on and … nothing worked.

The CPU didn't execute anything. Or executed a few instructions after which decided to go berserk; the PCW ASIC issuing incorrect wait states, feeding uninitialized memory to the CPU (instead of the boot ROM content).

I ripped off all the LA cables, same behavior. I've put the old (genuine Z80) processor back in, it booted (phew, at least I didn't kill the host). Back to my T80, nothing.

After a while I realized: most of what changed was plugging and unplugging things: could it be that I have a faulty connection? Rechecking the soldering job, I quickly have found a pin that wasn't soldered in at all. Fixing that, I was back where I had been.

It's soooo easy to waste hours on something trivial like this!

That of course didn't get rid of all the ghosts, only chased away the ones that crept up recently. The unpredictable boot behavior and the whole business about not booting in general remained.

At this point, I'm going to save you the details. The short of it is that I spent hours chasing failure after failure, trying to capture traces just to find the symptom change every time I thought I got close to understanding what was going on.

After I sufficiently exhausted both myself and my ideas, I was left with a few explanations:

Option 1: I have signal integrity issues
Option 2: I don't have memory refresh
Option 3: Ground bounce
Option 4: ????

Option 2 can be removed from the list immediately: refresh is generated by the ULA, not by the CPU in this system. There's no difference between the original Z80 (working) and the UnIC based (broken) system in that regard.

So noise or ground-bounce. Given that the PCB I've made is only a two-layer one, that grounding is done through diodes and that in general signal-integrity was the least of my concerns when laying out the board, both are equally likely.

Out with the scope! I quickly determined – and I'm sorry, I've lost the scope-captures in the shuffle – that indeed the signals were rather noisy.

To understand the next discussion, it's important to know how power is delivered to UnIC. Here's the simplified schematic for the relevant section:

You can see two diodes, D1 and D2 which separate power (and ground) as seen by the external world and by the internal FPGA. For now, the important one is D1. There is some voltage dropping on this diode. This drop is current-dependent, in other words, the internal FPGA GND is 'bouncing around' relative to the ground of the host system.

To check if ground-bounce could be an issue I simply connected FPGA GND to PCW GND. This was done with a simple jumper wire which is not by any means ideal. Reset, and... The machine went through the whole boot process. Bars and all! It didn't get all the way to CP/M, but oh, boy, did it make a difference!

Granted, not exactly working, but ... not dead either.

At this point I decided that I'm probably dealing with enough signal-integrity problems that the development board is hopeless. I had to wait for the form-factor build - still making its way though the build process - to show up.

Tests in form-factor builds

I have the timeline jumbled up a bit here, but I have written about my trials and tribulations of getting the form-factor board behaving like I intended them to before.

With that out of the way, I was ready to pick up the pace. Initial tests showed instability still. Shorting FPGA GND to PCW GND improved things dramatically: I still couldn't get all the way into CP/M, but much closer. The screen corruption gave way to a hang in the PIP utility. However, I could boot into pretty much anything else: LocoScript, Batman, even SymbOS works just fine:

Plane vanilla CP/M also boots, it's only PIP that throws a fit. So, at this point, I seem to have two problems:

1. Signal integrity problems still; something that can be remedied this time by shorting the ground-diode out
2. Some problem that hangs PIP.

II'm going to work under the assumption that these are unrelated and the PIP problem is a real HDL issue that I will have to – sigh – somehow track down.

But, let's see if we can make forward progress on number one! The working hypothesis is that I have ground-bounce problems.

Ground-bounce happens when an output (or a lot of outputs) change state, dumping a lot of current into the ground pin. The impedance of the ground connection converts this current into a voltage drop, ‘lifting' the reference voltage of all pins up (or down) compared to true ground. This in turn changes the output voltage as seen by other circuitry or the voltage seen by inputs of the chip. If the change in reference is severe enough, it can cause zero-s to be read as one, or the other way around.

The impedance of the ground pin can come in many forms (mostly inductive or resistive). Lets consider the easier one: resistance. In my particular case, I'm also mostly worried about elevated ground voltage, so currents flowing into the ground pin. When can that happen?

If an output pin drives an external pull-up resistor, the pull-up current will flow through the ground-pin of UnIC. How bad can that be? Well, it depends of course, but in my particular case, all data pins, A7, MREQ_N, IORQ_N, M1_N and RD_N have external 10k pull-ups. If the CPU tries to write 0x00 somewhere (which is not all that rare a thing to do), it would need to override nine of these lines (maybe 10 if A7 happens to be low as well). To drive a 10kOhm resistance from 5V to 0V, one would need 0.5mA. This is the current draw of each of the potentially 10 outputs, so the total static current is 5mA. This 5mA (as we'll soon see) is converted to ~14mV drop on the ground diode, a BAT54. That doesn't seem all that much: if that much extra noise throws anything off, we were already in trouble without it.

The dynamic currents are more concerning: for a short time, when changing a wire from 1 to 0, one needs to discharge the associated wire (as well as the input capacitance of all the chips connected to it). During this time, the current is mainly limited by the drive-strength of the driver, which I set to 4mA. But how many can change at the same time? Let's say it's half of the address pins as well as half of the data pins; a total of 12. This results in a brief current spike of 48mA (let's round it up to 50mA). That corresponds to about 140mV of extra ground bounce. That is high enough to be potentially problematic.

This is the time, I decided to actually look into the I-V curve of my BAT54 diode. Remember, this is a schottky diode, so it's forward drop should be around 0.2V, at least for low currents.

Hmm... This is not nice. Not nice at all! Sure, it starts off at around 240mV for very low currents, but after that, it's pretty much linear. I should have been alerted to this earlier, when I had power supply troubles, but alas, better late then never. Granted, I don't draw 200mA through this device, but I could easily see 0.5V drops at the ~~60mA mark. Looking at this curve, now I understand why.

Lets compare this curve to what a well-behaving TTL logic should be doing:

The low output level should be lower than 0.4V, but it better as hell be below 0.8. These voltages correspond to to 32mA and 192mA currents. Adding some dynamic currents to the mix, it's not inconceivable that we're violating TTL specs. No wonder then that shorting out the diode should improve things.

You might be wondering why I didn't just attach probes to the diode and measure things earlier. Well, because I was dumb, that is why. I wasn't thinking about this as a problem before and now that I'm convinced that the circuit will need to change measuring it doesn't make all that much of a difference.

Finding a new diode

Changing the circuit I must, but in what way? I think what I need is a different, better diode: something that stays closer to what a schottky diode should look like. Something like this:

Of course there was a reason for picking the BAV54: it was available in a very compact form-factor. In other words not only do I really need to find a better diode, I need to find something that comes in a very small package. The best so far is the SDM1L20DCP3. Unfortunately I don't have a simulation model for it, but the forward voltage drop spec is in the datasheet:

Concentrating on the 25C curve, we see it crossing the 400mV mark at 0.5A. Plenty. It is a dual diode in a 1006 package (that is 1.0x0.6mm). Still significantly larger then what I had (more expensive too), but not impossible to imagine to squeeze on the board.

The next step is to prototype the solution by using some other, similarly spec-ed diode, in a package that I can manage with my aging eyes and hands. I have a bag of the

MBRS140s.

It's a huge part, but as far as forward voltage drop goes, it's almost identical:

For this exercise, I only need to replace one, the one on the ground pin. Here's what the board looks like with the new diode bolted in place:

But does it work? Yes, yes it does! It works just as well as the shorted ground variant: all but PIP are happy.

This time I did take scope-captures of a few things:

Here you see the voltage drop across the diode in yellow, the 4MHz clock in blue and the address pin A0 in purple. The scales are different, but there are a few interesting things to note:

1. There is about 200mV of static drop across the diode
2. This drop doesn't move all that much around, maybe 20mV. Yet, it does move around and it's movement is tightly correlated with the clock, showing that the changing load does make a difference.
3. There is significant noise coupled into both the ground and the signal pins. These too are correlated to the clock edges, so they are probably dynamic load changes generating transients. The fact that they are not identical on the data and the ground (they would need to be 10x of one another on the picture due to the different scales) though seem to point to a different coupling mechanism then ground-bounce. Sure, some of it might be that, but most of it can't.

This third point is validated by swapping in the original Z80:

You see similar amount of noise in similar places on the same signal pin. Of course, in this case the internal and external ground cannot be measured separately, so the yellow curve just flat-lines.

I'm not even sure how much of this noise seen on the purple curve is real. It could be probe grounding, which I have to admit, I wasn't terribly careful about here.

Is that it? It certainly seems that replacing the diodes with something that's better at being, well, a diode certainly helps. It pulls the circuit operation within spec and remedies the problems I was having with signal quality.

Implementing it the proper way of course is a board re-spin, but maybe I can kludge something together. Before that though, let's verify one more thing: the reason for the diode I've used here (and selected as a replacement) have so much lower forward voltage drop is that they are high-current devices: rated to 1A currents, not the 0.2A of the BAT54. High current comes at a cost though: increased capacitance. This of course is not a problem for the power pins (which are forward biased), but could be for all the other signal lines.

The BAT54 has a capacitance of 10pF, the SDM1L20 has 18pF. Since I have two connected to each data-pin, that doubles to 36pF. Compare that to the 5…15pF specified in the Z80 datasheet. High, but is it too high? I don't think this is going to be all that problematic: a 36pF capacitance can be discharged from 3.3V to 0 by a 4mA current (the lowest drive-strength selectable on the FPGA) in about 2.5ns. This, compared to the 30ns rise- and fall-times of a 4MHz Z80 is not substantial.

Lessons learned

This was an unexpected trip. I was prepared to see problems, just not of this kind. I didn't expect schottky diodes to be so much different from one another and simply neglected to check their forward-voltage I-V curves before the design. That could have saved a lot of grief and money.

There certainly were other signal-integrity problems on the dev-board version as well, something I could have eliminated by not being cheap and going for a 4-layer PCB. That way I could have realized the ground-bounce problems and saved the pain of going through a form-factor build with incorrect part selection.

On a more positive note: it seems I can make forward progress with the board I have: I have a way of applying a fix that is a little less universal then I hoped for (i.e. the new diode I need has to be on the GND pin), but otherwise representative of what the next iteration will look like. It allows me to work on finding what PIPs problem is and finalizing other aspects of the board.

A word from our sponsor

I used PCBWay for manufacturing both the PCB and - this time - putting the components on the board. They are a contract-manufacturing shop in China, specializing in prototype and low-volume production builds. Their main competence is PCB and PCBa manufacturing, but they handle all manners of other activities, including 3D printing, stencil creation, CNC machining and injection molding. In my experience they make high quality products, so - even if they weren't sponsoring me - I would gladly recommend them.