A Testbench Race I Should Have Caught

Every so often a reader reports that the testbench for Tutorial 19 fails, with the comparisons coming out one sample off. Back in 2018 I said “nobody has reported this before” and asked which version of Vivado they were using. Recently a reader named detiber found a workaround, and when I finally sat down and looked at it, the bug was mine. It’s a nice little example of something I wrote about years ago in Tutorial 3, so I’m going to own up to it here.

The code

The first I2S sample out of the transmitter is junk, because the I2S clock is already running before any data shows up. So the testbench waits until it has sent some data, waits for the start of a left channel word, and then turns on the output checkers.

  event enable_output_checking;
  reg output_checking_enabled = 0;
  initial
    begin
      @(enable_output_checking);
      @(negedge ws);
      repeat (2) @(posedge sck);
      output_checking_enabled = 1;
    end

The checkers are ordinary always blocks clocked on the same sck. Here’s the right channel one, trimmed down:

  always @(posedge sck)
    if (output_checking_enabled && !wsd[0] && wsp)
      util.check(shift,expected_right.dequeue(0)&mask);

The race

Two rising edges of sck after ws falls, the word boundary pulse wsp is still high. On that edge two blocks wake up: the initial block, which sets output_checking_enabled, and the checker, which reads it. Verilog doesn’t say which one runs first.

The ws signal changes on the falling edge of sck, and the testbench’s little I2S receiver delays it through wsd on rising edges, so wsp is a one clock pulse at each word boundary. With repeat (2) the flag is set on the second rising edge, the same edge where the checker still sees wsp high.

  • If the checker runs first, it sees the flag as 0 and skips. The first comparison is the left word, which is what I intended.
  • If the initial block runs first, the checker sees a 1, compares the junk right channel word, and pulls an entry off the expected queue. From then on every right channel comparison is one sample behind.

The simulator I used in 2015 happened to pick the first order. Newer versions of XSim evidently pick the second. Neither one is wrong; my testbench was.

To prove it, I forced the bad ordering in Icarus Verilog by setting the flag a nanosecond before that edge instead of on it. The original testbench went from passing with 797 comparisons to failing with 398 errors out of 798, and every received word matched the previous expected word. That’s exactly what readers were reporting.

The fix

detiber’s workaround was to change repeat (2) to repeat (3). That works: by the third edge wsp is low, so the checkers can’t fire no matter who goes first. But it steps around the race rather than removing it. The real fix is one character:

      output_checking_enabled <= 1;

With a nonblocking assignment, every block triggered on that edge sees the old value, and the new value shows up after the edge. Simulation order no longer matters.

The lesson

This is the same rule from Tutorial 3, and it applies to testbench code just as much as RTL: if a value is written on a clock edge and read by another block on the same edge, use a nonblocking assignment. A testbench that only passes because of the order your simulator happens to pick is a testbench that will break when you upgrade.

The fixed i2s_transmit_test.v is in the tutorial repository. Thanks to detiber and Zdzislaw for reporting it, and sorry it took me so long.

Putting New Files in the Right Place: The Vivado Edition

Vivado has the ability to create and manage your own IP, which is a good thing. But there are some real gotchas when doing something like this, the first being that it doesn’t seem to place HDL files in the correct locations.

I have been running some FPGA workshops using the IP Packager in Vivado and having been having a lot of problems when creating new files in the IP component. Here’s an example:

From a Vivado project select Tools→Create and Package IP… Click Next, select Create a new AXI4 peripheral, click Next, just use the defaults for the next page, click Next again, use the defaults for the AXI information and click Next again, select Edit IP, then click Finish.

This will put in an IP Packager project. Edit the top level file myip_v1_0. I’m using Verilog but it works the same for VHDL. Instantiate a new dummy module at the bottom of the file:

// Add user logic here
dummy dummy (.clk(s00_axi_aclk));
// User logic ends

files_fig1Save the file and look at the design sources. You will see that there is now a new undefined module in the design called dummy. It should look like the figure to the right:

Now, we would like to create the new module. So we go to File→Add Sources… and create the new source file. Select Add or create design sources and click Next. Click Create File …, type the file name dummy and click OK. Click Finish. When the Define Module dialog pops up, just click OK. Click Yes to the dialog saying you haven’t changed anything. You now have a file defined for your dummy module.

files_fig2Ready to package that IP? Go to the Package IP tab in the Project Manager window. Then, select File Groups in the Packaging Steps window. You should see a window like the figure on the right.

Now click on the Merge changes from File Groups Wizard link. You will then get a link saying 2 warnings. Click on that, and you will see two warnings about your dummy.v file being on a path which is not relative to your IP root directory. What has happened is that Vivado placed the new file outside the IP component. If you go ahead and package your IP, you cannot use the packaged IP because it will be missing this file. Seems like a Vivado (2014.4) bug to me.

So how should we fix this? I don’t know how to move files in Vivado, but we can create the file in a different location when we make the new file. To do that, first delete the dummy file and remove it from disk. Just select the file, then right-click and select Remove File from Project. Select the checkbox to also delete it from disk and click OK.

files_fig3Now go to Add Sources again, and after clicking Create File…, specify the file name dummy.v, then select a new file location. files_fig4Navigate to your ip_repo area and into myip_1_0. Select the hdl directory, and click Select. Then click OK and then Finish to create the file. This will place the file in the proper location in the IP repository.

When you merge your changes in the File Groups step, you should not get any warnings. You can then package your IP and use it in your designs.

Hopefully this will be fixed in the next release of Vivado.