#
Notes on ITC'99 Benchmarks

##
Scott Davidson

These notes constitute the publicly available information on the ISCAS'99 benchmarks. Note that several of the benchmarks come from companies that do not want their identities known.

Where the benchmarks are publicly available, I give a link to the appropriate website. This is for informational purposes only - the benchmarks should be taken from the ITC'99 Benchmark website so that there is control of the versions used.

Direct questions about the benchmarks to me (scott.davidson@eng.sun.com). I will answer what I can. The donors do not wish to support these designs, and I have no special knowledge of their internals.

Some of these measures are rough. The exact number of gates is not very important when dealing with large circuits. Since many of the designs are expressed in terms of cells, the gate count will always be approximate, and will depend on the library used.

## Naming Conventions

I use the convention I99 to name the designs. The tag will give some indication of the type of design.

I99 means ASIC or other IC designs.

I99T means the set of benchmarks from Politecnico Torino. The refers to their numbering system.

I99S refers to the RTL version of the ISCAS89 benchmarks created by the University of Michigan. here stands for the numbering from ISCAS89. Other interesting but simple sequential benchmarks will be assigned numbers in this sequence.

I99C stands for potentially interesting combinational circuits. One from industry is provided in the initial release.

As of March 1999 not all of these circuits have been checked out. I am hoping for assistance from universities for this effort.

## THE BENCHMARKS

### I991:

This is a 5 - 6k Gate design, with 2 phase latches. The library is built out of simple primitives (NAND, nor, latches, etc.) and should be easy to model. There are no internal memories.

No scan is included in this design.

- Inputs: 120
- outputs: 250
- inouts: 14

Language: Structural verilog

Note: There is a 5 input mux driven by complementary enables. This is reputed to be fun to test.

### I992:

This is a pipeline ASIC of approximately 20,000 gates. It is an industrial design. The DFT strategy is ad hoc, there is no scan included.

The design uses a four phase clock. There are also several internal tristate busses. There are no internal memories.

The LSI Logic library was used, but a verilog library for all cells used is included.

A testbench and a test vector set exist for the design, and is available.

- Inputs: 31
- Inouts: 176
- Outputs: 88

Language: structural verilog.

Status: ready for release.

### I993:

This is an industrial ASIC, and includes embedded memories. It is represented in structural verilog, and uses the Mitsubishi library.

There are also several clock domains in this design.

This benchmark needs work, and is not yet ready for release.

- Inputs: 72
- Outputs: 32
- Inouts: 30

### I994 - The Torch Processor:

(see: http://www-flash.Stanford.EDU/torch/ )

Torch is an experimental superscalar microprocessor architecture, developed at Stanford.

Torch includes two integer execution units, a six-port register file, a floating point unit with register file, and instruction and data caches.

The processor is described in Verilog at the RTL level. I believe this processor was used as a simulation benchmark by a trade journal (I think ISD).

Layout information appears to be included, though I am not an expert and do not know the extent of the information.

I do not know the gatecount. However, this is a benchmark of significant size and complexity, implementing a superset of the MIPS R2000/R3000 instruction set.

A set of test vectors and simulators are included. VCS was used for the simulation.

No scan is included.

Status:

Ready to be downloaded, as there are no proprietary information issues. However, we need someone to produce a baseline gate level implementation. However, this seems an excellent test case for RTL level tools, as there are many RTL level modules to work on, including, at the top level:

- cp0: Co-processor 0
- decexec: Execution Unit
- extint: External Interface
- fp: Floating Point Unit
- ifetch: Fetcher/Instruction Cache
- ldsto: Load Store Unit
- regfile: Register File
- system: Top level module including the L2 cache and a gate array to service non-cacheable requests and magical locations.

### I995 - CMU DSP:

The CMU DSP is a DSP modeled after the Motorola DSP56002 processor. Some features have been removed to make it not commercially viable.

The CMU DSP is available in structural Verilog, and in a partially functional behavioral level Verilog. There is a gate level version also, synthesized from the structural RTL. This last has not been thoroughly verified yet, but it should be more than adequate for test work. Test vectors are included also. The distributors of this benchmark are sure that there are at least a few bugs left in the design.

The CMU DSP does have scan inserted in the structural Verilog. We will need a volunteer to remove it.

The CMU DSP is available at: http://www.ece.cmu.edu/~lowpower/benchmarks.html

A presentation on the CMU DSP is also available.

This includes the documentation, which will be also available on the benchmark website. Since we will be modifying the benchmark, and because the CMU version is still in progress, benchmarkers should use the version on the ITC99 Benchmark website.

The gate count is approx. 14,550. It has internal memories, and data and control paths, and thus is a realistic design.

Version 1.0 was due in December, but the documentation has not been updated since November, and is incomplete.

Status:

The code is freely available from the web site, and can be looked at before the ITC'99 benchmarks are released. Unfortunately a proprietary library is used in the RTL (Epoch), and we must see if we can technology map to our standard library.

### I99T<1-22> - Torino Benchmarks:

(Information available on the ITC'99 benchmark site.)

These consist of 22 RTL level sequential benchmarks, ranging from 45 gates and 5 FFs to 98,000 gates and 6600 flip-flops. Some of the larger benchmarks are composed of several of the smaller benchmarks connected together. They are all simple in the sense of being single clock designs, without internal memory or tristate busses.

They are useful in testing core ATPG algorithms on larger designs.

A table of these benchmarks, and supporting documentation, appear below:

####

###

###

          +-------+------+----+----+-------------+----------+---+
          | gates | #FF  | PI | PO | #VHDL lines | #process |   |
    +-----+-------+------+----+----+-------------+----------+---+
    | b01 |    45 |    5 |  4 |  2 |     110     |     1    | r | 4_10 (9293)
    +-----+-------+------+----+----+-------------+----------+---+
    | b02 |    25 |    4 |  3 |  1 |      70     |     1    | r | 4_isbcd (9293)
    +-----+-------+------+----+----+-------------+----------+---+
    | b03 |   150 |   30 |  6 |  4 |     141     |     1    | r | arbitro (9293)
    +-----+-------+------+----+----+-------------+----------+---+
    | b04 |   480 |   66 | 13 |  8 |      80     |     1    | r | minmax (9394)
    +-----+-------+------+----+----+-------------+----------+---+
    | b05 |   608 |   34 |  3 | 36 |     319     |     3    | r | elab_ram (9293)
    +-----+-------+------+----+----+-------------+----------+---+
    | b06 |    66 |    9 |  4 |  6 |     128     |     1    | r | gest_int (9293)
    +-----+-------+------+----+----+-------------+----------+---+
    | b07 |   382 |   51 |  3 |  8 |      92     |     1    | r | retta (9293)
    +-----+-------+------+----+----+-------------+----------+---+
    | b08 |   168 |   21 | 11 |  4 |      89     |     1    | r | ric_incl (9293)
    +-----+-------+------+----+----+-------------+----------+---+
    | b09 |   131 |   28 |  3 |  1 |     103     |     1    | r | 67246 (9495)
    +-----+-------+------+----+----+-------------+----------+---+
    | b10 |   172 |   17 | 13 |  6 |     167     |     1    | s | ostraco (9495)
    +-----+-------+------+----+----+-------------+----------+---+
    | b11 |   366 |   30 |  9 |  6 |     110     |     1    | r | 77449 (9596)
    +-----+-------+------+----+----+-------------+----------+---+
    | b12 |  1000 |  121 |  7 |  6 |     567     |     4    | s | simon (9596)
    +-----+-------+------+----+----+-------------+----------+---+
    | b13 |   309 |   53 | 12 | 10 |     296     |     5    | s | meteo (9495)
    +-----+-------+------+----+----+-------------+----------+---+
    | b14 |  3461 |  247 | 34 | 54 |     518     |     1    | f | viper
    +-----+-------+------+----+----+-------------+----------+---+
    | b15 |  6931 |  447 | 37 | 70 |     648     |     3    | f | 80386
    +-----+-------+------+----+----+-------------+----------+---+--------
    |     |    30 |    4 |  5 |    |             |     4    |   | 4_2
    |     +-------+------+----+    |             +----------+   |
    |     |    35 |    4 | 11 |    |             |     4    |   | 4_2_e
    |     +-------+------+----+    |             +----------+   |
    |     |    57 |    8 |  6 |    |             |     8    |   | 8_3
    |     +-------+------+----+    |             +----------+   |
    |     |    78 |    8 | 27 |    |             |     8    |   | 8_3_e
    |     +-------+------+----+    |             +----------+   |
    |     |   106 |   16 |  7 |    |             |    16    |   | 16_4
    |     +-------+------+----+    |             +----------+   |
    |     |   170 |   16 | 67 |    |             |    16    |   | 16_4_e
    |     +-------+------+----+    |             +----------+   |
    |     |   140 |   16 |  7 |    |             |    16    |   | 16_4n
    |     +-------+------+----+    |             +----------+   |
    | b16 |   147 |   16 |  8 |  1 |      68 @   |    16    | n | 16_5n
    |     +-------+------+----+    |             +----------+   |
    |     |   123 |   20 |  7 |    |             |    20    |   | 20_4
    |     +-------+------+----+    |             +----------+   |
    |     |   211 |   20 | 83 |    |             |    20    |   | 20_4_e
    |     +-------+------+----+    |             +----------+   |
    |     |   169 |   20 |  9 |    |             |    20    |   | 20_6n
    |     +-------+------+----+    |             +----------+   |
    |     |   180 |   26 | 11 |    |             |    26    |   | 26_8
    |     +-------+------+----+    |             +----------+   |
    |     |   249 |   28 | 11 |    |             |    28    |   | 28_8n
    |     +-------+------+----+    |             +----------+   |
    |     |   197 |   28 | 19 |    |             |    28    |   | 28_16
    |     +-------+------+----+    |             +----------+   |
    |     |   233 |   30 | 33 |    |             |    30    |   | 30_30
    |     +-------+------+----+    |             +----------+   |
    |     |   508 |   64 | 19 |    |             |    64    |   | 64_16n
    +-----+-------+------+----+----+-------------+----------+---+
    | b17 | 21191 | 1407 | 38 | 97 |     135 *   |     6    | n |
    +-----+-------+------+----+----+-------------+----------+---+
    | b18 | 49293 | 3308 | 37 | 30 |      94 *   |     1    | n |
    +-----+-------+------+----+----+-------------+----------+---+
    | b19 | 98726 | 6618 | 47 | 40 |      65 *   |     2    | n |
    +-----+-------+------+----+----+-------------+----------+---+
    | b20 |  7741 |  494 | 34 | 22 |    1040     |     3    | n |
    +-----+-------+------+----+----+-------------+----------+---+
    | b21 |  7931 |  494 | 34 | 22 |      63 *   |     1    | n |
    +-----+-------+------+----+----+-------------+----------+---+
    | b22 | 12128 |  709 | 34 | 22 |    1547     |     4    | n |
    +-----+-------+------+----+----+-------------+----------+---+
    							  |
    @ uses generate						  >- r : reti
    * uses components					  >- s : spa
    							  >- f : ftp
    							  >- n : new

    Notes:

     B16

    'b16' is a multiple benchmark where the general structure is a shift register.
    The basic cell is a D flip flop (i) that may be loaded with the value from
    the i-1 flip flop ('pi' if i=0) like a common shift register if 'sel'
    assumes a particular value. In 'n' benchmarks (e.g. 16_4n), the flip flop may
    be loaded with the value of the last flip flop in the chain with another 'sel'
    value to obtain an 'X' loading. For the others combinations of 'sel' the flip
    flop maintains its value.
    The circuits structure is one of the following:

     a. N_M

     b. N_M_e

     c. N_Mn

    where N is the number of cells (flip flop) in the chain and M is the number of
    wires for 'sel'. In the first and in the third case, the same 'sel' bus is
    shared by all the cells, while in the second case every cell has its own 'sel'
    bus.

     B17

    'b17' is the first benchmark built up by interconnecting smaller circuits to
    obtain benchmarks with a certain dimension.

    'b17' is composed by 3 'b15' communicating each other by two data register. The
    first and the third also communicate with the external world.

    B18

    'b18' is composed by 2 'b14' and 2 'b17'. Two groups b17-b14 are cross connected
    on datapath. The 2 b17-datain are the 2 b14-dataout mixed with the b18-datain.
    The address buses are mixed by mux and xor ports to the b18-dataout.

    B19

    'b19' is composed by 2 'b18' cross connected on datain-dataout buses. The input
    is mixed with internal buses to obtain the 'b18' inputs. The output is the mix
    of the 'b18' outputs.

    B20

    'b20' contains two not completely identical copies of the process in 'b14'.
    The minimal differences are some swap between '+' and '-' and some swap
    between choices in cases. In this way, the 2 processes generate different
    netlists after the synthesis.

    B21

    'b21' follows the same philosophy of 'b20', but the 2 netlist from 'b14'
    are generated with different parameters used during the synthesis.

    B22

    'b22' is like 'b20' but it contains 3 processes 'b14'.

    Each benchmark is delivered as RTL level VHDL and gate level EDIF.

    A presentation on the benchmarks can be found  here.

    The benchmarks are available directly from  their website, and will be mirrored at the ITC99 website.

    Status:

    These benchmarks are ready for use.  I have run one as an experiment
    on a commercial ATPG with good results.

     I99S - RTL Versions of ISCAS85 and ISCAS89 benchmarks from University of
    Michigan:

    Mark Hansen, when at the University of Michigan, derived RTL versions
    of some of the ISCAS benchmarks.  These can be found at  their website.

    There are now 4 high level combinational benchmarks and three high
    level sequential benchmarks, s208.1, s298 and s344/s349.

    They have all the flaws of the original benchmarks - single clock,
    simple, and unrealistic, but are useful in comparing high level and
    gate level ATPG, and in testing algorithms.

    Status:  Ready to download.

    I99C1 - An interesting combinational benchmark:

    This benchmark is extracted from an industrial design, and is said to
    cause problems for industrial ATPGs.

    The netlist is provided as gate level Verilog, using a generic
    library.  A set of vectors is also provided.

    Status:

    I have run the circuit on an industrial ATPG, getting very low fault
    coverage.  I have not yet verified my ATPG library with the provided
    vectors, so the low fault coverage might be a bug.  I hope to do this
    by the time of the release, but others are welcome to help.

    Home