Collection of Digital Design Benchmarks

Comprehensive set of logic design and optimization circuits


Description

A comprehensive set of circuits has been formed by combining circuits from different benchmark sets. By putting these circuits together, a more comprehensive, but unified and well-arranged set of example circuits was formed, from which a user can select circuits (or the whole benchmarks) upon his wishes and needs. The collection comprises of several sets, which, even though they sometimes contain the same circuits, are customized to particular needs of the user.

Performed transformations

These steps have been done to construct the collection:
  1. Circuits from all benchmark sets were converted to the BLIF format. Then all the circuits were accumulated, while their origin (original benchmark name) was indicated by the file name (see the naming convention). Circuits that could not be converted to BLIF without a major loss of information, e.g., those described as finite-state machines (FSMs) in the KISS format, multi-valued descriptions (MV-PLA), etc., were removed from the set.
  2. These circuits were read and written back by ABC. By this, several actions have been automatically done:
    a) circuits incorrectly converted to BLIF were removed from the set (this was especially the case of some Altera Cookbook circuits),
    b) extremely large circuits were removed, since ABC was not able to process them (this was the case of some large EPFL circuits),
    c) circuits with combinational feedback were removed,
    d) hierarchical descriptions have been flattened,
    e) blackboxes that appeared during the conversion to BLIF were removed,
    f) dangling gates were removed,
    g) circuits having external don't cares have been removed, since ABC does not support them.
  3. The ABC command 'short_names' was applied. As a result, a uniform signal naming convention was introduced to all circuits. Even though this may seem to be useless and even unwanted, it helps to avoid problems with signal names that are not supported by some other tools, or they are invalid identifiers in VHDL.
  4. The circuits were processed by the 'sweep' command using ABC or SIS. By this, unnecessary buffers and inverters were removed, and constant signals propagated. Here the circuits are distinguished by their origin; some circuits were processed by ABC, some by SIS. The reason for this ambiguity was that ABC directly produces "negated nodes", where found advantageous. This may completely hinder the structure (and purpose) of some circuits described by a PLA.
  5. The circuits were "cleaned" by our custom tool. Particularly:
    a) constant and void outputs were removed,
    b) primary outputs fed by a primary input only were removed, as well as primary inputs feeding a primary output only.
  6. Where applicable, each circuit was split into connected components. Especially this is the case of sequential circuits from which combinational parts were extracted, see the following transformation.
  7. Combinational parts of the circuits were extracted by the ABC command 'comb', making them combinational ones with pseudo-primary inputs/outputs. In this step, the circuit may break into several unconnected components, which are extracted by the step mentioned above.
  8. Small circuits are filtered out. Sometimes it happens that circuit parts obtained in Step 6) are too small, e.g., one gate only. Such circuits are thus useless for purpose of experimental evaluation. Next, the user may decide to use only circuits bigger than some threshold. Therefore, there are offered sets with several size thresholds (5 and 50 gate equivalents).
  9. When circuits from different benchmark sets are combined, one particular circuit may appear more than once in the mixed set. Therefore, three equivalence checking procedures were performed:
    a) File equivalence – exactly equivalent circuit descriptions were detected.
    b) Structural equivalence – the circuits were processed by the ABC command 'strash', by which semi-canonical AIGs were produced, and the results were checked for equivalence. Therefore, structurally equivalent circuits were detected. Obviously, the a) class of equivalence is a subset of this class. Note that the 'strash' command actually introduces some structure. Thus, it may happen, e.g., that a circuit originally described in a PLA format, will be found equivalent to a multi-level netlist, when this netlist was directly produced from the PLA by ABC. However, multi-level, structurally significantly different descriptions will not be identified as equal.
    c) Functional equivalence – functional equivalence of circuits was checked by the ABC command 'cec'. Thus, the result is independent of the structure. Obviously, the b) class of equivalence is a subset of this class.
    The respective equivalences are recorded in text files and equivalence matrices (a symmetric rectangular matrix of dimensions [n, n], where n is the number of circuits in the whole set. A '1' entry at the position [i, j] indicates equivalence of the i-th and j-th circuit).
    There are subsets of the original circuit set available, where there is only one representative of each equivalence set present (alphabetically the first one). Thus, it is upon the user whether he wishes to use the complete set with possible duplicities, or a set with duplicities removed.
  10. The circuits were converted to structural VHDL.

Circuit Naming Convention

The original benchmark set is indicated in the circuits" names as a prefix.
As described in step 6), the circuits were split into connected components, thus into several files. The ordinary number of the component is indicated by a suffix.
As a result, the circuit"s naming convention is as follows:

benchmark__name__partnumber.extension

For example, the circuit pair coming from the LGSynth"91 benchmark set was split into two components, which were named as (in the BLIF format):

LGSynth91__pair__part0.blif
LGSynth91__pair__part1.blif

Circuit Origin

Prefix Description Sweep
Cookbook Circuits from Altera Cookbook ABC
EPFL EPFL benchmark ABC
Espresso Espresso examples SIS
Generic Artificially generated generic structures ABC
Illinois Illinois circuits ABC
ISCAS ISCAS'85 and ISCAS'89 benchmarks ABC
ITC99 ITC'99 benchmark ABC
IWLS2005 IWLS'2005 benchmark ABC
IWLS93 IWLS'93 benchmark SIS
LEKO LEKO examples ABC
LEKU LEKU examples ABC
LGSynth91-PLA LGSynth'91 (IWLS'91) benchmark SIS
LGSynth91 LGSynth'91 (IWLS'91) benchmark SIS
MCNC-Comb MCNC benchmark SIS
MCNC-Seq MCNC benchmark SIS
Other Other, undocumented circuits ABC, SIS
QUIP QUIP benchmark ABC

Collection Structure and direct download

The files are stored in a hierarchical directory structure as follows:
2.0	                                     ; the collection version, original circuit descriptions
    Flat (1185) 	                     ; flattened hierarchy, processed by ABC step - 1), 2), 3)
        sweep (1164)         	             ; processed by 'sweep' and cleaned - steps 4), 5)
            Components (1836)                ; divided into connected components - step 6)
                Size-5 (1346)                ; minimum size 5 gate equivalents - step 8)
                 Size-50 (1103)              ; minimum size 50 gate equivalents - step 8)
            Comb (1164)	                     ; combinational parts extracted - step 7)
                Components (31086)
                Size-5 (2045)
                Size-50 (1508)
            Unique-file (955)                ; only unique files - step 9a)
                Comb (955)
                    Components (30701)
                        Size-5 (1793)
                        Size-50 (1316)
                Components (1535)
                    Size-5 (1094)
                    Size-50 (911)
            Unique-struct (944)               ; only structurally unique circuits - step 9b)
                Comb (944)
                    Components (30685)
                        Size-5 (1779)
                        Size-50 (1312)
                Components (1519)
                    Size-5 (1080)
                    Size-50 (907)
            Unique-funct (883)                ; only functionally unique circuits - step 9c)
                Comb (883)
                    Components (30588)
                        Size-5 (1719)
                        Size-50 (1257)
                Components (1452)
                    Size-5 (1020)
                    Size-50 (875)
The numbers of circuits in the each set are shown in the figure too, in parentheses.
Each directory contains respective zipped BLIF and VHDL files, statistical data, and a short description of the set.

File, structural, and functional duplicities

Information on circuit equivalences (see Section 4, step 9) is present in the sweep directory.
Particularly, here are the list of equivalences:
  • Equivalt files
  • Structurally equivalent descriptions
  • Functionally equivalent descriptions
  • Circuits properties

    Circuit details (numbers of inputs, outputs, flip-flops, literals, ...) are provided in each directory in the stats.csv file.

    Reference

    P. Fišer and J. Schmidt, "A Comprehensive Set of Logic Synthesis and Optimization Examples," in Proc. of 12th Int. Workshop on Boolean Problems (IWSBP), Freiberg (Germany), September 22-23, 2016, pp. 151-158. pdf