TACLeBench

  • Good, realistic benchmark suites are essential for the evaluation and comparison of code-level timing analysis techniques. Existing benchmarking suites for CLTA do not cover multi-core software. TACLeBench provides a freely available and comprehensive benchmark suite for timing analysis, featuring complex multi-core benchmarks in the near future. TACLeBench will be continuously extended by novel benchmarks, especially by parallel multi-task/multi-core benchmarks. The overall goal is to establish TACLeBench as the standard benchmarking suite for timing analysis worldwide.

    TACLeBench is a collection of currently 102 benchmark programs from several different research groups and tool vendors around the world. These benchmarks are provided as ANSI-C 99 source codes. The source codes are 100% self-contained – no dependencies to system-specific header files via #include directives exist, eventually used functions from math libraries are also provided in the form of C source code. Furthermore, almost all benchmarks are processor-independent and can be compiled and evaluated for any kind of target processor. The only exception is PapaBench which uses architecture-dependent I/O addresses and which currently supports Atmel AVR processors.

    Since TACLeBench addresses the needs imposed by timing analysis tools, all benchmarks are completely annotated with flow facts. These flow facts are directly incorporated into the ANSI-C source codes using Pragmas at a high level of abstraction. For example, the following source code snippet states that the shown loop iterates between 50 and 100 times, depending on the data dependency via variable maxIter:

      _Pragma( "loopbound min 50 max 100" )

      for ( i = 1; i <= maxIter; i++ )

        Array[i] = i * fact * KNOWN_VALUE;

    You can find a complete description of the used flow fact language in the download section of this website. A more detailed description of the TACLeBench collection can be found in the following paper:

    Heiko Falk, Sebastian Altmeyer, Peter Hellinckx, Björn Lisper, Wolfgang Puffitsch, Christine Rochange, Martin Schoeberl, Rasmus Bo Sørensen, Peter Wägemann and Simon Wegener.
    TACLeBench: A Benchmark Collection to Support Worst-Case Execution Time Research.
    In Proceedings of the 16th International Workshop on Worst-Case Execution Time Analysis (WCET), pages 2:1-2:10, Toulouse / France, July 2016.
    DOI 10.4230/OASIcs.WCET.2016.2

    If you would like to share your very own benchmarks with us, feel free to contact Heiko Falk (Heiko (dot) Falk (at) tuhh (dot) de) in order to have your source codes included in TACLeBench.

  • Sequential Benchmarks:

    Name Description Code Size (LoC) 1 Origin
    adpcm_decoder ADPCM decoder 406 MRTC
    adpcm_encoder ADPCM encoder 434 MRTC
    adpcm_g721_board_test ADPCM transcoder 661 DSPStone
    adpcm_g721_verify ADPCM verification 673 DSPStone
    ammunition C compiler arithmetic stress test 2508 misc
    anagram Word anagram computation 440 misc
    audiobeam Audio beam former 985 StreamIt
    basicmath_small Basic mathematical calculations (cubic function solving, integer square roots, angle conversion) 1001 MiBench
    binarysearch Binary search of 15 integers 35 MRTC
    bitcount Couting number of bits in an integer array 202 MiBench
    bitonic Bitonic sorting network 56 StreamIt
    bsort100 Bubblesort program 54 MRTC
    cjpeg_jpeg6b_transupp JPEG image transcoding routines 1599 MediaBench
    cjpeg_jpeg6b_wrbmp JPEG image bitmap writing code 1296 MediaBench
    codecs_codhuff Huffman encoding with a file source to compress 373 misc
    codecs_codrle1 RLE type 1 encoding with a file source to compress 110 misc
    codecs_dcodhuff Huffman decoding with a file source to decompress 221 misc
    codecs_dcodrle1 RLE type 1 decoding with a file source to decompress 60 misc
    complex_multiply_fixed Multiplication of complex numbers 22 DSPStone
    complex_multiply_float Multiplication of complex numbers 22 DSPStone
    complex_update_fixed Multiply-add with complex numbers 26 DSPStone
    complex_update_float Multiply-add with complex numbers 28 DSPStone
    compressdata Data compression program adopted from SPEC95 203 MRTC
    convolution_fixed Convolution filter 27 DSPStone
    convolution_float Convolution filter 27 DSPStone
    countnegative Counts non-negative numbers in a matrix 73 MRTC
    cover Artificial code with lots of different control flow paths 231 MRTC
    crc CRC computation on 40 bytes of data 68 MRTC
    dijkstra All pairs shortest path 227 MiBench
    dot_product_fixed Dot product of vectors 22 DSPStone
    dot_product_float Dot product of vectors 24 DSPStone
    duff Duff's device 44 MRTC
    edn FIR filter calculations 204 MRTC
    epic Efficient pyramid image coder 994 MediaBench
    expint Series expansion for computing an exponential integral function. 62 MRTC
    fac Faculty function 21 MRTC
    fdct Fast discrete cosine transform 143 MRTC
    fft_16_7 16-point FFT, 7 bits per twiddle 149 DSPStone
    fft_16_13 16-point FFT, 13 bits per twiddle 156 DSPStone
    fft_1024_7 1024-point FFT, 7 bits per twiddle 282 DSPStone
    fft_1024_13 1024-point FFT, 13 bits per twiddle 311 DSPStone
    fft1 1024-point FFT using the Cooly-Turkey algorithm 140 MRTC
    fibcall Simple iterative Fibonacci calculation 22 MRTC
    filterbank Filter bank for multirate signal processing 75 StreamIt
    fir FIR filter over a sample of 700 items 225 MRTC
    fir2dim_fixed 2-dimensional FIR filter convolution 81 DSPStone
    fir2dim_float 2-dimensional FIR filter convolution 81 DSPStone
    fir_fixed FIR filter 40 DSPStone
    fir_float FIR filter 40 DSPStone
    fmref Software FM radio with equalizer 680 StreamIt
    g721_encode CCITT G.721 encoder 901 misc
    g723_encode CCITT G.723 encoder 898 misc
    gsm Complete GSM 06.10 provisional standard codec 2382 MediaBench
    gsm_decode GSM 06.10 provisional standard decoder 1368 MediaBench
    gsm_encode GSM 06.10 provisional standard encoder 1940 MediaBench
    h264dec_ldecode_block H.264 block decoding functions 1574 MediaBench
    h264dec_ldecode_macroblock H.264 macroblock decoding 1144 MediaBench
    iir_biquad_N_sections_fixed Biquad IIR 4 sections filter 42 DSPStone
    iir_biquad_N_sections_float Biquad IIR 4 sections filter 43 DSPStone
    iir_biquad_one_section_fixed Biquad IIR 1 section filter 25 DSPStone
    iir_biquad_one_section_float Biquad IIR 1 section filter 25 DSPStone
    insertsort Insertion sort 61 MRTC
    janne_complex Artificial code where the inner loop's max. iterations depend on the outer loop 61 MRTC
    jfdctint Discrete-cosine transformation on a 8x8 pixel block 221 MRTC
    lcdnum Read ten values, output half to LCD 62 MRTC
    lms LMS adaptive signal enhancement 160 MRTC
    lms_fixed Least-Mean-Square filter 44 DSPStone
    lms_float Least-Mean-Square filter 44 DSPStone
    ludcmp LU decomposition 86 MRTC
    matmult 20x20 matrix multiplication 57 MRTC
    matrix1x3_fixed 1x3 matrix multiplication 23 DSPStone
    matrix1x3_float 1x3 matrix multiplication 41 DSPStone
    matrix1_fixed Generic matrix multiplication 48 DSPStone
    matrix1_float Generic matrix multiplication 49 DSPStone
    md5 Message digest algorithm 336 NetBench
    minver Floating point matrix inversion 159 MRTC
    mpeg2 MPEG2 motion estimation 1533 MediaBench
    n_complex_updates_fixed Multiply-add with vectors of complex numbers 38 DSPStone
    n_complex_updates_float Multiply-add with vectors of complex numbers 38 DSPStone
    n_real_updates_fixed Multiply-add with vectors of real numbers 29 DSPStone
    n_real_updates_float Multiply-add with vectors of real numbers 29 DSPStone
    ndes Complex embedded code 407 MRTC
    petrinet Petri net simulation 500 MRTC
    pm Pattern match kernel 558 misc
    prime Prime number test 34 MRTC
    qsort Quick sort of string arrays 2716 MiBench
    qsort-exam Non-recursive quicksort 84 MRTC
    qurt Root computation of quadratic equations 88 MRTC
    real_update_fixed Multiply-add with real numbers 21 DSPStone
    real_update_float Multiply-add with real numbers 24 DSPStone
    recursion Artificial recursive code 16 MRTC
    rijndael_decoder Rijndael AES decryption 3043 MiBench
    rijndael_encoder Rijndael AES encryption 1024 MiBench
    select Select the Nth largest number in a floating point array 62 MRTC
    sha NIST secure hash algorithm 509 MiBench
    sqrt Square root function implemented by Taylor series 45 MRTC
    st Statistics program (sum, mean, variance, std. deviation, correlation) 106 MRTC
    startup_fixed V32 modem startup code 99 DSPStone
    statemate Statechart simulation of a car window lift control 1053 MRTC
    susan MR image recognition algorithm 1984 MiBench
    test3 Artificial WCET analysis stress test 3985 misc

    Parallel Benchmarks:

    Name Description #Tasks Code Size (LoC) 1 Origin
    Debie DEBIE-1 instrument observing micro-meteoroids and small space debris 8 6094 Tidorum Ltd
    PapaBench UAV autopilot and fly-by-wire software 10 6644 Paparazzi

     

    1 "Code Size (LoC)" represents the lines of code of all source code files belonging to a benchmark, empty lines and comments are not counted.

     

  • You can obtain all TACLeBench source code files via the TACLeBench github repository. The use of the TACLeBench codes is subject to the license and copyright terms stated in each individual benchmark code.

    A specification of the flow fact language used to annotate the TACLeBench source codes for WCET analysis is available for download here.