Portable Testsuite to check a C / C++ Compiler and standard runtime libraries

Many compiler and the related standard libraries provide tests, but they are not easy usable as portable/standalone conformance check for other compiler or for other runtime library implementations.

Goals for stdtests

• A reusable license
• Only needs a Compiler and a posix shell to run the testsuite
• Portable: Works on many systems with many compiler and runtime libraries
• Check C Compiler and the C Runtime library
• Check the POSIX Runtime library
• Check C++ Compiler and the C++ Runtime library
• Create a nice Table from multiple Results

Expected usecase

• Extract the testsuite
• configure the testsuite to test a compiler / runtime library for a given standard
• make: run the testsuite and generate a result file
• make table: generate a table for multiple results

Example usage

• std: newest
• Compiler: tcc
• runtime-library: system-default

mkdir results
cd results
CC=tcc ../configure --disable-c++
make

• Now there should be an document with the results of the current test

Portability

• Current tested environments

  Target OS	tested Compiler
  Linux	tcc / gcc / g++ / owcc / clang / zig cc / zig c++
  Windows	x86_64-win32-tcc / x86_64-w64-mingw32-gcc / x86_64-w64-mingw32-g++

• Hints / Links for later use:

  • With many years delay, a command line switch arrived in MSVC 2015 Update 3 for selecting the c++ version

Checking C Compiler and the C Runtime library

STD	C2y	C23	C17	C11	C99	C95 (C90+AMD1)	C89/C90 / ANSI-C

Checking POSIX Runtime library

Currently, POSIX.1-2024 ist referenced. Older versions will follow.

Checking C++ Compiler and the C++ Runtime library

STD	C++2c	C++23	C++20	C++17	C++14	C++11	C++03	C++98

Some examples of other tables with Features or Test-Results

• gcc Status of C99 features in GCC / C11 Status in GCC / C++ Standards support in GCC
• clang C Support in clang / C++ Support in Clang
• msvc C/C++ Support in MSVC
• dmc C++0x language Implementation
• owcc C99 Compliance / C++ Language / C++ Library / C++ Library Status
• cppreference.com What's new in C99 / C99 Implementation status / C23 Implementation status / C++ Implementation status

Some links to compiler / library tests

Name	URL
clang	test top dir
gcc	test top dir
mcpp	preprocessor
musl	libc-tests
glibc	libc

---

C Reference Documentation

• ISOC SC22/WG14 C standards commitee
• ISO SC22/WG14 docs
• ISO SC22/WG14 document log
• C Project status and milestones
• C23: Early C2y draft from 02.02.2024 was N3220
• C17: Early C23 draft from 11.11.2018 was N2310

Released C Standards

std	__STDC_VERSION__	Latest Draft	Release
C2y	N/A	N3301	N/A
C23	202311	N3096	ISO/IEC 9899:2024
C17	201710	N2176	ISO/IEC 9899:2018
C11	201112	N1570	ISO/IEC 9899:2011
C99	199901	N1256	ISO/IEC 9899:1999
C95	199409	c89_na1: c94	ISO/IEC 9899:1990/AMD1:1995
C90	not defined	Missing	ISO/IEC 9899:1990
C89	not defined	c89 draft	ANSI X3.159-1989

---

POSIX Reference Documentation

POSIX (Portable Operating System Interface) specifies many parts of an Operating System.

• Unix standards overview

Interesting for this project is the API part of the Base Specification.

Released versions

The Open Group Base Specification is also released as the Single UNIX Specification and also released as the ISO/IEC 9945-1 standard.

_POSIX_VERSION	_XOPEN_VERSION	The Open Group Base Specification	View	Downloads
202405	800	POSIX.1-2024, Issue 8, 2024 Edition	View online	[Downloads N/A yet]
200809	700	POSIX.1-2017, Issue 7, 2018 Edition	View online	susv4-2018
200809	700	POSIX.1-2008, Issue 7, 2016 Edition	View online	susv4tc2
200809	700	POSIX.1-2008, Issue 7, 2013 Edition	View online	susv4tc1
200809	700	POSIX.1-2008, Issue 7, 2008 Edition	View online	susv4
200112	600	POSIX.1-2001, Issue 6, 2004 Edition	View online	susv3
199506		POSIX.1-1996
199009		POSIX.1-1990
198808		POSIX.1-1988

• Single UNIX Specification (SUSv*)

  • UNIX V7, Version 4
  • UNIX 03, Version 3
  • UNIX 98, Version 2
  • UNIX 95, Version 1

• POSIX 2024 Publication status at iso.org: 40.99 (since 19.Sep.2024)

  Changed	Status	Status-Text
  N/A	60.60	International Standard published
  N/A	60.00	International Standard under publication
  N/A	50.60	Close of voting. Proof returned by secretariat
  N/A	50.20	Proof sent to secretariat or FDIS ballout initiated: 8 weeks
  N/A	50.00	Final text received or FDIS registered for formal approval
  19.Sep.2024	40.99	Full report circulated: DIS approved for registration as FDIS

  • ISO/IEC/IEEE 9945:2024
  • ISO/IEC/IEEE 9945:2009
  • ISO/IEC/IEEE 9945-1:2003
  • ISO/IEC/IEEE 9945-1:2002
  • ISO/IEC/IEEE 9945-1:1996
  • ISO/IEC/IEEE 9945-1:1990

---

C++ Reference Documentation

• ISOCPP SC22/WG21 C++ standards commitee
• C++ standard core language issues list
• C++ standard library issues list
• C++26 First draft was N4958
• C++23 is available at the iso.org Website since 19.Oct.2024

Released C++ Standards

std	__cplusplus	Latest Draft	Release
C++26	N/A	N4993	N/A
C++23	202302	N4950	ISO/IEC 14882:2023
C++20	202002	N4849	ISO/IEC 14882:2020
C++17	201703	N4659	ISO/IEC 14882:2017
C++14	201402	N3797	ISO/IEC 14882:2014
C++11	201103	N3242	ISO/IEC 14882:2011
C++03	200300	c++03 final	ISO/IEC 14882:2003
C++98	199711	nr. unknown	ISO/IEC 14882:1998

---

Maintainer Mode hints

• use 'bootstrap' to regenerate the build system files
• use 'configure' with "--enable-maintainer-mode" to rebuild all programs all tests groups are enabled by default:
  • C
  • C++ (alias names: cxx / cplusplus)
  • posix
  • libc
  • libc++ (alias name: libcxx)

---