Hi, I did plenty of work towards getting SLES-16 reproducible. Also some patches outside of it. E.g. all old llvm versions got fixed now. And I wrote https://lists.reproducible-builds.org/pipermail/rb-general/2025-July/003846.html Here are the autogenerated bits: last month's status: https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/thread/U55TFJTYPTDALD4NB7KV4SRFSLGGJKRV/ Last months' reproducible builds project updates (including my work): https://reproducible-builds.org/reports/2025-06/ I uploaded https://rb.zq1.de/compare.factory-20250731/ today https://rb.zq1.de/spec/glossar.txt explains the meaning of below values: total-packages: 16055 (+16) build-tried: 15989 (+15) build-failed: 85 (-26) build-n-a: 162 (+0) build-succeeded: 15742 (+41) build-official-failed+na: 19 (-1) build-compare-failed: 169 (-3) build-compare-succeeded: 15573 (+44) verify-failed: 880 (-82) verified-semi-reproducible: 14895 (+124) verified-bit-identical: 14643 (+141) bit-by-bit-identical: 15486 (+38) not-bit-by-bit-identical: 256 (+5) not-bit-by-bit-identicalcheck: 256 (+3) https://rb.zq1.de/compare.factory-20250731/graph.png shows the change over time https://rb.zq1.de/compare.factory-20250731/unreproduciblerings.txt lists very unreproducible core packages (bootstrap+DVD) Of the badly unreproducible packages, 0 were in ring0 21 were in ring1 That makes it 21/4043 => 0.52 % which is below the overall average of 169/15742 => 1.07 % 256/15742 => 1.63 % of packages are not perfectly reproducible package notes: ==> ./OpenImageIO/.rb.notes <== FTBFS-j1 ==> ./binutils/.rb.notes <== => SR 1291583 PGO ==> ./calibre/.rb.notes <== CPU-dependent on sse4.1 from Qt5 QImage scaler /usr/share/icons/hicolor/64x64/mimetypes/text-lrs.png ==> ./curl/.rb.notes <== => https://github.com/curl/curl/pull/17665 FTBFS-2036 ==> ./dpdk/.rb.notes <== => SR 1296685 dpdk pesign ==> ./emacs/.rb.notes <== => SR 1292097 by Werner Fink some CPU-dependence remaining ==> ./fcitx-libpinyin/.rb.notes <== => SR 1296657 1296614 ASLR small diffs in pinyin_index.bin ==> ./fcitx5-zhuyin/.rb.notes <== => SR 1296656 1296613 ASLR See ibus-libzhuyin ==> ./firecracker/.rb.notes <== => https://github.com/firecracker-microvm/firecracker/pull/5298 rust HashMap by Andrea Manzini ==> ./gnucash/.rb.notes <== => SR 1293976 = https://bugs.gnucash.org/show_bug.cgi?id=799623 FTBFS-2038 --build-opt=--vm-custom-opt=-rtc base=2042-01-25T00:00:00 ==> ./gstreamer-plugins-rs/.rb.notes <== = https://bugzilla.opensuse.org/show_bug.cgi?id=1237097 = https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/599 date ; rust/llvm asm diff [build] jobs = 1 doesn't help ==> ./gtk2-engine-aurora/.rb.notes <== #=> https://src.opensuse.org/lxde/gtk2-engine-aurora/pulls/1 %jobs ==> ./gtk2-engine-cleanice/.rb.notes <== => SR 1289734 1284149 gtk2-engine-cleanice %jobs ==> ./gtk2-engines/.rb.notes <== => SR 1289736 1284006 gtk2-engines %jobs ==> ./hawk2/.rb.notes <== = https://bugzilla.opensuse.org/show_bug.cgi?id=1230275 date, gzip-mtime, other? .cache files vary for unknown reason ==> ./ibus-libzhuyin/.rb.notes <== => SR 1296576 1296344 ASLR 2 random bytes in output /usr/lib64/ibus-libzhuyin/data/pinyin_index.bin ==> ./java-21-openjdk/.rb.notes <== race influence in libjvm.so debuginfo => SR 1294290 1293272 FTBFS-2035-03-30 ==> ./kea/.rb.notes <== => SR 1296356 => https://src.opensuse.org/dhcp/kea/pulls/9 = https://github.com/mesonbuild/meson/issues/14850 = https://bugzilla.suse.com/show_bug.cgi?id=1246670 Variations in /usr/share/kea/meson-info/intro-dependencies.json and /usr/share/kea/meson-info/intro-targets.json - are these files even needed? ==> ./kwctl/.rb.notes <== minor rust ==> ./libcamera/.rb.notes <== => SR 1291502 => https://src.opensuse.org/jengelh/libcamera/pulls/6 signatures ==> ./libpinyin/.rb.notes <== => SR 1296340 1296340 = [libpinyin](https://bugzilla.opensuse.org/show_bug.cgi?id=1180734 ASLR 2 byte binary diff in ==> ./libsemigroups/.rb.notes <== = https://github.com/libsemigroups/libsemigroups/issues/767 FTBFS-j1 again ==> ./libzypp/.rb.notes <== #93 - Provider_test (SEGFAULT) [ 6396s] 93/97 Test #93: Provider_test ....................***Exception: SegFault 2.18 sec ==> ./llvm15/.rb.notes <== => SR 1292354 1292274 = https://bugzilla.opensuse.org/show_bug.cgi?id=1199076 order ==> ./llvm19/.rb.notes <== => SR 1292275 1292247 llvm19 date, %jobs https://bugzilla.opensuse.org/show_bug.cgi?id=1199076 ==> ./llvm20/.rb.notes <== => SR 1293548 1293268 date => SR 1290617 %jobs ==> ./mariadb/.rb.notes <== FTBFS-2040-02-27 mariadb-11.8.2/mysql-test/std_data/server-cert.pem ==> ./maven-archiver/.rb.notes <== => https://github.com/apache/maven-archiver/pull/273 date by Fridrich Strba ==> ./memcached/.rb.notes <== = https://bugzilla.suse.com/show_bug.cgi?id=1246407 FTBFS-2038 => https://github.com/memcached/memcached/pull/1074 FTBFS-2038 ==> ./moditect/.rb.notes <== => https://github.com/moditect/moditect/pull/263 by Fridrich Strba ==> ./mumble/.rb.notes <== => https://github.com/mumble-voip/mumble/pull/6872 = https://github.com/mumble-voip/mumble/issues/6818 FTBFS found range good=1751542431 bad=1751544355 ==> ./nodejs22/.rb.notes <== FTBFS-2031-12-09 ; CPU found range good=1954600556 bad=1954602480 ==> ./nwg-dock-hyprland/.rb.notes <== see nwg-drawer - possible toolchain issue ==> ./nwg-drawer/.rb.notes <== = https://bugzilla.opensuse.org/show_bug.cgi?id=1246700 ASLR? unknown 1-byte variation - same as nwg-dock-hyprland 00000000010cb9f0 <_gotk4_gtk3_Widget_ConnectCanActivateAccel>: ==> ./openhtj2k/.rb.notes <== FTBFS-CPU --build-opt=--vm-custom-opt=-cpu qemu64 failed ==> ./perl-HTTP-CookieJar/.rb.notes <== => SR 1293766 1293664 => https://github.com/dagolden/HTTP-CookieJar/pull/16 FTBFS-2041 --build-opt=--vm-custom-opt=-rtc base=2042-01-25T00:00:00 failed ==> ./perl-XML-LibXML/.rb.notes <== #=> SR 1288338 minor mtime (from %check): filterdiff 'rpm -qpvl' R*/perl-XML-LibXML-2.0210-1.1.x86_64.rpm ==> ./product-composer/.rb.notes <== => https://github.com/openSUSE/obs-build/commit/e3812b225f8b6fb66301c8dbee9e5bf242939f8a by Adrian Schroeter fixed in obs-build-20250708 ==> ./pympress/.rb.notes <== => SR 1295211 date+time from python-Babel : fix in rb:test --- old /usr/lib/python3.11/site-packages/pympress/share/locale/zh_TW/LC_MESSAGES/pympress.mo (hex) ==> ./python-Babel/.rb.notes <== => SR 1295211 date+time toolchain for pympress ==> ./python-M2Crypto/.rb.notes <== FTBFS 2034-09-14 found range good=2041798028 bad=2041799952 ==> ./python-numpy/.rb.notes <== => SR 1296687 1296304 => https://github.com/numpy/SVML/pull/7 = https://bugzilla.suse.com/show_bug.cgi?id=1245203 random tmpfile RECORD -numpy/_core/_multiarray_umath.cpython-312-x86_64-linux-gnu.so,sha256=k0KyrrcvbGOrtqoDXcpR5zSAQ1qgcxF0uWrnSGYY55c,10668392^M ==> ./python-paho-mqtt/.rb.notes <== FTBFS-j1 remaining => SR 1290153 SSL expired =https://github.com/eclipse/paho.mqtt.python/pull/854 ==> ./python-rdflib/.rb.notes <== => SR 1291184 by Daniel Garcia = https://bugzilla.suse.com/show_bug.cgi?id=1245096 = https://github.com/RDFLib/rdflib/issues/2645 => https://github.com/danigm/rdflib/commit/11dbb5092e21a91dfee17843795830b309c68afc#diff-afaae5660cefaa165b93d109383 ==> ./python-rencode/.rb.notes <== = https://bugzilla.opensuse.org/show_bug.cgi?id=1246916 CPU -march=native ; link order of __pyx_k_* symbols e.g. __pyx_k_Cython ==> ./python-uv/.rb.notes <== FTBFS? ==> ./python312/.rb.notes <== test_ssl ==> ./python314/.rb.notes <== variations in %{sitedir}/test/__pycache__ .pyc ==> ./rabbitmq-server/.rb.notes <== = rabbitmq-server https://bugzilla.opensuse.org/show_bug.cgi?id=1245558 timestamps in zip from elixir toolchain ; --enable-deterministic-build helps a bit FTBFS https://build.opensuse.org/package/show/network:messaging:amqp/rabbitmq-server ==> ./rage-encryption/.rb.notes <== => https://github.com/str4d/rage/pull/580 version update to replace HashMap -=> SR 1291462 1285623 = https://github.com/kellpossible/cargo-i18n/pull/151 = https://github.com/str4d/rage/issues/568 = https://bugzilla.suse.com/show_bug.cgi?id=1244083 rust/llvm asm diff from i18n-embed-fl-0.9.2 HashMap ==> ./re-flex/.rb.notes <== => https://bugzilla.opensuse.org/show_bug.cgi?id=1246164 re-flex CPU ==> ./rsign2/.rb.notes <== => https://codeberg.org/cunix/vendored_licenses_packager/pulls/1 = https://bugzilla.opensuse.org/show_bug.cgi?id=1245269 random .spec expansion ==> ./s390-tools/.rb.notes <== embeds rebuild counter ==> ./scummvm/.rb.notes <== => SR 1284741 1284696 ASLR in debuginfo ; --enable-release helps /usr/bin/scummvm ==> ./socat/.rb.notes <== FTBFS-j1 ==> ./starship/.rb.notes <== => https://github.com/baoyachi/shadow-rs/pull/235 https://github.com/baoyachi/shadow-rs/issues/234 toolchain, date ==> ./timescaledb/.rb.notes <== => SR 1284769 1282546 timescaledb uname -r 88f00 706f7374 67726573 716c5f76 65727369 postgresql_versi ==> ./vtk/.rb.notes <== => SR 1266058 1231633 minor jar mtime /usr/lib64/java/vtk.jar ==> ./xen/.rb.notes <== embeds rebuild-counter via %release => SR 1202206 = https://bugzilla.opensuse.org/show_bug.cgi?id=1230856 date+time in debugsource