Subject: Bug#775235: Repeatable failure to start on i386 running under kvm

From: Steve McIntyre <[email-masked]>
Date: Mon, 12 Jan 2015 21:36:35 +0000

Package: gnome-shell
Version: 3.14.2-3+b1
Severity: serious

Hi,

During testing of the Jessie d-i RC1 CD build, I installed lots and
lots of different variations of virtual systems using KVM. All the
others worked OK given sufficient packages (access to a mirror, etc.),
but installing Gnome on i386 left me with a failure. Installation was
successful, but on boot the virtual system fails to show an X login.

Instead of seeing a gdm prompt, I get several seconds of black screen
followed by an "Oh no! Something has gone wrong." message. Rebooting
into single-user mode, I grabbed /var/log/messages (attached). This
failure is easily repeatable, every boot.

(I don't know for definite which package is to blame, but gnome-shell
looks a likely candidate from the log. Please feel free to reassign as
appropriate, obviously!)

The following system information is from the host, not the VM...

-- System Information:
Debian Release: 7.8
  APT prefers stable
  APT policy: (500, 'stable'), (500, 'oldstable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 3.2.0-4-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

---

From: Bernhard Übelacker <[email-masked]>
Date: Mon, 19 Jan 2015 17:01:24 +0100

Hello,
tried to reproduce it by following steps:
- installed on amd64 host with qemu-system-i386 [1] and CD1 [2]
  and network mirror with Gnome desktop and ssh server.
- got the same message "Oh no! Something has gone wrong." (see attached picture)

Tried to debug into the issue:

# start VM without X by adding in grub "3" to the kernel parameters
# ssh into the VM

root@debian:~# apt-get update
root@debian:~# apt-get install gdb gnome-shell-dbg libglib2.0-0-dbg libgtk-3-0-dbg libllvm3.5-dbg libgl1-mesa-dri-dbg libcogl20-dbg libclutter-1.0-dbg dpkg-dev
root@debian:~# dpkg-reconfigure x11-common  # allow every user to start the X server

benutzer@debian:~$ apt-get source gnome-shell
benutzer@debian:~$ apt-get source libmutter0e
benutzer@debian:~$ apt-get source libllvm3.5
benutzer@debian:~$ X :0&
benutzer@debian:~$ export DISPLAY=:0

benutzer@debian:~$ gdb --args /usr/bin/gnome-shell --replace
(gdb) run
    ...
    Gjs-Message: JS LOG: Failed to launch ibus-daemon: Kindprozess »ibus-daemon« konnte nicht ausgeführt werden (Datei oder Verzeichnis nicht gefunden)

    (gnome-shell:1020): mutter-WARNING **: STACK_OP_RAISE_ABOVE: window 0x4f00800016 not in stack
    ...
    (gnome-shell:1020): mutter-WARNING **: STACK_OP_RAISE_ABOVE: window 0x4f00800016 not in stack
    ...
    LLVM ERROR: Do not know how to split the result of this operator!
    ...
    [Inferior 1 (process 1020) exited with code 01]

benutzer@debian:~$ gdb --args /usr/bin/gnome-shell --replace
(gdb) directory gnome-shell-3.14.2/src:mutter-3.14.2/src/core:mutter-3.14.2/src/meta:glib2.0-2.42.1/glib:llvm-toolchain-3.5-3.5/lib/Support
(gdb) set height 0
(gdb) set width 0
(gdb) b llvm::report_fatal_error
(gdb) run

Breakpoint 1, llvm::report_fatal_error (Reason=0xb2889a04 "Do not know how to split the result of this operator!\n", GenCrashDiag=true) at /build/llvm-toolchain-3.5-Ypkkuu/llvm-toolchain-3.5-3.5/lib/Support/ErrorHandling.cpp:60
60      void llvm::report_fatal_error(const char *Reason, bool GenCrashDiag) {
(gdb) bt
#0  llvm::report_fatal_error (Reason=0xb2889a04 "Do not know how to split the result of this operator!\n", GenCrashDiag=true) at /build/llvm-toolchain-3.5-Ypkkuu/llvm-toolchain-3.5-3.5/lib/Support/ErrorHandling.cpp:60
#1  0xb1b31861 in llvm::DAGTypeLegalizer::SplitVectorResult (this=0xbfffe0b0, N=0x90ff620, ResNo=0) at /build/llvm-toolchain-3.5-Ypkkuu/llvm-toolchain-3.5-3.5/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp:555
#2  0xb1b15cb3 in llvm::DAGTypeLegalizer::run (this=0xbfffe0b0) at /build/llvm-toolchain-3.5-Ypkkuu/llvm-toolchain-3.5-3.5/lib/CodeGen/SelectionDAG/LegalizeTypes.cpp:255
#3  0xb1b16476 in llvm::SelectionDAG::LegalizeTypes (this=0x8e8e7e0) at /build/llvm-toolchain-3.5-Ypkkuu/llvm-toolchain-3.5-3.5/lib/CodeGen/SelectionDAG/LegalizeTypes.cpp:1113
#4  0xb1bc58ed in llvm::SelectionDAGISel::CodeGenAndEmitDAG (this=0x8e8e678) at /build/llvm-toolchain-3.5-Ypkkuu/llvm-toolchain-3.5-3.5/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:687
#5  0xb1bc5e39 in llvm::SelectionDAGISel::SelectBasicBlock (this=0x8e8e678, Begin=..., End=..., HadTailCall=@0x8f5c74c: false) at /build/llvm-toolchain-3.5-Ypkkuu/llvm-toolchain-3.5-3.5/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:604
#6  0xb1bc8e07 in llvm::SelectionDAGISel::SelectAllBasicBlocks (this=0x8df8d30, Fn=...) at /build/llvm-toolchain-3.5-Ypkkuu/llvm-toolchain-3.5-3.5/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:1239
#7  0xb1bca381 in llvm::SelectionDAGISel::runOnMachineFunction (this=0x8e8e678, mf=...) at /build/llvm-toolchain-3.5-Ypkkuu/llvm-toolchain-3.5-3.5/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:451
#8  0xb1debc3c in (anonymous namespace)::X86DAGToDAGISel::runOnMachineFunction (this=0x8e8e678, MF=...) at /build/llvm-toolchain-3.5-Ypkkuu/llvm-toolchain-3.5-3.5/lib/Target/X86/X86ISelDAGToDAG.cpp:168
#9  0xb1f4a188 in llvm::MachineFunctionPass::runOnFunction (this=0x8e8e678, F=...) at /build/llvm-toolchain-3.5-Ypkkuu/llvm-toolchain-3.5-3.5/lib/CodeGen/MachineFunctionPass.cpp:33
#10 0xb195e4b2 in llvm::FPPassManager::runOnFunction (this=0x90039d0, F=...) at /build/llvm-toolchain-3.5-Ypkkuu/llvm-toolchain-3.5-3.5/lib/IR/LegacyPassManager.cpp:1545
#11 0xb195e56f in llvm::legacy::FunctionPassManagerImpl::run (this=0x9005528, F=...) at /build/llvm-toolchain-3.5-Ypkkuu/llvm-toolchain-3.5-3.5/lib/IR/LegacyPassManager.cpp:1494
#12 0xb195e64f in llvm::legacy::FunctionPassManager::run (this=0x90023b0, F=...) at /build/llvm-toolchain-3.5-Ypkkuu/llvm-toolchain-3.5-3.5/lib/IR/LegacyPassManager.cpp:1412
#13 0xb280574d in llvm::JIT::jitTheFunctionUnlocked (this=0x9025e98, F=0x8f15798) at /build/llvm-toolchain-3.5-Ypkkuu/llvm-toolchain-3.5-3.5/lib/ExecutionEngine/JIT/JIT.cpp:493
#14 0xb2805cff in llvm::JIT::runJITOnFunctionUnlocked (this=0x9025e98, F=0x8f15798) at /build/llvm-toolchain-3.5-Ypkkuu/llvm-toolchain-3.5-3.5/lib/ExecutionEngine/JIT/JIT.cpp:472
#15 0xb2805f0f in llvm::JIT::getPointerToFunction (this=0x9025e98, F=0x8f15798) at /build/llvm-toolchain-3.5-Ypkkuu/llvm-toolchain-3.5-3.5/lib/ExecutionEngine/JIT/JIT.cpp:529
#16 0xb1832c17 in llvm::ExecutionEngine::getPointerToGlobal (this=0x9025e98, GV=0x8f15798) at /build/llvm-toolchain-3.5-Ypkkuu/llvm-toolchain-3.5-3.5/lib/ExecutionEngine/ExecutionEngine.cpp:557
#17 0xb18349a0 in LLVMGetPointerToGlobal (EE=0x9025e98, Global=0x8f15798) at /build/llvm-toolchain-3.5-Ypkkuu/llvm-toolchain-3.5-3.5/lib/ExecutionEngine/ExecutionEngineBindings.cpp:337
#18 0xb37753cf in gallivm_jit_function (gallivm=0x91760c8, func=0x8f15798) at ../../../../../src/gallium/auxiliary/gallivm/lp_bld_init.c:594
#19 0xb39df1b4 in generate_variant (lp=<optimized out>, key=<optimized out>, shader=<optimized out>) at ../../../../../../src/gallium/drivers/llvmpipe/lp_state_fs.c:2634
#20 llvmpipe_update_fs (lp=0x808c000) at ../../../../../../src/gallium/drivers/llvmpipe/lp_state_fs.c:3166
#21 0xb39d7e01 in llvmpipe_update_derived (llvmpipe=0x808c000) at ../../../../../../src/gallium/drivers/llvmpipe/lp_state_derived.c:186
#22 0xb39b6fa4 in llvmpipe_draw_vbo (pipe=0x808c000, info=0xbffff074) at ../../../../../../src/gallium/drivers/llvmpipe/lp_draw_arrays.c:70
#23 0xb36904dd in cso_draw_vbo (cso=0x81dbfd8, info=0xbffff074) at ../../../../../src/gallium/auxiliary/cso_cache/cso_context.c:1428
#24 0xb36905d0 in cso_draw_arrays_instanced (cso=0x81dbfd8, mode=6, start=0, count=4, start_instance=0, instance_count=1) at ../../../../../src/gallium/auxiliary/cso_cache/cso_context.c:1465
#25 0xb35a9536 in draw_quad (st=<optimized out>, st=<optimized out>, color=<optimized out>, num_instances=<optimized out>, z=inf, y1=<optimized out>, x1=<optimized out>, y0=<optimized out>, x0=<optimized out>) at ../../../../src/mesa/state_tracker/st_cb_clear.c:206
#26 clear_with_quad (clear_buffers=<optimized out>, ctx=<optimized out>) at ../../../../src/mesa/state_tracker/st_cb_clear.c:347
#27 st_Clear (ctx=0x8198b78, mask=18) at ../../../../src/mesa/state_tracker/st_cb_clear.c:512
#28 0xb346585d in _mesa_Clear (mask=16640) at ../../../../src/mesa/main/clear.c:226
#29 0xb65425e5 in _cogl_framebuffer_gl_clear (framebuffer=0x821c608, buffers=3, red=1, green=1, blue=1, alpha=1) at ./driver/gl/cogl-framebuffer-gl.c:977
#30 0xb658229b in _cogl_framebuffer_clear_without_flush4f (framebuffer=0x821c608, buffers=3, red=1, green=1, blue=1, alpha=1) at ./cogl-framebuffer.c:238
#31 0xb6582c13 in cogl_framebuffer_clear4f (framebuffer=0x821c608, buffers=3, red=1, green=1, blue=1, alpha=1) at ./cogl-framebuffer.c:388
#32 0xb6582f19 in cogl_framebuffer_clear (framebuffer=0x821c608, buffers=3, color=0xbffff388) at ./cogl-framebuffer.c:457
#33 0xb6552da5 in cogl_clear (color=0xbffff388, buffers=3) at ./cogl.c:94
#34 0xb6ffac7b in _clutter_stage_do_pick (stage=0x821b530, x=512, y=384, mode=(unknown: 2995296772)) at ./clutter-stage.c:1541
#35 0xb6fd1184 in _clutter_input_device_update (device=0x81d3ad0, sequence=0x0, emit_crossing=1) at ./clutter-input-device.c:886
#36 0xb6fdb4d0 in _clutter_process_event_details (context=<optimized out>, event=<optimized out>, stage=<optimized out>) at ./clutter-main.c:2558
#37 _clutter_process_event (event=0x8894e00) at ./clutter-main.c:2752
#38 0xb6ff6e42 in _clutter_stage_process_queued_events (stage=0x821b530) at ./clutter-stage.c:1082
#39 0xb6fdce2a in master_clock_process_events (master_clock=<optimized out>, stages=<optimized out>) at ./clutter-master-clock.c:372
#40 clutter_clock_dispatch (source=0x868bd80, callback=0x0, user_data=0x0) at ./clutter-master-clock.c:589
#41 0xb6c44da4 in g_main_dispatch (context=<optimized out>) at /build/glib2.0-EvFudu/glib2.0-2.42.1/./glib/gmain.c:3111
#42 g_main_context_dispatch (context=0x0) at /build/glib2.0-EvFudu/glib2.0-2.42.1/./glib/gmain.c:3710
#43 0xb6c450c9 in g_main_context_iterate (context=0x8070958, block=150325068, block@entry=1, dispatch=1, self=<optimized out>) at /build/glib2.0-EvFudu/glib2.0-2.42.1/./glib/gmain.c:3781
#44 0xb6c45479 in g_main_loop_run (loop=0x822fe88) at /build/glib2.0-EvFudu/glib2.0-2.42.1/./glib/gmain.c:3975
#45 0xb78e1b3a in meta_run () at core/main.c:473
#46 0x0804994a in main (argc=1, argv=0xbffff6f4) at main.c:463
(gdb) print N
$1 = (llvm::SDNode *) 0x90ff620
(gdb) print *N
$2 = {<llvm::FoldingSetImpl::Node> = {NextInFoldingSetBucket = 0x8e08c29}, <llvm::ilist_node<llvm::SDNode>> = {<llvm::ilist_half_node<llvm::SDNode>> = {Prev = 0x90ffff8}, Next = 0x8fb86a0}, NodeType = 38, OperandsNeedDelete = 0, HasDebugValue = 0, SubclassData = 0, NodeId = 0, OperandList = 0x90ff650, ValueList = 0x8e8eab0, UseList = 0x8e03604, NumOperands = 2, NumValues = 1, debugLoc = {LineCol = 0, ScopeIdx = 0}, IROrder = 99}

SelectionDAGNodes.h:
    class SDNode : public FoldingSetNode, public ilist_node<SDNode> {
      unsigned getOpcode()  const { return (unsigned short)NodeType; }

ISDOpcodes.h:
    namespace llvm {
      namespace ISD {
        enum NodeType {
          INTRINSIC_WO_CHAIN,

(gdb) print (int)llvm::ISD::INTRINSIC_WO_CHAIN
$5 = 38

The error message is thrown because in the switch statement
in llvm::DAGTypeLegalizer::SplitVectorResult
is no case llvm::ISD::INTRINSIC_WO_CHAIN.
This switch statement evaluates the NodeType of the variable N.

Hope this helps someone with more knowledge that area.

Kind regards,
Bernhard

[1] qemu-system-i386 --enable-kvm -m 1G -drive file=debian-jessie-i386-gnome-install.qemu.img,cache=writeback -redir tcp:2222::22 -net nic -net user -cdrom debian-jessie-DI-rc1-i386-CD-1.iso
[2] http://cdimage.debian.org/cdimage/jessie_di_rc1/i386/jigdo-cd/debian-jessie-DI-rc1-i386-CD-1.jigdo

---

From: Bernhard Übelacker <[email-masked]>
Date: Tue, 20 Jan 2015 19:41:22 +0100

Hello,
came across launchpad bug #1360241 [1] which discusses the same error.
There it comes from ubuntu-ui-toolkit tests.

There they did revert their mesa package to depend on llvm-3.4 instead
of llvm-3.5.

So did I and recompiled mesa to use llvm-3.4 (see attached patch).
And with these packages installed the error message was gone and the login
screen is shown and a login possible.

Going back to current jessie packages depending on llvm-3.5 lead to
getting the error message again.

Kind regards,
Bernhard

[1] https://bugs.launchpad.net/ubuntu/+source/llvm-toolchain-3.5/+bug/1360241

Rebuilding the mesa packages:

root@debian:/home/benutzer# apt-get install debhelper quilt pkg-config libdrm-dev libx11-dev x11proto-gl-dev libxxf86vm-dev libexpat1-dev libxfixes-dev libxdamage-dev libxext-dev libvdpau-dev autoconf automake libtool x11proto-dri2-dev x11proto-dri3-dev x11proto-present-dev libx11-xcb-dev libxcb-dri2-0-dev libxcb-glx0-dev libxcb-xfixes0-dev libxcb-dri3-dev libxcb-present-dev libxcb-randr0-dev libxcb-sync-dev libxshmfence-dev libudev-dev flex bison llvm-3.4-dev libelf-dev libwayland-dev libclang-3.4-dev libclc-dev

benutzer@debian:~$ mkdir mesa; cd mesa
benutzer@debian:~/mesa$ apt-get source libgl1-mesa-dri
benutzer@debian:~/mesa$ cd mesa-10.3.2
benutzer@debian:~/mesa/mesa-10.3.2$ patch -p1 --dry-run < ../switch-to-3.4.patch    # change llvm 3.5 to 3.4 like https://bugs.launchpad.net/ubuntu/+source/llvm-toolchain-3.5/+bug/1360241
benutzer@debian:~/mesa/mesa-10.3.2$ dpkg-buildpackage -b

root@debian:/home/benutzer/mesa# dpkg -i libegl1-mesa_10.3.2-1_i386.deb libegl1-mesa-drivers_10.3.2-1_i386.deb libgbm1_10.3.2-1_i386.deb libgl1-mesa-dri_10.3.2-1_i386.deb libgl1-mesa-dri-dbg_10.3.2-1_i386.deb libgl1-mesa-glx_10.3.2-1_i386.deb libglapi-mesa_10.3.2-1_i386.deb libopenvg1-mesa_10.3.2-1_i386.deb libwayland-egl1-mesa_10.3.2-1_i386.deb libxatracker2_10.3.2-1_i386.deb
    #reboot, error is gone

root@debian:/home/benutzer# apt-get install --reinstall libegl1-mesa libegl1-mesa-drivers libgbm1 libgl1-mesa-dri libgl1-mesa-dri-dbg libgl1-mesa-glx libglapi-mesa libopenvg1-mesa libwayland-egl1-mesa libxatracker2
    #reboot, error is visible again
diff -Nurp mesa-10.3.2.orig/debian/control mesa-10.3.2/debian/control
--- mesa-10.3.2.orig/debian/control	2015-01-20 16:09:09.000000000 +0100
+++ mesa-10.3.2/debian/control	2015-01-20 16:15:14.772000000 +0100
@@ -37,10 +37,10 @@ Build-Depends:
  libudev-dev [linux-any],
  flex,
  bison,
- llvm-3.5-dev (>= 1:3.5-1) [amd64 i386 kfreebsd-amd64 kfreebsd-i386 armhf],
+ llvm-3.4-dev [amd64 i386 kfreebsd-amd64 kfreebsd-i386 armhf],
  libelf-dev [amd64 i386 kfreebsd-amd64 kfreebsd-i386 armhf],
  libwayland-dev (>= 1.2.0) [linux-any],
- libclang-3.5-dev (>= 1:3.5-1) [amd64 i386 armhf],
+ libclang-3.4-dev [amd64 i386 armhf],
  libclc-dev [amd64 i386 armhf],
 Vcs-Git: git://git.debian.org/git/pkg-xorg/lib/mesa
 Vcs-Browser: http://git.debian.org/?p=pkg-xorg/lib/mesa.git
diff -Nurp mesa-10.3.2.orig/debian/rules mesa-10.3.2/debian/rules
--- mesa-10.3.2.orig/debian/rules	2015-01-20 16:09:09.000000000 +0100
+++ mesa-10.3.2/debian/rules	2015-01-20 16:11:47.244000000 +0100
@@ -116,7 +116,7 @@ else
   ifneq (,$(filter $(DEB_HOST_ARCH),amd64 i386 kfreebsd-amd64 kfreebsd-i386 armhf))
 	GALLIUM_DRIVERS += radeonsi
 	confflags_GALLIUM += --enable-gallium-llvm
-	confflags_GALLIUM += ac_cv_path_LLVM_CONFIG=llvm-config-3.5
+	confflags_GALLIUM += ac_cv_path_LLVM_CONFIG=llvm-config-3.4
   endif
 
 	confflags_DIRECT_RENDERING = --enable-driglx-direct

---

From: Bernhard Übelacker <[email-masked]>
Date: Wed, 25 Feb 2015 23:07:05 +0100

Hello,
first a question about this merging with bug #770130 and #776911.
These bugs seem to happen on real hardware with some intel
graphics card and giving an error:
    "*ERROR* pipe A underrun".

But bug #775235 was explicitly opened by Steve McIntyre to be
run inside a i386 KVM and gives an error:
    "LLVM ERROR: Do not know how to split the result of this operator!"

Therefore I wonder if these should really be merged?

----------------

Second I searched a little bit further and came across
this patch discussion: https://freedesktop.org/patch/34445/

This is the start of the patch description by Maarten Lankhorst:
    This fixes a crash when llvmpipe tries to use sse instructions,
    but llvm detects a cpu that doesn't support them.
    
    Fixes for example piglit/bin/amd_seamless_cubemap_per_texture -fbo -auto
    on i386 when run inside "qemu -cpu qemu32", which would otherwise error with:
    "LLVM ERROR: Do not know how to split the result of this operator!"

Looks very similiar to our issue here.
Therefore I build a own local build of libgl1-mesa-dri and that was
successfully solving the issue inside the i386 KVM.

Unfortunately I cannot find this patch ever applied upstream.
The discussion reads (as far as I could follow) that there are
probably better ways to detect CPU features.
Also cannot say which impact it would have on every other systems.

----------------

Did some additional starts of the VM (with debian built packages) to verify:
    qemu-system-i386 -cpu core2duo               works 
    qemu-system-i386 -cpu pentium2               works
    qemu-system-i386 --enable-kvm -cpu host      works
    qemu-system-i386 --enable-kvm -cpu kvm32     works
    qemu-system-i386 --enable-kvm -cpu qemu32    fails
    qemu-system-i386 --enable-kvm                fails
    qemu-system-x86_64 --enable-kvm              fails

Unfortunately "qemu32" seems to be the default.

Kind regards,
Bernhard

These steps should make a local build of libgl1-mesa-dri with the mentioned patch applied:
    mkdir mesa; cd mesa
    apt-get source libgl1-mesa-dri
    cd mesa-10.3.2/
    wget "https://freedesktop.org/patch/34445/raw/" -O debian/patches/08_gallivm_force_sse_instructions_for_llvm_3.5+.patch
    echo 08_gallivm_force_sse_instructions_for_llvm_3.5+.patch >> debian/patches/series
    # install build dependencies
    dpkg-buildpackage -b
    cd ..
    su
        dpkg -i libgl1-mesa-dri_10.3.2-1_i386.deb libgl1-mesa-dri-dbg_10.3.2-1_i386.deb

---

From: PT FINDORA INTERNUSA <[email-masked]>
Date: Tue, 21 Sep 2021 21:15:02 -0700

Hello,

Good morning,

We have gone through your samples from a partner and Here is our  Order 
List. Please do bear in mind that we are very much in  need of this 
order, quote your competitive prices.

Kindly send the Order confirmation.

Your early reply will be much appreciated.

Best Regards,

Maryanah Erwin.

PT FINDORA INTERNUSA

Jln Pahlawan 66 Kec. Arjawinangun

45162 CIREBON West-Java INDONESIA

tel : +62 231 357334

fax: +62 231 357260

email: [email-masked]

---

From: Fabio Pedretti <[email-masked]>
Date: Mon, 18 Mar 2024 11:36:55 +0100

It looks like the src/gallium/auxiliary/gallivm/lp_bld_misc.cpp file
in mesa was heavily updated for managing newer llvm versions, even if
mesa is still using getHostCPUName and still not getCPUTargetFeatures.

Michael, do you think there is still something to do in mesa?

Steve and Bernhard, are you still able to reproduce the original issue?