Bug 561607

Summary: Crashes in kernel if dom0-cpus != 0 in /etc/xen/xend-config.sxp
Product: [openSUSE] openSUSE 11.2 Reporter: Zsolt Sági <novell.admin>
Component: XenAssignee: Jan Beulich <jbeulich>
Status: RESOLVED DUPLICATE QA Contact: E-mail List <qa-bugs>
Severity: Major    
Priority: P5 - None CC: jdouglas
Version: Final   
Target Milestone: ---   
Hardware: x86-64   
OS: openSUSE 11.2   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Zsolt Sági 2009-12-08 13:29:22 UTC 
User-Agent:       Mozilla/5.0 (X11; U; Linux x86_64; hu-HU; rv:1.9.1.5) Gecko/20091103 SUSE/3.5.5-1.1.2 Firefox/3.5.5

[ 3733.402455] BUG: soft lockup - CPU#2 stuck for 61s! [xenwatch_cb:15363]
[ 3733.402455] Modules linked in: xt_tcpudp xt_pkttype xt_physdev ipt_LOG xt_limit netbk blkbk blkback_pagemap blktap xenbus_be edd bridge stp llc xt_NOTRACK ipt_REJECT xt_state iptable_raw iptable_filter nf_conntrack_netbios_ns nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ip_tables ip6_tables x_tables microcode fuse loop dm_round_robin scsi_dh_rdac dm_multipath scsi_dh dm_mod ch osst st i5k_amb ide_pci_generic ata_generic i5000_edac iTCO_wdt 8250_pnp shpchp i2c_i801 iTCO_vendor_support 8250 pci_hotplug e1000e usbhid hid pcspkr edac_core qla2xxx ata_piix i2c_core serio_raw serial_core button sg ext4 jbd2 crc16 uhci_hcd ehci_hcd xenblk cdrom xennet fan processor pata_acpi piix ide_core aacraid thermal thermal_sys hwmon
[ 3733.402455] CPU 2:
[ 3733.402455] Modules linked in: xt_tcpudp xt_pkttype xt_physdev ipt_LOG xt_limit netbk blkbk blkback_pagemap blktap xenbus_be edd bridge stp llc xt_NOTRACK ipt_REJECT xt_state iptable_raw iptable_filter nf_conntrack_netbios_ns nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ip_tables ip6_tables x_tables microcode fuse loop dm_round_robin scsi_dh_rdac dm_multipath scsi_dh dm_mod ch osst st i5k_amb ide_pci_generic ata_generic i5000_edac iTCO_wdt 8250_pnp shpchp i2c_i801 iTCO_vendor_support 8250 pci_hotplug e1000e usbhid hid pcspkr edac_core qla2xxx ata_piix i2c_core serio_raw serial_core button sg ext4 jbd2 crc16 uhci_hcd ehci_hcd xenblk cdrom xennet fan processor pata_acpi piix ide_core aacraid thermal thermal_sys hwmon
[ 3733.402455] Pid: 15363, comm: xenwatch_cb Not tainted 2.6.31.5-0.1-xen #1 Sun Blade X6250 Server
[ 3733.402455] RIP: e030:[<ffffffff8005f032>]  [<ffffffff8005f032>] lock_timer_base+0x32/0x90
[ 3733.402455] RSP: e02b:ffff8803e6c89c10  EFLAGS: 00000246
[ 3733.402455] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffff80778370
[ 3733.402455] RDX: 0000000000000003 RSI: ffff8803e6c89c50 RDI: ffffc90000035280
[ 3733.402455] RBP: ffff8803e6c89c40 R08: ffffffff807813b0 R09: 0000000000000000
[ 3733.402455] R10: ffff8803e6c89cf0 R11: 00000000451b3acc R12: ffffc90000035280
[ 3733.402455] R13: ffff8803e6c89c50 R14: 0000000000000000 R15: ffffffff80778600
[ 3733.402455] FS:  00007f2e7f4fd6f0(0000) GS:ffffc90000020000(0000) knlGS:0000000000000000
[ 3733.402455] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 3733.402455] CR2: 00007f2e7f0b7020 CR3: 0000000000003000 CR4: 0000000000002660
[ 3733.402455] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 3733.402455] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 3733.402455] Call Trace:
[ 3733.402455]  [<ffffffff8005f0bc>] try_to_del_timer_sync+0x2c/0x90
[ 3733.402455]  [<ffffffff8005f14a>] del_timer_sync+0x2a/0x50
[ 3733.402455]  [<ffffffff8046758f>] mce_cpu_callback+0x122/0x1aa
[ 3733.402455]  [<ffffffff80471de7>] notifier_call_chain+0x57/0xb0
[ 3733.402455]  [<ffffffff80075a1c>] __raw_notifier_call_chain+0x1c/0x40
[ 3733.402455]  [<ffffffff8045b90f>] _cpu_down+0xaf/0x310
[ 3733.402455]  [<ffffffff8045bbf7>] cpu_down+0x87/0xb0
[ 3733.402455]  [<ffffffff8046a42c>] vcpu_hotplug+0xce/0x102
[ 3733.402455]  [<ffffffff8046a4ab>] handle_vcpu_hotplug_event+0x4b/0x61
[ 3733.402455]  [<ffffffff80306c4c>] xenwatch_handle_callback+0x2c/0x80
[ 3733.402455]  [<ffffffff8006fb96>] kthread+0xb6/0xc0
[ 3733.402455]  [<ffffffff8000d38a>] child_rip+0xa/0x20
[ 3798.900886] BUG: soft lockup - CPU#2 stuck for 61s! [xenwatch_cb:15363]
[ 3798.900886] Modules linked in: xt_tcpudp xt_pkttype xt_physdev ipt_LOG xt_limit netbk blkbk blkback_pagemap blktap xenbus_be edd bridge stp llc xt_NOTRACK ipt_REJECT xt_state iptable_raw iptable_filter nf_conntrack_netbios_ns nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ip_tables ip6_tables x_tables microcode fuse loop dm_round_robin scsi_dh_rdac dm_multipath scsi_dh dm_mod ch osst st i5k_amb ide_pci_generic ata_generic i5000_edac iTCO_wdt 8250_pnp shpchp i2c_i801 iTCO_vendor_support 8250 pci_hotplug e1000e usbhid hid pcspkr edac_core qla2xxx ata_piix i2c_core serio_raw serial_core button sg ext4 jbd2 crc16 uhci_hcd ehci_hcd xenblk cdrom xennet fan processor pata_acpi piix ide_core aacraid thermal thermal_sys hwmon
[ 3798.900886] CPU 2:
[ 3798.900886] Modules linked in: xt_tcpudp xt_pkttype xt_physdev ipt_LOG xt_limit netbk blkbk blkback_pagemap blktap xenbus_be edd bridge stp llc xt_NOTRACK ipt_REJECT xt_state iptable_raw iptable_filter nf_conntrack_netbios_ns nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ip_tables ip6_tables x_tables microcode fuse loop dm_round_robin scsi_dh_rdac dm_multipath scsi_dh dm_mod ch osst st i5k_amb ide_pci_generic ata_generic i5000_edac iTCO_wdt 8250_pnp shpchp i2c_i801 iTCO_vendor_support 8250 pci_hotplug e1000e usbhid hid pcspkr edac_core qla2xxx ata_piix i2c_core serio_raw serial_core button sg ext4 jbd2 crc16 uhci_hcd ehci_hcd xenblk cdrom xennet fan processor pata_acpi piix ide_core aacraid thermal thermal_sys hwmon
[ 3798.900886] Pid: 15363, comm: xenwatch_cb Not tainted 2.6.31.5-0.1-xen #1 Sun Blade X6250 Server
[ 3798.900886] RIP: e030:[<ffffffff8005f07f>]  [<ffffffff8005f07f>] lock_timer_base+0x7f/0x90
[ 3798.900886] RSP: e02b:ffff8803e6c89c10  EFLAGS: 00000246
[ 3798.900886] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffff80778370
[ 3798.900886] RDX: 0000000000000003 RSI: ffff8803e6c89c50 RDI: ffffc90000035280
[ 3798.900886] RBP: ffff8803e6c89c40 R08: ffffffff807813b0 R09: 0000000000000000
[ 3798.900886] R10: ffff8803e6c89cf0 R11: 00000000451b3acc R12: ffffc90000035280
[ 3798.900886] R13: ffff8803e6c89c50 R14: 0000000000000000 R15: ffffffff80778600
[ 3798.900886] FS:  00007f2e7f4fd6f0(0000) GS:ffffc90000020000(0000) knlGS:0000000000000000
[ 3798.900886] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 3798.900886] CR2: 00007f2e7f0b7020 CR3: 0000000000003000 CR4: 0000000000002660
[ 3798.900886] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 3798.900886] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 3798.900886] Call Trace:
[ 3798.900886]  [<ffffffff8005f0bc>] try_to_del_timer_sync+0x2c/0x90
[ 3798.900886]  [<ffffffff8005f14a>] del_timer_sync+0x2a/0x50
[ 3798.900886]  [<ffffffff8046758f>] mce_cpu_callback+0x122/0x1aa
[ 3798.900886]  [<ffffffff80471de7>] notifier_call_chain+0x57/0xb0
[ 3798.900886]  [<ffffffff80075a1c>] __raw_notifier_call_chain+0x1c/0x40
[ 3798.900886]  [<ffffffff8045b90f>] _cpu_down+0xaf/0x310
[ 3798.900886]  [<ffffffff8045bbf7>] cpu_down+0x87/0xb0
[ 3798.900886]  [<ffffffff8046a42c>] vcpu_hotplug+0xce/0x102
[ 3798.900886]  [<ffffffff8046a4ab>] handle_vcpu_hotplug_event+0x4b/0x61
[ 3798.900886]  [<ffffffff80306c4c>] xenwatch_handle_callback+0x2c/0x80
[ 3798.900886]  [<ffffffff8006fb96>] kthread+0xb6/0xc0
[ 3798.900886]  [<ffffffff8000d38a>] child_rip+0xa/0x20
[ 3841.096112] INFO: task xend:14313 blocked for more than 120 seconds.
[ 3841.096123] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 3841.096129] xend          D ffffffffffffffff     0 14313  14309 0x00000000
[ 3841.096136]  ffff8803e55ebe08 0000000000000282 ffff8803e55ebd48 ffff8803e55ebd88
[ 3841.096144]  00000000ffffffff ffff8803e55ebdd0 000000000000a380 ffff8803d67be668
[ 3841.096152]  000000000000a380 000000000000a380 000000000000a380 0000000000007d00
[ 3841.096159] Call Trace:
[ 3841.096189]  [<ffffffff8046d145>] __mutex_lock_slowpath+0xe5/0x1b0
[ 3841.096196]  [<ffffffff8046c9cc>] mutex_lock+0x2c/0x60
[ 3841.096204]  [<ffffffff80051745>] get_online_cpus+0x35/0x70
[ 3841.096211]  [<ffffffff8006b488>] schedule_on_each_cpu+0x48/0x130
[ 3841.096220]  [<ffffffff800e1063>] lru_add_drain_all+0x23/0x40
[ 3841.096227]  [<ffffffff800fb355>] sys_mlock+0x65/0x130
[ 3841.096235]  [<ffffffff8000c868>] system_call_fastpath+0x16/0x1b
[ 3841.096257]  [<00007f59e0a2bed7>] 0x7f59e0a2bed7
[ 3841.096262] INFO: task xenwatch_cb:15364 blocked for more than 120 seconds.
[ 3841.096266] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 3841.096270] xenwatch_cb   D 000000003bfd46d0     0 15364      2 0x00000000
[ 3841.096277]  ffff8803d7e89d10 0000000000000246 ffff8803d7e89c50 ffff8803d7e89c90
[ 3841.096284]  ffffffff80781388 ffff8803d7e89cd8 000000000000a380 ffff8803d56383e8
[ 3841.096291]  000000000000a380 000000000000a380 000000000000a380 0000000000007d00
[ 3841.096298] Call Trace:
[ 3841.096304]  [<ffffffff8046d145>] __mutex_lock_slowpath+0xe5/0x1b0
[ 3841.096310]  [<ffffffff8046c9cc>] mutex_lock+0x2c/0x60
[ 3841.096315]  [<ffffffff800516b3>] cpu_maps_update_begin+0x23/0x40
[ 3841.096322]  [<ffffffff8045bbb5>] cpu_down+0x45/0xb0
[ 3841.096327]  [<ffffffff8046a42c>] vcpu_hotplug+0xce/0x102
[ 3841.096334]  [<ffffffff8046a4ab>] handle_vcpu_hotplug_event+0x4b/0x61
[ 3841.096342]  [<ffffffff80306c4c>] xenwatch_handle_callback+0x2c/0x80
[ 3841.096349]  [<ffffffff8006fb96>] kthread+0xb6/0xc0
[ 3841.096356]  [<ffffffff8000d38a>] child_rip+0xa/0x20
[ 3841.096361] INFO: task xenwatch_cb:15365 blocked for more than 120 seconds.
[ 3841.096365] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 3841.096370] xenwatch_cb   D 00000000b05e58a5     0 15365      2 0x00000000
[ 3841.096376]  ffff8803e6787d10 0000000000000246 ffff8803e6787c50 ffff8803e6787c90
[ 3841.096383]  ffffffff80781388 ffff8803e6787cd8 000000000000a380 ffff8803ea2c68a8
[ 3841.096390]  000000000000a380 000000000000a380 000000000000a380 0000000000007d00
[ 3841.096397] Call Trace:
[ 3841.096403]  [<ffffffff8046d145>] __mutex_lock_slowpath+0xe5/0x1b0
[ 3841.096408]  [<ffffffff8046c9cc>] mutex_lock+0x2c/0x60
[ 3841.096414]  [<ffffffff800516b3>] cpu_maps_update_begin+0x23/0x40
[ 3841.096420]  [<ffffffff8045bbb5>] cpu_down+0x45/0xb0
[ 3841.096425]  [<ffffffff8046a42c>] vcpu_hotplug+0xce/0x102
[ 3841.096431]  [<ffffffff8046a4ab>] handle_vcpu_hotplug_event+0x4b/0x61
[ 3841.096437]  [<ffffffff80306c4c>] xenwatch_handle_callback+0x2c/0x80
[ 3841.096443]  [<ffffffff8006fb96>] kthread+0xb6/0xc0
[ 3841.096448]  [<ffffffff8000d38a>] child_rip+0xa/0x20
[ 3841.096454] INFO: task xenwatch_cb:15367 blocked for more than 120 seconds.
[ 3841.096458] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 3841.096462] xenwatch_cb   D ffffffffffffffff     0 15367      2 0x00000000
[ 3841.096469]  ffff8803e89a3d10 0000000000000246 ffffffff808b46b0 ffff8803e89a3c90
[ 3841.096476]  0000000000000001 ffff8803e89a3cd8 000000000000a380 ffff8803e8bbc3e8
[ 3841.096483]  000000000000a380 000000000000a380 000000000000a380 0000000000007d00
[ 3841.096490] Call Trace:
[ 3841.096495]  [<ffffffff8046d145>] __mutex_lock_slowpath+0xe5/0x1b0
[ 3841.096501]  [<ffffffff8046c9cc>] mutex_lock+0x2c/0x60
[ 3841.096506]  [<ffffffff800516b3>] cpu_maps_update_begin+0x23/0x40
[ 3841.096512]  [<ffffffff8045bbb5>] cpu_down+0x45/0xb0
[ 3841.096517]  [<ffffffff8046a42c>] vcpu_hotplug+0xce/0x102
[ 3841.096523]  [<ffffffff8046a4ab>] handle_vcpu_hotplug_event+0x4b/0x61
[ 3841.096529]  [<ffffffff80306c4c>] xenwatch_handle_callback+0x2c/0x80
[ 3841.096535]  [<ffffffff8006fb96>] kthread+0xb6/0xc0
[ 3841.096541]  [<ffffffff8000d38a>] child_rip+0xa/0x20
[ 3841.096546] INFO: task xenwatch_cb:15369 blocked for more than 120 seconds.
[ 3841.096550] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 3841.096554] xenwatch_cb   D 000000006ba6c164     0 15369      2 0x00000000
[ 3841.096561]  ffff8803e36cbd20 0000000000000246 0000000000000001 ffff8803e36cbca0
[ 3841.096568]  ffff8803e36cbc50 ffff8803e36cbce8 000000000000a380 ffff8803d51e0b68
[ 3841.096575]  000000000000a380 000000000000a380 000000000000a380 0000000000007d00
[ 3841.096582] Call Trace:
[ 3841.096587]  [<ffffffff8046d145>] __mutex_lock_slowpath+0xe5/0x1b0
[ 3841.096593]  [<ffffffff8046c9cc>] mutex_lock+0x2c/0x60
[ 3841.096598]  [<ffffffff800516b3>] cpu_maps_update_begin+0x23/0x40
[ 3841.096604]  [<ffffffff80468684>] cpu_up+0x55/0x93
[ 3841.096610]  [<ffffffff8046a408>] vcpu_hotplug+0xaa/0x102
[ 3841.096615]  [<ffffffff8046a4ab>] handle_vcpu_hotplug_event+0x4b/0x61
[ 3841.096622]  [<ffffffff80306c4c>] xenwatch_handle_callback+0x2c/0x80
[ 3841.096627]  [<ffffffff8006fb96>] kthread+0xb6/0xc0
[ 3841.096633]  [<ffffffff8000d38a>] child_rip+0xa/0x20
[ 3841.096638] INFO: task xenwatch_cb:15371 blocked for more than 120 seconds.
[ 3841.096642] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 3841.096647] xenwatch_cb   D 0000000051fa4cc8     0 15371      2 0x00000000
[ 3841.096653]  ffff8803db5d5d10 0000000000000246 ffffffff808b46b0 ffff8803db5d5c90
[ 3841.096660]  0000000000000001 ffff8803db5d5cd8 000000000000a380 ffff8803e4cc66e8
[ 3841.096667]  000000000000a380 000000000000a380 000000000000a380 0000000000007d00
[ 3841.096674] Call Trace:
[ 3841.096679]  [<ffffffff8046d145>] __mutex_lock_slowpath+0xe5/0x1b0
[ 3841.096685]  [<ffffffff8046c9cc>] mutex_lock+0x2c/0x60
[ 3841.096691]  [<ffffffff800516b3>] cpu_maps_update_begin+0x23/0x40
[ 3841.096696]  [<ffffffff8045bbb5>] cpu_down+0x45/0xb0
[ 3841.096701]  [<ffffffff8046a42c>] vcpu_hotplug+0xce/0x102
[ 3841.096707]  [<ffffffff8046a4ab>] handle_vcpu_hotplug_event+0x4b/0x61
[ 3841.096713]  [<ffffffff80306c4c>] xenwatch_handle_callback+0x2c/0x80
[ 3841.096719]  [<ffffffff8006fb96>] kthread+0xb6/0xc0
[ 3841.096724]  [<ffffffff8000d38a>] child_rip+0xa/0x20
[ 3841.096730] INFO: task xenwatch_cb:15373 blocked for more than 120 seconds.
[ 3841.096734] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 3841.096738] xenwatch_cb   D 0000000015973942     0 15373      2 0x00000000
[ 3841.096744]  ffff8803d7973d10 0000000000000246 ffffffff808b46b0 ffff8803d7973c90
[ 3841.096752]  0000000000000001 ffff8803d7973cd8 000000000000a380 ffff8803e89ee4a8
[ 3841.096759]  000000000000a380 000000000000a380 000000000000a380 0000000000007d00
[ 3841.096766] Call Trace:
[ 3841.096771]  [<ffffffff8046d145>] __mutex_lock_slowpath+0xe5/0x1b0
[ 3841.096777]  [<ffffffff8046c9cc>] mutex_lock+0x2c/0x60
[ 3841.096782]  [<ffffffff800516b3>] cpu_maps_update_begin+0x23/0x40
[ 3841.096788]  [<ffffffff8045bbb5>] cpu_down+0x45/0xb0
[ 3841.096793]  [<ffffffff8046a42c>] vcpu_hotplug+0xce/0x102
[ 3841.096799]  [<ffffffff8046a4ab>] handle_vcpu_hotplug_event+0x4b/0x61
[ 3841.096805]  [<ffffffff80306c4c>] xenwatch_handle_callback+0x2c/0x80
[ 3841.096811]  [<ffffffff8006fb96>] kthread+0xb6/0xc0
[ 3841.096816]  [<ffffffff8000d38a>] child_rip+0xa/0x20
[ 3841.096822] INFO: task xenwatch_cb:15375 blocked for more than 120 seconds.
[ 3841.096826] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 3841.096830] xenwatch_cb   D 00000000c9a1aaa1     0 15375      2 0x00000000
[ 3841.096836]  ffff8803e0b57d10 0000000000000246 ffff8803e0b57c58 ffff8803e0b57c90
[ 3841.096843]  ffff8803e0b57c58 ffff8803e0b57cd8 000000000000a380 ffff8803d520c528
[ 3841.096850]  000000000000a380 000000000000a380 000000000000a380 0000000000007d00
[ 3841.096857] Call Trace:
[ 3841.096862]  [<ffffffff8046d145>] __mutex_lock_slowpath+0xe5/0x1b0
[ 3841.096868]  [<ffffffff8046c9cc>] mutex_lock+0x2c/0x60
[ 3841.096874]  [<ffffffff800516b3>] cpu_maps_update_begin+0x23/0x40
[ 3841.096880]  [<ffffffff8045bbb5>] cpu_down+0x45/0xb0
[ 3841.096885]  [<ffffffff8046a42c>] vcpu_hotplug+0xce/0x102
[ 3841.096890]  [<ffffffff8046a4ab>] handle_vcpu_hotplug_event+0x4b/0x61
[ 3841.096897]  [<ffffffff80306c4c>] xenwatch_handle_callback+0x2c/0x80
[ 3841.096902]  [<ffffffff8006fb96>] kthread+0xb6/0xc0
[ 3841.096908]  [<ffffffff8000d38a>] child_rip+0xa/0x20
[ 3864.399395] BUG: soft lockup - CPU#2 stuck for 61s! [xenwatch_cb:15363]
[ 3864.399395] Modules linked in: xt_tcpudp xt_pkttype xt_physdev ipt_LOG xt_limit netbk blkbk blkback_pagemap blktap xenbus_be edd bridge stp llc xt_NOTRACK ipt_REJECT xt_state iptable_raw iptable_filter nf_conntrack_netbios_ns nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ip_tables ip6_tables x_tables microcode fuse loop dm_round_robin scsi_dh_rdac dm_multipath scsi_dh dm_mod ch osst st i5k_amb ide_pci_generic ata_generic i5000_edac iTCO_wdt 8250_pnp shpchp i2c_i801 iTCO_vendor_support 8250 pci_hotplug e1000e usbhid hid pcspkr edac_core qla2xxx ata_piix i2c_core serio_raw serial_core button sg ext4 jbd2 crc16 uhci_hcd ehci_hcd xenblk cdrom xennet fan processor pata_acpi piix ide_core aacraid thermal thermal_sys hwmon
[ 3864.399395] CPU 2:
[ 3864.399395] Modules linked in: xt_tcpudp xt_pkttype xt_physdev ipt_LOG xt_limit netbk blkbk blkback_pagemap blktap xenbus_be edd bridge stp llc xt_NOTRACK ipt_REJECT xt_state iptable_raw iptable_filter nf_conntrack_netbios_ns nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ip_tables ip6_tables x_tables microcode fuse loop dm_round_robin scsi_dh_rdac dm_multipath scsi_dh dm_mod ch osst st i5k_amb ide_pci_generic ata_generic i5000_edac iTCO_wdt 8250_pnp shpchp i2c_i801 iTCO_vendor_support 8250 pci_hotplug e1000e usbhid hid pcspkr edac_core qla2xxx ata_piix i2c_core serio_raw serial_core button sg ext4 jbd2 crc16 uhci_hcd ehci_hcd xenblk cdrom xennet fan processor pata_acpi piix ide_core aacraid thermal thermal_sys hwmon
[ 3864.399395] Pid: 15363, comm: xenwatch_cb Not tainted 2.6.31.5-0.1-xen #1 Sun Blade X6250 Server
[ 3864.399395] RIP: e030:[<ffffffff8005f07f>]  [<ffffffff8005f07f>] lock_timer_base+0x7f/0x90
[ 3864.399395] RSP: e02b:ffff8803e6c89c10  EFLAGS: 00000246
[ 3864.399395] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffff80778370
[ 3864.399395] RDX: 0000000000000003 RSI: ffff8803e6c89c50 RDI: ffffc90000035280
[ 3864.399395] RBP: ffff8803e6c89c40 R08: ffffffff807813b0 R09: 0000000000000000
[ 3864.399395] R10: ffff8803e6c89cf0 R11: 00000000451b3acc R12: ffffc90000035280
[ 3864.399395] R13: ffff8803e6c89c50 R14: 0000000000000000 R15: ffffffff80778600
[ 3864.399395] FS:  00007f2e7f4fd6f0(0000) GS:ffffc90000020000(0000) knlGS:0000000000000000
[ 3864.399395] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 3864.399395] CR2: 00007f2e7f0b7020 CR3: 0000000000003000 CR4: 0000000000002660
[ 3864.399395] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 3864.399395] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 3864.399395] Call Trace:
[ 3864.399395]  [<ffffffff8005f0bc>] try_to_del_timer_sync+0x2c/0x90
[ 3864.399395]  [<ffffffff8005f14a>] del_timer_sync+0x2a/0x50
[ 3864.399395]  [<ffffffff8046758f>] mce_cpu_callback+0x122/0x1aa
[ 3864.399395]  [<ffffffff80471de7>] notifier_call_chain+0x57/0xb0
[ 3864.399395]  [<ffffffff80075a1c>] __raw_notifier_call_chain+0x1c/0x40
[ 3864.399395]  [<ffffffff8045b90f>] _cpu_down+0xaf/0x310
[ 3864.399395]  [<ffffffff8045bbf7>] cpu_down+0x87/0xb0
[ 3864.399395]  [<ffffffff8046a42c>] vcpu_hotplug+0xce/0x102
[ 3864.399395]  [<ffffffff8046a4ab>] handle_vcpu_hotplug_event+0x4b/0x61
[ 3864.399395]  [<ffffffff80306c4c>] xenwatch_handle_callback+0x2c/0x80
[ 3864.399395]  [<ffffffff8006fb96>] kthread+0xb6/0xc0
[ 3864.399395]  [<ffffffff8000d38a>] child_rip+0xa/0x20
[ 3929.898618] BUG: soft lockup - CPU#2 stuck for 61s! [xenwatch_cb:15363]
[ 3929.898618] Modules linked in: xt_tcpudp xt_pkttype xt_physdev ipt_LOG xt_limit netbk blkbk blkback_pagemap blktap xenbus_be edd bridge stp llc xt_NOTRACK ipt_REJECT xt_state iptable_raw iptable_filter nf_conntrack_netbios_ns nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ip_tables ip6_tables x_tables microcode fuse loop dm_round_robin scsi_dh_rdac dm_multipath scsi_dh dm_mod ch osst st i5k_amb ide_pci_generic ata_generic i5000_edac iTCO_wdt 8250_pnp shpchp i2c_i801 iTCO_vendor_support 8250 pci_hotplug e1000e usbhid hid pcspkr edac_core qla2xxx ata_piix i2c_core serio_raw serial_core button sg ext4 jbd2 crc16 uhci_hcd ehci_hcd xenblk cdrom xennet fan processor pata_acpi piix ide_core aacraid thermal thermal_sys hwmon
[ 3929.898618] CPU 2:
[ 3929.898618] Modules linked in: xt_tcpudp xt_pkttype xt_physdev ipt_LOG xt_limit netbk blkbk blkback_pagemap blktap xenbus_be edd bridge stp llc xt_NOTRACK ipt_REJECT xt_state iptable_raw iptable_filter nf_conntrack_netbios_ns nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ip_tables ip6_tables x_tables microcode fuse loop dm_round_robin scsi_dh_rdac dm_multipath scsi_dh dm_mod ch osst st i5k_amb ide_pci_generic ata_generic i5000_edac iTCO_wdt 8250_pnp shpchp i2c_i801 iTCO_vendor_support 8250 pci_hotplug e1000e usbhid hid pcspkr edac_core qla2xxx ata_piix i2c_core serio_raw serial_core button sg ext4 jbd2 crc16 uhci_hcd ehci_hcd xenblk cdrom xennet fan processor pata_acpi piix ide_core aacraid thermal thermal_sys hwmon
[ 3929.898618] Pid: 15363, comm: xenwatch_cb Not tainted 2.6.31.5-0.1-xen #1 Sun Blade X6250 Server
[ 3929.898618] RIP: e030:[<ffffffff8005f02d>]  [<ffffffff8005f02d>] lock_timer_base+0x2d/0x90
[ 3929.898618] RSP: e02b:ffff8803e6c89c10  EFLAGS: 00000246
[ 3929.898618] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffff80778370
[ 3929.898618] RDX: 0000000000000003 RSI: ffff8803e6c89c50 RDI: ffffc90000035280
[ 3929.898618] RBP: ffff8803e6c89c40 R08: ffffffff807813b0 R09: 0000000000000000
[ 3929.898618] R10: ffff8803e6c89cf0 R11: 00000000451b3acc R12: ffffc90000035280
[ 3929.898618] R13: ffff8803e6c89c50 R14: 0000000000000000 R15: ffffffff80778600
[ 3929.898618] FS:  00007f2e7f4fd6f0(0000) GS:ffffc90000020000(0000) knlGS:0000000000000000
[ 3929.898618] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 3929.898618] CR2: 00007f2e7f0b7020 CR3: 0000000000003000 CR4: 0000000000002660
[ 3929.898618] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 3929.898618] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 3929.898618] Call Trace:
[ 3929.898618]  [<ffffffff8005f0bc>] try_to_del_timer_sync+0x2c/0x90
[ 3929.898618]  [<ffffffff8005f14a>] del_timer_sync+0x2a/0x50
[ 3929.898618]  [<ffffffff8046758f>] mce_cpu_callback+0x122/0x1aa
[ 3929.898618]  [<ffffffff80471de7>] notifier_call_chain+0x57/0xb0
[ 3929.898618]  [<ffffffff80075a1c>] __raw_notifier_call_chain+0x1c/0x40
[ 3929.898618]  [<ffffffff8045b90f>] _cpu_down+0xaf/0x310
[ 3929.898618]  [<ffffffff8045bbf7>] cpu_down+0x87/0xb0
[ 3929.898618]  [<ffffffff8046a42c>] vcpu_hotplug+0xce/0x102
[ 3929.898618]  [<ffffffff8046a4ab>] handle_vcpu_hotplug_event+0x4b/0x61
[ 3929.898618]  [<ffffffff80306c4c>] xenwatch_handle_callback+0x2c/0x80
[ 3929.898618]  [<ffffffff8006fb96>] kthread+0xb6/0xc0

Reproducible: Always
Comment 1 Zsolt Sági 2009-12-08 13:32:21 UTC 
The xen kernel (al least the 64 bit version on Sun Blade X6250) usually crashes either way after running for a few days, but a dom0-cpus value other than 0 causes an immadiate crash.
Comment 3 Jan Beulich 2009-12-10 07:42:31 UTC 
.

*** This bug has been marked as a duplicate of bug 558663 ***