Bug 1090458

Can you boot with older kernel and confirm that WiFi still works as expected?
If yes, which kernel worked and which broken?  Please give dmesg outputs taken on both working and non-working kernels.

If the old kernel is also broken, it can be rather an issue of kernel-firmware package update instead.

In anyway, please upload the hwinfo output, too.

Created attachment 767910 [details]
hwinfo output

Hi Takashi, nice to meet you again

Please check attached hwinfo.tar.gz, I've added here 2 output: one is full ("hwinfo") and another one is for netcards only ("hwinfo --netcard")

Regarding elder kernels: yesterday I updated my system and eldest kernel I have at the moment is 4.16.1. Could you please tell me where to get any kernel from 4.15.x (4.16.0 definitely has the same problem as 4.16.1/4.16.2)? I could also try with elder version of kernel-firmware but can't understand how to get&test it as well.

Thanks.  Also, could you give the full dmesg output when WiFi crashes?

TW repo doesn't keep the old packages, unfortunately.  Sometimes one older version may remain for some time, but it's not persistent.

So, for testing purpose, now I'm building the last 4.15.x openSUSE kernel in OBS home:tiwai:kernel:4.15 repo.  It'll take some time (an hour or so), and give it a try later.

Getting the older kernel-firmware is also tricky, you'd need to rebuild too.  Maybe we should begin with checking which packages have been updated.

Do you remember about which date the WiFi worked?  You can take a look at /var/log/zypp/history and see which kernel-default and kernel-firmware packages have been updated.

Thanks for the advice. Unfortunately I also can't remember exactly when this problem started, all I can say at the moment that in the end of March I had no such problem.

Looked at /var/log/zypp/history I found that:
- April 01 2018: kernel-firmware-20180320-1.1 was installed together with kernel-default-4.15.14-1.6
- April 04 2018: kernel-firmware-20180402-1.1 was installed together with kernel-default-4.16.0-1.5

Approximately problem started since 04.04.18 and definitely it existed with kernel-firmware-20180402-1.1 and kernel-default-4.16.0-1.5, I'm sure as before yesterday's update I had 4.16.0 kernel and been trying to check if problem exist here or not. It was here.

I think we can try with kernel-firmware first, in my understanding it will impact less things that old kernel, isn't it?

Created attachment 767926 [details]
dmesg output

Hi Takashi, trust you're well

I think I found the root cause: that's firmware and not kernel. I found this in dmesg output:

[    5.661061] ath10k_pci 0000:3e:00.0: enabling device (0000 -> 0002)
[    5.662044] ath10k_pci 0000:3e:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0
[    5.984890] ath10k_pci 0000:3e:00.0: Direct firmware load for ath10k/pre-cal-pci-0000:3e:00.0.bin failed with error -2
[    5.984901] ath10k_pci 0000:3e:00.0: Direct firmware load for ath10k/cal-pci-0000:3e:00.0.bin failed with error -2
[    5.985538] ath10k_pci 0000:3e:00.0: Direct firmware load for ath10k/QCA6174/hw2.1/firmware-6.bin failed with error -2
[    5.989291] ath10k_pci 0000:3e:00.0: qca6174 hw2.1 target 0x05010000 chip_id 0x003405ff sub 1a56:1525
[    5.989293] ath10k_pci 0000:3e:00.0: kconfig debug 0 debugfs 0 tracing 0 dfs 0 testmode 0
[    5.989586] ath10k_pci 0000:3e:00.0: firmware ver SW_RM.1.1.1-00157-QCARMSWPZ-1 api 5 features ignore-otp,no-4addr-pad crc32 10bf8e08
[    6.055008] ath10k_pci 0000:3e:00.0: board_file api 2 bmi_id N/A crc32 ae2e275a
[    7.228251] ath10k_pci 0000:3e:00.0: htt-ver 3.1 wmi-op 4 htt-op 3 cal otp max-sta 32 raw 0 hwcrypto 1
[    7.343807] ath10k_pci 0000:3e:00.0 wlp62s0: renamed from wlan0

As you can see problem happening in the very beginning of boot so other troubles with WiFi might be explained with this problem: all things happening because of no firmware loaded.

I tried to gather additional info as per your recommendation for other bug (probably you remember we've been working with the Bluetooth bug on the same laptop) but was failed. Could you please tell me how to get additional info from the device? This time it's PCI and not USB device.

Well, the non-existing firmware isn't a problem.  The driver supports a new API version 6 while it falls back to API version 5.  Currently the API v6 firmware isn't publicly available, as it seems, so the behavior you've seen is correct.

Now I took a look at the recent ath10k1 development, I found a fix in 4.17-rc1 that is likely relevant with your problem.

55cc11da69895a680940c1733caabc37be685f5e
    Revert "ath10k: send (re)assoc peer command when NSS changed"

I'm going to build a kernel with this fix for testing.

A test kernel is being built on OBS home:tiwai:bnc1090458 repo.
It'll appear at
  http://download.opensuse.org/repositories/home:/tiwai:/bnc1090458/standard/

The kernel contains the possible fix for WiFi crash, and the revert of the previous BT fix.  So, BT may be broken as is with this kernel.

In this bug entry, let's concentrate on the WiFi part.  If the test kernel is confirmed to work for WiFi, I'll push the corresponding fix for TW kernel.

Hi Takashi,

Can't see x86_64 in this repo, could you please check if that will be available?

It's still being built.  You can check the build status via build.opensuse.org.
  https://build.opensuse.org/project/show/home:tiwai:bnc1090458

Hi Takashi,

kernel-default build failed, could you please check it?

Looks like some error in OBS.  I retriggered the build again.  Let's see whether it works now.

Takashi,

Now x86_64 built fine but when I'm trying to install kernel-default 4.16.4-2.1 from your repo zypper gives me a lot of warnings (for more than 2K of packets) and I'm affraid this kernel will breaks my system. Could you please tell me how to try this kernel without system disruption?

(In reply to Ivan Levshin from comment #12)
> Takashi,
> 
> Now x86_64 built fine but when I'm trying to install kernel-default
> 4.16.4-2.1 from your repo zypper gives me a lot of warnings (for more than
> 2K of packets) and I'm affraid this kernel will breaks my system. Could you
> please tell me how to try this kernel without system disruption?

Could you give the exact messages?  Are you sure that you're installing x86_64 one, right?

I've been trying wyth YaST when I got those messages, finally I installed it with zypper without any problem

Now WiFi seems to be running fine - I still need some time for monitoring. Right now I can say that firmware load error still here:

ivan@mynote:~> dmesg|grep ath10k
[    5.571975] ath10k_pci 0000:3e:00.0: enabling device (0000 -> 0002)
[    5.572932] ath10k_pci 0000:3e:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0
[    5.896645] ath10k_pci 0000:3e:00.0: Direct firmware load for ath10k/pre-cal-pci-0000:3e:00.0.bin failed with error -2
[    5.896654] ath10k_pci 0000:3e:00.0: Direct firmware load for ath10k/cal-pci-0000:3e:00.0.bin failed with error -2
[    5.896976] ath10k_pci 0000:3e:00.0: Direct firmware load for ath10k/QCA6174/hw2.1/firmware-6.bin failed with error -2
[    5.900202] ath10k_pci 0000:3e:00.0: qca6174 hw2.1 target 0x05010000 chip_id 0x003405ff sub 1a56:1525
[    5.900204] ath10k_pci 0000:3e:00.0: kconfig debug 0 debugfs 0 tracing 0 dfs 0 testmode 0
[    5.900524] ath10k_pci 0000:3e:00.0: firmware ver SW_RM.1.1.1-00157-QCARMSWPZ-1 api 5 features ignore-otp,no-4addr-pad crc32 10bf8e08
[    5.964525] ath10k_pci 0000:3e:00.0: board_file api 2 bmi_id N/A crc32 ae2e275a
[    7.137174] ath10k_pci 0000:3e:00.0: htt-ver 3.1 wmi-op 4 htt-op 3 cal otp max-sta 32 raw 0 hwcrypto 1
[    7.243263] ath10k_pci 0000:3e:00.0 wlp62s0: renamed from wlan0

But WiFi seems to running fine.

So far, so good.  The firmware load error is no real problem as long as it's about the firmware-6.bin and other *-pci-*.bin.

That's cool but what was the root cause if not firmware?

(In reply to Ivan Levshin from comment #16)
> That's cool but what was the root cause if not firmware?

It was a commit in the ath10k driver and that was reverted in 4.17-rc1.
It should have been sent to stable 4.16.y tree, too.  I'll submit it to Greg later.

Many thanks, Takashi

The fix patch was merged in stable branch and will be included in the next TW kernel.  Let's close.