Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support for the S0ix sleep state #6411

Open
e6lk7dqzm83p opened this issue Feb 17, 2021 · 139 comments
Open

Support for the S0ix sleep state #6411

e6lk7dqzm83p opened this issue Feb 17, 2021 · 139 comments
Assignees
@HW42
Labels
C: power management C: Xen hardware support P: major Priority: major. Between "default" and "critical" in severity. T: enhancement Type: enhancement. A new feature that does not yet exist or improvement of existing functionality. waiting for upstream This issue is waiting for something from an upstream project to arrive in Qubes. Remove when closed.

Comments

@e6lk7dqzm83p
Copy link

e6lk7dqzm83p commented Feb 17, 2021 •
edited by andrewdavidwong
Loading

Current status

Initial support of S0ix is available for testing; see #6411 (comment).

Editor's note: The original issue description below is somewhat out-of-date. This issue is now narrowly focused on support for the S0ix sleep state (aka "Modern Standby," "Low Power S0 Idle," "InstantGo," "Connected Standby," etc.) found in newer CPUs (especially Tiger Lake and later).

The problem you're addressing (if any)

It appears Qubes does not support Intel's latest version of processors.

Describe the solution you'd like

Support for Intel's latest version of their processors (i.e., tiger lake)

Where is the value to a user, and who might that user be?

People who want to benefit from the significant performance upgrades of Tiger Lake

Describe alternatives you've considered

N/A

Additional context

N/A

Relevant documentation you've consulted

N/A (I've checked GitHub for any open issues on this but either I've missed it or my SearchFu is weak)

Related, non-duplicate issues

N/A
@e6lk7dqzm83p e6lk7dqzm83p added P: default Priority: default. Default priority for new issues, to be replaced given sufficient information. T: enhancement Type: enhancement. A new feature that does not yet exist or improvement of existing functionality. labels Feb 17, 2021
@e6lk7dqzm83p
Copy link
Author

@marmarek I believe this has to do with the current kernel Qubes is using?

@DemiMarie
Copy link

That would not surprise me. R4.1 has several advantages here, including being able to include multiple kernels in the install ISO.

@e6lk7dqzm83p e6lk7dqzm83p changed the title Support for Intel Tiger Lake Processoes Support for Intel Tiger Lake Processors Feb 17, 2021
@e6lk7dqzm83p
Copy link
Author

@DemiMarie, does that mean R4.1 will support Tiger Lake? Sorry if I'm being obtuse.

@DemiMarie
Copy link

@e6lk7dqzm83p that’s a question that @marmarek would be better positioned to answer, sorry.

@e6lk7dqzm83p
Copy link
Author

No problem; I appreciate the insight!

@andrewdavidwong andrewdavidwong added this to the TBD milestone Feb 17, 2021
@marmarek
Copy link
Member

I believe this is close related to #5374, which this time is more about Xen than Linux. @e6lk7dqzm83p do you see some specific error message?

@e6lk7dqzm83p
Copy link
Author

Apologies, I do not. I was talking to System76 about about their Qubes support with the newer chipsets and was informed that Tiger Lake was not currently supported due to a kernel issue (I'm trying to ensure Qubes support before I purchase my new computer). I was trying to see what the status of that issue was and couldn't find any documentation here so I opened the request.

Sorry if that was in error/off protocol.

@andrewdavidwong andrewdavidwong added C: Xen needs diagnosis Requires technical diagnosis from developer. Replace with "diagnosed" or remove if otherwise closed. T: bug Type: bug report. A problem or defect resulting in unintended behavior in something that exists. and removed C: kernel T: enhancement Type: enhancement. A new feature that does not yet exist or improvement of existing functionality. labels Feb 19, 2021
@andrewdavidwong andrewdavidwong modified the milestones: TBD, Release 4.0 updates Feb 19, 2021
@marmarek
Copy link
Member

Generally, for Intel systems >=10th gen I'd recommend trying Qubes 4.1. Some may work with Qubes 4.0.x too, but backporting some of the hardware support parts (especially on Xen part) is impractical.

marmarek added a commit to marmarek/qubes-vmm-xen that referenced this issue Feb 20, 2021
@marmarek
Backport HPET fix for recent Intel systems
075e6b1 
This fixes "IO-APIC + timer doesn't work!" boot crash

Fixes QubesOS/qubes-issues#6372
Fixes QubesOS/qubes-issues#5374
Related to QubesOS/qubes-issues#6411
Closed
@ideologysec
Copy link

Any chance of seeing a new testing release of 4.1 built soon? The last one I know of in the repo is 6+ months old, especially with the HPET fix just landed, it'd be great to have an updated installer.

@ideologysec
Copy link

Managed to find https://openqa.qubes-os.org/group_overview/1

Not documented anywhere that I can find, but it does appear to be the output of the automated build system from time to time. Last build for 4.1 was ~7 days ago, won't contain these fixes, and failed. What is the current channel for getting the latest 4.1 iso, if any other than this or the kernel.org mirrors?

@marmarek
Copy link
Member

openQA is the right place for looking for test builds. But note the CI system is not a trusted build environment, so do not use those builds for anything else than testing.

@fnerdman
Copy link

@marmarek when I tested TGL system76 hardware with qubes 4.0 I was able to boot it via modification of the coreboot firmware. Checkout this patch: system76/coreboot#42
What I was not able to fix were suspend issues, as Intel has dropped S3 support with TGL and moved to s0ix only. Does Qubes 4.1 with Xen 4.14 support s0ix suspend?

@marmarek
Copy link
Member

Does Qubes 4.1 with Xen 4.14 support s0ix suspend?

Sadly, not.

@DemiMarie
Copy link

Does Qubes 4.1 with Xen 4.14 support s0ix suspend?

Sadly, not.

I wonder if we should reconsider staying on a single Xen version for an entire release. This leads to rather awkward hardware support problems.

Does Xen 4.15 support s0ix?

@marmarek
Copy link
Member

No, and I'm just learning how absurdly unfriendly to virtualization power management on newest Intel platform is.

marmarek added a commit to QubesOS/qubes-core-admin that referenced this issue May 1, 2024
@marmarek
Update dependencies for s0ix support
4e52773 
QubesOS/qubes-issues#6411
@qubesos-bot qubesos-bot mentioned this issue May 1, 2024
Closed
@marmarek
Copy link
Member

marmarek commented May 1, 2024 •
edited
Loading

Initial version (the thing discussed in the last few comments) is merged and available in current-testing repository.
Specifically, it needs:

  • qubes-core-dom0 >= 4.2.25 (and its dependencies)
  • kernel-latest >= 6.8.8 for dom0
  • kernel-latest-qubes-vm >= 6.8.8 (and use it at least for sys-net and sys-usb)
  • qubes-core-agent >= 4.2.32 (in template(s) for sys-net and sys-usb)

Then, the feature is opt-in. To use it, you need to:

  1. detach "Thunderbolt 4 NHI" devices from sys-usb (they won't be used by dom0 either)
  2. qvm-features dom0 suspend-s0ix 1
  3. reboot

Ensure /sys/power/mem_sleep has s2idle selected. It will the case on systems not supporting S3 anyway, but on others may need to be selected manually.

The system is expected to sleep about 90% of the time (not 100% as native Linux does on some systems), which will affect how long it will stay suspended on battery. We will work on the remaining 10%, but in the meantime it is what it is.
The amount of time the system was in the proper suspend state can be seen in /sys/kernel/debug/pmc_core/substate_residencies (compare values before suspend and after, and interpret in relation to how long it was suspended on the wall clock). There are other interesting files in this directory that may help debugging issues, especially substate_requirements file.

@ekaflaereht
Copy link

ekaflaereht commented May 2, 2024 •
edited
Loading

Many thanks for working on this. I have tried it on my HP 845 G10 with a Ryzen 7840U (similar platform as the framework 13) .

These is my environment:

  • qubes-core-dom0 4.2.25-1
  • kernel-latest 6.8.8-1
  • kernel-latest-qubes-vm 6.8.8-1 (used by sys-net and sys-usb)
  • qubes-core-agent 4.2.32-1+deb12u1 (used by sys-net and sys-usb)
  1. USB4/Thunderbolt NHI controlers were not attached to sys-usb because they were breaking suspend for me before
  2. qvm-features dom0 suspend-s0ix reports 1 after enabling it as described above

I do not have the sys dir /sys/kernel/debug/pmc_core dir. Do I have to use some kernel cmd line parameter for it?

The power consumption in suspend did not change after the process of updating qubes-core-agent, qubes-core-dom0, enabling the qvm feature and rebooting.

What should be the result of these changes? Suspend was working before even though s2idle was reported in /sys/power/mem_sleep

As I said I was using the latest kernel before because in March I have reported here that the power consumption in suspend with Qubes OS was 3.6 Watt, but now it is measured on the AC adapter between 2.78 and 2.86 Watt.
So it seems something has improved since March but not with the latest changes and it is not on the level as with a regular Linux distribution yet.

Let me know what else is relevant or if I should test something.

@marmarek
Copy link
Member

marmarek commented May 2, 2024

Many thanks for working on this. I have tried it on my HP 845 G10 with a Ryzen 7840U (similar platform as the framework 13) .

The current iteration focuses on Intel. AMD might get some improvements as a "side effect", but no promises...

I do not have the sys dir /sys/kernel/debug/pmc_core dir. Do I have to use some kernel cmd line parameter for it?

AMD has it somewhere else, check /sys/kernel/debug/amd_pcm. That said, we have some patches to allow dom0 access to some MSR relevant for pmc_core reporting, AMD may need similar work too.

The power consumption in suspend did not change after the process of updating qubes-core-agent, qubes-core-dom0, enabling the qvm feature and rebooting.

The main expected result is that system can suspend and successfully wakeup. The secondary effect should be that suspended system uses ~90% less power than non-suspended one. So, if I understand correctly, you get the first one (even before this update), but not the second one.

@ekaflaereht
Copy link

AMD has it somewhere else, check /sys/kernel/debug/amd_pcm

Yes, true, that does exists and it has an s0ix_stats file but that one reports 0 for all three values before and after a short suspend of 30 seconds or so.

When I check the same value on another Ryzen system with a regular Linux distribution it reports quite high values, so I guess s0ix sleep state is not really supported yet on Xen with AMD cpus but I was wondering what kind of suspend is then used in QubesOS?

That said, we have some patches to allow dom0 access to some MSR relevant for pmc_core reporting, AMD may need similar work too.

Yes, the amd_s2idle.py script reports some MSR related issue. And of course it complains about missing Thunderbold support, but I have commented out the check.

The main expected result is that system can suspend and successfully wakeup. The secondary effect should be that suspended system uses ~90% less power than non-suspended one. So, if I understand correctly, you get the first one (even before this update), but not the second one.

That is correct, suspend and resume was working after I have removed the USB 4 entries from sys-usb.

@marmarek
Copy link
Member

marmarek commented May 5, 2024

Another data point: HP EliteBook 1040 G9
Here firmware advertises both S3 and S0ix support, but S3 is quite broken (the best I can achieve is system waking up, but keyboard doesn't work, just touchpad works; most of the time it either doesn't suspend, or doesn't wake up at all).
With S0ix enabled, the situation is much better. System properly goes to sleep (power LED blinks), and also wakes up. But the residency counters are not great: after 60s sleep, it got 25.5s in S0i3 and ~0.1s in S0i2.

Note the laptop has Mediatek LTE modem (14c3:4d75) that gave quite a bit of troubles with suspend, even on native Linux, but a recent-ish firmware update made it work. It was left in dom0 for this test (attached to its proper driver: mtk_t7xx), but the interface was down. This may or may not matter for this test.

@DemiMarie
Copy link

@marmarek: is it possible to disable the modem in firmware?

@HW42
Copy link

HW42 commented May 6, 2024

[...] Specifically, it needs:

  • qubes-core-dom0 >= 4.2.25 (and its dependencies)
  • kernel-latest >= 6.6.8 for dom0
  • kernel-latest-qubes-vm >= 6.6.8 (and use it at least for sys-net and sys-usb)
  • qubes-core-agent >= 4.2.32 (in template(s) for sys-net and sys-usb)

And libvirt and stubdom, but just enabling current-testing and updating should get all you need.

The amount of time the system was in the proper suspend state can be seen in /sys/kernel/debug/pmc_core/substate_residencies (compare values before suspend and after, and interpret in relation to how long it was suspended on the wall clock).

All the time values there are in µs (10**-6 seconds).

@HW42
Copy link

HW42 commented May 6, 2024 •
edited
Loading

Another data point: HP EliteBook 1040 G9

Thanks for testing. That's a 12th gen Core i, based on a quick online search, right?

[...] With S0ix enabled, the situation is much better. System properly goes to sleep (power LED blinks), and also wakes up. But the residency counters are not great: after 60s sleep, it got 25.5s in S0i3 and ~0.1s in S0i2.

Can you test with leaving the xHCI controller for the Thunderbolt ports assigned to dom0 and handled by the xhci driver there.

@HW42
Copy link

HW42 commented May 6, 2024 •
edited
Loading

When I check the same value on another Ryzen system with a regular Linux distribution it reports quite high values, so I guess s0ix sleep state is not really supported yet on Xen with AMD cpus but I was wondering what kind of suspend is then used in QubesOS?

That sounds like you are using suspend to idle, but just don't get S0ix low power mode residency. A bit simplified: With S3 the OS requested from the firmware to put the system in the suspend mode. On the other hand with S0ix the OS needs to put all devices in their required low power state via the devices' drivers and then let the CPU idle. The power management controller of the CPU/chipset/SoC will then detect that all requirements for the lower power states (S0ix) are met and put the system in it. But if for some reason the requirements are not met the system will just idle, until it gets a wakeup and resumes.

As noted for the moment we will first focus on Intel systems. If you want to debug your AMD system, the first thing I would check if the CPU reaches it's lowest package C-state and getting all MSR access allowed that's needed for AMD's pmc debug output.

So, if I understand correctly, you get the first one (even before this update), but not the second one.

That is correct, suspend and resume was working after I have removed the USB 4 entries from sys-usb.

Just FYI: The part needed for correct wakeup (at least on the tested Intel systems), was already merged earlier than the test, see QubesOS/qubes-linux-kernel#826

@marmarek
Copy link
Member

marmarek commented May 6, 2024

And libvirt and stubdom, but just enabling current-testing and updating should get all you need.

qubes-core-dom0 has versioned dependencies on them

Thanks for testing. That's a 12th gen Core i, based on a quick online search, right?

yes

Can you test with leaving the xHCI controller for the Thunderbolt ports assigned to dom0 and handled by the xhci driver there.

No improvement.

@marmarek: is it possible to disable the modem in firmware?

Yes. Doesn't help.

@qubesos-bot qubesos-bot mentioned this issue May 7, 2024
Closed
@slchris
Copy link

slchris commented May 10, 2024

Initial version (the thing discussed in the last few comments) is merged and available in current-testing repository. Specifically, it needs:

  • qubes-core-dom0 >= 4.2.25 (and its dependencies)
  • kernel-latest >= 6.8.8 for dom0
  • kernel-latest-qubes-vm >= 6.8.8 (and use it at least for sys-net and sys-usb)
  • qubes-core-agent >= 4.2.32 (in template(s) for sys-net and sys-usb)

Then, the feature is opt-in. To use it, you need to:

  1. detach "Thunderbolt 4 NHI" devices from sys-usb (they won't be used by dom0 either)
  2. qvm-features dom0 suspend-s0ix 1
  3. reboot

Ensure /sys/power/mem_sleep has s2idle selected. It will the case on systems not supporting S3 anyway, but on others may need to be selected manually.

The system is expected to sleep about 90% of the time (not 100% as native Linux does on some systems), which will affect how long it will stay suspended on battery. We will work on the remaining 10%, but in the meantime it is what it is. The amount of time the system was in the proper suspend state can be seen in /sys/kernel/debug/pmc_core/substate_residencies (compare values before suspend and after, and interpret in relation to how long it was suspended on the wall clock). There are other interesting files in this directory that may help debugging issues, especially substate_requirements file.

Thanks for working on this, I have tried it on my Thinkpad X1 Carbon Gen 12, so far it looks like suspend is functioning correctly, including all usb devices and network devices(including LET devices) keyboards, mice, etc. after waking up.

@marmarek marmarek moved this from In progress to In review in Current team tasks May 11, 2024
@scsich
Copy link

scsich commented May 14, 2024

Initial version (the thing discussed in the last few comments) is merged and available in current-testing repository. Specifically, it needs:

  • qubes-core-dom0 >= 4.2.25 (and its dependencies)
  • kernel-latest >= 6.8.8 for dom0
  • kernel-latest-qubes-vm >= 6.8.8 (and use it at least for sys-net and sys-usb)
  • qubes-core-agent >= 4.2.32 (in template(s) for sys-net and sys-usb)

Then, the feature is opt-in. To use it, you need to:

  1. detach "Thunderbolt 4 NHI" devices from sys-usb (they won't be used by dom0 either)
  2. qvm-features dom0 suspend-s0ix 1
  3. reboot

Ensure /sys/power/mem_sleep has s2idle selected. It will the case on systems not supporting S3 anyway, but on others may need to be selected manually.
The system is expected to sleep about 90% of the time (not 100% as native Linux does on some systems), which will affect how long it will stay suspended on battery. We will wo[ 149.600999] PM: suspend entry (s2idle)
[ 149.624415] Filesystems sync: 0.023 seconds
[ 149.669942] Freezing user space processes
[ 149.671809] Freezing user space processes completed (elapsed 0.001 seconds)
[ 149.671814] OOM killer disabled.
[ 149.671815] Freezing remaining freezable tasks
[ 149.774551] Freezing remaining freezable tasks completed (elapsed 0.102 seconds)
[ 149.774569] printk: Suspending console(s) (use no_console_suspend to debug)
[ 149.842499] proc_thermal_pci 0000:00:04.0: failed to save offset (-5)
[ 150.976639] PM: suspend devices took 1.201 seconds
[ 151.030094] ACPI: EC: interrupt blocked
[ 151.032152] pciback 0000:00:07.0: xen_pciback: Forcing device suspend
[ 151.035651] pciback 0000:00:14.0: xen_pciback: Backend-side device suspend not enabled
[ 151.035815] pciback 0000:00:0d.0: xen_pciback: Backend-side device suspend not enabled
[ 151.036221] pciback 0000:00:07.2: xen_pciback: Forcing device suspend
[ 151.036288] pciback 0000:00:14.3: xen_pciback: Backend-side device suspend not enabled
[ 151.046853] pciback 0000:00:0d.3: xen_pciback: Device suspended. It's now in D3cold
[ 151.046889] pciback 0000:00:07.0: xen_pciback: Device suspended. It's now in D3cold
[ 151.049619] pciback 0000:00:07.2: xen_pciback: Device suspended. It's now in D3cold
[ 151.049630] pciback 0000:00:0d.2: xen_pciback: Device suspended. It's now in D3cold
[ 328.528320] ACPI: EC: interrupt unblocked
[ 328.672713] nvme nvme0: Shutdown timeout set to 10 seconds
[ 328.675123] i915 0000:00:02.0: [drm] GT0: GuC firmware i915/adlp_guc_70.bin version 70.13.1
[ 328.675129] i915 0000:00:02.0: [drm] GT0: HuC firmware i915/tgl_huc.bin version 7.9.3
[ 328.679123] nvme nvme0: 14/0/0 default/read/poll queues
[ 328.691708] i915 0000:00:02.0: [drm] GT0: HuC: authenticated for all workloads
[ 328.692790] i915 0000:00:02.0: [drm] GT0: GUC: submission enabled
[ 328.692795] i915 0000:00:02.0: [drm] GT0: GUC: SLPC enabled
[ 328.693368] i915 0000:00:02.0: [drm] GT0: GUC: RC enabled
[ 328.993096] PM: resume devices took 0.330 seconds
[ 328.995826] OOM killer enabled.
[ 328.995830] Restarting tasks ...

rk on the remaining 10%, but in the meantime it is what it is. The amount of time the system was in the proper suspend state can be seen in /sys/kernel/debug/pmc_core/substate_residencies (compare values before suspend and after, and interpret in relation to how long it was suspended on the wall clock). There are other interesting files in this directory that may help debugging issues, especially substate_requirements file.

Thanks for working on this, I have tried it on my Thinkpad X1 Carbon Gen 12, so far it looks like suspend is functioning correctly, including all usb devices and network devices(including LET devices) keyboards, mice, etc. after waking up.

Same machine here, but I'm curious if @slchris is getting residency values above zero. I am not, however suspend seems to work (slow breathing blinken lights). @marmarek I'm not sure if the PCI backend suspend not enabled messages are expected from dom0 dmesg. But I have pulled all NMI devices from sys-usb/net and remaining devices are USB controllers and WIFI respectively. Followed all remaining steps quoted.

[  149.600999] PM: suspend entry (s2idle)
[  149.624415] Filesystems sync: 0.023 seconds
[  149.669942] Freezing user space processes
[  149.671809] Freezing user space processes completed (elapsed 0.001 seconds)
[  149.671814] OOM killer disabled.
[  149.671815] Freezing remaining freezable tasks
[  149.774551] Freezing remaining freezable tasks completed (elapsed 0.102 seconds)
[  149.774569] printk: Suspending console(s) (use no_console_suspend to debug)
[  149.842499] proc_thermal_pci 0000:00:04.0: failed to save offset (-5)
[  150.976639] PM: suspend devices took 1.201 seconds
[  151.030094] ACPI: EC: interrupt blocked
[  151.032152] pciback 0000:00:07.0: xen_pciback: Forcing device suspend
[  151.035651] pciback 0000:00:14.0: xen_pciback: Backend-side device suspend not enabled
[  151.035815] pciback 0000:00:0d.0: xen_pciback: Backend-side device suspend not enabled
[  151.036221] pciback 0000:00:07.2: xen_pciback: Forcing device suspend
[  151.036288] pciback 0000:00:14.3: xen_pciback: Backend-side device suspend not enabled
[  151.046853] pciback 0000:00:0d.3: xen_pciback: Device suspended. It's now in D3cold
[  151.046889] pciback 0000:00:07.0: xen_pciback: Device suspended. It's now in D3cold
[  151.049619] pciback 0000:00:07.2: xen_pciback: Device suspended. It's now in D3cold
[  151.049630] pciback 0000:00:0d.2: xen_pciback: Device suspended. It's now in D3cold
[  328.528320] ACPI: EC: interrupt unblocked
[  328.672713] nvme nvme0: Shutdown timeout set to 10 seconds
[  328.675123] i915 0000:00:02.0: [drm] GT0: GuC firmware i915/adlp_guc_70.bin version 70.13.1
[  328.675129] i915 0000:00:02.0: [drm] GT0: HuC firmware i915/tgl_huc.bin version 7.9.3
[  328.679123] nvme nvme0: 14/0/0 default/read/poll queues
[  328.691708] i915 0000:00:02.0: [drm] GT0: HuC: authenticated for all workloads
[  328.692790] i915 0000:00:02.0: [drm] GT0: GUC: submission enabled
[  328.692795] i915 0000:00:02.0: [drm] GT0: GUC: SLPC enabled
[  328.693368] i915 0000:00:02.0: [drm] GT0: GUC: RC enabled
[  328.993096] PM: resume devices took 0.330 seconds
[  328.995826] OOM killer enabled.
[  328.995830] Restarting tasks ...

Substate requirements:

                       Element |    S0i2.0 |    S0i3.0 |    Status |
pmc0:            USB2PLL_OFF_STS |  Required |  Required |           |
pmc0: PCIe/USB3.1_Gen2PLL_OFF_STS |  Required |  Required |           |
pmc0:       PCIe_Gen3PLL_OFF_STS |  Required |  Required |           |
pmc0:            OPIOPLL_OFF_STS |  Required |  Required |           |
pmc0:              OCPLL_OFF_STS |  Required |  Required |           |
pmc0:            MainPLL_OFF_STS |           |  Required |           |
pmc0:            MIPIPLL_OFF_STS |  Required |  Required |           |
pmc0:      Fast_XTAL_Osc_OFF_STS |           |  Required |           |
pmc0:        AC_Ring_Osc_OFF_STS |  Required |  Required |           |
pmc0:            SATAPLL_OFF_STS |  Required |  Required |           |
pmc0:       XTAL_USB2PLL_OFF_STS |           |  Required |           |
pmc0:                CSME_PG_STS |  Required |  Required |           |
pmc0:                SATA_PG_STS |  Required |  Required |           |
pmc0:                xHCI_PG_STS |  Required |  Required |           |
pmc0:               UFSX2_PG_STS |  Required |  Required |           |
pmc0:                 OTG_PG_STS |  Required |  Required |           |
pmc0:                 SPA_PG_STS |  Required |  Required |           |
pmc0:                 SPB_PG_STS |  Required |  Required |           |
pmc0:                 SPC_PG_STS |  Required |  Required |           |
pmc0:                THC0_PG_STS |  Required |  Required |           |
pmc0:                THC1_PG_STS |  Required |  Required |           |
pmc0:              GBETSN_PG_STS |  Required |  Required |           |
pmc0:                 GBE_PG_STS |  Required |  Required |           |
pmc0:                LPSS_PG_STS |  Required |  Required |           |
pmc0:                ADSP_D3_STS |           |  Required |           |
pmc0:               xHCI0_D3_STS |  Required |  Required |           |
pmc0:               xDCI1_D3_STS |  Required |  Required |           |
pmc0:                  IS_D3_STS |  Required |  Required |           |
pmc0:             GBE_TSN_D3_STS |  Required |  Required |           |
pmc0:          CPU_C10_REQ_STS_0 |  Required |  Required |           |
pmc0:             CNVI_REQ_STS_6 |           |  Required |           |
pmc0:              ISH_REQ_STS_7 |           |  Required |           |
pmc0:    MPHY_Core_DL_REQ_STS_16 |  Required |  Required |           |
pmc0:   Break-even_En_REQ_STS_17 |  Required |  Required |           |
pmc0:    Auto-demo_En_REQ_STS_18 |  Required |  Required |           |
pmc0: Int_Timer_SS_Wake0_Pol_STS |  Required |  Required |           |
pmc0: Int_Timer_SS_Wake1_Pol_STS |  Required |  Required |           |
pmc0: Int_Timer_SS_Wake2_Pol_STS |  Required |  Required |           |
pmc0: Int_Timer_SS_Wake3_Pol_STS |  Required |  Required |           |
pmc0: Int_Timer_SS_Wake4_Pol_STS |  Required |  Required |           |
pmc0: Int_Timer_SS_Wake5_Pol_STS |  Required |  Required |           |

@marmarek
Copy link
Member

@scsich is the requirements file dump after suspend? if so, it didn't really suspended... the thing that ultimately matter is how much power it uses in this state, with S0ix the blinking LED is even less reliable indicator of that than it was before :/
I guess you can suspend it and observe if it's still warm after some time. Or how long does it last on battery in this state. Or, if you have some power meter, measure it (but in this case, do it with fully charged battery, to avoid charging interfering with the measurement).

@kainz
Copy link

kainz commented May 14, 2024

It is the various internal power meter values you want to monitor in the /sys/class/power_supply/BAT* hierarchy, as well as the sibling hierarchy devices for AC and/or power delivery. Namely, the current_* values for delivery/draw from charger type devices, power_now for current draw from battery devices, and energy_now for the best guess at current battery charge levels.

Note that your exact platform values, provided monitor endpoints, and accuray may vary -- I would also recommend referring to the power_supply class documentation available in your nearest kernel source, or at https://www.kernel.org/doc/html/latest/power/power_supply_class.html .

@ekaflaereht
Copy link

ekaflaereht commented Jun 4, 2024 •
edited
Loading

Just as a heads up. I know that you are not working on AMD systems yet, but it seems that enabling suspend-s0ix in Qubes actually breaks USB after suspend on these systems while it is working fine before (if you do not include the Thunderbolt USB devices in sys-usb).

I did not debug it yet, because I just realized it. I have enabled suspend-s0ix as described in my post above and everything seemed fine but at the same time no change in power consumption as stated.

But what I realized now is that if you suspend, that sys-usb loses all the USB devices, like the webcam or externally connected. At least they are not shown in the UI anymore.

If I restart sys-usb it does not succeed but seems to hang. When I kill it, sys-usb restarts but then the whole system crashes and reboots, like it does when the Thunderbolt devices are attached to it.

Initially I thought it was related to the latest kernel, but the same happened even with the stable kernel.
The issue only disappeared after unsetting the suspend-s0ix and a reboot.

Let me know if you need any data, but since you are not working on AMD yet and it should be easily reproducible with the Framework 13, this is more a heads up to not enable the option by default or add a hardware recognition.

@GammaSQ
Copy link

GammaSQ commented Jun 24, 2024

I'm running on Intel Raptor Lake, sleep doesn't work for me. In fact, it seems VMs don't sleep at all now. Waking from sleep my USB-peripherals all work immediately, without confirmingattachment.

my setup:

  • qubes-core-dom0 4.2.30
  • kernel-latest 6.9.2
  • kernel-latest-qubes-vm 6.9.2
  • qubes-core-agent (in sys-usb & sys-net, fedora-40-xfce) 4.2.33
  • qvm-features dom0 suspend-s0ix reports 1

I detached all but a generic USB-controller from sys-usb and furthermore tried sleeping after manually shutting down sys-usb and sys-net. Reading /sys/kernel/debug/pmc_core/substate_residencies always report 0 for all three states.

@marmarek
Copy link
Member

marmarek commented Jun 24, 2024 via email

@GammaSQ
Copy link

GammaSQ commented Jun 25, 2024

sorry, yes, s2idle is set:

$ sudo cat /sys/power/mem_sleep
[s2idle]

substate_requirements (sorry, can't interpret any of this):

                       Element |    S0i2.0 |    S0i3.0 |    S0i2.1 |    Status |
pmc0:            USB2PLL_OFF_STS |  Required |  Required |  Required |           |
pmc0: PCIe/USB3.1_Gen2PLL_OFF_STS |  Required |  Required |  Required |           |
pmc0:       PCIe_Gen3PLL_OFF_STS |  Required |  Required |  Required |           |
pmc0:            OPIOPLL_OFF_STS |  Required |  Required |  Required |           |
pmc0:              OCPLL_OFF_STS |  Required |  Required |  Required |           |
pmc0:            MainPLL_OFF_STS |           |  Required |           |           |
pmc0:            MIPIPLL_OFF_STS |  Required |  Required |  Required |           |
pmc0:      Fast_XTAL_Osc_OFF_STS |           |  Required |           |           |
pmc0:        AC_Ring_Osc_OFF_STS |  Required |  Required |  Required |           |
pmc0:            SATAPLL_OFF_STS |  Required |  Required |  Required |           |
pmc0:       XTAL_USB2PLL_OFF_STS |           |  Required |           |           |
pmc0:                CSME_PG_STS |  Required |  Required |  Required |           |
pmc0:                SATA_PG_STS |  Required |  Required |  Required |           |
pmc0:                xHCI_PG_STS |  Required |  Required |  Required |           |
pmc0:               UFSX2_PG_STS |  Required |  Required |  Required |           |
pmc0:                 OTG_PG_STS |  Required |  Required |  Required |           |
pmc0:                 SPA_PG_STS |  Required |  Required |  Required |           |
pmc0:                 SPB_PG_STS |  Required |  Required |  Required |           |
pmc0:                 SPC_PG_STS |  Required |  Required |  Required |           |
pmc0:                 LSX_PG_STS |           |           |  Required |           |
pmc0:                P2SB_PG_STS |           |           |  Required |           |
pmc0:                 PSF_PG_STS |           |           |  Required |           |
pmc0:                 SBR_PG_STS |           |           |  Required |           |
pmc0:              OPIDMI_PG_STS |           |           |  Required |           |
pmc0:                THC0_PG_STS |  Required |  Required |  Required |           |
pmc0:                THC1_PG_STS |  Required |  Required |  Required |           |
pmc0:              GBETSN_PG_STS |  Required |  Required |  Required |           |
pmc0:                 GBE_PG_STS |  Required |  Required |  Required |           |
pmc0:                LPSS_PG_STS |  Required |  Required |  Required |           |
pmc0:                ADSP_D3_STS |           |  Required |           |           |
pmc0:                SATA_D3_STS |           |           |  Required |           |
pmc0:               xHCI0_D3_STS |  Required |  Required |  Required |           |
pmc0:               xDCI1_D3_STS |  Required |  Required |  Required |           |
pmc0:                  IS_D3_STS |  Required |  Required |  Required |           |
pmc0:                THC0_D3_STS |           |           |  Required |           |
pmc0:                THC1_D3_STS |           |           |  Required |           |
pmc0:             GBE_TSN_D3_STS |  Required |  Required |  Required |           |
pmc0:      GPIO_COM0_VNN_REQ_STS |           |           |  Required |           |
pmc0:      GPIO_COM1_VNN_REQ_STS |           |           |  Required |           |
pmc0:      GPIO_COM2_VNN_REQ_STS |           |           |  Required |           |
pmc0:      GPIO_COM3_VNN_REQ_STS |           |           |  Required |           |
pmc0:      GPIO_COM4_VNN_REQ_STS |           |           |  Required |           |
pmc0:      GPIO_COM5_VNN_REQ_STS |           |           |  Required |           |
pmc0:          Audio_VNN_REQ_STS |           |           |  Required |           |
pmc0:            ISH_VNN_REQ_STS |           |           |  Required |           |
pmc0:           CNVI_VNN_REQ_STS |           |           |  Required |           |
pmc0:           eSPI_VNN_REQ_STS |           |           |  Required |           |
pmc0:        Display_VNN_REQ_STS |           |           |  Required |           |
pmc0:          SMBUS_VNN_REQ_STS |           |           |  Required |           |
pmc0:           CSME_VNN_REQ_STS |           |           |  Required |           |
pmc0:        SMLINK0_VNN_REQ_STS |           |           |  Required |           |
pmc0:        SMLINK1_VNN_REQ_STS |           |           |  Required |           |
pmc0:          CLINK_VNN_REQ_STS |           |           |  Required |           |
pmc0:            DCI_VNN_REQ_STS |           |           |  Required |           |
pmc0:           CSME_VNN_REQ_STS |           |           |  Required |           |
pmc0:          CPU_C10_REQ_STS_0 |  Required |  Required |  Required |           |
pmc0:      PCIe_LPM_En_REQ_STS_3 |           |           |  Required |           |
pmc0:             CNVI_REQ_STS_6 |           |  Required |           |           |
pmc0:              ISH_REQ_STS_7 |           |  Required |           |           |
pmc0:    MPHY_Core_DL_REQ_STS_16 |  Required |  Required |  Required |           |
pmc0:   Break-even_En_REQ_STS_17 |  Required |  Required |  Required |           |
pmc0:    Auto-demo_En_REQ_STS_18 |  Required |  Required |  Required |           |
pmc0:           LSX_Wake0_En_STS |           |           |  Required |           |
pmc0:           LSX_Wake1_En_STS |           |           |  Required |           |
pmc0:           LSX_Wake2_En_STS |           |           |  Required |           |
pmc0:           LSX_Wake3_En_STS |           |           |  Required |           |
pmc0:           LSX_Wake4_En_STS |           |           |  Required |           |
pmc0:           LSX_Wake5_En_STS |           |           |  Required |           |
pmc0:           LSX_Wake6_En_STS |           |           |  Required |           |
pmc0:           LSX_Wake7_En_STS |           |           |  Required |           |
pmc0:  Int_Timer_SS_Wake0_En_STS |           |           |  Required |           |
pmc0: Int_Timer_SS_Wake0_Pol_STS |  Required |  Required |  Required |           |
pmc0:  Int_Timer_SS_Wake1_En_STS |           |           |  Required |           |
pmc0: Int_Timer_SS_Wake1_Pol_STS |  Required |  Required |  Required |           |
pmc0:  Int_Timer_SS_Wake2_En_STS |           |           |  Required |           |
pmc0: Int_Timer_SS_Wake2_Pol_STS |  Required |  Required |  Required |           |
pmc0:  Int_Timer_SS_Wake3_En_STS |           |           |  Required |           |
pmc0: Int_Timer_SS_Wake3_Pol_STS |  Required |  Required |  Required |           |
pmc0:  Int_Timer_SS_Wake4_En_STS |           |           |  Required |           |
pmc0: Int_Timer_SS_Wake4_Pol_STS |  Required |  Required |  Required |           |
pmc0:  Int_Timer_SS_Wake5_En_STS |           |           |  Required |           |
pmc0: Int_Timer_SS_Wake5_Pol_STS |  Required |  Required |  Required |           |

@HazelHex
Copy link

HazelHex commented Jun 27, 2024 •
edited
Loading

Another datapoint: HP EliteBook 650 G9 i5-1235U
Qubes 4.2.2-rc1, kernel-latest and kernel-latest-qubes-vm 6.9.2-1
qubes-core-dom0 4.2.31-1, qubes-core-agent in template 4.2.35-1

With S3 sleep - all the same symptoms as marmarek has with 1040 G9 (so S3 not working at all in Windows/Linux/Qubes)

The strange thing is, that /sys/kernel/debug/pmc_core/substate_residencies and substate_requirements does not even list S0i3 state for me:

Substate Residency
S0i2.0   0

Also these counters are zero after 60 minutes suspend. Also all requirements for the lonely S0i2.0 state are "Required", and the last column Status is all empty.

But the system suspends, led is blinking and wakes up without a problem.

Edit: Actually there is a problem as after the wakeup the network doesn't work anymore. In the sys-net console there are some errors from iwlwifi (RFIm is deactivated, reason = 4) and e1000e (Failed to disable ULP). Maybe they are not errors, but notifications that devices are going to sleep. Nevertheless, network icon in the tray is red and unclickable after suspend.

Also in sys-usb there are a lot of errors thunderbolt 0000:00:07.0: 0: timeout reading config space 1 from 0x39. But I actually have thunderbolt disabled in the BIOS ("Thunderbolt" setting is enabled, but only in USB-C and DP mode). So in Qubes I have no "Thunderbolt 4 NHI" devices.

Edit2: Sorry, I have had Thunderbolt 4 NHI and USB devices attached to sys-usb, but I have removed them already. The situation is still the same even with the current-testing kernel-latest and kernel-latest-qubes-vm 6.9.4-1 - the system seems does not sleep for real, even when the power led slowly blinks, so residency is zero and the laptop stays hot. I also get the "Backend-side device suspend not enabled" errors from xen_pciback.

@marmarek
Copy link
Member

marmarek commented Jun 27, 2024 via email

@HazelHex
Copy link

So here are my logs after suspend:

[nomad@dom0 ~]$ sudo dmesg
...
[  300.061162] PM: suspend entry (s2idle)
[  300.063840] Filesystems sync: 0.002 seconds
[  300.077353] Freezing user space processes
[  300.079580] Freezing user space processes completed (elapsed 0.002 seconds)
[  300.079589] OOM killer disabled.
[  300.079592] Freezing remaining freezable tasks
[  300.178383] Freezing remaining freezable tasks completed (elapsed 0.098 seconds)
[  300.178404] printk: Suspending console(s) (use no_console_suspend to debug)
[  300.218505] proc_thermal_pci 0000:00:04.0: failed to save offset (-5)
[  300.515981] PM: suspend devices took 0.338 seconds
[  300.554324] ACPI: EC: interrupt blocked
[  300.554719] pciback 0000:00:1f.6: xen_pciback: Backend-side device suspend not enabled
[  300.561405] pciback 0000:00:14.3: xen_pciback: Backend-side device suspend not enabled
[  300.561795] pciback 0000:00:14.0: xen_pciback: Backend-side device suspend not enabled
[  300.561834] pciback 0000:00:0d.0: xen_pciback: Backend-side device suspend not enabled
[  300.561964] pciback 0000:00:07.0: xen_pciback: Forcing device suspend
[  300.578925] pciback 0000:00:07.0: xen_pciback: Device suspended. It's now in D3cold
[  300.580677] pciback 0000:00:0d.2: xen_pciback: Device suspended. It's now in D3cold
[  357.184131] ACPI: EC: interrupt unblocked
[  357.486639] i2c_hid_acpi i2c-ELAN07FF:00: i2c_hid_get_input: incomplete report (14/65280)
[  357.490818] i915 0000:00:02.0: [drm] GT0: GuC firmware i915/adlp_guc_70.bin version 70.20.0
[  357.490825] i915 0000:00:02.0: [drm] GT0: HuC firmware i915/tgl_huc.bin version 7.9.3
[  357.491822] nvme nvme1: D3 entry latency set to 8 seconds
[  357.499723] nvme nvme1: 10/0/0 default/read/poll queues
[  357.506388] nvme nvme1: Ignoring bogus Namespace Identifiers
[  357.507697] i915 0000:00:02.0: [drm] GT0: HuC: authenticated for all workloads
[  357.508712] i915 0000:00:02.0: [drm] GT0: GUC: submission enabled
[  357.508716] i915 0000:00:02.0: [drm] GT0: GUC: SLPC enabled
[  357.508914] nvme nvme0: D3 entry latency set to 8 seconds
[  357.509232] i915 0000:00:02.0: [drm] GT0: GUC: RC enabled
[  357.575137] nvme nvme0: 10/0/0 default/read/poll queues
[  357.766904] PM: resume devices took 0.284 seconds
[  357.769133] OOM killer enabled.
[  357.769232] Restarting tasks ... 
[  357.769236] mei_hdcp 0000:00:16.0-b638ab7e-94e2-4ea2-a552-d1c54b627f04: bound 0000:00:02.0 (ops i915_hdcp_ops [i915])
[  357.771773] done.
[  357.771782] random: crng reseeded on system resumption
[  357.772429] mei_pxp 0000:00:16.0-fbf6fcf1-96cf-4e2e-a6a6-1bab8cbe36b1: bound 0000:00:02.0 (ops i915_pxp_tee_component_ops [i915])
[  357.789893] PM: suspend exit
...

[nomad@dom0 ~]$ uname -a
Linux dom0 6.9.4-1.qubes.fc37.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Jun 17 16:27:28 GMT 2024 x86_64 x86_64 x86_64 GNU/Linux

[nomad@dom0 ~]$ cat /sys/power/mem_sleep 
[s2idle] deep

[nomad@dom0 ~]$ qvm-features dom0
suspend-s0ix  1

[nomad@dom0 ~]$ qvm-pci
BACKEND:DEVID  DESCRIPTION                                                                                                     USED BY
dom0:00_00.0   Host bridge: Intel Corporation                                                                                  
dom0:00_02.0   VGA compatible controller: Intel Corporation Alder Lake-UP3 GT2 [Iris Xe Graphics]                              
dom0:00_04.0   Signal processing controller: Intel Corporation Alder Lake Innovation Platform Framework Processor Participant  
dom0:00_06.0   PCI bridge: Intel Corporation 12th Gen Core Processor PCI Express x4 Controller #0                              
dom0:00_06.2   PCI bridge: Intel Corporation 12th Gen Core Processor PCI Express x4 Controller #2                              
dom0:00_07.0   PCI bridge: Intel Corporation Alder Lake-P Thunderbolt 4 PCI Express Root Port #0                               
dom0:00_08.0   System peripheral: Intel Corporation 12th Gen Core Processor Gaussian & Neural Accelerator                      
dom0:00_0d.0   USB controller: Intel Corporation Alder Lake-P Thunderbolt 4 USB Controller                                     
dom0:00_0d.2   USB controller: Intel Corporation Alder Lake-P Thunderbolt 4 NHI #0                                             
dom0:00_14.0   USB controller: Intel Corporation Alder Lake PCH USB 3.2 xHCI Host Controller                                   sys-usb (no-strict-reset=True)
dom0:00_14.2   RAM memory: Intel Corporation Alder Lake PCH Shared SRAM                                                        
dom0:00_14.3   Network controller: Intel Corporation Alder Lake-P PCH CNVi WiFi                                                sys-net
dom0:00_15.0   Serial bus controller: Intel Corporation Alder Lake PCH Serial IO I2C Controller #0                             
dom0:00_16.0   Communication controller: Intel Corporation Alder Lake PCH HECI Controller                                      
dom0:00_1d.0   PCI bridge: Intel Corporation Alder Lake PCI Express Root Port #9                                               
dom0:00_1e.0   Communication controller: Intel Corporation Alder Lake PCH UART #0                                              
dom0:00_1e.2   Serial bus controller: Intel Corporation Alder Lake SPI Controller                                              
dom0:00_1f.0   ISA bridge: Intel Corporation Alder Lake PCH eSPI Controller                                                    
dom0:00_1f.3   Audio device: Intel Corporation Alder Lake PCH-P High Definition Audio Controller                               
dom0:00_1f.4   SMBus: Intel Corporation Alder Lake PCH-P SMBus Host Controller                                                 
dom0:00_1f.5   Serial bus controller: Intel Corporation Alder Lake-P PCH SPI Controller                                        
dom0:00_1f.6   Ethernet controller: Intel Corporation Ethernet Connection (16) I219-LM                                         sys-net (no-strict-reset=True)
dom0:02_00.0   Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller PM9B1                            
dom0:2d_00.0   Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller 980                              

[nomad@dom0 ~]$ sudo cat /sys/kernel/debug/pmc_core/substate_residencies
Substate   Residency      
S0i2.0     0              

[nomad@dom0 ~]$ sudo cat /sys/kernel/debug/pmc_core/substate_requirements
                       Element |    S0i2.0 |    Status |
pmc0:            USB2PLL_OFF_STS |  Required |           |
pmc0: PCIe/USB3.1_Gen2PLL_OFF_STS |  Required |           |
pmc0:       PCIe_Gen3PLL_OFF_STS |  Required |           |
pmc0:            OPIOPLL_OFF_STS |  Required |           |
pmc0:              OCPLL_OFF_STS |  Required |           |
pmc0:            MIPIPLL_OFF_STS |  Required |           |
pmc0:        AC_Ring_Osc_OFF_STS |  Required |           |
pmc0:            SATAPLL_OFF_STS |  Required |           |
pmc0:                CSME_PG_STS |  Required |           |
pmc0:                SATA_PG_STS |  Required |           |
pmc0:                xHCI_PG_STS |  Required |           |
pmc0:               UFSX2_PG_STS |  Required |           |
pmc0:                 OTG_PG_STS |  Required |           |
pmc0:                 SPA_PG_STS |  Required |           |
pmc0:                 SPB_PG_STS |  Required |           |
pmc0:                 SPC_PG_STS |  Required |           |
pmc0:                THC0_PG_STS |  Required |           |
pmc0:                THC1_PG_STS |  Required |           |
pmc0:              GBETSN_PG_STS |  Required |           |
pmc0:                 GBE_PG_STS |  Required |           |
pmc0:                LPSS_PG_STS |  Required |           |
pmc0:               xHCI0_D3_STS |  Required |           |
pmc0:               xDCI1_D3_STS |  Required |           |
pmc0:                  IS_D3_STS |  Required |           |
pmc0:             GBE_TSN_D3_STS |  Required |           |
pmc0:          CPU_C10_REQ_STS_0 |  Required |           |
pmc0:    MPHY_Core_DL_REQ_STS_16 |  Required |           |
pmc0:   Break-even_En_REQ_STS_17 |  Required |           |
pmc0:    Auto-demo_En_REQ_STS_18 |  Required |           |
pmc0: Int_Timer_SS_Wake0_Pol_STS |  Required |           |
pmc0: Int_Timer_SS_Wake1_Pol_STS |  Required |           |
pmc0: Int_Timer_SS_Wake2_Pol_STS |  Required |           |
pmc0: Int_Timer_SS_Wake3_Pol_STS |  Required |           |
pmc0: Int_Timer_SS_Wake4_Pol_STS |  Required |           |
pmc0: Int_Timer_SS_Wake5_Pol_STS |  Required |           |

After suspend the network tray icon is red and if I try to restart the sys-net (I shutdown the sys-firewall first) it errors out after about a minute Got empty response from qubesd. See journalctl in dom0 for details. Here is the journal:

[nomad@dom0 ~]$ sudo journalctl --no-pager --since -5m
...
Jun 28 15:48:21 dom0 libvirtd[2234]: internal error: Failed to shutdown domain '1' with libxenlight
Jun 28 15:48:22 dom0 qubesd[2371]: unhandled exception while calling src=b'dom0' meth=b'admin.vm.Shutdown' dest=b'sys-net' arg=b'' len(untrusted_payload)=0
Jun 28 15:48:22 dom0 qubesd[2371]: Traceback (most recent call last):
Jun 28 15:48:22 dom0 qubesd[2371]:   File "/usr/lib/python3.11/site-packages/qubes/api/__init__.py", line 297, in respond
Jun 28 15:48:22 dom0 qubesd[2371]:     response = await self.mgmt.execute(
Jun 28 15:48:22 dom0 qubesd[2371]:                ^^^^^^^^^^^^^^^^^^^^^^^^
Jun 28 15:48:22 dom0 qubesd[2371]:   File "/usr/lib/python3.11/site-packages/qubes/api/admin.py", line 882, in vm_shutdown
Jun 28 15:48:22 dom0 qubesd[2371]:     await self.dest.shutdown(force=force, wait=wait)
Jun 28 15:48:22 dom0 qubesd[2371]:   File "/usr/lib/python3.11/site-packages/qubes/vm/qubesvm.py", line 1365, in shutdown
Jun 28 15:48:22 dom0 qubesd[2371]:     self.libvirt_domain.shutdown()
Jun 28 15:48:22 dom0 qubesd[2371]:   File "/usr/lib/python3.11/site-packages/qubes/app.py", line 103, in wrapper
Jun 28 15:48:22 dom0 qubesd[2371]:     return attr(*args, **kwargs)
Jun 28 15:48:22 dom0 qubesd[2371]:            ^^^^^^^^^^^^^^^^^^^^^
Jun 28 15:48:22 dom0 qubesd[2371]:   File "/usr/lib64/python3.11/site-packages/libvirt.py", line 2972, in shutdown
Jun 28 15:48:22 dom0 qubesd[2371]:     raise libvirtError('virDomainShutdown() failed')
Jun 28 15:48:22 dom0 qubesd[2371]: libvirt.libvirtError: internal error: Failed to shutdown domain '1' with libxenlight
...

And if I reboot Qubes after that it is waiting very long time on Job qubes-core.service/stop running.

@marmarek marmarek mentioned this issue Jul 22, 2024
Open
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Sep 6, 2024
@marmarek
Set desired suspend mode in /sys/power/mem_sleep
9c2405a 
If S0ix isn't explicitly enabled, default to S3. This means also writing
'deep' to /sys/power/mem_sleep (some systems defaults to s2idle
nowadays). Try to not override user choice done via mem_sleep_default=
kernel option (if 'enable-s0ix' is not set).

At some point, S0ix will be automatically enabled if hardware expects
it, but it isn't this time yet.

QubesOS/qubes-issues#6411
@qubesos-bot qubesos-bot mentioned this issue Sep 6, 2024
Closed
marmarek added a commit to QubesOS/qubes-core-admin that referenced this issue Sep 6, 2024
@marmarek
Set desired suspend mode in /sys/power/mem_sleep
ae37419 
If S0ix isn't explicitly enabled, default to S3. This means also writing
'deep' to /sys/power/mem_sleep (some systems defaults to s2idle
nowadays). Try to not override user choice done via mem_sleep_default=
kernel option (if 'enable-s0ix' is not set).

At some point, S0ix will be automatically enabled if hardware expects
it, but it isn't this time yet.

QubesOS/qubes-issues#6411

(cherry picked from commit 9c2405a)
@qubesos-bot qubesos-bot mentioned this issue Sep 7, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@HW42 HW42

Labels
C: power management C: Xen hardware support P: major Priority: major. Between "default" and "critical" in severity. T: enhancement Type: enhancement. A new feature that does not yet exist or improvement of existing functionality. waiting for upstream This issue is waiting for something from an upstream project to arrive in Qubes. Remove when closed.
Projects
Current team tasks
Status: In review
Milestone
No milestone
Development

No branches or pull requests