Nvidia Optimus on Linux

Introduction

A previous article on this site, The Best Distributions for Problematic NVIDIA Optimus Hybrid Graphics, discussed some issues in configuring an NVIDIA Optimus laptop for use with Linux -- in particular, complications arising when the HDMI and USB-C/Thunderbolt/Display Port video outputs are internally wired to the NVIDIA GPU instead of the integrated GPU -- as well as some of the solutions available on various Linux distributions to manage the switching to/from the discrete NVIDIA and the integrated graphics processors. That article referenced an NVIDIA GeForce GTX 1050 Ti Mobile GPU, part of the series of NVIDIA GPUs codenamed Pascal, released in 2016.

During the period this generation of NVIDIA GPUs were current, and for a time before, Optimus laptops running Linux could be used in several modes.^[1]

• integrated only mode, where from boot, the NVIDIA card is off and the driver its driver is prevented from loading
• discrete only mode, where from boot, the NVIDIA card is powered on and the driver its loaded
• a dynamic hybrid mode, where the integrated GPU is used by default, but with graphics rendering offloaded to the discrete card and then transferred to the integrated GPU for display

In all cases where switching modes was desired, external tools or manual methods were required to load or unload the appropriate driver and manage the power state of the GPU. One popular external utility was Bumblebee which set the default mode to hybrid graphics and managed the power state of the NVIDIA GPU using the bbswitch kernel module to enable and disable the NVIDIA card as necessary. With Bumblebee applications which could benefit from rendering by the NVIDIA card were started with the commands optirun or primusrun (see below).

Bumblebee was limited if external displays were connected to ports that were directly connected to the NVIDIA GPU, a limitation which motivated the previous article. Other Optimus management utilities which could switch between the various graphics modes -- such as Optimus Manager for Arch and SUSEPrime for openSUSE -- were also available at this time, some also using bbswitch to manage the power state of the NVIDIA GPU,

Since the Pascal generation of GPU, a major new capability -- referred to as Runtime D3 Power management by NVIDIA -- has been added to the proprietary NVIDIA driver for Linux that allows the power state of NVIDIA processors to be managed internally by the kernel driver module in the hybrid mode. Beginning with the Turing generation of NVIDIA processors, the first of which were shipped circa 2019, when used with NVIDIA drivers versions 435.XX and later, the NVIDIA driver became capable of managing the ACPI device power state of NVIDIA processors in hybrid mode, without resorting to external utilities and methods, such as the bbswitch kernel module, to manage the power state of the NVIDIA processor.

While the update to the Linux driver kernel module allows ACPI device power state management in hybrid mode, Optimus laptops on linux still require external utilities, such as the previously mentioned Optimus Manager and SUSEPrime -- described below, along with other utilities for other distributions, or manual methods to switch modes.

Unfortunately, the added capability and corresponding changes in the GPUs causes a problem when using some of the Optimus management tools with the newer cards -- something I discovered in my initial use of the Lenovo Legion 5i Pro, reviewed in Lenovo Legion 5i Pro 16ITH6. When using these tools on this laptop, which includes an NVIDIA GeForce RTX 3050 Mobile (NVIDIA GA107BM, Ampere generation micro-architecture) in its Optimus configuration, the laptop would power off immediately after switching graphics cards, as described in the review. This problem, apparently the result of incompatible ACPI calls with newly added Runtime D3 Power Management capability in newer NVIDIA cards and driver on Linux, requires a change in approach to configuring the Optimus management tools on laptops with the newer NVIDIA processors.

This article begins with a description of ACPI power states to provide insight into the meaning of "D3" in "Runtime D3 Power Management". It then describes the new Runtime D3 Power Management capability, Optimus management tools available for various distributions, and the change in approach required when using these tools with the post Turing generation NVIDIA cards compared to the previous cards.

ACPI Power States

The "D3" in "Runtime D3 Power Management" refers to one possible ACPI device power state of the NVIDIA GPU.

ACPI is a standard architecture-independent framework developed in the mid 1990's that defines a subsystem partially incorporated in operating systems and partially implemented in hardware in order to provide methods for configuration and power management of computer hardware. The subsystem acts as an intermediate interface between the OS and hardware allowing configuration and power management without issuing native firmware commands from the OS.

The system defines a hierarchical representation of ACPI devices on the system bus to configure and manage them. ACPI can also access the internal hierarchical representation of devices such as those on the PCI and USB buses, and use this representation to access them for configuration and power management. It defines four power states for the platform as a whole called Global Power States (G0, G1, G2, G3 -- in order of highest power consumption to lowest), four sleeping states for the platform as a whole, Sleeping States (S1, S2, S3, S4 -- in order of highest power consumption to lowest), and four power states for the individual devices that comprise the computer (D0, D1, D2, D3 -- in order of highest power consumption to lowest). In some devices an additional device power state, D3hot, may be supported, in which the power consumption of this state is between D2 and D3. In such devices the D3 state can also be referred to as D3cold, the difference between D3 (or D3cold) and D3hot being, generally, that power is completely removed in the D3cold state requiring a complete re-initialization of the device. Not all devices support all of the device power states, although some are required by the specification. These states and their relationships are shown in the following diagram (click image for a larger version in a new tab) and described below.

Global Power States

The global power states are defined using six criteria:

• whether software is running
• latency in application response to external events
• power consumption level
• whether the system must be rebooted to return to working state
• whether it is safe to disassemble the system
• whether the state can be entered electronically (along with software) or mechanically

Generally, the G0 global power state is the fully working state of the system with the lowest latency and the largest power consumption. In the G1 state the system is sleeping in one of four sleep modes, S1 - S4 (S0 is equivalent to the G0 global system state) . As the X in GX increases the power consumption decreases, while latency -- the time to return to a fully working state -- increases. Similarly, as the X in SX increases power consumption decreases, while latency increases.

The G2 global power state is a "Soft Off" state into which the system is entered through software or electronic means, i.e., not through a mechanical switch. This state is also known as a S5 state and is one of the possible states entered into during hibernation after the system context is stored in non-volatile storage (see below).

The G3 global power state is the state entered with a power switch. In this state no current is running through the system and is the only state in which the system is safe to disassemble.

Sleep States

Transitions from the G0 to the G1 global power state occurs as a result of an event that can include, for example, a lid close switch activation. When the G1 state is entered, the system can go into one of the four sleep states (S1 -S4) each of which trades off power consumption versus wake latency and the amount of the system context that is preserved. A transition from one sleep state to another can also occur, but only after an intermediate transition to the G0 (also S0) global state. When a particular sleep state is entered, the system sets devices to a device power state that is compatible with that particular sleep state.

One item of interest regarding sleep states is the that the S4 sleep state is also a type of hibernation. When the system is requested to enter the hibernation state in a laptop in which the OS uses systemd-sleep, depending on the value of configuration item "Hibernate Mode"" in /etc/systemd/sleep.conf, after the context is preserved in non-volatile storage, the system can either wait for OEM specific hibernation handling (S4) or immediately shutdown (specifically, enter the G2 global power state or S5). The value of the configuration item is a space separated list of the hibernation mode in order of preference. In the specific case of the Lenovo Legion 5i Pro in which rEFInd is set in the firmware as the default boot manager, if the first value of Hibernate Modeis "platform", after hibernation is requested, the system exits to the rEFInd interface, requiring two cycles of manual shutdown, although on the next boot the system resumes from a hibernation state. If the first value of the configuration item is set as shutdown the system immediately shuts down, entering the G2 global power state (S5) as is normally expected, and at the next power on, the system resumes from hibernation. The default state of the configuration file is shown in the following listing. Note that the comments suggest overriding configuration items in the directory, /etc/systemd/sleep.conf.d, which I did to modify the behavior during hibernation.

#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it under the
#  terms of the GNU Lesser General Public License as published by the Free
#  Software Foundation; either version 2.1 of the License, or (at your option)
#  any later version.
#
# Entries in this file show the compile time defaults. Local configuration
# should be created by either modifying this file, or by creating "drop-ins" in
# the sleep.conf.d/ subdirectory. The latter is generally recommended.
# Defaults can be restored by simply deleting this file and all drop-ins.
#
# See systemd-sleep.conf(5) for details.

[Sleep]
#AllowSuspend=yes
#AllowHibernation=yes
#AllowSuspendThenHibernate=yes
#AllowHybridSleep=yes
#SuspendMode=
#SuspendState=mem standby freeze
#HibernateMode=platform shutdown
#HibernateState=disk
#HybridSleepMode=suspend platform shutdown
#HybridSleepState=disk
#HibernateDelaySec=120min

Device Power States

The device power state is defined using four criteria:

• power consumption
• the amount of the device context is retained by hardware
• the actions that must be taken by the device driver to restore the device
• the latency in restoring the device

In the G0 state, each device in the system can be in any one of the device power states supported by the device, independent of other devices in the system. In this global power state, a device in the D0 state -- where full functionality of the device is available with maximum device power consumption -- can be in one of four device performance states (P1, P2, P3, P4) each of which balance the device performance with the device power consumption at various levels such that power can be conserved without transitioning the device to one of the lower device power states. In the G1 global power state, devices may only be in device states compatible with the G1 global power state.

In general, as X in the device power state DX increases, power consumption decreases, the amount of context preserved decreases, and the latency in restoring the device increases.

The processor is also considered a device but has a special terminology in describing its state, C0 - C3 instead of D0 - D3. Within the C0 processor power state, the processor can be in one of four performance states P0 - P1, where performance may be traded for decreased power consumption.

Device Classes

The characteristics of each device power state, and whether a particular device power state is supported by a device depends on the device class. The ACPI specification defines a Default Device class with characteristics that apply to all devices. The default device class supports the D0 and D3 power states only. Other device classes are:

• Audio Device class
• COM Port Device class
• Display Device class
• Input Device class
• Modem Device class
• Network Device class
• PC Card Controller Device class
• Storage Device class

Subclasses may also be defined for each device class. The NVIDIA GPU of interest in this article is the Video Controller (Graphics Adapter) subclass of the Display Device class. The complete set of subclasses of the Display Device class is:

• CRT Monitor
• External Digital Display
• Standard TV Devices and Analog HDTVs
• Full Screen Display Devices
• Video Controller (Graphics Adapters)

In addition to the D0 and D3 device states required by the Default Device class, other device classes will support any or all of the other device power states mentioned, depending on the specific device class and subclass. The supported device power states of devices in the Lenovo Legion 5i Pro are indicated in the following listing showing some relevant entries in the systemd journal during boot of Arch Linux.

 90%  17:47:01  USER: brook HOST: ARCH-16ITH6   
 ~  ❯$ sudo journalctl -xb -1 | grep -e PME#
Dec 30 15:41:37 ARCH-16ITH6 kernel: pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
Dec 30 15:41:37 ARCH-16ITH6 kernel: pci 0000:00:06.0: PME# supported from D0 D3hot D3cold
Dec 30 15:41:37 ARCH-16ITH6 kernel: pci 0000:00:07.0: PME# supported from D0 D3hot D3cold
Dec 30 15:41:37 ARCH-16ITH6 kernel: pci 0000:00:07.2: PME# supported from D0 D3hot D3cold
Dec 30 15:41:37 ARCH-16ITH6 kernel: pci 0000:00:0d.0: PME# supported from D3hot D3cold
Dec 30 15:41:37 ARCH-16ITH6 kernel: pci 0000:00:0d.2: PME# supported from D0 D1 D2 D3hot D3cold
Dec 30 15:41:37 ARCH-16ITH6 kernel: pci 0000:00:0d.3: PME# supported from D0 D1 D2 D3hot D3cold
Dec 30 15:41:37 ARCH-16ITH6 kernel: pci 0000:00:14.0: PME# supported from D3hot D3cold
Dec 30 15:41:37 ARCH-16ITH6 kernel: pci 0000:00:14.3: PME# supported from D0 D3hot D3cold
Dec 30 15:41:37 ARCH-16ITH6 kernel: pci 0000:00:16.0: PME# supported from D3hot
Dec 30 15:41:37 ARCH-16ITH6 kernel: pci 0000:00:17.0: PME# supported from D3hot
Dec 30 15:41:37 ARCH-16ITH6 kernel: pci 0000:00:1d.0: PME# supported from D0 D3hot D3cold
Dec 30 15:41:37 ARCH-16ITH6 kernel: pci 0000:00:1d.6: PME# supported from D0 D3hot D3cold
Dec 30 15:41:37 ARCH-16ITH6 kernel: pci 0000:00:1f.3: PME# supported from D3hot D3cold
Dec 30 15:41:37 ARCH-16ITH6 kernel: pci 0000:01:00.0: PME# supported from D0 D3hot
Dec 30 15:41:37 ARCH-16ITH6 kernel: pci 0000:58:00.0: PME# supported from D0 D1 D2 D3hot D3cold

ACPI and NVIDIA Optimus

The significance of the above discussion of ACPI power states, and the ACPI system itself to Optimus laptops, is apparent in the following listing of entries in the systemd journal written during booting of Arch Linux on the Lenovo Legion 5i Pro. The first line shows that OS power management can switch the global power state from G0 (indicated by the equivalent S0) to sleep states S3, S4, and to global power state G2 (indicated by the equivalent S5). The second line refers to the NVIDIA GeForce RTX 3050 Mobile GPU, which is at PCI address 0000:00:01.0, and indicates it supports the device power states D0, D3hot, and D3cold.

Feb 12 01:22:03 ARCH-16ITH6 kernel: ACPI: PM: (supports S0 S3 S4 S5)
Feb 12 01:22:03 ARCH-16ITH6 kernel: pci 0000:00:01.0: PME# supported from D0 D3hot D3cold

When the laptop is in the G0 global power state and it is operating in hybrid graphics mode and the NVIDIA graphics processor is required for an application that needs higher graphics performance, the device power state of the GPU is set to D0 to provide the full functionality of the processor. When the NVIDIA GPU is not needed, or when the system is put into one of the sleep states S1 - S4, within the G1 global power state, the device power state of the NVIDIA GPU is set to any of the lower power consumption device states compatible with the specific sleep state. In practice this is limited to only the D3hot and D3cold states as the NVIDIA GPU only supports these two low power states and not the D1 or D2 device power states.

Optimus

As NVIDIA describes it:

NVIDIA Optimus [was] a legacy, game-changing feature. Before its invention, a laptop either ran via low-performance integrated graphics, or high-performance power-intensive GPUs. There was no middle ground, and most importantly, no way to dynamically switch which processor was driving the internal display. Instead, users had to reboot their laptop, and potentially change BIOS settings.

NVIDIA Optimus introduced a way to dynamically switch between the two graphics processors. The integrated graphics would always drive the display. Profiles from our GeForce drivers told the system when a game or app would benefit from high-performance graphics and GPU acceleration, and the GeForce GPU would be tagged in to render the 3D game and pass the rendered frames to the integrated graphics to display. When the game or app was shut down, display duties went back to the integrated graphics, lengthening the laptop’s battery life.

The dynamic graphics switching capabilities of Optimus as described above by NVIDIA reflect the state of Optimus on Windows for most of the history of Optimus. Only recently has this dynamic capability been available in Linux directly through the NVIDIA driver, without using external tools to manage the power state of the NVIDIA GPU, since the Turing micro-architecture generation of NVIDIA GPUs and since version 435.XX of the driver. Although the power management is now automatic in hybrid graphics mode, it requires the use of appropriate environment variables when starting applications to use the GPU.

The current evolution of the technology, Advanced Optimus:

improves upon the original by introducing a hardware-based dynamic display switch that allows GeForce GPUs to directly control the display, rather than routing via the IGP. Bypassing integrated graphics increases game performance, reduces system latency when gaming, and enables the use of G-SYNC variable refresh rate displays, improving users’ experiences.

The switch mentioned in the above quotation seems to be implemented in the Lenovo Legion 5i Pro in the EFI interface screen shown in the following image. The NVIDIA GPU can be set to drive the EDP directly by setting the value of the drop down selection of the "Graphics Device" configuration device to "Discrete Graphic". This setting also ensures that the integrated card is powered off at boot, as we see in the following listing. Setting it to "Dynamic Graphics" will ensure that both the integrated and discrete GPU will be powered on and available and cause the integrated card to be used by default, and the NVIDIA GPU as needed.

This setting in the firmware interface would seem to work in conjunction with the setting in the operating system such that if "Discrete Graphic" is set in the firmware, the NVIDIA driver will be available if either hybrid mode or discrete mode is set in the OS; however, if "Dynamic Graphics" is set in the firmware, then the "Dynamic Graphics" (or hybrid mode) must be set in the OS. This works as intended in Windows with OS side settings configured through the Lenovo Vantage utility, however this is not the case in Linux where optimus management utilities -- at least in their current state of development -- expect the OS to be booted with both cards available. The following listing showing the entries created in the systemd journal by Optimus Manager with its startup mode set to NVIDIA demonstrates the issue.

 100% 19:08:40 USER: brook HOST: ARCH-16ITH6
PCD: 5s ~ ❯$ sudo journalctl -xb -1 | grep python3
Dec 31 19:04:11 ARCH-16ITH6 python3[741]: [12] INFO: # Daemon pre-start hook
Dec 31 19:04:11 ARCH-16ITH6 python3[741]: [12] INFO: Removing /etc/X11/xorg.conf.d/10-optimus-manager.conf (if present)
Dec 31 19:04:11 ARCH-16ITH6 python3[741]: [13] INFO: Copying /etc/optimus-manager/optimus-manager.conf to
/var/lib/optimus-manager/tmp/config_copy.conf
Dec 31 19:04:11 ARCH-16ITH6 python3[741]: [14] INFO: Startup mode is: nvidia
Dec 31 19:04:11 ARCH-16ITH6 python3[741]: [14] INFO: Writing state {'type': 'pending_pre_xorg_start', 'requested_mode':
'nvidia', 'current_mode': None}
Dec 31 19:04:11 ARCH-16ITH6 python3[741]: [14] INFO: Daemon pre-start hook completed successfully.
Dec 31 19:04:11 ARCH-16ITH6 python3[741]: [14] INFO: Calling Xorg pre-start hook.
Dec 31 19:04:11 ARCH-16ITH6 python3[774]: [6] INFO: # Xorg pre-start hook
Dec 31 19:04:11 ARCH-16ITH6 python3[774]: [6] INFO: Previous state was: {'type': 'pending_pre_xorg_start',
'requested_mode': 'nvidia', 'current_mode': None}
Dec 31 19:04:11 ARCH-16ITH6 python3[774]: [6] INFO: Requested mode is: nvidia
Dec 31 19:04:11 ARCH-16ITH6 python3[794]: modinfo: ERROR: Module bbswitch not found.
Dec 31 19:04:12 ARCH-16ITH6 python3[774]: [825] INFO: Available modules: ['nouveau', 'acpi_call', 'nvidia',
'nvidia_drm', 'nvidia_modeset', 'nvidia_uvm']
Dec 31 19:04:12 ARCH-16ITH6 python3[774]: [825] INFO: Unloading modules ['nouveau'] (if loaded)
Dec 31 19:04:12 ARCH-16ITH6 python3[774]: [827] INFO: switching=none, nothing to do
Dec 31 19:04:13 ARCH-16ITH6 python3[774]: [1060] ERROR: Xorg pre-start setup error
Dec 31 19:04:13 ARCH-16ITH6 python3[774]: Traceback (most recent call last):
Dec 31 19:04:13 ARCH-16ITH6 python3[774]: File
"/usr/lib/python3.10/site-packages/optimus_manager/hooks/pre_xorg_start.py", line 51, in main
Dec 31 19:04:13 ARCH-16ITH6 python3[774]: setup_kernel_state(config, prev_state, requested_mode)
Dec 31 19:04:13 ARCH-16ITH6 python3[774]: File "/usr/lib/python3.10/site-packages/optimus_manager/kernel.py", line 22,
in setup_kernel_state
Dec 31 19:04:13 ARCH-16ITH6 python3[774]: _nvidia_up(config, hybrid=(requested_mode == "hybrid"))
Dec 31 19:04:13 ARCH-16ITH6 python3[774]: File "/usr/lib/python3.10/site-packages/optimus_manager/kernel.py", line 81,
in _nvidia_up
Dec 31 19:04:13 ARCH-16ITH6 python3[774]: if not pci.is_nvidia_visible():
Dec 31 19:04:13 ARCH-16ITH6 python3[774]: File "/usr/lib/python3.10/site-packages/optimus_manager/pci.py", line 67, in
is_nvidia_visible
Dec 31 19:04:13 ARCH-16ITH6 python3[774]: bus_ids = get_gpus_bus_ids(notation_fix=False)
Dec 31 19:04:13 ARCH-16ITH6 python3[774]: File "/usr/lib/python3.10/site-packages/optimus_manager/pci.py", line 100, in
get_gpus_bus_ids
Dec 31 19:04:13 ARCH-16ITH6 python3[774]: raise PCIError("Cannot find the integrated GPU. Is this an Optimus system ?")
Dec 31 19:04:13 ARCH-16ITH6 python3[774]: optimus_manager.pci.PCIError: Cannot find the integrated GPU. Is this an
Optimus system ?
Dec 31 19:04:13 ARCH-16ITH6 python3[774]: [1061] INFO: Removing /etc/X11/xorg.conf.d/10-optimus-manager.conf (if
present)
Dec 31 19:04:13 ARCH-16ITH6 python3[774]: [1061] INFO: Writing state {'type': 'pre_xorg_start_failed', 'switch_id':
'20221231T190411', 'requested_mode': 'nvidia'}
Dec 31 19:04:13 ARCH-16ITH6 python3[1020]: [1] INFO: # Commands daemon
Dec 31 19:04:13 ARCH-16ITH6 python3[1020]: [1] INFO: Opening UNIX socket
Dec 31 19:04:13 ARCH-16ITH6 python3[1020]: [1] INFO: Awaiting commands
Dec 31 19:06:13 ARCH-16ITH6 python3[1020]: [120149] INFO: Process stop requested
Dec 31 19:06:13 ARCH-16ITH6 python3[1020]: [120149] INFO: Closing and removing the socket...
Dec 31 19:06:13 ARCH-16ITH6 python3[1020]: [120150] INFO: Goodbye !

 100% 19:08:44 USER: brook HOST: ARCH-16ITH6
~ ❯$

The ultimate indicator of the issue is the line:

Dec 31 19:04:13 ARCH-16ITH6 python3[774]: optimus_manager.pci.PCIError: Cannot find the integrated GPU. Is this an Optimus system ?

in which we see that Optimus Manager expects the integrated GPU to be available, but it is not because of the Advanced Optimus switch being set to "Discrete Graphics" in the firmware. This means that even when the Optimus Manager mode is set to NVIDIA, before the X Window system starts Optimus Manager sets up graphics in hybrid mode, and only after the display manager starts, does it make the NVIDIA GPU available. The result is a seemingly stalled boot before the display manager starts.

The dynamic graphics switching capabilities of Optimus as described above generally by NVIDIA, and below more specifically, is a significant improvement in Windows. However, as the issue with Optimus Manager with respect to Advanced Optimus demonstrates, the tools have yet to completely catch up with Windows.

Optimus on Linux Before Turing Generation of NVIDIA GPUs

Before Turing and version 435 of the NVIDIA driver, on Linux the NVIDIA driver, while able to switch the performance states (P0 and P4) could not switch device power states from D0 to one of the lower power states. Also, the dynamic offloading of graphics rendering to the discrete GPU -- described in the first quotation from NVIDIA, above -- could not be done automatically. Both of these functions -- which were performed automatically by the NVIDIA driver on Windows -- required the use of external utilities on Linux.

One of the most popular pre-Turing Optimus external utilities for Linux was Bumblebee -- currently not actively maintained, although still available in package repositories. I used this Optimus solution on all distributions installed on my first NVIDIA Optimus laptop, the Acer V15 Nitro, and later on its replacement, the Dell G5, at least initially, before switching to the other solutions mentioned in this and the previous article.

Bumblebee provided an Optimus solution in which the integrated graphics processor would be set as the default primary graphics processor, and the one that would power the built-in display. When higher performance graphics capability was required, an application would be started with the optimus command, as in

optirun [optirun-options] application-command [application-parameters]

causing the application's graphics to be rendered by the dedicated NVIDIA processor then transferred to the integrated processor to be displayed on the integrated graphics processor. Note, that this would be done automatically in Windows.

With Bumblebee power management of the NVIDIA processor was managed externally using bbswitch, a kernel module, also developed by the Bumblebee project. The module was able to enable the NVIDIA card when required and disable it when not required through methods standardized in the ACPI Specification. It also provided methods to manually enable and disable the NVIDIA GPU, provided the driver was manually loaded beforehand.

In the long run, Bumblebee was not a viable solution for me -- without extensive tinkering -- if using an external display in Optimus laptops where the HDMI and DisplayPort (USB-C/Thunderbolt) connectors were directly wired to the NVIDIA card. This was the motivation for the previous NVIDIA article, The Best Distributions for Problematic NVIDIA Optimus Hybrid Graphics, which resulted in the discovery of other Optimus utilities for the Dell G5 which allowed use of the external display.

These other Optimus solutions included Optimus Manager for Arch and derivatives, SUSEPrime for openSUSE, nvidia-prime, a utility that integrates into the NVIDIA Settings GUI, for Ubuntu and derivatives, and a tool similar to Optimus Manager called nvidia-optimus-manager for Solus. At least two of these utilities -- Optimus Manager and SUSEPrime -- used the bbswitch modules to a certain extent for managing the power state of the NVIDIA processor. Optimus Manager only uses bbswitch at the user's discretion as of one of four alternative methods of managing the power state of the NVIDIA processor, one of which is to use the driver's internal methods, if applicable. SUSEPrime on the other hand relied on bbswitch as its only means of managing the power state of the NVIDIA processor.

On the Dell after replacing Bumblebee with Optimus Manager and SUSEPrime the switching between the NVIDIA and integrated GPUs was possible with reboots. This allowed use of the external display when necessary by switching to NVIDIA, or when higher performance was required with the availability of an external power source, and the integrated card for lower power consumption. A hybrid mode was also available with these tools, such that the X Windows system processes used the NVIDIA card but applications would be started with the appropriate environment variables for rendering to be offloaded to the NVIDIA GPU. Of course, both of these methods used external methods to manage the device power state of the NVIDIA card and Runtime D3 Power Management of the NVIDIA card was not available.

The key characteristic of this era of Optimus on Linux when comparing to the later post-Turing is that power management in the hybrid mode was performed by external tools or manually, and not by the NVIDIA driver itself. The method of offloading the rendering also changed from starting applications to be rendered by the discrete card with the optirun or primusrun command to setting an environment variable when starting the application.

Before the Turing generation of NVIDIA cards and the inclusion of Runtime D3 Power Management capability in the driver for Linux, external tools such as bbswitch were required to manage the device power state of the NVIDIA GPU and manage the offloading of the rendering to the discrete GPU, functions automatically performed by the driver in Windows as Optimus is described by NVIDIA. The legacy hybrid graphics functionality could also be realized where either the NVIDIA or integrated card could be used exclusively per session.

Optimus on Linux After Turing Generation GPUs

Since the Turing micro-architecture generation, the basic Optimus operating modes have changed in that a true integrated only mode -- in which the NVIDIA GPU is both powered off and its proprietary driver is not loaded -- is not available. The discrete GPU only mode is still available, however, as is the hybrid mode in which both GPUs are on with drivers loaded, and rendering is offloaded to the NVIDIA GPU for applications which need better graphics performance. The difference between the previous and the current hybrid mode is that external tools to manage the power state of the NVIDIA processor, such as bbswitch, have become obsolete in hybrid mode, as now the NVIDIA driver is capable of changing the ACPI device power states.

In fact, using these older tools with a newer NVIDIA Optimus laptop, for example using Optimus Manager with bbswitch as the user configurable method for managing the power state of the NVIDIA processor, or using SUSEPrime, according to its documentation (as of December 2021) which required the use of bbswitch, will cause undesired behavior. As mentioned in the review of the Lenovo LEGION 5i PRO 16ITH6 [82JF002RUS], when using Optimus Manager configured to use bbswitch on Garuda Linux -- an Arch derivative, or when using SUSE Prime on openSUSE Tumbleweed, the entire laptop would power down when switching to integrated mode.

This behavior on the newer Optimus laptops with post Turing cards can be attributed to the newer ACPI power management architecture in post-Turing cards and the inclusion in the NVIDIA Linux driver kernel module of the capability to put the NVIDIA processor in the ACPI D3 power state. NVIDIA describes the power management capabilities, generally, in Chapter 22. PCI-Express Runtime D3 (RTD3) Power Management of the NVIDIA Accelerated Linux Graphics Driver README and Installation Guide, the only detailed documentation that discusses the internal workings of the driver and the GPU:

NVIDIA GPUs have many power-saving mechanisms. Some of them will reduce clocks and voltages to different parts of the chip, and in some cases turn off clocks or power to parts of the chip entirely, without affecting functionality or while continuing to function, just at a slower speed. However, the lowest power states for NVIDIA GPUs require turning power off to the entire chip, often through ACPI calls. Obviously, this impacts functionality. Nothing can run on the GPU while it is powered off. Care has to be taken to only enter this state when there are no workloads running on the GPU and any attempts to start work or any memory mapped I/O (MMIO) access must be preceded with a sequence to first turn the GPU back on and restore any necessary state.

The first part of the above presumably refers to selecting an appropriate device performance state (P0 - P3) for the GPU while the GPU is in the D0 device power state, as device power states D1 and D2 are not supported by the NVIDIA GPUs (at least not on the GeForce RTX 3050) and the second part refers to transitioning the GPU from the D0 device power state to the D3. While the NVIDIA driver can put the GPU into the lowest device power state, the ACPI D3 state, a state where the device does not consume any power, it is not possible to power it off in such a way that only the integrated GPU is used for graphics rendering and only the integrated GPU's driver is loaded. The lowest power state is only entered when the NVIDIA driver recognizes that the GPU is not being used by any application at a certain moment and then puts it in the lowest power state.

In addition, while the NVIDIA documentation does not state this, the ACPI specification requires that devices, when in the D3 state, even if not consuming any power, must be able to receive and recognize signals from the larger ACPI system that requests devices to transition from the D3 power state to the fully powered D0 power state. It seems that tools like bbswitch make ACPI calls that are not compatible with this requirement and also forces the global power state to G

The implication of this is that for Turing micro-architecture generation (GeForce GTX 1600 series, GeForce RTX 2000 series) and Ampere micro-architecture generation (GeForce RTX 3000 series) NVIDIA GPUs, the NVIDIA driver must always be loaded so that it can manage the power state of the NVIDIA processor using its internal ACPI methods. Furthermore, the only graphics modes that are available are the discrete mode in which the NVIDIA processor is used exclusively, and a hybrid mode in which the NVIDIA processor is used when a particular application needs higher graphics performance and the integrated processor is used otherwise. This mode also enables the runtime power management capabilities of the NVIDIA GPU to put it in the various low power device states, including ACPI D3 such that the NVIDIA processor consumes no power.

The implication of this is that for Turing micro-architecture generation (GeForce GTX 1600 series, GeForce RTX 2000 series) and Ampere micro-architecture generation (GeForce RTX 3000 series) NVIDIA GPUs, the NVIDIA driver must always be loaded so that it can manage the power state of the NVIDIA processor using its internal ACPI methods. Furthermore, the only graphics modes that are available are the discrete mode in which the NVIDIA processor is used exclusively, and a hybrid mode in which the NVIDIA processor is used when a particular application needs higher graphics performance and the integrated processor is used otherwise. This mode also enables the runtime power management capabilities of the NVIDIA GPU to put it in the various low power device states, including ACPI D3 such that the NVIDIA processor con3sumes no power.

The limitation to only either the hybrid or the discrete mode is true even in Windows, where for example, the Legion 5i Pro's Lenovo provided utility for system configuration, Lenovo Vantage, only allows a selection between Dynamic and Discrete graphics modes, where Dynamic is the hybrid mode.

RTD3 Power Management Settings and Status Information

NVIDIA Driver Kernel Module Power Management Options

The NVIDIA driver allows modification of the RTD3 Power Management through parameters set as kernel module parameters. These are:

• NVreg_DynamicPowerManagement
• NVreg_DynamicPowerManagementVideoMemoryThreshold

Parameter	Parameter Values Values
NVreg_DynamicPowerManagement	0x00 Disables RTD3 power management. Driver will use GPU built-in power management. 0x01 Coarse-grained power control. 0x02 Fine-grained power control. 0x03 Default setting. For Ampere and later, equivalent to fine-grained power control; for pre-Ampere disables RTD3 Power Management.
NVreg_DynamicPowerManagementVideoMemoryThreshold	Any positive integer up to 1024 The video memory utilization threshold, expressed in MB. The video memory will not be turned off if utilization is above the set value. Default is 200 MB. Higher values will increase latency during device power state transitions. 0 Prevents video memory from being turned off.

Power State

The NVIDIA driver also provides a filesystem hierarchy interface for querying the status of the power management at /proc/driver/nvidia/gpus/domain:bus:device.function/power, where domain:bus:device.function is the PCI address of the device including its function. The address of the NVIDIA GPU on the Lenovo Legion 5i Pro is 0000:01:00.0. The last 0 is the VGA controller function of the PCI device; other functions of the GPU are 1, 2, and 3, respectively, the audio driver function, USB xHCI controller, and the USB Type-C UCSI controller drivers. Issuing a cat command with the filesystem path as the argument produces output as in the following listing.

 87%  19:19:23  USER: brook HOST: ARCH-16ITH6   
 ~  ❯$ cat /proc/driver/nvidia/gpus/0000:01:00.0/power
Runtime D3 status:          Enabled (fine-grained)
Video Memory:               Off

GPU Hardware Support:
 Video Memory Self Refresh: Supported
 Video Memory Off:          Supported

Power Limits:
 Default:                   N/A milliwatts
 GPU Boost:                 N/A milliwatts

 87%  19:19:27  USER: brook HOST: ARCH-16ITH6   
 ~  ❯$
		

The above listing also shows that the NVIDIA RTD3 power management is active in the fine-grained mode and that the GPU is in the D3 device power state, where it is not consuming any power -- indicated by the N/A under power limits. If the GPU is in the D0 device power state, values displayed under Power Limits would have actual numeric values, as shown in the following listing.

 100%  21:00:22  USER: brook HOST: 16ITH6-openSUSE   
PCD: 3s ~  ❯$ cat /proc/driver/nvidia/gpus/0000:01:00.0/power
Runtime D3 status:          Enabled (fine-grained)
Video Memory:               Active

GPU Hardware Support:
 Video Memory Self Refresh: Supported
 Video Memory Off:          Supported

Power Limits:
 Default:                   60000 milliwatts
 GPU Boost:                 4294967295 milliwatts

 100%  21:01:30  USER: brook HOST: 16ITH6-openSUSE   
 ~  ❯$

Distributions' Optimus Solutions

The most practical way to use a post Turing generation Optimus laptop on Linux, currently, is to use the various Optimus management tools available for, and provided by distributions. These tools can be used to automatically set the graphics mode that should be enabled at boot, selecting between a discrete GPU only, integrated GPU only (pre-Turing GPUs only), and hybrid graphics modes. Hybrid mode will allow the Runtime D3 Power Management capability of the NVIDIA driver to adjust the ACPI device power state to the lowest power consumption device power state when the NVIDIA GPU is not actively in use by an application, using the integrated GPU otherwise; as such this mode is optimal when the battery is supplying power. Discrete mode will use the NVIDIA GPU exclusively to render video for all applications all the time, providing rendering for all applications including the desktop environment.^[2]

Unfortunately, as the Optimus management utilities expect the integrated GPU to be available from boot, Advanced Optimus, which can explicitly set the NVIDIA GPU to be used to display to the EDP after rendering, as discussed above, can not be used at this time.

As discussed above, in pre-Turing generation Optimus laptops, the Optimus management utilities could also be used to switch to the integrated mode in addition to the hybrid and discrete modes. However, with these pre-Optimus laptops the Runtime D3 Power Management of the NVIDIA driver will not be available in hybrid mode and other external utilities would have to be used for power management.

The solutions do vary in whether they provide a GUI, a CLI or both. They also differ in the ancillary functionality they provide such as whether the graphics mode can be set automatically based on the available power source during boot, or whether the NVIDIA driver kernel module options can be set by the utility or they have to be set manually by editing configuration files in /etc/modprobe.d.

Also, despite generalizing the functionality of the pre and post Turing GPU capabilities, the NVIDIA GPU architectures themselves have differences that cause differences in the configuration required between the different post Turing micro-architectures, as documented in Chapter 22. PCI-Express Runtime D3 (RTD3) Power Management of NVIDIA Accelerated Linux Graphics Driver README and Installation Guide

These differences notwithstanding, they do have a common element as the core functionality is to ensure the NVIDIA driver and the NVIDIA GPU are available when switching to the discrete GPU, which has its own requirements regardless of the Optimus management utility or the generation of post Turing GPU. This requirement is the enablement of some systemd services which are distributed with the NVIDIA driver package to be enabled. These services, which facilitate power management by the NVIDIA driver, are illustrated in the following listing showing the output of systemctl list-unit-files.

 89%  19:42:39  USER: brook HOST: 16ITH6-openSUSE   
 ~  ❯$ sudo systemctl list-unit-files --type=service | grep nvidia
[sudo] password for root: 
nvidia-hibernate.service                     enabled         disabled
nvidia-persistenced.service                  enabled         disabled
nvidia-powerd.service                        enabled         disabled
nvidia-resume.service                        enabled         disabled
nvidia-suspend.service                       enabled         disabled

Most critically, without the nvidia-suspend.service, nvidia-hibernate.service, or nvidia-resume.service being enabled in a Linux installation using any of these utilities, the system would not be able to properly suspend, hibernate, or resume.

The various Optimus management utilities for some popular distributions are described below.

openSUSE

openSUSE provides the SUSEPrime CLI utility -- actually a shell script -- for managing Optimus laptops. It is capable of switching, from any other graphics mode, to discrete mode, integrated mode, and hybrid mode -- called "offload"" by the utility. It can also set the default mode to use at every boot, as well as specifying a different mode at the next boot, overriding the default.

It relies on the bbswitch kernel module to manage the power state of the NVIDIA GPU, but only for pre-Turing GPUs. Usage varies between post and pre Turing GPUs, for reasons explained previously, the differences being that the integrated graphics mode is not available in post-Turing GPUs (and setting it should not be attempted, as discussed previously) and that the utility relies on the bbswitch to manage the power state in pre-Turing NVIDIA GPUs. As such, the bbswitch package must be installed if using the utility with pre-Turing GPUs, but not with post-Turing GPUs.

The basic command of the utility is prime-select, which without any sub-commands or arguments produces the help output, shown in the following image of a Konsole window. Also shown is the command used with a sub-command to display the current graphics mode.

The mode can also be specified during boot by setting the value of the kernel parameter nvidia.prime in the GRUB interface, overriding the current boot mode which may have been set with, for example,

prime-select boot offload

or

prime-select next-boot nvidia

For example, including

nvidia.prime=offload

in the kernel command line during boot will set the hybrid mode for the current boot, overriding whatever was set with prime-select boot or prime-select next-boot.

Installation

1. Install NVIDIA driver using one of the alternative processes described in openSUSE Wiki: SDB:NVIDIA drivers. I used the following steps to install the driver on the Lenovo Legion 5i Pro with the NVIDIA GeForce RTX-3050 Mobile, an Ampere micro-architecture GPU.
   1. Add the NVIDIA repository:

      sudo zypper ar -r -p 75 -n "NVIDIA Graphics Drivers" https://download.nvidia.com/opensuse/tumbleweed NVIDIA

   2. Install the NVIDIA driver:

      sudo zypper in nvidia-video-G06

      Note that the package name has been recently changed, possibly to de-emphasize X11 for when Wayland becomes fully supported by the NVIDIA driver (even since starting this article), although the openSUSE Wiki doesn't reflect this name change. When I originally installed the package it was named x11-video-nvidia-G06, so the appropriate command at that time was:

      sudo zypper in x11-video-nvidiaG06

   3. Install the graphics libraries corresponding to the driver package version:

      sudo zypper in nvidia-glG06

For the Lenovo Legion 5i Pro, I originally installed the "G05" set of packages, but when the "G06" set was released, I uninstalled all "G05" packages and installed "G06" packages. This led to more stability, I assume because the "G06" set included the newer 5XX version of the NVIDIA driver, whereas the "G05" set only included the 4XX version. The openSUSE wiki is somewhat misleading because it states that the "G05" set includes drivers for the current NVIDIA GPUs and the "G06" set includes all of the latest NVIDIA GPUs but, it also states that "G05" contains the 4XX driver version and "G06", the 5XX version.
2. Install the SUSEPrime package:

   sudo zypper in suse-prime

Until recently, installing suse-prime would also install bbswitch so that it can be used to switch the NVIDIA GPU off. Currently, this has to be installed separately, but only for pre-Turing generation GPUs. For reasons described previously, it should not be installed in post-Turing NVIDIA GPUs.

The following listing which shows the output of zypper se -i nvidia displays the packages related to the NVIDIA driver ultimately installed, some as dependencies to explicitly installed packages. Note that, as briefly mentioned earlier, there have been changes to NVIDIA related packages since this listing was produced, including a name change of the NVIDIA driver related packages such as a name change of the driver package from x11-video-nvidiaG0X to nvidia-video-G0X and a movement of some files previously provided by a certain package now to another package.

 100%  18:30:02  USER: brook HOST: 16ITH6-openSUSE   
 ~  ❯$ sudo zypper se -i nvidia
Loading repository data...
Reading installed packages...

S  | Name                      | Summary                                                               | Type
---+---------------------------+-----------------------------------------------------------------------+--------
i+ | kernel-firmware-nvidia    | Kernel firmware files for Nvidia Tegra and graphics drivers           | package
i+ | libnvidia-egl-wayland1    | The EGLStream-based Wayland external platform                         | package
i  | nvidia-computeG06         | NVIDIA driver for computing with GPGPU                                | package
i  | nvidia-computeG06-32bit   | 32bit NVIDIA driver for computing with GPGPU                          | package
i  | nvidia-gfxG06-kmp-default | NVIDIA graphics driver kernel module for GeForce 700 series and newer | package
i+ | nvidia-glG06              | NVIDIA OpenGL libraries for OpenGL acceleration                       | package
i  | nvidia-glG06-32bit        | 32bit NVIDIA OpenGL libraries for OpenGL acceleration                 | package
i+ | x11-video-nvidiaG06       | NVIDIA graphics driver for GeForce 700 series and newer               | package
i  | x11-video-nvidiaG06-32bit | 32bit NVIDIA graphics driver for GeForce 700 series and newer         | package

Installed Files

The installation of the NVIDIA driver and SUSEPrime packages, in addition to the files that provide their core functionalities, writes several important other system files. One set of these is the systemd units files previously mentioned. Others are the kernel module configuration files in /usr/lib/modprobe.d/ where module options are set, dracut configuration files in /usr/lib/dracut/dracut.conf.d/, and udev rules in /usr/lib/udev/rules.d. These files and the packages that own them (as of October 2022) are listed in the following table.

Type	Installed Files
Kernel Module Options	File Installed by package /usr/lib/modprobe.d/50-nvidia-default.conf nvidia-gfxG06-kmp-default /usr/lib/modprobe.d/nvidia-default.conf nvidia-gfxG06-kmp-default /usr/lib/modprobe.d/09-nvidia-modprobe-bbswitch-G04.conf suse-prime /usr/lib/modprobe.d/09-nvidia-modprobe-pm-G05.conf suse-prime
udev rules	File Installed by package /usr/lib/udev/rules.d/90-nvidia-udev-pm-G05.rules suse-prime
systemd units	File Installed by package /usr/lib/systemd/system/nvidia-hibernate.service x11-video-nvidia-G06 /usr/lib/systemd/system/nvidia-persistenced.service x11-video-nvidia-G06 /usr/lib/systemd/system/nvidia-powerd.service x11-video-nvidia-G06 /usr/lib/systemd/system/nvidia-resume.service x11-video-nvidia-G06 /usr/lib/systemd/system/nvidia-suspend.service x11-video-nvidia-G06 /usr/lib/systemd/system/prime-select.service suse-prime
dracut configuration files	File Installed by package /usr/lib/dracut/dracut.conf.d/60-nvidia-default.conf nvidia-gfxG06-kmp-default /usr/lib/dracut/dracut.conf.d/90-nvidia-dracut-G05.conf suse-prime

Some of the files installed by the NVIDIA driver and SUSEPrime packages have names that are not consistent with the actual driver set installed, i.e., G06 package set may have a file name that indicates G05. The following set of images of YaST's software management component show the files installed by the NVIDIA driver -- before and after the name change -- and the SUSEPrime packages. After the NVIDIA driver package name change, other changes were made in packaging of the NVIDIA related packages. For example, the systemd unit files that were previously part of the NVIDIA driver package are now in other related packages.

Configuration

Overall configuration of SUSEPrime simply involves ensuring the prime-select.service is enabled, in case it isn't done automatically during package installation. The services installed by the NVIDIA driver package should also be enabled.

Configuration of the NVIDIA driver's Runtime D3 Power Management functionality depends on the generation of post-Turing GPU and the kernel version. As stated in Chapter 22. PCI-Express Runtime D3 (RTD3) Power Management of NVIDIA Accelerated Linux Graphics Driver README and Installation Guide, the RTD3 Power Management is not enabled by default in earlier versions of post-Turing GPUs, and must be enabled manually for the GPU's video controller function with a command such as

echo auto > /sys/bus/pci/devices/0000\:01\:00.0/power/control

Also, as stated in Chapter 22 of the NVIDIA Readme, this can be automated with udev rules. The suse-prime package provides the udev rules file /usr/lib/udev/rules.d/90-nvidia-udev-pm-G05.rules, indicated in the above table, that effectuates this automation. These are useful for post-Turing, pre-Ampere Optimus laptops or those that are running Linux kernel versions before 5.5, but not necessary on when running later kernels or post-Ampere NVIDIA GPUs.

So, the only real configuration required is setting the desired driver kernel module options.

NVIDIA Driver Kernel Module Options

The default installation of the SUSEPrime package sets one kernel module option, NVreg_DynamicPowerManagement setting its value to 0x01 in the file /usr/lib/modprobe.d/09-nvidia-modprobe-pm-G05.conf. This and the other possible NVIDIA kernel module options were described in RTD3 Power Management Settings and Status Information -> NVIDIA Driver Kernel Module Power Management Options, where we saw that this option and value sets the RTD3 Power Management to "coarse-grained control". To set a different value for this option, it is necessary override it by setting a different value to the parameter in a file in /etc/modprobe.d/. We can do this by copying /usr/lib/modprobe.d/09-nvidia-modprobe-pm-G05.conf to /etc/modprobe.d/09-nvidia-modprobe-pm-G05.conf and setting a different value, for example, 0x02 which sets RTD Power Management to "fine-grained control". The image below shows a split Konsole window with the top pane showing the contents of the default in /usr/lib/modprobe.d/09-nvidia-modprobe-pm-G05.conf, and the bottom showing the module option set to a different value, overriding the default.

Graphical Tool

TBA

Arch (and Derivatives)

One of the best Optimus management utilities -- supporting both NVIDIA and AMD discrete GPUs -- is found in the AUR for Arch Linux and derivatives, in the regular repositories of some derivatives, and in third-party repositories such as the Chaotic-AUR, namely, Optimus Manager. The tool, written in Python, is capable of -- like SUSEPrime -- switching from any other graphics mode, to discrete mode, integrated mode. Also like SUSEPrime, it can set the default mode to use at every boot, as well as specifying a different mode at the next boot, overriding the default. As with SUSEPrime, because the integrated graphics mode is not available in post-Turing GPUs, setting the graphics mode to integrated is not possible.

However, unlike SUSEPrime, it uses a very robust configuration system which allows modifying the operation of the utility by setting parameter values in the configuration files. Numerous parameters are available that among others, allow the user to specify which graphics mode should be set depending on the available power source, i.e., battery or AC power, to specify the method to use to power on/off the NVIDIA GPU from among four possible methods (this is not usable for post-Turing GPUS), to specify the NVIDIA driver option values (not just NVreg_DynamicPowerManagement) such that Optimus Manager writes the necessary files to set these values, and other parameters that would be included in the Xorg configuration file.

The configuration system starts with a default file provided by the package in /usr/share/optimus-manager.conf which can be overridden by values in /etc/optimus-manager/optimus-manager.conf and by a file with the appropriate values passed as a command line option.

The output of optimus-manager --help is shown in the following image.

Similar to SUSEPrime, the mode can be specified for the current boot, overriding the default boot mode, which may have been set using one of the utility's configuration options by appending the kernel parameter optimus-manager.startup to the kernel command line in the GRUB interface. Possible values of the parameter are integrated (only possible with pre-Turing GPUs), hybrid, nvidia, or amd. For example, appending

optimus-manager.startup=hybrid

to the kernel command line will set the graphics mode to hybrid for the current boot.

Installation

Optimus Manager is packaged as optimus-manager in the AUR. A development version is available as optimus-manager-git. Another development version for systems which use the OpenRC init system is also available in the package optimus-manager-openrc-git. Packages of a graphical interface to Optimus Manager, Optimus Manager Qt, are available as optimus-manager-qt, optimus-manager-qt-git, and optimus-manager-qt-kde.

The third-party repository for Arch based systems, Chaotic-AUR conveniently provides a prebuilt packages of the utility -- optimus-manager and optimus-manager-git, and its graphical interface -- optimus-manager-qt and optimus-manager-qt-git. The output from the paru AUR helper shown in the following image displays all of these available packages.

I installed the stable versions of the utility and its graphical interface using paru, which as configured on my system, prefers the Chaotic-AUR packages over the AUR packages with

paru -S optimus-manager optimus-manager-qt

This is shown in the following image.

The files written by the optimus-manager package are displayed in the following listing showing the output of pacman -Fl optimus-manager

 100%  19:53:45  USER: brook HOST: ARCH-16ITH6   
 ~  ❯$ sudo pacman -Fl optimus-manager
optimus-manager etc/
optimus-manager etc/lightdm/
optimus-manager etc/lightdm/lightdm.conf.d/
optimus-manager etc/lightdm/lightdm.conf.d/20-optimus-manager.conf
optimus-manager etc/optimus-manager/
optimus-manager etc/optimus-manager/nvidia-disable.sh
optimus-manager etc/optimus-manager/nvidia-enable.sh
optimus-manager etc/optimus-manager/xorg/
optimus-manager etc/optimus-manager/xorg/hybrid-mode/
optimus-manager etc/optimus-manager/xorg/hybrid-mode/integrated-gpu.conf
optimus-manager etc/optimus-manager/xorg/hybrid-mode/nvidia-gpu.conf
optimus-manager etc/optimus-manager/xorg/integrated-mode/
optimus-manager etc/optimus-manager/xorg/integrated-mode/integrated-gpu.conf
optimus-manager etc/optimus-manager/xorg/nvidia-mode/
optimus-manager etc/optimus-manager/xorg/nvidia-mode/integrated-gpu.conf
optimus-manager etc/optimus-manager/xorg/nvidia-mode/nvidia-gpu.conf
optimus-manager etc/optimus-manager/xsetup-hybrid.sh
optimus-manager etc/optimus-manager/xsetup-integrated.sh
optimus-manager etc/optimus-manager/xsetup-nvidia.sh
optimus-manager etc/sddm.conf.d/
optimus-manager etc/sddm.conf.d/20-optimus-manager.conf
optimus-manager usr/
optimus-manager usr/bin/
optimus-manager usr/bin/optimus-manager
optimus-manager usr/bin/prime-offload
optimus-manager usr/bin/prime-switch
optimus-manager usr/lib/
optimus-manager usr/lib/modprobe.d/
optimus-manager usr/lib/modprobe.d/optimus-manager.conf
optimus-manager usr/lib/python3.10/
optimus-manager usr/lib/python3.10/site-packages/
optimus-manager usr/lib/python3.10/site-packages/optimus_manager-1.4-py3.10.egg-info/
optimus-manager usr/lib/python3.10/site-packages/optimus_manager-1.4-py3.10.egg-info/PKG-INFO
optimus-manager usr/lib/python3.10/site-packages/optimus_manager-1.4-py3.10.egg-info/SOURCES.txt
optimus-manager usr/lib/python3.10/site-packages/optimus_manager-1.4-py3.10.egg-info/dependency_links.txt
optimus-manager usr/lib/python3.10/site-packages/optimus_manager-1.4-py3.10.egg-info/entry_points.txt
optimus-manager usr/lib/python3.10/site-packages/optimus_manager-1.4-py3.10.egg-info/top_level.txt
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__init__.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/__init__.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/__init__.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/acpi_data.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/acpi_data.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/checks.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/checks.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/config.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/config.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/daemon.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/daemon.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/envs.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/envs.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/kernel.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/kernel.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/kernel_parameters.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/kernel_parameters.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/log_utils.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/log_utils.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/pci.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/pci.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/processes.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/processes.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/sessions.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/sessions.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/var.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/var.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/xorg.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/__pycache__/xorg.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/acpi_data.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/checks.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/client/
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/client/__init__.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/client/__pycache__/
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/client/__pycache__/__init__.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/client/__pycache__/__init__.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/client/__pycache__/args.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/client/__pycache__/args.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/client/__pycache__/client_checks.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/client/__pycache__/client_checks.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/client/__pycache__/error_reporting.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/client/__pycache__/error_reporting.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/client/__pycache__/utils.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/client/__pycache__/utils.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/client/args.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/client/client_checks.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/client/error_reporting.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/client/utils.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/config.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/config_schema.json
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/daemon.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/envs.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hacks/
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hacks/__init__.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hacks/__pycache__/
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hacks/__pycache__/__init__.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hacks/__pycache__/__init__.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hacks/__pycache__/manjaro.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hacks/__pycache__/manjaro.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hacks/manjaro.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/__init__.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/__pycache__/
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/__pycache__/__init__.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/__pycache__/__init__.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/__pycache__/post_daemon_stop.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/__pycache__/post_daemon_stop.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/__pycache__/post_resume.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/__pycache__/post_resume.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/__pycache__/post_xorg_start.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/__pycache__/post_xorg_start.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/__pycache__/pre_daemon_start.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/__pycache__/pre_daemon_start.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/__pycache__/pre_suspend.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/__pycache__/pre_suspend.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/__pycache__/pre_xorg_start.cpython-310.opt-1.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/__pycache__/pre_xorg_start.cpython-310.pyc
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/post_daemon_stop.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/post_resume.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/post_xorg_start.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/pre_daemon_start.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/pre_suspend.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/hooks/pre_xorg_start.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/kernel.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/kernel_parameters.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/log_utils.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/pci.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/processes.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/sessions.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/var.py
optimus-manager usr/lib/python3.10/site-packages/optimus_manager/xorg.py
optimus-manager usr/lib/systemd/
optimus-manager usr/lib/systemd/logind.conf.d/
optimus-manager usr/lib/systemd/logind.conf.d/10-optimus-manager.conf
optimus-manager usr/lib/systemd/system-sleep/
optimus-manager usr/lib/systemd/system-sleep/optimus-manager.py
optimus-manager usr/lib/systemd/system/
optimus-manager usr/lib/systemd/system/optimus-manager.service
optimus-manager usr/share/
optimus-manager usr/share/licenses/
optimus-manager usr/share/licenses/optimus-manager/
optimus-manager usr/share/licenses/optimus-manager/LICENSE
optimus-manager usr/share/optimus-manager.conf

 100%  19:53:48  USER: brook HOST: ARCH-16ITH6   
 ~  ❯$ 

Note that the AUR and paru have to be enabled on an Arch system, as described in An Arch Linux Installation on a Btrfs Filesystem with Snapper for System Snapshots and Rollbacks and the Chaotic-AUR repository has to be enabled as described prominently on its web page.

Configuration

Configuration of Optimus Manger and some of the NVIDIA kernel module options is entirely in the editing of the configuration file. The default configuration options found in /usr/share/optimus-manager.conf can be overridden by first copying the default file to /etc/optimus-manager/optimus-manager.conf and editing values as appropriate for the system.

The most important configuration options is the switching parameter in the [optimus] section of the file. This parameter specifies whether Optimus Manager should attempt to manage the power state of the NVIDIA GPU and the method to use. This must be set to none in Optimus laptops with post-Turing NVIDIA GPUs for reasons discussed earlier in this article, and to avoid the consequences, which were described earlier in this article and in Lenovo LEGION 5i PRO 16ITH6 Review [82JF002RUS]. For pre-Turing GPUs, the method of managing the NVIDIA GPU power state can be specified. On my Dell G5 and Acer V15, I have used the bbswitch and nouveau options without issues. The configuration file is well commented and describes the parameter, as shown in the following listing:

# This parameter defines the method used to power switch the Nvidia card. See the documentation
# for a complete description of what each value does. Possible values :
#
# - nouveau : load the nouveau module on the Nvidia card.
# - bbswitch : power off the card using the bbswitch module (requires the bbswitch dependency).
# - acpi_call : try various ACPI method calls to power the card on and off (requires the acpi_call dependency)
# - custom: use custom scripts at /etc/optimus-manager/nvidia-enable.sh and /etc/optimus-manager/nvidia-disable.sh
# - none : do not use an external module for power management. For some laptop models it's preferable to
#          use this option in combination with pci_power_control (see below).
switching=none

The above listing, besides showing the comments that describe the option, shows that the switching option is set to "none" on my Legion 5 Pro.

The values of the NVIDIA kernel module options NVreg_DynamicPowerManagement and NVreg_DynamicPowerManagementVideoMemoryThreshold can be set directly in the Optimus Manager configuration file through the Optimus-Manager configuration parameters dynamic_power_management and dynamic_power_management_memory_threshold, respectively, with the utility ensuring that the values are then set in the appropriate files in /etc/modprobe.d/. Again, the configuration file comments describe the options very well:

# Enable Runtime D3 (RTD3) Power Management in the Nvidia driver. While in hybrid mode,
# this feature allows the Nvidia card to go into a low-power mode if it's not in use.
#
# IMPORTANT NOTES:
# - The feature is still experimental
# - It's only supported on laptops with a Turing GPU or above, and an Intel Coffee Lake CPU
# or above (not sure about the state of support for AMD CPUs).
# - if your Nvidia card also has an audio chip (for HDMI) or a USB port wired to it, they may not
# function properly while in low-power mode
#
# For more details, see
# https://download.nvidia.com/XFree86/Linux-x86_64/460.39/README/dynamicpowermanagement.html
#
# Available options:
# - no (the default): RTD3 power management is disabled.
# - coarse: the card only goes to low-power if no application is using it.
# - fine: the card is also allowed to go to low-power if applications are using it but have not
# actively submitted GPU work in some amount of time.
dynamic_power_management=fine

# The Nvidia driver handles power to the video memory separately from the rest of GPU.
# When dynamic_power_management=fine, this options controls the threshold of memory utilization
# (in Megabytes) under which the memory is put in a low-power state.
# Values over 200MB are ignored. Leave blank for the default (200MB).
# Setting this value to 0 keeps the memory powered at all times.
dynamic_power_management_memory_threshold=

The listing shows that dynamic_power_management, which specifies the value of NVreg_DynamicPowerManagement is set to "fine", corresponding to a value for the kernel parameter of "0x02". The dynamic_power_management_memory_threshold which specifies the value of NVreg_DynamicPowerManagementVideoMemoryThreshold is left blank, which corresponds to the default value of the kernel parameter of 200MB. Both of these kernel parameters were discussed in the section RTD3 Power Management Settings and Status Information -> NVIDIA Driver Kernel Module Power Management Options, above.

This capability of setting the NVIDIA RTD3 Power Management kernel module options is one of those that set it apart from SUSEPrime and other Optimus Manager utilities for Linux which require setting these values manually by editing the modprobe configuration files.

Graphical Tool

A graphical interface to the Optimus Manager utility, called Optimus Manager Qt is available from a separate developer. As the name suggests, it is built with Qt and thus suitable only for the Plasma desktop environment. This GUI is available through the previously mentioned packages in the AUR and the third-party Chaotic-AUR, and possibly the regular repositores of Arch derivatives. The GUI is not only capable of switching from one graphics mode to another but is able to manipulate the configuration of Optimus Manager as if the configuration file was edited. A Plasma panel applet can initiate the switch or launch the full featured GUI.

Ubuntu

Ubuntu provides the simplest and most user friendly Optimus management utility. The primary interface to the utility is integrated into the NVIDIA Settings GUI as an additional screen in NVIDIA Settings, activated by the PRIME Profiles. In this interface, the user only needs to activate the radio button corresponding to the desired graphics mode and log out. Note that on Optimus laptops with a post-Turing NVIDIA GPU, the integrated mode ("Intel") is grayed out and not available for reasons described previously. This interface is shown in the following image.

The functionality integrated into NVIDIA Settings, collectively called NVIDIA Prime by Ubuntu, seems to be provided by -- as far as I can tell from a brief examination of the files provided by the relevant packages -- the Python script /usr/bin/prime-select and the shell scripts /usr/bin/prime-supported, /sbin/prime-switch, /sbin/prime-offload, and /sbin/get-quirk-options. A binary executable, /usr/bin/gpu-manager is also called by the prime-switch script.

Some of the screens of NVIDIA Settings are shown in the following set of images.

CLI

Although the Optimus management GUI is prominent, consistent with the general perception of Ubuntu being user friendly through the use of GUI tools versus CLI tools, Ubuntu's NVIDIA Prime provides a CLI interface for switching graphics modes with the prime-select script. The basic command is:

prime-select < argument >

where the argument can be "nvidia", "intel", or "on-demand". Another argument, "query" is available to output the current graphics mode. As stated before, the "intel" argument can not be used on laptops with post-Turing Optimus laptops.

Installation

The NVIDIA driver, NVIDIA Settings, and NVIDIA Prime can be installed simply by using the Additional Drivers component of the Software and Updates application -- provided that installation from the restricted set of packages was enabled during the initial installation of Ubuntu -- by selecting the appropriate item. Before installing the NVIDIA related the open source Nouveau driver will be shown as being used. Selecting, for example, "driver metapackage from nvidia-driver-525" will install this version of the NVIDIA proprietary driver and all related utilities including NVIDIA Prime. The following image shows Additional Drivers component of the Software and Updates on an Optimus laptop after the installation.

Alternatively, the proprietary driver and all necessary utilities can be installed in a terminal, as follows.

1. This requires first determining the available packages suitable for the GPU with

   sudo ubuntu-drivers devices

   The output, as shown in the following listing, will first identify and describe the hardware then provide a list of available drivers, with the recommended option indicated. In the listing the package "nvidia-driver-525" is recommended.

    100%  08:57:56  USER: brook HOST: UBUNTU-16ITH6   
    ~  ❯$ sudo ubuntu-drivers devices
   == /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0 ==
   modalias : pci:v000010DEd000025E2sv000017AAsd00003E87bc03sc00i00
   vendor   : NVIDIA Corporation
   model    : GA107BM [GeForce RTX 3050 Mobile]
   driver   : nvidia-driver-470 - distro non-free
   driver   : nvidia-driver-515 - distro non-free
   driver   : nvidia-driver-525-open - distro non-free
   driver   : nvidia-driver-525 - distro non-free recommended
   driver   : nvidia-driver-525-server - distro non-free
   driver   : nvidia-driver-515-open - distro non-free
   driver   : nvidia-driver-470-server - distro non-free
   driver   : nvidia-driver-515-server - distro non-free
   driver   : nvidia-driver-510 - distro non-free
   driver   : xserver-xorg-video-nouveau - distro free builtin
   
   
   

2. Then install the driver and the related packages with:

   sudo apt install nvidia-driver-525 nvidia-settings nvidia-prime

Installed Files

The Optimus management scripts providing functionality for NVIDIA prime are installed by the nvidia-prime package. NVIDIA settings is installed by the nvidia-settings package. Other secondary system files necessary for NVIDIA Optimus, such as systemd unit files are installed by nvidia-driver-common-525, a dependency of nvidia-driver-525 (for NVIDIA driver version 525). Some of these secondary files are listed in the following table.

Type	Installed Files
Kernel Module Options	File Installed by package /lib/modprobe.d/nvidia-runtimepm.conf Not written by package install process. File generated by Ubiquity during installation with UbuntuDrivers. Kernel options written by prime-select script. /lib/modprobe.d/nvidia-drivers-common.conf nvidia-kernel-common-525 (dependency of nvidia-driver-525)
udev rules	File Installed by package /lib/udev/rules.d/71-nvidia.rules nvidia-kernel-common-525 (dependency of nvidia-driver-525)
systemd units	File Installed by package /lib/systemd/system/nvidia-hibernate.service nvidia-kernel-common-525 (dependency of nvidia-driver-525) /lib/systemd/system/nvidia-resume.service nvidia-kernel-common-525 (dependency of nvidia-driver-525) /lib/systemd/system/nvidia-suspend.service nvidia-kernel-common-525 (dependency of nvidia-driver-525)
initramfs configuration files	File Installed by package

One notable item in the table is the file that contains NVIDIA RTD3 Power Management kernel options, in Ubuntu's case, /lib/modprobe.d/nvidia-runtimepm.conf. Unlike with SUSEPrime and Optimus Manager (which use /usr/lib/modprobe.d/), this file is not written as part of a package installation, is initially written by Ubiquity -- the Ubuntu installer -- as is indicated by its log, using the UbuntuDrivers Python module to identify graphics hardware, and modified to write a specific option by prime-select. For GPUs that support the RTD3 Power Management, the script sets the value of NVreg_DynamicPowerManagement to 0x02, equivalent to "fine-grained" power management.

Configuration

In order to modify the default generated by prime-select , it is necessary to create a file in /etc/modprobe.d/ that sets the values.

1. Create the file /etc/modprobe.d/09-nvidia-modprobe.conf
2. Add the line

   options nvidia NVreg_DynamicPowerManagement=0x02

   to the file, replacing the 0x02 with an appropriate value as described above in RTD3 Power Management Settings and Status information -> NVIDIA Driver Kernel Module Power Management Options
3. After reboot, fine-grained RTD3 Power Management should be enabled.

PopOS

As an Ubuntu derivative, PopOS uses the same NVIDIA Settings addition to switch graphics modes. The distribution also provides additional GUI tools that are integrated by default into the GNOME shell panel system menu, and integrated optionally into a Plasma panel through the PopOS KSys76 Power widget, shown in the following image. Of course, the same limitation regarding switching to integrated mode for Post-Turing GPUs applies.

Fedora/Red Hat

TBA

References

1. NVIDIA chip code names and their corresponding official GeForce number
2. List of NVIDIA graphics processing units
3. GeForce
4. How Advanced Optimus Delivers Max Performance With Optimal Battery Life
5. Advanced Configuration and Power Interface (ACPI) Specification
6. Device Power States
8. Bumblebee-Project/bbswitch GitHub Page
9. NVIDIA Accelerated Linux Graphics Driver README and Installation Guide
10. Chapter 22. PCI-Express Runtime D3 (RTD3) Power Management of NVIDIA Accelerated Linux Graphics Driver README and Installation Guide
11. Chapter 16. Configuring a Notebook of NVIDIA Accelerated Linux Graphics Driver README and Installation Guide
12. Chapter 35. PRIME Render Offload
13. Appendix B. X Config Options of NVIDIA Accelerated Linux Graphics Driver README and Installation Guide
14. openSUSE Forum Thread: No video after sleep (NVIDIA)
15. SUSE Prime GitHub Page
16. openSUSE Wiki Page on SUSE Prime
17. openSUSE Wiki Page on NVIDIA Drivers
18. Optimus Manager GitHub Page
19. Ubuntu Wiki Page: BinaryDriverHowto/Nvidia