Supporting 1G hugepages requires the Kernel to support 1G Hugepages. This is done by applying a boot Kernel parameter.
- Create a MachineConfig (MC) to apply to the Node type that will be using the 1G hugepages. The following example set 1G hugepages for all worker nodes
50-kargs-1g-hugepages.yaml
:apiVersion: machineconfiguration.openshift.io/v1 kind: MachineConfig metadata: labels: machineconfiguration.openshift.io/role: worker name: 50-kargs-1g-hugepages spec: kernelArguments: - default_hugepagesz=1G - hugepagesz=1G # - hugepages=32
- Apply the MC to the cluster
This exaple will update the Kernel parameters for all the Worker Nodesoc create -f 50-kargs-1g-hugepages.yaml
Create hugepages-1g_tuning.yaml
with the desired configuration. This is a way to pass Tuned parameters that are handled by the Node Tuning Operator.
Huge Pages are allocated explicitly by the vm.nr_hugepages
sysctl parameter.
The following example enables 1G hugepages for any Node with a label of hugepages-1g
:
apiVersion: tuned.openshift.io/v1
kind: Tuned
metadata:
name: hugepages-1g
namespace: openshift-cluster-node-tuning-operator
spec:
profile:
- data: |
[main]
summary=Configuration for hugepages
include=openshift-node
[vm]
transparent_hugepages=never
[sysctl]
# The Default Hugepage size 2MB (1024*2)
# Alternate Hugepage size 1G (1024*1024) -- need to be set with MachineConfig kernelArguments
# vm.nr_hugepages is the Number of Hugepages to allocate
# Examples:
# If need 1G pool from 2MB hugepagesize then (1024*1024)/2048 = 512
# If need 2G pool from 1GB hugepagesize then (1024*1024*2)/(1024*1024)=2
vm.nr_hugepages=2
name: node-hugepages-1g
recommend:
- match:
- label: hugepages-1g
priority: 30
profile: node-hugepages-1g
Create the new Tuned profile
oc create -f hugepages-1g_tuning.yaml
Apply the hugepages-1g
label to the Nodes to be used with the 1G Hugepage
oc label node <node_using_hugepages> hugepages-1g=true
After the MC applies the config and reboots the node, it should report the new parameters
[root@worker-0 ~]# cat /proc/cmdline
BOOT_IMAGE=/ostree/rhcos-ee73255863c1e88baaa06eac3fff2d753f20da9a49139224bb88dbfe2a7ba712/vmlinuz-4.18.0-80.11.2.el8_0.x86_64 console=tty0 console=ttyS0,115200n8 rootflags=defaults,prjquota rw root=UUID=6101ba79-239a-439e-b891-6315c6c4b7bd ostree=/ostree/boot.1/rhcos/ee73255863c1e88baaa06eac3fff2d753f20da9a49139224bb88dbfe2a7ba712/0 coreos.oem.id=metal ignition.platform.id=metal default_hugepagesz=1G hugepagesz=1G
[root@worker-0 ~]#
The /proc/meminfo
should report the available pages and page size
# grep -i huge /proc/meminfo
AnonHugePages: ###### ##
ShmemHugePages: 0 kB
HugePages_Total: 2
HugePages_Free: 2
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: #### ##
Hugetlb: #### ##
The Node description should report the new size
$ oc describe node worker-0.ocp4poc.example.com | grep -i huge
hugepages-1g=true
hugepages-###: ###
hugepages-###: ###