CGroups

Resource exhaustion is one of the common issues while running production machines. There are cases where running servers crash due to other process using high memory or any other process running a high CPU intensive code. It is always good if we have a way to control resource usage. On larger systems kernel resource controllers (also called as Control groups (CGroups)) can be usefull to help priority applications to get the necessary resources thus limiting resources for other applications.

In this article we will see how we can use Cgroups to manage resources. According to kernel documentation, Control Groups provide a mechanism for aggregating/partitioning sets of tasks, and all their future children, into hierarchical groups with specialized behavior.

CGroups is a linux kernel feature to limit, account and isolate resource (CPU, memory, disk, I/O) usages of process groups. Using this we can get a control over allocating, prioritizing, managing and monitoring system resources. The Cgroup can also be taught as a generic framework where resource controller can be plugged in which is then used to manage different resources of the system.

The resource controller can be a memory, CPU, Network or a Disk I/O controller. As the name suggests each of this controller performs the necessary functions like memory controller managing the memory of the processes.

The type of the resources that can be managed by CGroups include the following,

·         Blkio (Storage) - Limits total input and output access to storage devices (such as hard disks, USB drives, and so on)

·         CPU (Processor Scheduling) - Assigns the amount of access a cgroup has to be scheduled for processing power

·         Memory - Limits memory usage by task. It also creates reports on memory resources used.

·         Cpuacct (Process accounting) – Reports on CPU usage. This information can be leveraged to charge clients for the amount of processing power they use

·         Cpuset (CPU assignment) - On systems with multiple CPU cores, assigns a task to a particular set of processors and associated memory

·         Freezer (Suspend/resume) - Suspends and resumes cgroup tasks

·         net_cls (Network bandwidth) - Limits network access to selected cgroup tasks

There are some other resources that are even managed by CGroups. Check the docs for more details.

Installation

The easiest way to work with CGroups is to install the libcgroup package which contains the necessary packages and utilities for using CGroups.libcgroup is a library that abstracts the control group file system in Linux.

[root@vx111a work]# yum list installed | grep libcgroup

libcgroup.x86_64                       0.41-8.el7                      @anaconda

libcgroup-tools.x86_64               0.41-8.el7                      @anaconda

Install the libcgroup library and we can start from there.

Configuration

Cgroups are implemented using a file system-based model—just as you can traverse the /proc tree to view process-related information, you can examine the hierarchy at /cgroup to examine current control group hierarchies, parameter assignments, and associated tasks.

Once the libcgroup package is installed we get 2 services

Cgconfig

Cgred

Cgconfig – The cgconfig service installed with the libcgroup packages provides a convenient way to create hierarchies, attach sub systems to the hierarchies and manage cgroups with in thise hierarchies. The service is not started by default.  The service reads the file /etc/cgconfig.conf and depending on the contents of the file it creates hierarchies, mounts necessary files systems, creates cgroups and sets the sub system parameters.

The default /etc/cgconfig.conf file installed with the libcgroup package creates and mounts an individual hierarchy for each subsystem, and attaches the subsystems to these hierarchies. In other words this is used to define control groups, their parameters and also mount points.

Once the cgconfig service is started, a virtual file system is mounted. This can be either /cgroup or /sys/fs/cgroup.

​[root@vx111a conf.d]# ll /sys/fs/cgroup/
total 0
drwxr-xr-x 2 root root  0 Mar 16 13:35 blkio
lrwxrwxrwx 1 root root 11 Mar 15 14:24 cpu -> cpu,cpuacct
lrwxrwxrwx 1 root root 11 Mar 15 14:24 cpuacct -> cpu,cpuacct
drwxr-xr-x 5 root root  0 Mar 16 13:36 cpu,cpuacct
drwxr-xr-x 2 root root  0 Mar 15 14:24 cpuset
drwxr-xr-x 4 root root  0 Mar 15 14:24 devices
drwxr-xr-x 2 root root  0 Mar 15 14:24 freezer
drwxr-xr-x 2 root root  0 Mar 15 14:24 hugetlb
drwxr-xr-x 2 root root  0 Mar 16 13:36 memory
drwxr-xr-x 2 root root  0 Mar 15 14:24 net_cls
drwxr-xr-x 2 root root  0 Mar 15 14:24 perf_event
drwxr-xr-x 4 root root  0 Mar 15 14:24 systemd

The configuration file contains the group elements. The resource that needs to be managed is defined in the configuration file. A simple configuration looks as

group group1 {  cpu {         cpu.shares="800";     }     cpuacct {         cpuacct.usage="0";     }     memory {         memory.limit_in_bytes="4G";         memory.memsw.limit_in_bytes="6G";     } }

In the above snippet we defined a group with the name group1 in which we defined the sus systems that we want to manage. We defined the CPU and Memory Sub systems.

The cpu.shares parameter determines the share of CPU resources available to each process in all cgroups

The memory.limit_in_bytes parameter tells the amount of memory that this group has access to. The processes associated to this group will be given with 4GB limit of memory.

The memory.memsw.limit_in_bytes parameter specifies the total amount of memory and swap space processes may use. 

Cgred is a service that moves tasks into cgroups according to parameters set in the /etc/cgrules.conf file. This file contains list of rules which assign to a defined group/user a control group in a subsystem.

The configuration file contains the form

user    subsystems    control_group

a sample example would be like

*:java    memory  group1

In the above snippet we have defined a rule such that all java Process that starts will be added to the memory system under the group1. So all process started by java will have the limit of 4G memory as we defined in the /etc/cgconfig.conf file.

Start the Service

Start both the services using the commands,

Service cgconfig restart

Service cgred restart

Once the services are started we can check our configuration using the lscgroup command as

[root@vx111a conf.d]# lscgroup -g cpu:/
cpu,cpuacct:/
cpu,cpuacct:/group1
cpu,cpuacct:/browsers
memory:/group1

devices:/blockDevice

lssubsys - The command lssubsys -am lists all subsystems which are present in the system, mounted ones will be shown with their mount point:

[root@vx111a docker]# lssubsys
cpuset
cpu,cpuacct
memory
devices
freezer
net_cls
blkio
perf_event
hugetlb

In the next article we will see some of the use cases using the Cgroups. More to come. Happy Learning