IRQ Balance

The purpose of irqbalance (Interrupt ReQuest) is distribute interrupts across processors on a multiprocessor system in order to increase performance. You can check all of number of interrupts per cpu and each device in the cd /proc/interrupts directory.

Irqbalance is especially useful on systems with multi-core processors, as interrupts will typically only be serviced by the first core.

It is recommended that all interrupts generated by a specific device to be handled by the same CPU cores. IRQ fair distribution between all CPU cores is not recommended because when the interrupt goes to another fresh CPU core, the new CPU core will load the interrupt handler function from the main memory to the cache (time overhead).

Execute this “cat /proc/interrupts” to get the list of interrupts in your system. (this output for Genband S3 SBC)

Explanation for all columns;

Column 1: It describes number of IRQ
Column 2…n: Each CPU in the system has its own column and its own number of interrupts per IRQ.
Next column: It lists types of Interrupt.
Last column: It lists the driver or device that is associated with the interrupt.

Types of Interrupts;

• XT-PIC — This is the old AT computer interrupts.
• IO-APIC-edge — The voltage signal on this interrupt transitions from low to high, creating an edge, where the interrupt occurs and is only signaled once.
• IO-APIC-level — Generates interrupts when its voltage signal is high until the signal is low again.

Change Affinity Value for a Specific IRQ

The affinity value (CPU cores) for a specific IRQ is stored in the file “/proc/irq/IRQ_NUMBER/smp_affinity”. The value in this file is a hexadecimal bit-mask (The lowest order bit corresponding to the first CPU). For example to set the affinity value to the IRQ-14 (the example above), do this as root:

#Display the current affinity value: “cat /proc/irq/14/smp_affinity”.
#Change the affinity of the IRQ-14 to the first and third cores (0101). The corresponding hex value is 5. We write this number 5 in the file “/proc/irq/30/smp_affinity”, like this; “echo 5 > /proc/irq/14/smp_affinity”

if necessary to show another instance;

[root@anakin /proc]# echo 1 > /proc/irq/24/smp_affinity
[root@anakin /proc]# cat /proc/irq/24/smp_affinity
00000001

Now, let’s test it out and see what happens:

[root@anakin /proc]# cd /tmp/
[root@anakin /tmp]# tar -zcf test.tgz /usr/src/linux-2.4.2
tar: Removing leading `/’ from member names
[root@anakin /tmp]# tar -zxf test.tgz && rm -rf usr/
[root@anakin /tmp]# tar -zxf test.tgz && rm -rf usr/
[root@anakin /tmp]# tar -zxf test.tgz && rm -rf usr/
[root@anakin /tmp]# tar -zxf test.tgz && rm -rf usr/
[root@anakin /tmp]# tar -zxf test.tgz && rm -rf usr/
[root@anakin /tmp]# cat /proc/interrupts | grep 24:
24: 99719 86067 86012 86627 IO-APIC-level aic7xxx

Compare that to the previous run without having the IRQ bound to CPU0:

24: 87298 86066 86012 86626 IO-APIC-level aic7xxx

You can see interrupt number differences on the first cpu.

In short, this means you only have to worry about the “f” and you
can assume everything else is a “0”

GBSBC-1:/proc/irq/10 # cat smp_affinity
f
GBSBC-1:/proc/irq/10 #

“f” is the hexadecimal represenatation for the decimal number 15 (fifteen)
and the binary pattern of “1111”. Each of the places in the binary pattern
corresponds to a CPU in the server, which means we can use the following
chart to represent the CPU bit patterns:

Binary Hex
CPU 0 0001 1
CPU 1 0010 2
CPU 2 0100 4
CPU 3 1000 8

Also, if we see output of #cat /proc/irq/40/smp_affinity like fallowing example, it explains that descrite 32-bit groups and using commas to describe it. This mean is required on system more than 32 cores.

# cat /proc/irq/40/smp_affinity
ffffffff,ffffffff

Please do not hesitate to ask any question about irqbalance with using my e-mail.