回收 Linux cached memory

最近常被問到,當程式讀取大量資料, kernel 會使用大量記憶體當 cache,又不會在記憶體不足時進行回收,造成程式無法執行。這時該怎麼辨?

方法一:

echo 1 > /proc/sys/vm/drop_caches
    or
sysctl -w vm.drop_caches=1

這會觸發 kernel 回收用於 cache 的記憶體。

方法二:

fcntl(fd, F_SETFL, O_DIRECT)

在程式裡,為 file descriptor 設上 O_DIRECT,使 filesystem 避免 cache 該檔案的內容。這適用於某些大檔案。例如,播放隨便就上 Giga bytes 的影片的 media player,就能在 file descriptor 設定 O_DIRECT,避免吃掉大量的系統記憶體。

方法三:

sysctl -w vm.vfs_cache_pressure=n  (n > 100)

這會使 kernel 更勤於回收 cache。

以上作為參考。

O_DIRECT 的用途

Flags, enum (C) - Stack Overflow

another way of storing flags is to not bother with the underlying type at all. when using an enum, the enum values are stored by default into an unsigned int, which is 32 bits on a common computer. this gives you with only 32 possible flags: while certainly much, there are some cases where it is not sufficient.

now you can define your flag set this way:

typedef struct
{
   
int takes_damage : 1;
   
int grabbable    : 1;
   
int liquid       : 1;
   
int some_other   : 1;
} flags;

if you never encountered this, the ': 1' part tells the compiler to only use 1 bit to store this struct member.

now you can define a variable to hold the flags, and work with those flags:

flags myflags = {1,0,0,1}; // defines a variable holding a set of flags, with an initial value of takes_damage & some_other

myflags
.liquid = 1; // change the flags to include the liquid

if ( myflags.takes_damage ) // test for one flag
    apply_damage
();
if ( myflags.liquid && myflags.some_other ) // test for multiple flags
    show_strange_behavior
();

this method allows you to define any number of flags, without limitation, and you can extend your flag set at any time without fearing an overflow. the drawback is that testing a subset of the flags is more cumbersome and necessitate more code.

原本是在看 stackoverflow 上面對 enum 的常用法,無意中看到其中一個回覆,用「:1」可以告訴 compiler 只用 1 bit 去存…,這樣就不會被限制在一個 enum flag var 只用有 32 種 flags。

How can I hide the declaration of a struct in C? - Stack Overflow

In the header file:

typedef struct _point * Point;

After the compiler sees this it knows:

  • there is s struct called _point
  • there is a pointer type Point that can refer to a _point

The compiler does not know:

  • what the _point struct looks like
  • what members it contains
  • how big it is

And not only does the compiler not know it - we as programmers don't know it either. This means we can't write code that depends on those properties of _point, which means that our code may be more portable.

對於 C opaque pointer 的解譯

Strategy pattern - Wikipedia, the free encyclopedia

C

A struct in C can be used to define a class, and the strategy can be set using a function pointer. The following mirrors the Python example, and uses C99 features:

#include <stdio.h>
 
void print_sum(int n, int *array) {
  int total = 0;
  for (int i=0; i<n; i++) total += array[i];
  printf("%d", total);
}
 
void print_array(int n, int *array) {
  for (int i=0; i<n; i++) printf("%d ", array[i]);
}
 
int main(void) {
  typedef struct {
    void (*submit_func)(int n, int *array);
    char *label;
  } Button;
 
  // Create two instances with different strategies
  Button button1 = {print_sum, "Add 'em"};
  Button button2 = {print_array, "List 'em"};
 
  int n = 10, numbers[n];
  for (int i=0; i<n; i++) numbers[i] = i;
 
  button1.submit_func(n, numbers);
  button2.submit_func(n, numbers);
 
  return 0;
}