YJL

A few days ago, I ran df -h for no reasons as I usually did, the output as follows:

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs           72G   49G   20G  72% /
[...]

To my amaze or not to, the part of my brain which manages mathematics finally told me that 72G != 49G + 20G. I have been using Linux full-time for more than five years and probably have run df for a thousand times if not less. Finding the inconsistency is not really what confused me but why it took me so long to see the numbers.

I found the answer after consulted Google using columns of df, ie. "df size used avail". Yes, that's how I found the answer at my first try. I am still a master of Google.

The discrepancy comes from Reserved Blocks. From what I read, Extended File System (ext) since ext2, has such feature and the default reserved blocks are 5% of total blocks, marked by mke2fs.

-m reserved-blocks-percentage
  Specify the percentage of the filesystem blocks reserved for the
  super-user.  This avoids fragmentation, and allows root-owned daemons,
  such as syslogd(8), to continue to function correctly after non-privileged
  processes are prevented from writing to the filesystem.  The default
  percentage is 5%.

To see how many blocks are reserved, you can use tune2fs:

$ sudo tune2fs -l /dev/sda3 | grep -i block
Block count:              19037025
Reserved block count:     951851
Free blocks:              6019654
[...]
Block size:               4096
[...]

In my case, it's 19,037,025 * 0.05 = 951,851.25. Round to 951,851 blocks as you see in the output above. To do the math correctly, use 1K-blocks for calculations:

$ df | head -2
Filesystem     1K-blocks     Used Available Use% Mounted on
rootfs          74953252 50727968  20417880  72% /

The reserved size is 951,851 * 4 (Block size, 4096 = 4*1K-blocks) + 507,279,68 + 204,178,880 =  74,953,252. Mystery solved!

5% (3.6G) is really a lot, even for my tiny harddrive. Imagine a 1TB harddrive, that's 50G, almost as big as my harddrive. There are two ways to set the reserved amount of blocks using tune2fs, one by percentage, another by number of blocks.

For percentage using -m, the following code set to 1%, you can assign a float number.

$ sudo tune2fs -m 1 /dev/sda3
tune2fs 1.42 (29-Nov-2011)
Setting reserved blocks percentage to 1% (190370 blocks)

Or, the specific amount of block using -r, the following code set to around 500MB in 4K block size.

$ sudo tune2fs -r 129008 /dev/sda3
tune2fs 1.42 (29-Nov-2011)
Setting reserved blocks count to 129008

When creating an ext2/ext3/ext4 filesystem using mke2fs/mkfs.ext2/mkfs.ext3/mkfs.ext4, you can only use -m to specify a reversed percentage.

I came across Glances, a system monitoring program written in Python, as its name hints, it enables you to review system information, status, or resource usage in just a few glances. As you can see from the screen shot above, all information in one screen.

• CPU: Glances probably the only few programs which display the details of what CPU spends time on rather than just a simple percentage of utilization. How much time it's idling or waiting for IO? For example, iowait is an useful information, when disk is busy, more or less it could slow down the system for different reasons and by reading iowait, you know some heavy IO activity is going on.
• Load average: It's same as uptime gives, the load average of past 1, 5, and 15 minutes.
• Memory and Swap: Like CPU, more detail is shown, such as buffered, cached, active, or inactive memory. Similar to free command output.
• Network: Each interface's bandwidth usage, upload and download rates. You can press B to switch unit between bit/second and byte/second.
• Disk I/O: Read/Write rates on partitions and devices. Not only physical storage devices but also optical devices.
• Mount points: Similar information you get from df command, such as total size, used space, and available size.
• Processes: Like htop or top, you can use keys to choose sorting fields.

Glances shows information as much as they can be fitted in terminal window size. You can turn off sections by keys, separately, if you don't need them.

It also support server mode, a client can use XML-RPC call to get the system information in JSON format. It's possible to write your own client if you don't like default text-based client or you can write a GUI or web interface to display the data. It supports Linux, Mac OS, and Windows, although no default client for Windows, it will only runs as server due to no curses module available on Windows platform.

Glances is only one-year-old, first released on December 4, 2011, there are many possibilities for improvements. For example, configuration file for colors or disabling sections by default. Also custom fields for processes, and the list can go on and on.

I was intrigued by its name when I first heard of it and the idea behind this program is useful and simple. You don't need to run several programs to get all information of a system. You can have everything on one screen, even status of multiple systems at same time with client/server mode and terminal multiplexer.

I often need to extract an archive, sometimes, I just need to look into file content especially for source-tarball-only projects. Not necessarily mean I need to compile it, but there is also a benefit if you compile source in file system which resides in memory, such as tmpfs, it is fast.

My /tmp is tmpfs, not a file system on physical disk. Not only programs needs it, I rely on it, even it only has 1 GB size, it already helps a lot. I so wish my computer had like 16 GB memory, I could compile anything on it.

tmpfs is fast and by using it rather than normal file system do not reduce your hard disk life since it reduce the number of writes on physical disk. If you haven't taken advantage of it, you should from now on. Physical disk file system is really slow, comparing to tmpfs. I don't know how fast SSD could be, but I don't think it can beat tmpfs, it requires to go through disk interface before data reaches or is retrieved from memory of SSD disk.

It always takes me a while to figure out what is the correct option to tell archive program to extract to /tmp. So, here is a short list which may help me:

bunzip2 -c foobar.bz2 > /tmp/foobar
gunzip -c foobar.gz > /tmp/foobar
tar xf whatever.tar.supports.tar.ext -C /tmp
unrar x foobar.rar /tmp
unzip foobar.zip -d /tmp

You may ask, didn't you download archive to the directory already? If you has archive with packing convention, then you know there will be a new directory after you unpack it. But that's not always the case, especially when you have Zip packs. If you are not careful, you will end up having like a hundred of files in your current directory mixing with other files, then you swear about it if you just can't remove all files and redo.

So, I always create a new directory whenever I have suspected such thing might happen. I don't want to move the archive, so this extracting directory is needed. Of course, I can move working directory to the new directory, then use relative path to the archive, but why miss a chance to learn something you don't know?

However, tmpfs isn't all that good for all scenarios. Computer crashes and content in tmpfs isn't persistent. Once power is gone, tmpfs is phewwoof, too. One time, I lost a few pages of writing in LibreOffice, there was no recovery process. Heck, LibreOffice didn't even know about the crash, the temporary files were gone with the crash.

After that experience, I changed LibreOffice's temporary directory to a newly created directory which resides in physical disk. So, I wouldn't yell "Frak!"

I dont know if this ever happens to you, but I believe ever Linux users have. You are in a hurry to get computer booted and logged in, so you can check up an important message. Unfortunately, a partition has reach the maximal mount count and a forced checked is running. The check really takes time and you just sit there and watch that progress bar from 0% to finally 100%. Even you are not in a hurry, you still feel its like ages.

Not sure if there is a tray program, even there is, I wouldnt use. I wrote a short script for parsing the output of tune2fs for the left mounts.

#!/bin/bash

for part in /dev/sd??; do
  echo $part: $(($(sudo tune2fs -l $part 2>/dev/null | sed -n '/^Mount/ {x;n;G;y/\n/-/;s/[^0-9-]\+//g
p;q}')))
done > /etc/mountcounts

If you use Gentoo with baselayout2 (everyone should be using v2 already), then you can save it as /etc/local.d/mountcounts.start and grant it an executable permission. Or you can save it as or into /etc/rc.local if your distro uses it. Note that tune2fs requires root user to get the info of a partition.

Every time, you boot, a file /etc/mountcounts will be written with the left mounts, e.g.:

/dev/sda1: 23
/dev/sda2: 0
/dev/sda3: 5

I am lazy, so I just let the script to check all partitions and mute error message when checking swap partition. You can manually list all partitions which you really want to check.

When the count reaches 0, it means a forced check will be launched in next boot and you should be ready for that, i.e. press the power button before you go get your coffee.

I dont add any notifications, but you can write a simple startup script for your DE and check if the number reaches zero, then pop up a dialog. For me, I just include the file in my dzen-status.

I got the warning message (not error) when I tried to add a file with colon (:) in its filename to a Hg repo. The repo is basically private, I don't intent to publish it to public, but I think it's still worth to resolve the issue even I don't use Windows.

There are some restrictions on file naming on Windows/DOS file systems. The message was added by a patch and there was a bug report related to this issue. I don't know what will happen if you try to clone such repo, but I am guessing files with illegal names will not be updated into working directory.

From that bug report, there is no remapping to work around, so it is best not to use anything may cause issues on different file systems. By default, it's only a warning, you can still commit the addition action.

If you don't want to see such warning, you can update your hgrc by adding:

[ui]
portablefilenames = ignore

which will suppress the warning message. If you want to stop the actions may cause issues, then gives it abort. You can read more about this configuration in man 5 hgrc.