As of 2016-02-26, there will be no more posts for this blog. s/blog/pba/
Showing posts with label Portage. Show all posts

There is a thread about quitting Gentoo (not the only one thread), one point was mentioned is about the long compiling/merging time. I wonder how much exactly the time do we spend. All those are telling about what they feel, not really an accurate number is given.

Here is a quick one-liner I wrote it in a hurry:
echo $((86400 * $(total=0 ; sudo genlop -nt '*' | sed -n '/merge time/ {s/ hours\?/*3600/;s/ minutes\?/*60/;s/[^0-9*]\+/ /g;s/^ //;s/ $//;s/ /+/g;p}' | while read s; do let total+=$s; echo $total; done | tail -1) / $(($(date +%s) - $(sudo head -1 /var/log/emerge.log | cut -d: -f1)))))
On my Gentoo, it is 812 seconds a day, that's 13 minutes and 32 seconds.

I don't think that's too much, but the output of genlop doesn't include failed merges, though that doesn't happen often. Also, I don't have many unstable package installed and I don't use KDE or GNOME, or anything really a huge package. The biggest one and getting updated regularly on my Gentoo probably is Firefox, I don't have Chromium anymore.

My processor is a five-year-old Core 2 Duo 1.833MHz and I often don't merge with 2 cores, so I can still work on something, therefore the merge time can reduce up to 50%. The real merge time with full power of my computer can be between 406 and 812 seconds.

I don't know why people need to leave computer to merge overnight, use genlop to estimate and manually select one or two package with dependencies to merge one at a time. In background or during your lunch break.

I really think using long merging time is not so convincing reason if I was going to leave Gentoo. Also I don't understand why people have troubled with merging. I never got my Gentoo unbootable or under some strange situations.

I don't read anything tips I mentioned above from anywhere, I learned by exploring on my own. Playing with options, searching for packages, etc. If you can post on forums, ask how to use Portage efficient before you quit.

Don't be a quitter like that.

I am doing my weekly system update, GCC is being pulled in by revdep-rebuild for dev-libs/gmp. I forgot to supply the quiet flag and I have no idea how many are pulled. Didn't pay much attention but I do know two slots of GCC I have both are pulled, and I know GCC takes time to compile. But I really need to know when the emerge began.

So I use qlop and genlop because I still remember they both have such option, just didn't have chance to utilize them.


genlop gives a nice ETA as it can already give you an estimation of emerge time if you pipe the emerge's output to it.

10 more minutes to go... for 4.4.5, still have 4.3.x to be emerged. That would probably be another hour.

I recently found out I could use qlop -gH package or qlop -tH package to get merge time:

http://farm2.static.flickr.com/1010/5190755414_3fef86bebf.jpg

But it doesnt have an option to list packages merged in a session. So I wrote one to parse /var/log/emerge.log on its own, last-merge-time.sh:

http://farm2.static.flickr.com/1039/5190755456_e4530fd6a5.jpg

I tried to mimic the result format.

The merge time calculation is different than qlop, you might see difference in a few seconds. The script uses sed to filter unwanted merge log and keeps the last merge, then uses awk to format the output. You probably noticed that interrupted in the screenshot1 above, its a result of user interruption (pressing Ctrl+C) while merging. The timestamp is the start time, not the end time of merging as shown by qlop, I am just too lazy to change my code.

Of course, there is also the last sync time.

[1]The screenshot shows the result by feeding the script with hand-modified raw log.

Since I started to use eix-sync for Portage tree synchronization. I always found annoyed when eix-sync runs with emerge --sync, I used to run emerge --sync -q. I really dont need to see a long list of directories.

At first, I didnt find the solution and I still have no idea why I missed these from eix-sync -h:

-c CMD Run CMD instead of emerge --sync.
-C OPT Add OPT to the emerge --sync command (or whatever is used instead). This option can be used accumulatively.

So, the solution is clear:

eix-sync -C -q

If you need more options to be appended, run with -C '--foo --bar'. If you need to run emerge without --sync, use -c '--whatever'.

If you are lazy like me, add the following line to ~/etc/eixrc:

EIX_SYNC_OPTS="-C -q"

-C -q will be appended when you run eix-sync.

I am currently synchronize Portage tree with weekly basis. I used to show the timestamp (/usr/portage/metadata/timestamp.chk) directly in Conky and synchronize when its been more than 24 hours. I want to have a easy way to know if I should update. So, I wrote:

${color lightblue}Portage:$color ${execpi 600 thres=$((7*24*60*60));ts="$(cat /usr/portage/metadata/timestamp.chk)";dur=$(($(date +%s) - $(date -d "$ts" +%s)));((dur>=thres)) && echo $(date -d "$ts" +%F\ %T) || echo "\${execbar echo $((100*dur/thres))}"}

The command is:

thres=$((7*24*60*60))
ts="$(cat /usr/portage/metadata/timestamp.chk)"
dur=$(($(date +%s) - $(date -d "$ts" +%s)))
((dur>=thres))
  && echo $(date -d "$ts" +%F\ %T)
  || echo "\${execbar echo $((100*dur/thres))}"

It calculates the seconds since last update. If its been seven days, then it shows the time of last update, otherwise a conky execbar for showing how far away from seven days.

Gentoo is a source-based distribution. The main benefit of using Gentoo is you pick features of a software you need to compile if its configurable. Here is an example I just saw:

% emerge -pvuDNt world

These are the packages that would be merged, in reverse order:

Calculating dependencies... done!
[ebuild   R   ] www-client/chromium-7.0.503.1  USE="sse2 -cups -gnome -gnome-keyring%" 0 kB

Total: 1 package (1 reinstall), Size of downloads: 0 kB

The developer just made GNOME Keyring dependency optional if you use gnome USE flag. That cups USE tag is an example, too. When Chromium just added printing support, they have made CUPS dependency necessary. It got fixed quite fast, in just a few days, Gentoo developer fixed it.

Note

I couldnt find an issue about it on Chromium issues, but it seems fixed in upstream because I dont see any patch in Portage tree.

Anyway, after CUPS, GNOME Keyring suddenly jumped out from nowhere. I already had GNOME Keyring on my system, but I didnt have CUPS. So, I let Chromium to compile with it, I didnt have a choice. Now, we have new USE flag.

If you are using any binary-based distribution, unless you try to build on your own or with some tools, your system is always with a bunch of stuff you dont need. And you never could have even a vague idea what have been made.

I still have some packages depends on GNOME Keyring, I should check them up again, some are optional, some might have no need to be on my system.

% equery d gnome-keyring
[ Searching for packages depending on gnome-keyring... ]
dev-vcs/subversion-1.6.12 (gnome-keyring? gnome-base/gnome-keyring)
gnome-base/gnome-mount-0.8-r1 (<gnome-base/gnome-keyring-2.29.4)
gnome-base/libgnome-keyring-2.30.1 (>=gnome-base/gnome-keyring-2.29)
net-libs/libsoup-gnome-2.28.2 (gnome-base/gnome-keyring)
sys-auth/pambase-20100310 (gnome-keyring? &gt;=gnome-base/gnome-keyring-2.20[pam])
www-client/chromium-7.0.503.1 (&gt;=gnome-base/gnome-keyring-2.28.2)
                              (gnome-keyring? &gt;=gnome-base/gnome-keyring-2.28.2)

I always run emerge -pvuD world to check updates, then sudo emerge -quD world to update my Gentoo. Sometimes, I would see packages showing up but I couldnt understand why they were pulled for update. Here is a case:

https://farm5.staticflickr.com/4138/4886199841_fdd3a34807_o.png

chromium package is no doubt that it needs to be updated since its in world (rendered in bold style). But for eventlog, you would not think its a dependency of Chromium; and for librsvg, its not one because itd be updated after Chromium.

So, why do they show up? After reading the man page, --tree (or -t) option would tell you the reason.

https://farm5.staticflickr.com/4117/4886199913_e2da2e382c_o.png

I can see clearly now.

(Also posted to Gentoo forum)


Seeing a visualized dependency graph of a package should be always interesting. I read a post on Arch Linux Forums, which generates very beautiful images of your Arch Linux. I was thinking to add a support for Gentoo, but after reading the code, I decided to write my own completely.

Actually, I only did a half of work. My code doesn't not render the image but using Graphviz to the job. It requires packages gentoolkit and graphviz. There are few usage examples written at top of the script.

Here are two quick examples: First one is the Dependency of portage

dep graph of package portage on Twitpic

Second one is big, whole world highlighting python.

You can read the code and download it.

The result should be reflecting installed packages and used USE flags on packages at the time of installing. I didn't take a look at the results very deep, so I am not sure if they are 100% correct since I don't know if I interpreted the data from gentoolkit correctly or not.

I think it's fun to see the visualized dependency data. The code needs Graphviz to render the image, and I think I didn't configure it well. So if you generate whole world, the result is hard to read.

Hope you would like it and give me some feedback.