Fake referrer from YouTube video

It seems spammers are trying everything they can think to get more expose, not just simply using a website. They are using media as well.

Fake referrer from YouTube video

The screenshot above shows a YouTube video's referrers, on top of that, in that blue box, it's a screenshot from this blog's referrers. The YouTube video link links to that video, which is about casino. From YouTube stats, you can see there were 27 (4+23) visitors from blogger.com, I believe they are all targeted Blogger bloggers. When I first saw that referrer in Stats tab of dashboard, 99.999% I was sure it's faked since no way anyone would link my blog in their videos' descriptions or comments. But I still clicked on it to check it out. You can also see it has 303 likes and I bet they are all faked, too, but those 5 dislikes, I am certain that they are real ratings.

I was thinking to flag that video, but I couldn't. There is no suitable option to file an complaint, it makes sense. Someone might use this to attack normal video, in other words, we have to take this kind of spamming. Too bad, there is no Spam folder as in Gmail.

I have to quote this:

"Spam will be a thing of the past in two years' time." Bill Gates.

I really wish he was right. That was in 2004. Six years later, they are still around. Currently, if you don't see, they are just being hidden by algorithm. They are detected and put into spam folder. That's sad, because what needs to be done is to find who sends not to write some sophisticated program to detect, those companies fight back in wrong direction. They keep saying how many they have done. The truth is they don't, they just cover our eyes.

Collecting evidence and help to charge those spams. Say one spam mail for just one minute community service, that would definitely be enough.

Happy 11111011011!

I know it's pretty late to post this after four posts this year. I had not wanted to write one, same reason as I posted much less last month. (In fact, I am posting this because I am trying to make myself post one each day.)

In the beginning of last month, I started to think the whole thing about blogging and I slow down. I wasn't sure I would resume when this new year started and I still don't know if I will continue. Two years ago, I posted about my definition of a blogger in an old blog, it's clear to me, I am still not one that I want to be. I never have a great vision for my blog, never want it to be some top 1 in whatsoever ranking blah blah list. But I do want it (the content) to reach a certain good quality. Sadly, they are still far from that point. To put it in simple words, a nice personal blog is all I want to achieve.

Many of my posts are very short and lack of rich descriptive explanation. For instance, I rarely explain my source code. I simply paste the code, then end the post. I usually think the code is simple enough, everyone probably knows. Even they don't, but who really reads? So I don't explain. When I read other blogger's posts, programming blogs or other topics. I often find myself envy their writing skill, they can write a 500-word post for one simple thing as if they are writing the first page of The Lord of the Rings, but I could probably use one line to end the story. Two if I try harder.

I don't like to write stuff I don't use when it's about computer topics, I wrote things I would use as often as by daily-basis. Unfortunately, with simplicity and easy satisfaction (it's another way to describe superb laziness), I hardly have many things to talk about or if I do have something, they usually just have few lines to be finished off.

However I do have a few informative posts and they do get some valid referrers from forums or other people's blogs, but that's only a handful. I want to write something like "The first course of Bash scripting." Yes, I want to write one like that in a blog post or break up into few small pieces if I have to. This leads to blogging technique and consistence problems I also wonder, I am that kind of person who says yes one minute ago, then says no.

When writing a blog post, it's not easy to tag (label), you need to choice wisely. Also, referring to old posts is another hard task. I have been reading some blogs, those blog writers can link to their own posts flawlessly and seamlessly. By my observation, they don't write a new post just to link to old posts, they write follow-ups intentionally or not.

The only thing I can say positive is the layout and style of this blog. I spent quite some time to adjust and make it my way, I wrote my own HTML5 template (though it can't pass validator check, but it's not my fault!) and some tools which helped me to build up CSS/JS/font resources on Google App Engine.


Time to change the subject, the blogging obstacle isn't the only reason I slow down. I was caught up by Stargate, SG-1 and Atlantis, 15 seasons. So far, I have watched Star Trek (except TOS) and Stargate, I might watch Babylon 5 some day, then I will earn the Trinity of Scifi Space. Once again, I want to explore the final frontier!

So what else I shall share about last year? Right, many bloggers would list some statistics of last year in this kind of post, but I think I will just tell you some things I would like to share.

First thing, I have quit Twitter and Instant Messaging more than six months, I know I don't need them and I am right. Though, I did want to tweet some meaningless when I just quit, but I haven't for quite a few months. Twitter isn't a necessity anymore.

Also I became a lactarian (lacto vegetarian), who eats likes vegetarian but also eats dairy product (say Cheese!) and still no eggs. I decided to go for this instead of a true vegetarian because I believe dairy exists to make lives thrive. If so, there is no reason not to consume them. But eggs are different story to me, they could be supposed to bear new lives, although even in natural not all are fertilized, but I still don't want to eat them. I know you won't buy a fertilized egg from supermarket because they don't put boys and girls together, however still no to eggs. By the way, some do eat fertilized eggs.

I wanted to convert when I watched a tv show, I believe it's on Discovery channel. It's about farming tuna in the ocean. I watched them and suddenly I realized that they would be on some family's or restaurant's dining table. There were thousands, and that's how we have them for our fresh served seafood and delicious dinner.

Recently, I discovered a great tv show, Jimmy's Food Factory on BBC. The host shows viewers how supermarket food are produced in food factory. There are a lot of interesting facts when manufacturing mass food products I didn't know before. Some episodes are about raising turkeys for Thanksgiving eighteen months before, yum lambs (how could you stand with that when little lamb baa baa at you, and you make a roast lamb dish for your few years old kids?). You will see a lot of new innocent (they have no a clue what they would be) lives being bred and raised, then be sent to supermarket after they are put to death in a such humanity way. And for live seafood when you buy and cook at home, I feel terrible even more.

Anyway, becoming lactarian did make me know more about food and also made me reading the package. There is a very funny thing when I started to be one. I thought 'How am I going to live without butter?' I didn't know it's made of milk, I thought that's body fat.

I started to notice how many food products contain eggs or animal-by-product for texture or flavor. There are always some extracts or essences listed, then you have to put that back. As for eggs, I have given up toasts and some kinds of breads. I even tried to make eggless onion rings, many batters contain eggs. One thing out of my expectation, do you know marshmallow is one vegans have to avoid? I learned it from the show.

Also, it's really painful when your breakfast plate doesn't have nice crunchy bacon and creamy scrambled egg. Try without them only for one day, you will see how hard it will be.

I'm not trying to provoke something between vegan and meat-eater, I don't do this for health or religious reason, it's my personal choice and I don't really have problem with people who have different diet. In fact, I have problem with parents who allow or make kids vegan for whatever the parents think or believe, they shouldn't.


Now back to this blog, I might be writing some more. I am thinking some cheatsheets (because I can't remember what keys to press sometimes) or short guides or specific topic of programming. Anyway, keeping writing is the way to start.

Lastly, I want to share a nice scene from Losers, don't stop believing you can make things happened but you have to fire up for them first!

Plotting posting and commenting activities from Blogger exported XML using Python

I wanted to plot posting activity by month in bar chart. I was thinking to use Blogger API, but I didn't want to write JavaScript (prefer JS if using API) or request multiple times if there are more than 500 posts. So, I decided to do it using exported XML file.

Here is the current stat of this blog:

$ ./b.py blog-01-11-2011.xml 
2008-09  18 ###################
2008-10  25 ##########################
2008-11  57 ############################################################
2008-12  51 ######################################################
2009-01  32 ##################################
2009-02  13 #############
2009-03  27 ############################
2009-04  41 ###########################################
2009-05  14 ##############
2009-06   1 #
2009-07   1 #
2009-08   1 #
2009-09   1 #
2009-10  27 ############################
2009-11  18 ###################
2009-12  16 #################
2010-01  16 #################
2010-02   3 ###
2010-03   2 ##
2010-04   9 #########
2010-05  19 ####################
2010-06   2 ##
2010-07   1 #
2010-08  53 ########################################################
2010-09  62 ##################################################################
2010-10   8 ########
2010-11  51 ######################################################
2010-12  10 ##########
2011-01   3 ###

Total:  582 posts

$ ./b.py blog-01-11-2011.xml comment
2008-09   1 #
2008-10   6 ######
2008-11   6 ######
2008-12  38 ######################################
2009-01  23 #######################
2009-02   8 ########
2009-03  16 ################
2009-04  10 ##########
2009-05   3 ###
2009-06   3 ###
2009-07   1 #
2009-08   2 ##
2009-09   0 
2009-10   6 ######
2009-11   9 #########
2009-12   8 ########
2010-01   3 ###
2010-02   4 ####
2010-03   0 
2010-04   1 #
2010-05   0 
2010-06   0 
2010-07   0 
2010-08   0 
2010-09  66 ##################################################################
2010-10   4 ####
2010-11  12 ############
2010-12   9 #########

Total:  239 comments

The Python code:

#!/usr/bin/env python
# ./script.py export.xml [<post|comment> [$(tput cols)]]

from pyquery import PyQuery as pq
import sys

kind = 'post' if len(sys.argv) < 3 else sys.argv[2]
width = (78 if len(sys.argv) < 4 else int(sys.argv[3])) - 12

with open(sys.argv[1]) as f:
  b = pq(f.read().replace('feed xmlns=', 'feed xmlblahblahblah='), parser='xml')

entries = b('entry')
m_count = {}
for idx in range(len(entries)):
  if entries.eq(idx)('category[scheme$=kind]').eq(0).attr('term').split('#')[1] != kind:
    continue

  month = entries.eq(idx)('published').text()[:7]
  if month not in m_count:
    m_count[month] = 0
  m_count[month] += 1

m_count_keys = m_count.keys()
m_min = min(m_count_keys).split('-')
m_max = max(m_count_keys).split('-')
min_year, min_month = int(m_min[0]), int(m_min[1])
max_year, max_month = int(m_max[0]), int(m_max[1])
del m_count_keys, m_min, m_max

m_count_values = m_count.values()
max_count, min_count = max(m_count_values), min(m_count_values)
total_count = sum(m_count_values)
del m_count_values

for year in range(min_year, max_year+1):
  for month in range(1, 12+1):
    if year == min_year and month < min_month:
      continue
    if year == max_year and month > max_month:
      break
    key = '%d-%02d' % (year, month)
    count = m_count[key] if key in m_count else 0
    print '%s %3d %s' % (key, count, '#'*(width*count/max_count))
print
print 'Total: %4d %ss' % (total_count, kind)

It requires pyquery.

It can be done without pyquery, but it would be real pain in ass for me. I ran into with xmlns problem, there is a related bug report.

The script must be supplied with the XML file and you can specify second argument of what kind of entry you want to plot, which should be post or comment only. The third argument will be the text width. The default value is 78, you can use $(tput cols) or $COLUMNS to get current terminal text width if available.

I decided to use textual chart, it's good enough for me to see the results.

Really simple MPD client

I have a Bash script which connects to MPD server and display current song and also accepts some keys to control the player. I have stripped some functions from it a few times when I needed to write a Bash script to do something with MPD, e.g. showing Last.fm playcount in Conky. A few times, I wanted to know what metadata of MPD command currentsong provides, I quick hacked the script and made it printed out the returned data.

I think it's time to write a simple client to accept command and print out the response. Stripping that Bash script was my first thought, then I decided to write a Python script, instead. It's not because I prefer Python or something, just I hadn't written a single line Python code for a long time.

Before I showed you that Python code, there is one more thing I just found out. A really really simple client for MPD, which is telnet tool.

$ telnet 0 6600
Trying 0.0.0.0...
Connected to 0.
Escape character is '^]'.
OK MPD 0.15.0
currentsong
file: A/Alexandria Maillot/Just Another Girl/Just Another Girl.mp3
Time: 224
Artist: Alexandria Maillot
AlbumArtist: Alexandria Maillot
Title: Just Another Girl
Album: Just Another Girl
Track: 1/4
Genre: Pop
Composer: Alexandria Maillot and Joby Baker
Pos: 1
Id: 159
OK
stats
artists: 56
albums: 56
songs: 392
uptime: 124
playtime: 54
db_playtime: 84859
db_update: 1291807454
OK
pasus
ACK [5@0] {} unknown command "pasus"
pause
OK
play
OK
close
Connection closed by foreign host.

You don't really need mpc if you are a script nut with nc (netcat) or expect. If you don't need to process the response, you can always give commands to MPD, e.g.

echo "next" | telnet 0 6600

This will make MPD to play next song.

Now the Python code:

#!/usr/bin/env python

try:
  import readline
except ImportError:
  pass
import socket
import sys


HOST = 'localhost'
PORT = 6600
RECV_SIZE = 2**20 # One megabyte
try:
  s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
  print 'Connecting to %s:%d...' % (HOST, PORT)
  s.connect((HOST, PORT))
except socket.error:
  print 'Unable to connect.'
  sys.exit(1)

print s.recv(RECV_SIZE)
print '*** <Control+C> to exit ***\n'

while True:
  try:
    cmd = raw_input(">>> ")
    if cmd:
      s.send(cmd + '\n')
      if 'close' in cmd:
        break
      print s.recv(RECV_SIZE)
  except KeyboardInterrupt:
    break

print
s.close()

A sample interaction:

$ ./smpdc.py 
Connecting to localhost:6600...
OK MPD 0.15.0

*** <Control+C> to exit ***

>>> currentsong
file: A/Alexandria Maillot/Just Another Girl/Just Another Girl.mp3
Time: 224
Artist: Alexandria Maillot
AlbumArtist: Alexandria Maillot
Title: Just Another Girl
Album: Just Another Girl
Track: 1/4
Genre: Pop
Composer: Alexandria Maillot and Joby Baker
Pos: 1
Id: 159
OK

>>> playlist
0:A/A Lion Named Roar/Shoot It Straight.mp3
1:A/Alexandria Maillot/Just Another Girl/Just Another Girl.mp3
2:A/Ana Free/Keep On Walking.mp3
3:A/Ana Free/Questions In My Mind.mp3
4:S/Stars/Stars_Dead_Hearts.mp3
OK

>>> next
OK

>>> currentsong
file: A/Ana Free/Questions In My Mind.mp3
Time: 231
Artist: Ana Free
AlbumArtist: Ana Free
Title: Questions In My Mind
Album: Radian
Track: 1/5
Date: 2010
Genre: Pop Rock
Composer: Ana Gomes Ferreira and Blake Harris Brandes
Pos: 3
Id: 161
OK

>>> close

Bastard Tetris

I noticed this Bastard Tetris (bastet) when I re-synced Portage tree, it's a ncurses-based game, a very special one. I am not a fan of Tetris games. No, I didn't spend hours and hours sitting in front of game console or ancient computer trying to make one more line after another.

At first, when I read Portage tree description: "a simple, evil, ncurses-based Tetris(R) clone." I had no intention to try it out even I was curious about the word evil in the description. I did put it into my to-try list, but it's because of it's ncurses-based. However, when I was checking out its website, this got my interest when I read its description:

Have you ever thought that Tetris is evil because it never sends you that straight "I" brick you need to clear four rows? Well, Tetris(R) probably is not so malevolent, but Bastet certainly is. >:-> Bastet stands for "bastard tetris", and is a simple ncurses-based Tetris(R) clone for Linux. Instead of choosing the next block randomly, this fiendish program uses a special algorithm to give you the worst possible brick. Playing Bastet can be a very frustrating experience!

The last part woke up my inner cat. I got to try it! So I emerged it and this was from the first minute playing:

Bastard Tetris

I started to understand why it's evil... A few moments later:

Bastard Tetris

I am 100% sure, it's not only evil and a bastard but also definitely comes from hell. I stopped to pile up the squares, then I got other types of brick, but in the end:

Bastard Tetris

I only cleaned up one line. Believe me, I didn't try to end the game quick. It just didn't give me the bricks I want all the time.

What a !@#$%^&* game!

Using GNU gcal in dzen

I just discovered this much more powerful GNU gcal program. It has a lot of features that cal doesn't have. My base command to list calendar is

$ cal -3s
    December 2010         January 2011          February 2011   
Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa
          1  2  3  4                     1         1  2  3  4  5
 5  6  7  8  9 10 11   2  3  4  5  6  7  8   6  7  8  9 10 11 12
12 13 14 15 16 17 18   9 10 11 12 13 14 15  13 14 15 16 17 18 19
19 20 21 22 23 24 25  16 17 18 19 20 21 22  20 21 22 23 24 25 26
26 27 28 29 30 31     23 24 25 26 27 28 29  27 28               
                      30 31

The command for the similar result using gcal is

$ gcal .


                                2010/2011


      December                  January                   February
 Su Mo Tu We Th Fr Sa      Su Mo Tu We Th Fr Sa      Su Mo Tu We Th Fr Sa
           1  2  3  4                         1             1  2  3  4  5
  5  6  7  8  9 10 11       2  3  4  5  6  7  8       6  7  8  9 10 11 12
 12 13 14 15 16 17 18       9 10 11 12 13 14 15      13 14 15 16 17 18 19
 19 20 21 22 23 24 25      16 17 18 19 20 21 22      20 21 22 23 24 25 26
 26 27 28 29 30 31         23 24 25 26 27 28 29      27 28               
                           30 31

My old dzen script showed 3-month calendar in this way:

2010-12-07--01:31:24

Now it's much better:

dzen with GNU gcal

And the code is sort of simpler, you can see the diff. The basic command is

gcal --cc-holiday=TW+US_NY --holiday-list=short --highlighting=\<:\>:[:] .

First arguments lists what country holidays I need, I use two, Taiwan and New York State of US, you can use plus sign + to list more than one. I also change the highlighting characters, so I can do some dzen syntax later. I tried to give it a string instead of single character, but gcal doesn't accept that. So I used <>[] which don't seem to be used by gcal itself, then used sed to replace with the strings I really need. You can use same method for ANSI color escape code if you want. Note that the last period . is for 3-month format. There are more information in its info page, info gcal, or you can run gcal -hh -p.

I think the most amazing part of gcal is you can have a holiday list. I think you can also import your schedule from some source with a bit of coding help. But I don't have such need, therefore I didn't dig in.