YJL

tl;dr http://pipes.yahoo.com/pipeslivibetter/newrubygems

I have been monitoring Python Packages for quite some time, few days ago, I wanted to expand to the gems on RubyGems.org. Sadly, it doesnt even have any feeds for newly created packages.

The best option for me is to utilize the /api/v1/activity/latest of the Activity APIwhich has XML, JSON, and YAML formatswith the help from Yahoo! Pipes:

$ curl 'https://rubygems.org/api/v1/activity/latest.json'

You can grab the feed at http://pipes.yahoo.com/pipeslivibetter/newrubygems.

Thhttps://bitbucket.org/grandpas/e.bashe source looks like:

It only took me 10 minutes or less to figure out how to make such conversion. This was my first time to use Yahoo! Pipes to convert a JSON to an RSS feed.

Although it works well and is very simple, this isnt my ideal solution. I actually asked someone who knows about Ruby to add feed support and timestamp to RubyGems.orgs source code. Thats not going to happen by the person I asked, so if you know about Ruby and you have time, please think about adding such feed to it.

I am thinking to create a couple of issues, but not really sure if I should do that since I dont think I can even call myself a Ruby user.

Before that, well, this feed would do just fine.

The PyPI newest packages feed contains 40 entries, thats 40 new packages. As of writing, it spans back about 21 hours, 3 of 40 are those A simple printer of nested lists spams, for those I had even filed an issue to PyPI issue tracker, I was told:

Richard Jones: I regularly clean out these modules. Just try to ignore them.

As you can guess, I am really sick of those. Everyday, at least three are spams if not less. Wasting time to go through that feed, 7.5% of spam rate, thats not an insignificant number by any means. I just could not believe that I had waited for three months to do something.

So, here it is a cleaner feed¹ with the help of Yahoo! Pipes, once again:

PyPI Packages Filtered

You can see two filters, first one is for the spams, seconds are for other legit packages that I am not interested in. Off-topic: the quality of packages are usually low, some are not even ready, links send you to 404. I really dont know why those people bother wasting others time and theirs.

This action was actually prompted by someone who was asking about my Ultimate YJL feed, reminded me of the usefulness of Yahoo! Pipes, should have thought about using Yahoo! Pipes, this pipe only took me about 10 minutes to lay down after 3 months of wasting time on skipping through.

Only 26 packages got through, 14 are removed. 14 actions less to take every day. 14 keypress might not sound a lot but multiply by 365, its 5,110 presses a year. And if you are a mouse clicker, you are wasting even more.

What, mileage? Its a mouse not a flight!

I saw a posting on FriendFeed, the poster want a Super Feed which has contents of his FriendFeed Feed and Comments + Likes Feed. I think Yahoo Pipes can do this task easily, and it does. However, Pipes is extremely slow right now, it wasnt about three months ago. In other words, I havent used Pipes for three months, hope the issue is just temporary.

Here is the full view of this Pipe:

It uses three types of modules:

• Fetch Feed: On the top, they grabs your two feeds.
• Union: At middle, it combines all its inputs, which are your two feeds in this case.
• Sort: You need to sort the output of Union, because the output of Union is all items in first feed, then all items in second feed. That wasnt what we expected, it should be sorted by published date (time).

I think you can clone my pipe, and replace feed sources with yours (in Fetch Feed modules). If you cant, just drag-and-drop by following the figure above. You can also check out the RSS feed directly.