Wednesday, May 26, 2010

Download all the mp3 links in a page given the url with ruby and nokogiri

You'll need ruby and the gem nokogiri installed. Then run this


#!/usr/bin/env ruby

require 'nokogiri'
require 'open-uri'

if ARGV.size == 0
  puts "usage: #{File.basename(__FILE__)} 'url' ..."
  exit
end

ARGV.each do |url|
  doc = open(url) {|io| Nokogiri.HTML(io.read) }

  mp3_links = doc.xpath("//a").select {|link| link['href'] =~ /\.mp3$/ }
  mp3_links.each do |link|
    href = link['href']
    outname = File.basename(href)
    puts "Downloading: #{outname}"
    open(href) do |io|
      File.open(outname,'w') {|out| out.print(io.read) }
    end
  end
end

Wednesday, May 12, 2010

A debian/ubuntu gem environment of your very own

There are some differences with the way apt-get rubygems and downloaded and 'sudo ruby setup.rb' gems behave. Here is how to bring the directory structure in conformity with the apt-get version and have your gem's bin folder in your path. This is for using gems on a per-user basis.

Add these two lines to your ~/.bashrc file:
# this will export the path to whatever version of ruby you are using:
export GEM_HOME="$HOME/.gem/ruby/`ruby -e 'x=RUBY_VERSION;print(x=~/^1.8/ ? "1.8" : x)'`"
export PATH="$PATH:$GEM_HOME/bin"

Then 'source ~/.bashrc' if needed and you can use either version of rubygems indistinguishably.

Now, you can install gems with either rubygems, you don't need sudo privileges, and your gem executables will be active. Wallah!

gem install rake

Friday, May 7, 2010

Yet another reason to love Ubuntu, multiple folder extraction

Today I needed to extract 20+ folders and I thought, I don't want to right-click over 20 times, wouldn't it be cool if I could just select all the folders, right-click and extract? Well, it worked like a charm and made my day. Now if only Ubuntu could solve the problem of all of my folders being over 2.5 Gb and taking forever to extract.

Tuesday, May 4, 2010

Your own legal ssh personal server on comcast with Ubuntu 10.04

Legal Matters

It appears to me that a personal ssh server is legal and acceptable to use [at least if you are with Comcast].
 
The Comcast Terms of Service:

The relevant bullets under Technical restrictions are:
* use or run dedicated, stand-alone equipment or servers from the Premises that provide network content or any other services to anyone outside of your Premises local area network (“Premises LAN”), also commonly referred to as public services or servers. Examples of prohibited equipment and servers include, but are not limited to, e-mail, Web hosting, file sharing, and proxy services and servers;
* use or run programs from the Premises that provide network content or any other services to anyone outside of your Premises LAN, except for personal and non-commercial residential use;  (my emphasis)
Also, note that the ssh port (22) is not blocked by comcast.

How to setup a personal ssh server

This should work in many different distros and versions with only minor modification.

  1. sudo aptitude install openssh-server
  2. Since I have kids with weak passwords using my computer, I only want a couple accounts to be accessible. Edit /etc/ssh/sshd_config and add something like this:
    AllowUsers user1 user2 user3
    
    This will prevent other user accounts from being accessible.
  3. It is generally a good idea to use a static IP address so your router knows where to send the ssh traffic.  Right click your wireless icon on the panel and edit your connection.  Shown to the right is a setup that is compatible with a linksys router (i.e., the router IP address is 192.168.1.1).
  4. Forward port 22 traffic to your statically assigned PC (see image below for router specs).
  5. Use a service like dyndns.com to associate a static IP address with your dynamically assigned address.
  6. Run ddclient to update dyndns.com.
    sudo aptitude install ddclient
    Here is a configuration file (/etc/ddclient.conf) that works for me:
    daemon=600
    use=web, web=checkip.dyndns.com/, web-skip='IP Address'
    login=
    password=protocol=dyndns2
    server=members.dyndns.org
    wildcard=YES
    jtprince.dyndns.org, bwv549.homeip.net
     

Now, you should be able to ssh into your home computer from anywhere in the world. Also, please note that you can do just about anything with ssh access.