Custom HTTPS POST queries in Ruby

Complementing both the manual for the Net::HTTP class and Rail’s guide to rendering proxied pages (from the Rails wiki) here is a short script, which makes use of a custom made HTTPS POST query (btw. HTTP POST is basically the same, you just have to remove the ssl declaration and change the destination port).

First, the script makes a GET query to the target host (in the example, it is the login page for Wirtualna Polska webmail). In the next step it reads which cookies the website wants to set. After that, we can make our POST query. We send both filled in POST parameters (and hidden inputs too) and all of the cookies, which would normally be set in the browser. After that the script just prints out response headers and data (if there’s any).

The solution is simple, short and clean, although it took me few hours to find out that POST parameters must be a &-separated string and that the headers is a hash array. After that, it was a piece of cake.

Anyway, here’s the script:

#!/usr/bin/ruby

require 'net/http'
require 'net/https'

http = Net::HTTP.new('profil.wp.pl', 443)
http.use_ssl = true
path = '/login.html'

# GET request -> so the host can set his cookies
resp, data = http.get(path, nil)
cookie = resp.response['set-cookie']

# POST request -> logging in
data = ’serwis=wp.pl&url=profil.html&tryLogin=1&countTest=1&logowaniessl=1&login_username=blah&login_password=blah’
headers = {
  ‘Cookie’ => cookie,
  ‘Referer’ => ‘http://profil.wp.pl/login.html’,
  ‘Content-Type’ => ‘application/x-www-form-urlencoded’
}

resp, data = http.post(path, data, headers)

# Output on the screen -> we should get either a 302 redirect (after a successful login) or an error page
puts ‘Code = ‘ + resp.code
puts ‘Message = ‘ + resp.message
resp.each {|key, val| puts key + ‘ = ‘ + val}
puts data

7 Responses to “Custom HTTPS POST queries in Ruby”

  1. Greg Says:

    Hey, thanks for this, it’s really useful! Was wondering if you knew of any conveneint way to follow redirects?

  2. Paul Goscicki Says:

    It sould be fairly easy to combine my code with the redirect implementation from ruby-doc (scroll down one screen).

  3. Greg Says:

    Was playing with that for awhile and then I just discovered WWW::Mechanize. Have you seen this thing? It’s totally amazing. It was so easy, I took the liberty of rewriting your script using it like so:

    require ‘rubygems’
    require ‘mechanize’

    agent = WWW::Mechanize.new
    agent.user_agent_alias = ‘Mac Safari’

    page = agent.get(’http://profil.wp.pl/login.html’)

    form = page.forms[0]
    form.fields.find{|f| f.name == ‘login_username’}.value = “blah”
    form.fields.find{|f| f.name == ‘login_password’}.value = “blah”

    page = agent.submit(form)
    puts page.body

    So easy! You can also turn on verbose logging by changing that first line to:
    agent = WWW::Mechanize.new {|a| a.log = Logger.new(STDERR) }. Plus it does link-clicking and all kinds of other fancy stuff. It’s the best thing since sliced-bread.

  4. Paul Goscicki Says:

    Sounds very interesting. I must try it out.

  5. links for 2006-10-27 « Bloggitation Says:

    [...] Custom HTTPS POST queries in Ruby (tags: ruby programming) [...]

  6. Avishai Says:

    Wow, this turns out to be _extremely_ helpful!!! I WWW::Mechanize takes care of all the dirty work for you. I was able to use it to construct a pseudo-API to a site that doesnt have one

  7. pradeepta Says:

    Hi, I am trying to post form data from a jsp page which is using http method GET . How can I use it and get response .

Leave a Reply