You know you need to use Mechanize for your new Ruby project, but you’re wasting time and effort scouring the web for tutorials and examples, most of which are broken.
Documentation is hard to read and a little bit sucky. XPath isn’t giving you the results you want. ResponseCodeError exceptions, timeout errors, and blank pages you didn’t expect.
You’re spending so much time figuring out the code, it’s taking longer than expected to get everything working. It’s all so confusing and frustrating.
Surely there must be a better way? If only you could stop wasting time on Stack Overflow and figure out the Mechanize code you could get your project working!
What if… you knew what the different methods do and how to use them? What if… you could focus on writing code and getting results?
What if… you could get your new web scraping project finished and spend more time developing your skills, and working on your side projects? Imagine how that would help you advance your career. Wasting time searching the web for working code samples definitely isn’t going to advance your career!
If you can write a scraper which performs against any real world data, your new rate is $100 per hour. — Patrick McKenzie (patio11)
Web scraping is one of those practical development skills that has a variety of real-world applications. If a business (read: your employer or your client) can use it to save time or generate revenue then that makes it an in-demand skill worth knowing about.
Introducing the Ruby Mechanize Handbook, an easy to follow introduction to scraping websites with Ruby Mechanize.
What can you expect? With the Ruby Mechanize Handbook you’ll learn…
- How to crawl web pages and deal with some common navigation errors
- How to follow links and handle international domain names
- How to extract links, text, and microdata from the web page content
- How to download files and save all the images on a page
- How to export CSV files, PDF documents, and XLSX spreadsheets
- How to automate forms, upload files, select dropdowns, and toggle the right checkboxes
- How to save, load, and manually add cookies
- How to run your scraper code in background jobs with sidekiq
- How to throttle your scraper to support rate limited requests
- How to test your code without hitting the network
“The book is a a great way to quickly get up and running with web scraping for jr to experienced developers.” — William Kennedy
“It is really informative and a great start for beginners! I like how you expand the code base gradually and each line of code really taught me something!” — Faye Fang
“Reading your book was easier for me to grasp than digesting the documentation(s). Simply put, being overwhelmed with course work and a regular job it’s just been super handy to have your reference manual on hand so I can get to the meat without skimming!” — John Mayo
“Well worth the $39, it’s cut my web scraping project down considerably. Cheers!” — Jonathan Korty
Questions you might have
Who is this handbook for?
Ruby developers who want to learn about crawling and scraping websites with Ruby Mechanize. If you’ve been struggling to get your Mechanize code to work, this is for you.
What format is the handbook?
DRM-free PDF, ideal for on-screen reading on your desktop, your laptop, or your tablet. There are no plans to offer print, ePub, or Kindle versions for the foreseeable future.
What if I’m not happy with the handbook?
I offer a 100% unconditional money back guarantee. If you’re not sure you got your money’s worth from the handbook just email me and I’ll refund your purchase.
Is the payment process secure?
Yep. Credit card payments are processed securely through Stripe, with everything transmitted securely over SSL/TLS as you’d expect.
How do I get a site license for my team?
Do you want to buy a site license for your company, organization, or coworking space? Get a site license (25 seat maximum) for $299. Email firstname.lastname@example.org to buy.
About the author
My name is Tim, and I’m a Ruby developer just like you. I’ve built e-commerce websites and voting systems from scratch, released a number of Ruby gems, taught developers how to integrate payments and write cleaner code, implemented the OAuth signature algorithms twice, and written code in over 15 different programming languages.
I’ve made use of web scraping throughout my career, and I want to teach you how you can leverage this useful skill to increase your value as a Ruby developer.
Questions? Email email@example.com and I’ll reply to you personally.