Best Desktop and Cloud Based SEO Crawlers (2016)

A website crawler is a tool that every SEO will need to do their job effectively. SEOs are expected to know how the major search engines are crawling a particular site and identify errors, redirects, spider traps, or other issues. We’ve come a long way from GSiteCrawler and Xenu Link Slueth, browser extensions, and one page SEO audit graders that were the few options available in the past. This post is intended to build a greater understanding of the most popular web crawlers out there today.

botify web crawler

Botify (Paid)
Pricing: Lower tier package starts at $569 per month (for 5M URLs per month), however they also advertise SMB pricing options as low as $49 per month.
Why Botify: Crawl high volume of URLs for onpage data at the lowest cost per URL

It’s a cloud-based solution, so you don’t have any of the desktop software based memory limitations. Assuming you can afford it and your server can handle it, Botify can crawl up 100 URLs per second. That’s roughly 5M URLs in 14 hours! They claim to be the most powerful & comprehensive crawler and analysis tool for your website, and based on the feature set, it’s hard to argue against that. Continue reading

Manage Your Personal Budget With Google Sheets

The beginning of the year is a common time to make promises to yourself, and hopefully, this can help you keep those promises related to personal finance and budgeting.

This document started as a way for my wife and me to manage our budget while focusing resources on getting our student loans paid off. Trust me, I’m aware there are a number of (free-ish) tools out there, so if you already have a system that works, great. If not, or if you’re looking for a free alternative to the budgeting tool you’re using, then this post is for you.

The Personal Budget template described can be found here:

https://docs.google.com/spreadsheets/d/1fUjNr-n5u-GLzvmWLJk2_m9CYOIHSpmS-A8m3Jjz1-c/edit?usp=sharing

The following steps will help you through the process, to get your budget set up in around 20 minutes. Continue reading