Day 31 - waybackurls - 100 tools in 100 days!

July 14, 2022

1 . Introduction
2 . My Setup
3 . What is waybackurls?
4 . Why use waybackurls?
5 . How to use waybackurls?
6 . Summary

This post is designed to introduce you to the tool waybackurls.

Disclaimer : Please only use waybackurls for professional and educational reasons. Do not use this tool for nefarious or malicious reasons.

1. Introduction

Welcome to the thirty-first blog post of 100 tools in 100 days.

Find waybackurls here.

Tom Hudson created waybackurls, and a bunch of other cool tools you will see in future posts.

Find Tom Hudson here:

GitHub

Slightly Outdated Website

2. My Setup

For running the waybackurls tool, I used Kali Linux in a VMware Workstation 16 Player virtualized environment.

3. What is waybackurls?

waybackurls is a tool for grabbing URLs from the Internet Archives Way Back Machine. You can find the Way Back Machine here.

4. Why use waybackurls?

Finding old domains is usually a form of reconnaissance for any penetration tester. Using waybackurls a tester could grab all of the URLs from waybackmachine with a simple command.

The tester could output the URLs into a file where further sorting and analysis could be performed to find information from less current sections of the website.

5. How to use waybackurls?

waybackurls is written in the Go language and uses the go install feature for downloading and installing.

If you do not have Go installed or do not have your variables set up for running go binaries perform Step 0 first.

Step 0 (a):
Run the following command to install Go:

sudo apt install -y golang

Step 0 (b):
Next, update your .zshrc file (the file used for Kali, if you are not 
using .zshrc then update .bashrc). Type the following:

sudo nano ~/.zshrc

Press the down arrow to your export section and add the following:

export GOROOT=/usr/lib/go
export GOPATH=$HOME/go
export PATH=$GOPATH/bin:$GOROOT/bin:$PATH

Press Ctrl+X
Type: Y
Press: Enter <br>

Step 0 (c):
Update your source file for your current shell by typing the following into your terminal:

source ~/.zshrc

Step 1:
To verify GO was installed, in terminal type:

go version

Step 2:
Install waybackurls with the following command:

go install github.com/tomnomnom/waybackurls@latest

You should be able to use the command waybackurls at this point. If you cannot, a temporary fix is to run the binary from its path which is: /home/YOURUSERNAME/go/bin/waybackurls

Step 3:
For this demonstration, I am going to use the domain: neopets.com and 
output the URLs into a file named neopets. Here is the command:

waybackurls neopets.com > neopets

waybackurls was able to pull 14798 URLs into the file neopets! That is a lot of URLs to analyze.

With this information, a tester or analyst could use other various Linux commands to filter out information, or they could input this file into a data analysis tool to look for specific items of interest.

6. Summary

waybackurls is a simple URL grabber for the Way Back Machine written in the Go Language by Tom Hudson. Check out Tom’s GitHub here.

A penetration tester may use waybackurls to find information during reconnaissance that may help during the test.

An analyst may use waybackurls to interact with less updated versions of a website to find security issues.

I hope you enjoyed this blog post.

Thanks for reading!

If you have suggestions for what tool to cover next, contact me!