gau (get all urls)
which replaces Rengine's https://github.com/bp0lr/gauplus
getallurls (gau) fetches known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, Common Crawl, and URLScan for any given domain. Inspired by Tomnomnom's waybackurls.
Resources
Usage:
Examples:
$ printf example.com | gau $ cat domains.txt | gau --threads 5 $ gau example.com google.com $ gau --o example-urls.txt example.com $ gau --blacklist png,jpg,gif example.com