Skip to main content

tomnomnom tools in Rengine

https://github.com/tomnomnom/gf

The examples are GREAT!!!!!!

gf

A wrapper around grep to avoid typing common patterns.

What? Why?

I use grep a lot. When auditing code bases, looking at the output of meg, or just generally dealing with large amounts of data. I often end up using fairly complex patterns like this one:

▶ grep -HnrE '(\$_(POST|GET|COOKIE|REQUEST|SERVER|FILES)|php://(input|stdin))' *

It's really easy to mess up when typing all of that, and it can be hard to know if you haven't got any results because there are non to find, or because you screwed up writing the pattern or chose the wrong flags.

I wrote gf to give names to the pattern and flag combinations I use all the time. So the above command becomes simply:

▶ gf php-sources

github.com/tomnomnom/waybackurls

waybackurls

Accept line-delimited domains on stdin, fetch known URLs from the Wayback Machine for *.domain and output them on stdout.

Usage example:

▶ cat domains.txt | waybackurls > urls

Install:

▶ go install github.com/tomnomnom/waybackurls@latest

https://github.com/tomnomnom/unfurl

▶ unfurl -h
Format URLs provided on stdin

Usage:
  unfurl [OPTIONS] [MODE] [FORMATSTRING]

Options:
  -u, --unique   Only output unique values
  -v, --verbose  Verbose mode (output URL parse errors)

Modes:
  keys     Keys from the query string (one per line)
  values   Values from the query string (one per line)
  keypairs Key=value pairs from the query string (one per line)
  domains  The hostname (e.g. sub.example.com)
  paths    The request path (e.g. /users)
  apexes   The apex domain (e.g. example.com from sub.example.com)
  json     JSON encoded url/format objects
  format   Specify a custom format (see below)

Format Directives:
  %%  A literal percent character
  %s  The request scheme (e.g. https)
  %u  The user info (e.g. user:pass)
  %d  The domain (e.g. sub.example.com)
  %S  The subdomain (e.g. sub)
  %r  The root of domain (e.g. example)
  %t  The TLD (e.g. com)
  %P  The port (e.g. 8080)
  %p  The path (e.g. /users)
  %e  The path's file extension (e.g. jpg, html)
  %q  The raw query string (e.g. a=1&b=2)
  %f  The page fragment (e.g. page-section)
  %@  Inserts an @ if user info is specified
  %:  Inserts a colon if a port is specified
  %?  Inserts a question mark if a query string exists
  %#  Inserts a hash if a fragment exists
  %a  Authority (alias for %u%@%d%:%P)

Examples:
  cat urls.txt | unfurl keys
  cat urls.txt | unfurl format %s://%d%p?%q
Back to top