About This Resource

A command-line tool for extracting information from web pages and other online resources. It uses configurable steps to fetch content and select the data needed from it.

Saved Notes

Original notes saved with this link. They may describe an earlier version.

GitHub - bjesus/pipet: Swiss-army tool for scraping and extracting data from online assets, made for hackers

Pipet is a command line based web scraper. It supports 3 modes of operation - HTML parsing, JSON parsing, and client-side JavaScript evaluation. It relies heavily on existing tools like curl, and it uses unix pipes for extending its built-in capabilities.