acid/web-scraper

Go to file

acid c16b0160cb Merge branch 'master' of git.cusack.cloud:1922]:acid/web-scraper

2021-03-07 15:19:14 +00:00

Added tests

2021-03-07 14:58:01 +00:00

.env_sample

working

2021-03-05 16:16:51 +00:00

.gitignore

Refactored to functional style to increase reusability

2021-03-06 15:37:16 +00:00

.gitlab-ci.yml

Add .gitlab-ci.yml

2021-03-07 14:59:31 +00:00

README.md

added readme

2021-03-06 16:33:15 +00:00

requirements.txt

updated requirements

2021-03-07 15:19:02 +00:00

scrape.py

Added tests

2021-03-07 14:58:01 +00:00

test_scrape.py

Added tests

2021-03-07 14:58:01 +00:00

README.md

Web Scraper

Installation

Either download a copy of the script, or use to git clone to grab a copy

git clone https://git.cusack.cloud/acid/web-scraper.git

Create a virtual environment and then activate it

cd web-scraper
python3 -m venv .
source bin/activate

Install requirements

pip install -r requirements.txt

Usage

Copy the .env_sample to .env and fill in your twilio details to send SMS messages

Edit the config object in scrape.py to point to the URL to be monitored.