Skip to content
This repository has been archived by the owner on Feb 1, 2022. It is now read-only.

Add a requirements.txt for python dependencies #69

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 11 additions & 14 deletions README.de.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,23 +47,20 @@ community data (aggregates to distrcit - hopefully):
```landkreise/data/0<districtId>/<communityId>.csv```

### Requirements:
* Python3
* requests (often included)
* CacheControl[filecache] (persistent cache does not work yet)
* lockfile
* beautiful soup 4
* Python3
* Dependencies from `requirements.txt` which can be install as follows:

pip3 line:
```
pip3 install requests bs4 cachecontrol[filecache] lockfile
```
- by using pip:
```
pip3 install -r requirements.txt
```

Debian/Ubuntu packages:
```
sudo apt-get install python3 python3-bs4 python3-cachecontrol python3-lockfile
```
- by using Debian/Ubuntu packages:
```
sudo apt install python3 python3-bs4 python3-cachecontrol python3-lockfile python3-tqdm
```

* Makefile for Docker container also exists
Alternatively, you can use the Dockerfile to build a Docker container which has everything already installed.

### Writing / Mainting a scraper

Expand Down
29 changes: 13 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,24 +54,21 @@ community data (aggregates to distrcit - hopefully):
```landkreise/data/0<districtId>/<communityId>.csv```

### Requirements:
* Python3
* requests (often included)
* CacheControl[filecache] (persistent cache does not work yet)
* lockfile
* beautiful soup 4
* running all scrapers tqdm

pip3 line:
```
pip3 install requests bs4 cachecontrol[filecache] lockfile tqdm
```

Debian/Ubuntu packages:
```
sudo apt-get install python3 python3-bs4 python3-cachecontrol python3-lockfile python3-tqdm
```
* Python3
* Dependencies from `requirements.txt` which can be install as follows:

- by using pip:
```
pip3 install -r requirements.txt
```

- by using Debian/Ubuntu packages:
```
sudo apt install python3 python3-bs4 python3-cachecontrol python3-lockfile python3-tqdm
```

* Makefile for Docker container also exists
Alternatively, you can use the Dockerfile to build a Docker container which has everything already installed.

### Writing / Mainting a scraper

Expand Down
3 changes: 2 additions & 1 deletion landkreise/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ VOLUME /landkreise
# - add ~/.local/bin for pip --user
ENV PATH="/home/corona/.local/bin:${PATH}"
# - install dependencies
RUN pip3 install --user bs4 requests cachecontrol[filecache] lockfile tqdm
ADD requirements.txt /tmp/
RUN pip3 install --user --requirement /tmp/requirements.txt

CMD bash
5 changes: 5 additions & 0 deletions landkreise/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
bs4==0.0.1
CacheControl==0.12.6
lockfile==0.12.2
requests==2.24.0
tqdm==4.46.1