ben/gtfs-dagster
SHA256
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

added env.sample and updated README

Browse source
This commit is contained in:
Ben Varick 2025-11-05 18:44:07 -08:00
parent 38243834c5
commit ed220222a7
Signed by: ben
SSH key fingerprint: SHA256:jWnpFDAcacYM5aPFpYRqlsamlDyKNpSj3jj+k4ojtUo
2 changed files with 21 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

19
README.md
View file

@ -1,3 +1,20 @@
# gtfs-dagster
Dagster setup that scrapes GTFS and GTFS-RT for specified transit agencies and adds them to a DuckDB
Dagster setup that scrapes GTFS and GTFS-RT for specified transit agencies and adds them to a DuckDB
## Input
This reads from the `data/gtfs/agency_list.csv` file, edit this file to include the transit agencies that you want to scrape, add the relevant IDs from mobilitydatabase.org
## set to your environment
### Data directory
edit dagster.yaml to specify the correct data directory under run_launcher.
Right now it says: `/home/ben/code/gtfs-dagster/data:/opt/dagster/app/data`, change the first part to where you want the data to be stored.
### .env file
copy `env.sample` to `.env` and change the password
# Run it
`docker compose build`
`docker compose up -d`
access the Dagster web ui at 127.0.0.1:3001