20 lines
777 B
Markdown
20 lines
777 B
Markdown
# gtfs-dagster
|
|
|
|
Dagster setup that scrapes GTFS and GTFS-RT for specified transit agencies and adds them to a DuckDB
|
|
|
|
## Input
|
|
This reads from the `data/gtfs/agency_list.csv` file, edit this file to include the transit agencies that you want to scrape, add the relevant IDs from mobilitydatabase.org
|
|
|
|
## set your environment
|
|
### Data directory
|
|
edit dagster.yaml to specify the correct data directory under run_launcher.
|
|
|
|
Right now it says: `/home/ben/code/gtfs-dagster/data:/opt/dagster/app/data`, change the first part to where you want the data to be stored.
|
|
|
|
### .env file
|
|
copy `env.sample` to `.env` and change the database password and the mobility database refresh token
|
|
|
|
# Run it
|
|
`docker compose build`
|
|
`docker compose up -d`
|
|
access the Dagster web ui at 127.0.0.1:3001
|