ben/gtfs-dagster
SHA256
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

added agency_list.csv.sample and updated README

Browse source
This commit is contained in:
Ben Varick 2025-11-07 18:07:35 -08:00
parent 97aa28e003
commit e761964bea
Signed by: ben
SSH key fingerprint: SHA256:jWnpFDAcacYM5aPFpYRqlsamlDyKNpSj3jj+k4ojtUo
3 changed files with 10 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

3
.gitignore vendored
View file

@ -1,6 +1,9 @@
#Exclude the .env file
.env
# Exclude agency_list.csv
config/agency_list.csv
#Exclude contents of the postres_data directory
postgres_data
postgres_data/*

4
README.md
View file

@ -3,7 +3,7 @@
Dagster setup that scrapes GTFS and GTFS-RT for specified transit agencies and adds them to a DuckDB
## Input
This reads from the `config/agency_list.csv` file, edit this file to include the transit agencies that you want to scrape, add the relevant IDs from mobilitydatabase.org
This reads from the `config/agency_list.csv` file, copy `agency_list.csv.sample` to `agency_list.csv` and edit this file to include the transit agencies that you want to scrape, add the relevant IDs from mobilitydatabase.org
## set your environment
@ -13,6 +13,8 @@ copy `env.sample` to `.env` and change:
- MobilityDatabase.org API token
- Location of data, config, and postgres_data directories (default is in working directory)
# Run it
`docker compose build`
`docker compose up -d`

4
config/agency_list.csv.sample Normal file
View file

@ -0,0 +1,4 @@
Name,GTFS,GTFS-RT_vehicles,GTFS-RT_trips,GTFS-RT_alerts
Madison Metro,mdb-394,mdb-2097,mdb-2096,mdb-2095
Milwaukee County Transit System,mdb-2127,mdb-2130,mdb-2128,mdb-2129
Whatcom Transportation Authority,mdb-284,tld-4358-vp,tld-4358-tu,tld-4358-sa