gtfs-dagster/README.md

gtfs-dagster

Dagster setup that scrapes GTFS and GTFS-RT for specified transit agencies and adds them to a DuckDB

Input

This reads from the config/agency_list.csv file, copy agency_list.csv.sample to agency_list.csv and edit this file to include the transit agencies that you want to scrape, add the relevant IDs from mobilitydatabase.org

set your environment

.env file

copy env.sample to .env and change:

• Postgres database password - make it something random before the first run
• MobilityDatabase.org API token
• Location of data, config, and postgres_data directories (default is in working directory)

Run it

docker compose build docker compose up -d access the Dagster web ui at 127.0.0.1:3001