ben/gtfs-dagster
SHA256
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
gtfs-dagster/README.md
Ben Varick cb7247f2f1
edited README
2025-11-07 18:12:09 -08:00

751 B
Raw Blame History

gtfs-dagster

Dagster setup that scrapes GTFS and GTFS-RT for specified transit agencies and adds them to a DuckDB

Input

You define which agencies and feeds to scrape with the fileconfig/agency_list.csv To include the transit agencies that you want to scrape, add the relevant IDs from mobilitydatabase.org See config/agency_list.csv.sample for an example.

set your environment

.env file

copy env.sample to .env and change:

  • Postgres database password - make it something random before the first run
  • MobilityDatabase.org API token
  • Location of data, config, and postgres_data directories (default is in working directory)

Run it

docker compose build docker compose up -d access the Dagster web ui at 127.0.0.1:3001