gtfs-dagster

Dagster setup that scrapes GTFS and GTFS-RT for specified transit agencies and adds them to a DuckDB

Input

This reads from the data/gtfs/agency_list.csv file, edit this file to include the transit agencies that you want to scrape, add the relevant IDs from mobilitydatabase.org

set to your environment

Data directory

edit dagster.yaml to specify the correct data directory under run_launcher.

Right now it says: /home/ben/code/gtfs-dagster/data:/opt/dagster/app/data, change the first part to where you want the data to be stored.

.env file

copy env.sample to .env and change the password

Run it

docker compose build docker compose up -d access the Dagster web ui at 127.0.0.1:3001

731 B Raw Blame History