SNCF Travaux Extractor Implementation

Overview

This document describes the implementation of the SNCF Travaux Extractor for the OpenEventDatabase. The extractor fetches railway work schedules from the SNCF open data API and adds them to the database as events.

Implementation Details

The extractor is implemented in the file extractors/sncf_travaux.py. It consists of the following components:

API Integration: The extractor connects to the SNCF open data API to fetch railway work schedules.
Date Conversion: The extractor converts week numbers to dates, as the SNCF data provides the year and week number rather than explicit start and end dates.
Event Creation: The extractor creates event objects from the SNCF data, including all required properties for the OpenEventDatabase.
Database Integration: The extractor submits events to the OpenEventDatabase.

Key Functions

`fetch_sncf_data()`

This function fetches railway work planning data from the SNCF open data API. It handles HTTP errors, JSON decoding errors, and checks if the response contains a 'results' field.

def fetch_sncf_data():
    """
    Fetch railway work planning data from the SNCF open data API.
    
    Returns:
        list: A list of railway work records.
    """
    # Implementation details...

`week_to_date(year, week_number)`

This function converts a year and week number to a date range (start date and end date). It handles various input formats and edge cases.

def week_to_date(year, week_number):
    """
    Convert a year and week number to a date.
    
    Args:
        year (str or int): The year.
        week_number (str or int): The week number (1-53).
        
    Returns:
        tuple: A tuple containing (start_date, end_date) as ISO format strings.
    """
    # Implementation details...

`create_event(record)`

This function creates an event object from a SNCF record. It extracts relevant data from the record, converts the year and week number to start and end dates, and creates a GeoJSON Feature object with all the necessary properties.

def create_event(record):
    """
    Create an event object from a SNCF record.
    
    Args:
        record: A record from the SNCF API.
        
    Returns:
        dict: A GeoJSON Feature representing the event.
    """
    # Implementation details...

`submit_event(event)`

This function submits an event to the OpenEventDatabase. It connects to the database, inserts the geometry and event data, and handles various error cases.

def submit_event(event):
    """
    Submit an event to the OpenEventDatabase.
    
    Args:
        event: A GeoJSON Feature representing the event.
        
    Returns:
        bool: True if the event was successfully submitted, False otherwise.
    """
    # Implementation details...

Event Properties

The events created by the extractor include the following properties:

type: "scheduled" (as these are planned railway works)
what: "transport.railway.maintenance" (a descriptive category)
what:series: "SNCF Railway Maintenance" (to group related events)
where: The line name
label: A descriptive label
description: A detailed description of the work
start and stop: The start and end dates derived from the year and week number

Additional properties specific to railway works:

line_code: The line code
work_type: The type of work
interventions: The number of interventions
start_point and end_point: The start and end points (kilometer points)
structure: The managing structure
source: "SNCF Open Data"

Testing

A test script (test_sncf_travaux.py) is provided to test the functionality of the extractor without actually submitting events to the database. It tests the week_to_date(), create_event(), and fetch_sncf_data() functions with various inputs.

To run the test script:

python3 extractors/test_sncf_travaux.py

Usage

To run the extractor and add SNCF railway work events to the database:

./extractors/sncf_travaux.py

Notes

The extractor uses a placeholder location (center of France) for the event geometry. In a real implementation, you might want to geocode the line or use a predefined location.
The extractor assumes that the SNCF API returns data in the format described in the comments. If the API changes, the extractor may need to be updated.
The extractor handles various error cases, such as missing required fields, invalid week numbers, and database connection errors.

4.5 KiB Raw Blame History