add doc for deployment on web

This commit is contained in:
Tykayn 2025-09-18 14:50:30 +02:00 committed by tykayn
parent 339147e762
commit 3e9d3c838e
11 changed files with 309 additions and 469 deletions

4
.idea/backend.iml generated
View file

@ -1,7 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<module type="WEB_MODULE" version="4">
<component name="NewModuleRootManager">
<content url="file://$MODULE_DIR$" />
<content url="file://$MODULE_DIR$">
<excludeFolder url="file://$MODULE_DIR$/datasources" />
</content>
<orderEntry type="inheritedJdk" />
<orderEntry type="sourceFolder" forTests="false" />
</component>

View file

@ -0,0 +1,144 @@
# openeventdatabase.org
(Article de 2017 https://cq94.medium.com/openeventdatabase-org-f15ccf290537 )
Pour plus dinfo, contacter: christian.quest+oedb@gmail.com
Dernière avancée:
le premier prototype dAPI est opérationnel !
projet sur github https://github.com/openeventdatabase
début de documentation sur le wiki de lAPI https://github.com/openeventdatabase/backend/wiki
Le concept
- Un calcul ditinéraire peut avoir besoin de données en temps réel pour sajuster: bouchons, ralentissements, travaux, conditions météo. Cela permet de passer dune appli passive à une appli temp-réel.
- Que faire ce week-end ? Pour trouver facilement un loisir, une activité proche de chez soi… les spectacles, activités, horaires de ciné.
- Conserver des évènements historiques, ou lhistorique de géométries changeantes: anciens tracés de routes, champs de bataille, découpages administratifs, frontières… et les évènements passés plus ponctuels.
Dans tout ces exemples, trois informations sont nécessaires: quoi, où et quand.
Aucun projet à lheure actuelle ne propose de mettre en commun ce type de données. OpenStreetMap répond à quoi et où, mais pas à quand et na pas vocation ni à collecter des données “temps-réel”, ni à collecter des données historiques.
Le projet OpenEventDatabase tente de combler ce manque en proposant une base de données géo-temporelles pour échanger des informations de type “quoi / où / quand” de tout type.
Les usages sont très nombreux, comme détecter des “collisions” dévénements:
une prévision météo incompatible avec un événement sportif: annulation, report
un retard de train qui aura des conséquences sur dautres événements liés (correspondances affectées).
Pour figurer dans cette base, une donnée doit comporter les 3 composantes :
Quoi: description sémantique
type dévénement: type=*
scheduled : planifié (horaires de spectacles, de transports, travaux, chantiers)
unscheduled : accident, bouchon, retard/avance, etc
forecast : probabilité issu dune prévision (modèle météo, récurrence dévénements passés)
observed : une mesure météo…
autre ?
famille dévénement: what=*
culture (spectacles, concerts, cinéma, etc)
sport (match, entraînements, rencontres, etc)
transport (routier, ferroviaire, aérien, maritime, covoiturage, etc)
environnement (météo, pollution, etc)
autre ?
importance ?
Où: composante géographique
géométrie: geojson
lien avec OSM (where_osm) -> non stable
lien avec wikipédia (where_wikipedia)
lien avec wikidata (where_wikidata)
autre liens ?
Quand: composante temporelle au format ISO8601
gestion des répétitions, des intervalles, des fuseaux horaires
avec précision variable :
2014: année 2014
201401: janvier 2014
20140107: 7 janvier 2014
etc
Metadonnées
identifiant unique (uuid) attribué par lAPI
timestamp de création et de dernière mise à jour de la donnée (qui sert aussi au versionning) gérés par lAPI
source de la donnée (source par défaut et/ou source sur chaque attribut)
licence de la donnée (licence par défaut et/ou sur chaque attribut) la base peut collecter des données sous différentes licences ?)
API + dump
A linstar dOpenStreetMap, les données seront manipulables via une API, mais aussi sous forme de dumps pour permettre des analyses sur les historiques.
Get Christian Quests stories in your inbox
Join Medium for free to get updates from this writer.
LAPI doit permettre:
dajouter/modifier un évènement
de rechercher des évènements par combinaison de what/where/when
Les résultats de recherche pourront être proposés sous forme geojson (FeatureCollection) de flux RSS ou iCal.
Le format et lorganisation des données
La description pourra se faire à laide dattributs sur le modèle clé=valeur dOpenStreetMap
Exemples:
Une séance de cinéma
what=leisure.cinema.movie
type=scheduled
when=20140119T13:50+0100
where_osm=node:1428007260
what_name=Le Loup de Wall Street
where_name=Max Linder Panorama
wikipedia_what=fr:Le Loup de Wall Street
wikipedia_where=fr:Max Linder Panorama
Un départ de TGV
type=scheduled
what=public_transport.train.departure
when=20140119T06:23+0100
where_name=Paris-Gare de Lyon
where_uic_ref=8768600
what_operator=SNCF
what_ref=TGV6701
source=SNCF
source_where_wkt=OpenStreetMap
licence_where_wkt=ODbL/1.0
Un bouchon
type=unscheduled
what=traffic.jam
when=20140119 16:40
where_osm=way:68613064
where_name=A4 direction Paris
source_where= OpenStreetMap
licence_where=ODbL/1.0
Une alerte météo de vigilance orange pluie sur un département
type=forecast
what=weather.warning.rain
start=20140119T19:00+01:00
stop=20140120T06:00+01:00
where_name=Var
where_osm=relation:7390
source=meteo.fr
Les formats déchange pourront sappuyer sur: GeoJSON, XML ou autre.
Resources
http://www.geoevent.net/about/
http://blog.programmableweb.com/2014/02/14/olympics-are-now-a-global-api-event/
https://blog.twitter.com/2014/manhattan-our-real-time-multi-tenant-distributed-database-for-twitter-scale
http://www.journaldunet.com/solutions/cloud-computing/coulisses-techniques-de-twitter/

View file

@ -1,22 +0,0 @@
#!/bin/bash
# Script to start the server in the background and run tests
echo "Starting the server in the background..."
python3 backend.py > server.log 2>&1 &
SERVER_PID=$!
# Give the server a moment to start
sleep 2
echo "Running tests..."
python3 test_api.py
TEST_RESULT=$?
echo "Stopping the server (PID: $SERVER_PID)..."
kill $SERVER_PID
# Wait for the server to stop
sleep 1
echo "Test completed with exit code: $TEST_RESULT"
exit $TEST_RESULT

View file

@ -0,0 +1,91 @@
# Installation du service systemd pour uWSGI
Ce document explique comment installer et activer le service systemd pour faire fonctionner le serveur uWSGI de l'OpenEventDatabase backend.
## Prérequis
- Système Linux avec systemd
- Droits d'administrateur (sudo)
- PostgreSQL installé et configuré
- Python et dépendances installées (voir README.md)
## Installation du service
1. Copiez le fichier de service dans le répertoire systemd :
2. Assurez-vous que les permissions sont correctes :
3. Assurez-vous que l'utilisateur www-data a les permissions nécessaires sur le répertoire du projet :
4. Rechargez la configuration de systemd :
```bash
sudo cp oedb-uwsgi.service /etc/systemd/system/
sudo chmod 644 /etc/systemd/system/oedb-uwsgi.service
sudo chown -R www-data:www-data /home/poule/encrypted/stockage-syncable/www/development/html/oedb-backend
sudo systemctl daemon-reload
sudo systemctl enable oedb-uwsgi.service
sudo systemctl start oedb-uwsgi.service
```
## Activation et démarrage du service
1. Activez le service pour qu'il démarre automatiquement au démarrage du système :
```bash
```
2. Démarrez le service :
```bash
```
3. Vérifiez l'état du service :
```bash
sudo systemctl status oedb-uwsgi.service
```
## Gestion du service
- Pour arrêter le service :
```bash
sudo systemctl stop oedb-uwsgi.service
```
- Pour redémarrer le service :
```bash
sudo systemctl restart oedb-uwsgi.service
```
- Pour voir les logs du service :
```bash
sudo journalctl -u oedb-uwsgi.service
```
## Dépannage
Si le service ne démarre pas correctement, vérifiez les points suivants :
1. Assurez-vous que PostgreSQL est en cours d'exécution :
```bash
sudo systemctl status postgresql
```
2. Vérifiez les logs du service pour identifier les erreurs :
```bash
sudo journalctl -u oedb-uwsgi.service -n 50
```
3. Vérifiez que les chemins dans le fichier de service sont corrects et que l'utilisateur www-data a accès à ces chemins.
4. Assurez-vous que l'environnement virtuel Python est correctement configuré et que uwsgi est installé dans cet environnement.
5. Vérifiez que le script setup_db.sh s'exécute correctement et que la base de données est accessible avec les identifiants fournis.
## Personnalisation
Si vous souhaitez modifier la configuration du service, éditez le fichier `/etc/systemd/system/oedb-uwsgi.service` puis rechargez la configuration de systemd avec `sudo systemctl daemon-reload`.

View file

@ -0,0 +1,34 @@
# Caddyfile for OpenEventDatabase
# Configuration for oedb.cipherbliss.com and api.oedb.cipherbliss.com
# Main demo site - oedb.cipherbliss.com
oedb.cipherbliss.com {
# Log configuration
log {
output file /var/log/caddy/oedb.cipherbliss.com.log
format json
}
handle / {
reverse_proxy localhost:8080
}
}
# API site - api.oedb.cipherbliss.com
api.oedb.cipherbliss.com {
# Log configuration
log {
output file /var/log/caddy/api.oedb.cipherbliss.com.log
format json
}
# Handle all routes - proxy to backend
handle /* {
reverse_proxy localhost:8080
}
}

View file

@ -0,0 +1,29 @@
[Unit]
Description=uWSGI service for OpenEventDatabase backend
After=network.target postgresql.service
[Service]
User=www-data
Group=www-data
WorkingDirectory=/home/poule/encrypted/oedb-backend
Environment="DB_USER=cipherbliss"
Environment="POSTGRES_PASSWORD=tralalahihou"
ExecStartPre=/home/poule/encrypted/oedb-backend/setup_db.sh
ExecStart=/home/poule/encrypted/oedb-backend/venv/bin/uwsgi \
--http :8080 \
--wsgi-file backend.py \
--callable app \
--master \
--processes 4 \
--threads 2 \
--vacuum
Restart=on-failure
RestartSec=5s
KillSignal=SIGQUIT
Type=notify
NotifyAccess=all
StandardError=syslog
StandardOutput=syslog
[Install]
WantedBy=multi-user.target

View file

@ -5,6 +5,14 @@
-- Dumped from database version 9.5.3
-- Dumped by pg_dump version 9.5.3
CREATE ROLE "cipherbliss";
ALTER ROLE "cipherbliss" WITH LOGIN;
-- pass à changer selon les données du fichier .env
-- alter USER cipherbliss with password '1234';
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO cipherbliss;
GRANT ALL PRIVILEGES ON DATABASE oedb TO cipherbliss;
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
CREATE EXTENSION IF NOT EXISTS "postgis";

View file

@ -1,87 +0,0 @@
#!/usr/bin/env python3
"""
Test script to verify API endpoints are accessible.
"""
import requests
import sys
import json
import time
BASE_URL = "http://127.0.0.1:8080"
def test_endpoint(endpoint, method="GET", data=None):
"""Test an API endpoint and print the result."""
url = f"{BASE_URL}{endpoint}"
print(f"Testing {method} {url}...")
try:
if method == "GET":
response = requests.get(url)
elif method == "POST":
headers = {"Content-Type": "application/json"}
response = requests.post(url, data=json.dumps(data), headers=headers)
print(f"Status code: {response.status_code}")
if response.status_code < 400:
print("Success!")
else:
print(f"Error: {response.text}")
print("-" * 50)
return response.status_code < 400
except Exception as e:
print(f"Exception: {e}")
print("-" * 50)
return False
def main():
"""Run tests for all endpoints."""
# Wait for server to start
print("Waiting for server to start...")
max_retries = 5
retries = 0
while retries < max_retries:
try:
requests.get(f"{BASE_URL}/")
print("Server is running!")
break
except requests.exceptions.ConnectionError:
print(f"Server not ready yet, retrying in 2 seconds... ({retries+1}/{max_retries})")
retries += 1
time.sleep(2)
if retries == max_retries:
print("Could not connect to server after multiple attempts.")
print("Please make sure the server is running on http://127.0.0.1:8080")
return 1
success = True
# Test root endpoint
success = test_endpoint("/") and success
# Test event endpoint
success = test_endpoint("/event") and success
# Test event/search endpoint with POST
search_data = {
"geometry": {
"type": "Point",
"coordinates": [2.3522, 48.8566] # Paris coordinates
}
}
success = test_endpoint("/event/search", method="POST", data=search_data) and success
# Test stats endpoint
success = test_endpoint("/stats") and success
if success:
print("All tests passed!")
return 0
else:
print("Some tests failed!")
return 1
if __name__ == "__main__":
sys.exit(main())

View file

@ -1,293 +0,0 @@
#!/usr/bin/env python3
# Script to test the API query parameters
# This script tests each query parameter to ensure it works as expected
import os
import sys
import json
import requests
from datetime import datetime, timedelta
import time
# Function to load environment variables from .env file
def load_env_from_file():
if os.path.exists('.env'):
print("Loading environment variables from .env file...")
with open('.env', 'r') as f:
for line in f:
line = line.strip()
if line and not line.startswith('#'):
key, value = line.split('=', 1)
os.environ[key] = value
# Load environment variables from .env file
load_env_from_file()
# API base URL
API_BASE_URL = "http://localhost:8080"
# Current date (2025-09-15 as specified in the issue)
CURRENT_DATE = datetime(2025, 9, 15, 23, 0)
# Sample locations (Paris, Lyon, Marseille, Toulouse, Bordeaux, Lille)
SAMPLE_LOCATIONS = [
{"name": "Paris", "lon": 2.3522, "lat": 48.8566},
{"name": "Lyon", "lon": 4.8357, "lat": 45.7640},
{"name": "Marseille", "lon": 5.3698, "lat": 43.2965},
{"name": "Toulouse", "lon": 1.4442, "lat": 43.6047},
{"name": "Bordeaux", "lon": -0.5792, "lat": 44.8378},
{"name": "Lille", "lon": 3.0573, "lat": 50.6292}
]
# Event categories
EVENT_CATEGORIES = ["traffic", "nature", "weather", "sport", "conference", "party"]
# Function to make API request and handle errors
def make_api_request(endpoint, params=None):
url = f"{API_BASE_URL}/{endpoint}"
try:
response = requests.get(url, params=params)
response.raise_for_status() # Raise exception for 4XX/5XX responses
return response.json()
except requests.exceptions.RequestException as e:
print(f"Error making request to {url}: {e}")
return None
# Function to print test results
def print_test_results(test_name, success, results=None, error=None):
print(f"\n--- Test: {test_name} ---")
if success:
print("✅ SUCCESS")
if results:
if isinstance(results, dict) and 'features' in results:
print(f"Found {len(results['features'])} events")
for i, feature in enumerate(results['features'][:3]): # Show first 3 events
props = feature['properties']
print(f" {i+1}. {props.get('name', 'Unnamed')} ({props.get('id', 'No ID')})")
if len(results['features']) > 3:
print(f" ... and {len(results['features']) - 3} more events")
else:
print(json.dumps(results, indent=2))
else:
print("❌ FAILED")
if error:
print(f"Error: {error}")
# Test 1: Basic event retrieval (no parameters)
def test_basic_retrieval():
print("\n=== Testing Basic Event Retrieval ===")
results = make_api_request("event")
success = results is not None and 'features' in results
print_test_results("Basic Event Retrieval", success, results)
return success
# Test 2: Filter by 'what' parameter
def test_what_parameter():
print("\n=== Testing 'what' Parameter ===")
all_success = True
for category in EVENT_CATEGORIES:
results = make_api_request("event", {"what": category})
success = results is not None and 'features' in results
# Verify that all returned events have the correct category
if success and results['features']:
for feature in results['features']:
if 'events_what' in feature['properties'] and feature['properties']['events_what'] != category:
success = False
break
print_test_results(f"Filter by what={category}", success, results)
all_success = all_success and success
return all_success
# Test 3: Filter by 'when' parameter
def test_when_parameter():
print("\n=== Testing 'when' Parameter ===")
all_success = True
# Test with 'now'
results = make_api_request("event", {"when": "now"})
success = results is not None and 'features' in results
print_test_results("Filter by when=now", success, results)
all_success = all_success and success
# Test with 'today'
results = make_api_request("event", {"when": "today"})
success = results is not None and 'features' in results
print_test_results("Filter by when=today", success, results)
all_success = all_success and success
# Test with 'yesterday'
results = make_api_request("event", {"when": "yesterday"})
success = results is not None and 'features' in results
print_test_results("Filter by when=yesterday", success, results)
all_success = all_success and success
# Test with 'tomorrow'
results = make_api_request("event", {"when": "tomorrow"})
success = results is not None and 'features' in results
print_test_results("Filter by when=tomorrow", success, results)
all_success = all_success and success
# Test with 'lasthour'
results = make_api_request("event", {"when": "lasthour"})
success = results is not None and 'features' in results
print_test_results("Filter by when=lasthour", success, results)
all_success = all_success and success
# Test with 'nexthour'
results = make_api_request("event", {"when": "nexthour"})
success = results is not None and 'features' in results
print_test_results("Filter by when=nexthour", success, results)
all_success = all_success and success
# Test with ISO8601 date
iso_date = CURRENT_DATE.isoformat()
results = make_api_request("event", {"when": iso_date})
success = results is not None and 'features' in results
print_test_results(f"Filter by when={iso_date}", success, results)
all_success = all_success and success
return all_success
# Test 4: Filter by 'start' and 'stop' parameters
def test_start_stop_parameters():
print("\n=== Testing 'start' and 'stop' Parameters ===")
all_success = True
# Test with start only
start_date = (CURRENT_DATE - timedelta(days=10)).isoformat()
results = make_api_request("event", {"start": start_date})
success = results is not None and 'features' in results
print_test_results(f"Filter by start={start_date}", success, results)
all_success = all_success and success
# Test with stop only
stop_date = (CURRENT_DATE + timedelta(days=10)).isoformat()
results = make_api_request("event", {"stop": stop_date})
success = results is not None and 'features' in results
print_test_results(f"Filter by stop={stop_date}", success, results)
all_success = all_success and success
# Test with both start and stop
results = make_api_request("event", {"start": start_date, "stop": stop_date})
success = results is not None and 'features' in results
print_test_results(f"Filter by start={start_date} and stop={stop_date}", success, results)
all_success = all_success and success
# Test with relative keywords
results = make_api_request("event", {"start": "last7days", "stop": "next30days"})
success = results is not None and 'features' in results
print_test_results("Filter by start=last7days and stop=next30days", success, results)
all_success = all_success and success
return all_success
# Test 5: Filter by 'bbox' parameter
def test_bbox_parameter():
print("\n=== Testing 'bbox' Parameter ===")
# Create a bounding box around France
bbox = ["-5.0", "41.0", "10.0", "52.0"] # [E, S, W, N]
results = make_api_request("event", {"bbox": bbox})
success = results is not None and 'features' in results
print_test_results(f"Filter by bbox={','.join(bbox)}", success, results)
return success
# Test 6: Filter by 'near' parameter
def test_near_parameter():
print("\n=== Testing 'near' Parameter ===")
all_success = True
# Test with Paris coordinates
paris = SAMPLE_LOCATIONS[0]
near_params = [paris["lon"], paris["lat"], 50000] # 50km radius
results = make_api_request("event", {"near": near_params})
success = results is not None and 'features' in results
print_test_results(f"Filter by near={paris['lon']},{paris['lat']},50000", success, results)
all_success = all_success and success
# Test with Lyon coordinates
lyon = SAMPLE_LOCATIONS[1]
near_params = [lyon["lon"], lyon["lat"], 30000] # 30km radius
results = make_api_request("event", {"near": near_params})
success = results is not None and 'features' in results
print_test_results(f"Filter by near={lyon['lon']},{lyon['lat']},30000", success, results)
all_success = all_success and success
return all_success
# Test 7: Combine multiple parameters
def test_combined_parameters():
print("\n=== Testing Combined Parameters ===")
# Combine 'what' and 'when' parameters
params = {
"what": "traffic",
"when": "now"
}
results = make_api_request("event", params)
success = results is not None and 'features' in results
print_test_results("Filter by what=traffic and when=now", success, results)
return success
# Main function to run all tests
def main():
print("Starting API parameter tests...")
# Check if the server is running
try:
response = requests.get(f"{API_BASE_URL}/stats")
if response.status_code != 200:
print(f"❌ Server is not responding correctly. Status code: {response.status_code}")
return False
print("✅ Server is running and responding to requests.")
except requests.exceptions.ConnectionError:
print("❌ Cannot connect to the server. Make sure it's running on http://localhost:8080")
return False
# Run all tests
tests = [
("Basic Event Retrieval", test_basic_retrieval),
("'what' Parameter", test_what_parameter),
("'when' Parameter", test_when_parameter),
("'start' and 'stop' Parameters", test_start_stop_parameters),
("'bbox' Parameter", test_bbox_parameter),
("'near' Parameter", test_near_parameter),
("Combined Parameters", test_combined_parameters)
]
results = {}
all_success = True
for test_name, test_func in tests:
print(f"\n{'=' * 50}")
print(f"Running test: {test_name}")
try:
success = test_func()
results[test_name] = success
all_success = all_success and success
except Exception as e:
print(f"❌ Test failed with exception: {e}")
results[test_name] = False
all_success = False
# Print summary
print("\n\n" + "=" * 50)
print("TEST SUMMARY")
print("=" * 50)
for test_name, success in results.items():
status = "✅ PASSED" if success else "❌ FAILED"
print(f"{test_name}: {status}")
overall_status = "✅ ALL TESTS PASSED" if all_success else "❌ SOME TESTS FAILED"
print(f"\nOverall: {overall_status}")
return all_success
if __name__ == "__main__":
main()

View file

@ -1,42 +0,0 @@
#!/usr/bin/env python3
# Test script to verify database connectivity check
import os
import sys
import psycopg2
# Import the db_connect and check_db_connection functions from oedb.utils.db
sys.path.append(os.path.dirname(os.path.abspath(__file__)))
from oedb.utils.db import db_connect, check_db_connection
def test_db_connection():
"""Test the database connection check functionality"""
print("Testing database connection...")
# Test with valid connection
result = check_db_connection()
if result:
print("✅ Database connection successful")
else:
print("❌ Database connection failed with valid credentials")
# Test with invalid connection (by temporarily modifying environment variables)
original_host = os.environ.get("DB_HOST", "")
os.environ["DB_HOST"] = "nonexistent-host"
result = check_db_connection()
if not result:
print("✅ Correctly detected invalid database connection")
else:
print("❌ Failed to detect invalid database connection")
# Restore original environment
if original_host:
os.environ["DB_HOST"] = original_host
else:
os.environ.pop("DB_HOST", None)
print("Database connection test completed")
if __name__ == "__main__":
test_db_connection()

View file

@ -1,24 +0,0 @@
#!/usr/bin/env python3
"""
Test script to verify database connection.
This script attempts to connect to the PostgreSQL database using the same
connection logic as the main application.
"""
import sys
from oedb.utils.db import check_db_connection
from oedb.utils.logging import logger
def main():
"""Test database connection and print the result."""
logger.info("Testing database connection...")
if check_db_connection():
logger.success("Database connection successful!")
return 0
else:
logger.error("Database connection failed. Check your .env file and PostgreSQL configuration.")
return 1
if __name__ == "__main__":
sys.exit(main())