tykayn/oedb-backend
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

up demo

Browse source
This commit is contained in:
Tykayn 2025-09-16 01:01:32 +02:00 • committed by tykayn
parent 2157091778
commit afe83a9a3c
8 changed files with 1433 additions and 8 deletions
Download patch file Download diff file Expand all files Collapse all files

116
ANTI_SPAM_MEASURES.md Normal file
View file

@ -0,0 +1,116 @@
# Anti-Spam and Caching Measures
This document describes the anti-spam and caching measures implemented in the OpenEventDatabase API to protect against abuse and improve performance.
## Implemented Measures
### 1. Rate Limiting
Rate limiting is implemented using the `RateLimitMiddleware` class, which tracks request rates by IP address and rejects requests that exceed defined limits.
#### Key Features
- **Global Rate Limit**: By default, each IP address is limited to 60 requests per minute across all endpoints.
- **Endpoint-Specific Limits**:
- POST requests to `/event`: Limited to 10 requests per minute
- POST requests to `/event/search`: Limited to 20 requests per minute
- DELETE requests to `/event`: Limited to 5 requests per minute
- **Proper HTTP Responses**: When a rate limit is exceeded, the API returns a `429 Too Many Requests` response with a `Retry-After` header indicating when the client can try again.
- **Detailed Logging**: Rate limit violations are logged with details about the client IP, request method, path, and user agent for security analysis.
- **Development Mode**: Rate limiting is skipped for local requests (127.0.0.1, localhost) to facilitate development.
#### Implementation Details
The rate limiting middleware:
1. Tracks request timestamps by IP address
2. Cleans up old request timestamps that are outside the current time window
3. Counts recent requests within the time window
4. Rejects requests that exceed the defined limits
5. Handles IP addresses behind proxies by checking the `X-Forwarded-For` header
### 2. Caching
Caching is implemented using the `CacheMiddleware` class, which adds appropriate cache-control headers to responses based on the endpoint and request method.
#### Key Features
- **Global Default**: By default, GET requests are cached for 60 seconds.
- **Endpoint-Specific Caching**:
- GET requests to `/event`: Cached for 60 seconds
- GET requests to `/stats`: Cached for 300 seconds (5 minutes)
- GET requests to `/demo`: Cached for 3600 seconds (1 hour)
- POST requests to `/event/search`: Not cached
- **No Caching for Write Operations**: POST, PUT, DELETE, and PATCH requests are not cached.
- **No Caching for Error Responses**: Responses with status codes >= 400 are not cached.
- **Proper HTTP Headers**: The middleware adds appropriate `Cache-Control`, `Vary`, `Pragma`, and `Expires` headers.
#### Implementation Details
The caching middleware:
1. Determines the appropriate max-age value for the current request based on endpoint and method
2. Adds caching headers for cacheable responses
3. Adds no-cache headers for non-cacheable responses
## How These Measures Help
### Rate Limiting Benefits
1. **Prevents Abuse**: Limits the impact of malicious users trying to overload the system.
2. **Ensures Fair Usage**: Prevents a single user from consuming too many resources.
3. **Protects Against Brute Force Attacks**: Makes it harder to use brute force attacks against the API.
4. **Reduces Server Load**: Helps maintain server performance during traffic spikes.
### Caching Benefits
1. **Improves Performance**: Reduces server load by allowing clients to reuse responses.
2. **Reduces Bandwidth Usage**: Minimizes the amount of data transferred between the server and clients.
3. **Enhances User Experience**: Provides faster response times for frequently accessed resources.
4. **Optimizes Resource Usage**: Allows the server to focus on processing new requests rather than repeating the same work.
## Suggestions for Future Improvements
### Rate Limiting Enhancements
1. **API Key Authentication**: Implement API key authentication to identify users and apply different rate limits based on user roles or subscription levels.
2. **Graduated Rate Limiting**: Implement a graduated rate limiting system that reduces the rate limit after suspicious activity is detected.
3. **Distributed Rate Limiting**: Use a distributed cache (like Redis) to track rate limits across multiple server instances.
4. **Machine Learning for Abuse Detection**: Implement machine learning algorithms to detect and block abusive patterns.
5. **CAPTCHA Integration**: Add CAPTCHA challenges for suspicious requests.
6. **IP Reputation Checking**: Integrate with IP reputation services to block known malicious IPs.
### Caching Enhancements
1. **Server-Side Caching**: Implement server-side caching using a cache like Redis or Memcached to reduce database load.
2. **Cache Invalidation**: Implement a cache invalidation system to clear cached responses when the underlying data changes.
3. **Conditional Requests**: Support conditional requests using ETags and If-Modified-Since headers.
4. **Vary Header Optimization**: Optimize the Vary header to better handle different client capabilities.
5. **Cache Partitioning**: Implement cache partitioning based on user roles or other criteria.
6. **Content Compression**: Add content compression (gzip, brotli) to reduce bandwidth usage further.
## How to Monitor and Adjust
### Monitoring Rate Limiting
The rate limiting middleware logs detailed information about rate limit violations. You can monitor these logs to:
- Identify potential abuse patterns
- Adjust rate limits based on actual usage patterns
- Detect and block malicious IPs
### Adjusting Rate Limits
To adjust the rate limits, modify the `RateLimitMiddleware` class in `oedb/middleware/rate_limit.py`:
- Change the `window_size` and `max_requests` parameters in the constructor
- Modify the `rate_limit_rules` list to adjust endpoint-specific limits
### Monitoring Caching
To monitor the effectiveness of caching:
- Use browser developer tools to check if responses are being cached correctly
- Monitor server logs to see if the same requests are being processed repeatedly
- Use performance monitoring tools to measure response times
### Adjusting Caching
To adjust the caching settings, modify the `CacheMiddleware` class in `oedb/middleware/cache.py`:
- Change the `default_max_age` parameter in the constructor
- Modify the `caching_rules` list to adjust endpoint-specific caching durations

122
DEMO_POPUP_ENHANCEMENT.md Normal file
View file

@ -0,0 +1,122 @@
# Demo Page Popup Enhancement
## Overview
The `/demo` endpoint of the OpenEventDatabase API provides an interactive map that displays current events from the database. This document describes the enhancement made to the event popups on the map, which now display all properties of each event.
## Changes Made
Previously, the event popups only displayed a few selected properties:
- Event name (label)
- Date (when)
- Type (what)
- A link to more information (if available)
Now, the popups display **all properties** of each event, providing a more comprehensive view of the event data. The enhancements include:
1. **Complete Property Display**: All properties from the event's JSON object are now displayed in the popup.
2. **Organized Layout**: Properties are displayed in a table format with property names in the left column and values in the right column.
3. **Alphabetical Sorting**: Properties are sorted alphabetically for easier navigation.
4. **Scrollable Container**: A scrollable container is used to handle events with many properties without making the popup too large.
5. **Smart Formatting**:
- Objects are displayed as formatted JSON
- URLs are displayed as clickable links
- Null values are displayed as "null" in italics
- Other values are displayed as strings
## Example
When you click on an event marker on the map, a popup will appear showing all properties of the event. For example:
```
Event Name
createdate: 2023-09-15T12:00:00Z
id: 123e4567-e89b-12d3-a456-426614174000
lastupdate: 2023-09-15T12:00:00Z
source: https://example.com/event
start: 2023-09-15T12:00:00Z
stop: 2023-09-16T12:00:00Z
type: scheduled
what: sport.match.football
what:series: Euro 2024
where: Stadium Name
```
## Technical Implementation
The enhancement was implemented by modifying the JavaScript code in the `demo.py` file. The key changes include:
```javascript
// Create popup content
let popupContent = '<div class="event-popup">';
popupContent += `<h3>${properties.label || 'Event'}</h3>`;
// Display all properties
popupContent += '<div style="max-height: 300px; overflow-y: auto;">';
popupContent += '<table style="width: 100%; border-collapse: collapse;">';
// Sort properties alphabetically for better organization
const sortedKeys = Object.keys(properties).sort();
for (const key of sortedKeys) {
// Skip the label as it's already displayed as the title
if (key === 'label') continue;
const value = properties[key];
let displayValue;
// Format the value based on its type
if (value === null || value === undefined) {
displayValue = '<em>null</em>';
} else if (typeof value === 'object') {
displayValue = `<pre style="margin: 0; white-space: pre-wrap;">${JSON.stringify(value, null, 2)}</pre>`;
} else if (typeof value === 'string' && value.startsWith('http')) {
displayValue = `<a href="${value}" target="_blank">${value}</a>`;
} else {
displayValue = String(value);
}
popupContent += `
<tr style="border-bottom: 1px solid #eee;">
<td style="padding: 4px; font-weight: bold; vertical-align: top;">${key}:</td>
<td style="padding: 4px;">${displayValue}</td>
</tr>`;
}
popupContent += '</table>';
popupContent += '</div>';
```
## Benefits
This enhancement provides several benefits:
1. **More Information**: Users can now see all available information about an event without having to make additional API calls.
2. **Better Debugging**: Developers can more easily debug issues with event data by seeing all properties.
3. **Improved User Experience**: The organized layout and smart formatting make it easier to read and understand the event data.
4. **Transparency**: Users can see exactly what data is stored for each event.
## How to Test
1. Start the server:
```bash
python3 backend.py
```
2. Open a web browser and navigate to:
```
http://127.0.0.1:8080/demo
```
3. Click on any event marker on the map to see the enhanced popup with all properties displayed.
## Future Improvements
Potential future improvements for the popup display:
1. Add filtering options to show only certain categories of properties
2. Add the ability to copy property values to the clipboard
3. Add visualization for temporal data (e.g., timeline for start and stop times)
4. Add the ability to edit event properties directly from the popup
5. Add support for displaying images or other media that might be included in event properties

29
README.md
View file

@ -127,9 +127,34 @@ The OpenEventDatabase API allows you to search for events using various query pa
- Geographic location (`bbox`, `near`, `polyline`)
- And more...
#### Query Parameters
For detailed information about the available query parameters, examples, and response format, see the [API Query Parameters Documentation](API_QUERY_PARAMS.md).
Have a look at the swagger file.
`swagger.json`
#### Search Endpoint
The API includes a dedicated search endpoint (`/event/search`) that allows you to search for events using a GeoJSON geometry in the request body. This is particularly useful for complex spatial queries. For more information, see the [Search Endpoint Documentation](SEARCH_ENDPOINT.md).
#### Anti-Spam and Caching Measures
The API implements caching and rate limiting to improve performance and prevent abuse. For more information about these measures, see the [Anti-Spam and Caching Measures Documentation](ANTI_SPAM_MEASURES.md).
#### Testing
A comprehensive test plan is available for verifying the functionality of the API, including caching, rate limiting, and search features. See the [Test Plan](TEST_PLAN.md) for details.
#### Swagger Documentation
Have a look at the swagger file for a complete API reference:
`swagger.json`
# Todo
page de démo listant les évènements selon leur type, les afficher sur une carte.
créer une page de démo qui permet de modifier un évènement, faire un lien vers cette page quand on ouvre une popup d'évènement sur la page de /demo. et afficher une icone pour les marqueurs de carte selon le type d'évènement, définis en quelques uns et utilise les icones de bulma css.
vérifier le fonctionnement des endpoints de recherche avec les queryparameters, les mettre dans la page de démo.
## License

201
SEARCH_ENDPOINT.md Normal file
View file

@ -0,0 +1,201 @@
# OpenEventDatabase Search Endpoint Documentation
This document describes the `/event/search` endpoint of the OpenEventDatabase API, which allows you to search for events using a GeoJSON geometry.
## Overview
The search endpoint provides a way to search for events using a GeoJSON geometry in the request body. This is particularly useful when you need to search for events within a complex shape that cannot be easily expressed using query parameters like `bbox` or `near`.
## Endpoint
```
POST /event/search
```
## Request Format
The request body should be a JSON object containing a `geometry` field with a valid GeoJSON geometry. The geometry can be any valid GeoJSON geometry type (Point, LineString, Polygon, MultiPoint, MultiLineString, MultiPolygon, or GeometryCollection).
Example request body:
```json
{
"geometry": {
"type": "Polygon",
"coordinates": [
[
[2.3, 48.8],
[2.4, 48.8],
[2.4, 48.9],
[2.3, 48.9],
[2.3, 48.8]
]
]
}
}
```
## Query Parameters
The search endpoint supports all the same query parameters as the `/event` endpoint. These parameters can be used to further refine your search beyond the geometric constraints.
For detailed information about the available query parameters, see the [API Query Parameters Documentation](API_QUERY_PARAMS.md).
### Additional Parameters
In addition to the parameters documented in the API Query Parameters Documentation, the search endpoint also supports the following parameters:
#### `where:osm`
Filters events by OpenStreetMap ID.
- **Format**: String
- **Example**: `where:osm=R12345`
- **Description**: Returns events associated with the specified OpenStreetMap ID.
#### `where:wikidata`
Filters events by Wikidata ID.
- **Format**: String
- **Example**: `where:wikidata=Q90`
- **Description**: Returns events associated with the specified Wikidata ID.
## Response Format
The response format is the same as for the `/event` endpoint. It returns a GeoJSON FeatureCollection containing the events that match the query parameters and intersect with the provided geometry.
Example response:
```json
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"geometry": {
"type": "Point",
"coordinates": [2.35, 48.85]
},
"properties": {
"id": "123e4567-e89b-12d3-a456-426614174000",
"label": "Conference event in Paris",
"what": "event.conference",
"where": "Paris",
"start": "2025-09-10T23:00:00",
"stop": "2026-05-10T23:00:00",
"createdate": "2025-09-15T23:00:00",
"lastupdate": "2025-09-15T23:00:00"
}
}
],
"count": 1
}
```
## Examples
### Search for events within a polygon
```
POST /event/search
Content-Type: application/json
{
"geometry": {
"type": "Polygon",
"coordinates": [
[
[2.3, 48.8],
[2.4, 48.8],
[2.4, 48.9],
[2.3, 48.9],
[2.3, 48.8]
]
]
}
}
```
Returns events located within the specified polygon.
### Search for events within a polygon with additional filters
```
POST /event/search?what=sport.match&when=today
Content-Type: application/json
{
"geometry": {
"type": "Polygon",
"coordinates": [
[
[2.3, 48.8],
[2.4, 48.8],
[2.4, 48.9],
[2.3, 48.9],
[2.3, 48.8]
]
]
}
}
```
Returns sports matches taking place today within the specified polygon.
### Search for events near a point with a buffer
```
POST /event/search?buffer=5000
Content-Type: application/json
{
"geometry": {
"type": "Point",
"coordinates": [2.3522, 48.8566]
}
}
```
Returns events within 5km of the specified point (Paris).
## Caching and Rate Limiting
The OpenEventDatabase API implements caching and rate limiting to improve performance and prevent abuse.
### Caching
- POST requests to `/event/search` are not cached.
- The results of GET requests to `/event` are cached for 60 seconds.
- If you need fresh data, you can bypass the cache by adding a unique query parameter (e.g., `?_=timestamp`).
### Rate Limiting
- POST requests to `/event/search` are limited to 20 requests per minute per IP address.
- If you exceed this limit, you will receive a `429 Too Many Requests` response with a `Retry-After` header indicating when you can try again.
For more information about the anti-spam and caching measures implemented in the API, see the [Anti-Spam and Caching Measures Documentation](ANTI_SPAM_MEASURES.md).
## Error Handling
The search endpoint may return the following error responses:
- `400 Bad Request`: If the request body is not valid JSON or does not contain a `geometry` field.
- `429 Too Many Requests`: If you have exceeded the rate limit.
- `500 Internal Server Error`: If there is an error processing the request.
Error responses include a JSON object with an `error` field containing a description of the error.
Example error response:
```json
{
"error": "Request body must contain a 'geometry' field"
}
```
## Limitations
- Only the first 200 events are returned by default. Use the `limit` parameter to adjust this.
- Complex geometries may result in slower query performance.
- The API may return a 500 Internal Server Error if there are issues with the database connection or if the query is malformed.

216
TEST_PLAN.md Normal file
View file

@ -0,0 +1,216 @@
# Test Plan for OpenEventDatabase Enhancements
This document outlines a test plan for verifying the functionality of the caching, rate limiting, and search features implemented in the OpenEventDatabase API.
## Prerequisites
- The OpenEventDatabase server is running on `http://127.0.0.1:8080`
- The database is properly configured and contains some test events
- You have a tool for making HTTP requests (e.g., curl, Postman, or a web browser)
## 1. Testing Caching Behavior
### 1.1 Test GET /event Caching
1. Make a GET request to `/event`:
```bash
curl -v http://127.0.0.1:8080/event
```
2. Check the response headers for `Cache-Control` header with `max-age=60`
3. Make the same request again within 60 seconds and observe the response time (should be faster)
4. Make the same request with a unique query parameter to bypass the cache:
```bash
curl -v "http://127.0.0.1:8080/event?_=$(date +%s)"
```
5. Observe that the response is not served from cache (response time should be slower)
### 1.2 Test GET /stats Caching
1. Make a GET request to `/stats`:
```bash
curl -v http://127.0.0.1:8080/stats
```
2. Check the response headers for `Cache-Control` header with `max-age=300`
3. Make the same request again within 300 seconds and observe the response time (should be faster)
### 1.3 Test GET /demo Caching
1. Open `http://127.0.0.1:8080/demo` in a web browser
2. Check the network tab in the browser's developer tools for `Cache-Control` header with `max-age=3600`
3. Refresh the page within 3600 seconds and observe that resources are loaded from the browser cache
### 1.4 Test POST /event/search No-Caching
1. Make a POST request to `/event/search`:
```bash
curl -v -X POST -H "Content-Type: application/json" -d '{"geometry":{"type":"Point","coordinates":[2.3522, 48.8566]}}' http://127.0.0.1:8080/event/search
```
2. Check the response headers for `Cache-Control` header with `no-store, no-cache, must-revalidate, max-age=0`
3. Make the same request again and observe that the response is not served from cache (response time should be similar)
### 1.5 Test Error Response No-Caching
1. Make a request that will result in an error (e.g., invalid JSON):
```bash
curl -v -X POST -H "Content-Type: application/json" -d '{invalid json}' http://127.0.0.1:8080/event/search
```
2. Check the response headers for `Cache-Control` header with `no-store, no-cache, must-revalidate, max-age=0`
## 2. Testing Rate Limiting
### 2.1 Test Global Rate Limit
1. Make 61 GET requests to `/event` within 60 seconds:
```bash
for i in {1..61}; do curl -v http://127.0.0.1:8080/event; sleep 1; done
```
2. Observe that the 61st request returns a `429 Too Many Requests` response with a `Retry-After` header
### 2.2 Test POST /event Rate Limit
1. Make 11 POST requests to `/event` within 60 seconds:
```bash
for i in {1..11}; do curl -v -X POST -H "Content-Type: application/json" -d '{"type":"Feature","geometry":{"type":"Point","coordinates":[2.3522, 48.8566]},"properties":{"type":"scheduled","what":"test.event","start":"2025-09-16T00:00:00","stop":"2025-09-16T23:59:59","label":"Test Event"}}' http://127.0.0.1:8080/event; sleep 5; done
```
2. Observe that the 11th request returns a `429 Too Many Requests` response with a `Retry-After` header
### 2.3 Test POST /event/search Rate Limit
1. Make 21 POST requests to `/event/search` within 60 seconds:
```bash
for i in {1..21}; do curl -v -X POST -H "Content-Type: application/json" -d '{"geometry":{"type":"Point","coordinates":[2.3522, 48.8566]}}' http://127.0.0.1:8080/event/search; sleep 2; done
```
2. Observe that the 21st request returns a `429 Too Many Requests` response with a `Retry-After` header
### 2.4 Test DELETE /event Rate Limit
1. Create a test event and note its ID:
```bash
curl -v -X POST -H "Content-Type: application/json" -d '{"type":"Feature","geometry":{"type":"Point","coordinates":[2.3522, 48.8566]},"properties":{"type":"scheduled","what":"test.event","start":"2025-09-16T00:00:00","stop":"2025-09-16T23:59:59","label":"Test Event"}}' http://127.0.0.1:8080/event
```
2. Make 6 DELETE requests to `/event/{id}` within 60 seconds:
```bash
for i in {1..6}; do curl -v -X DELETE http://127.0.0.1:8080/event/{id}; sleep 10; done
```
3. Observe that the 6th request returns a `429 Too Many Requests` response with a `Retry-After` header
## 3. Testing Search Functionality
### 3.1 Test Basic Search with Point Geometry
1. Make a POST request to `/event/search` with a Point geometry:
```bash
curl -v -X POST -H "Content-Type: application/json" -d '{"geometry":{"type":"Point","coordinates":[2.3522, 48.8566]}}' http://127.0.0.1:8080/event/search
```
2. Verify that the response contains events near the specified point
### 3.2 Test Search with Polygon Geometry
1. Make a POST request to `/event/search` with a Polygon geometry:
```bash
curl -v -X POST -H "Content-Type: application/json" -d '{"geometry":{"type":"Polygon","coordinates":[[[2.3, 48.8],[2.4, 48.8],[2.4, 48.9],[2.3, 48.9],[2.3, 48.8]]]}}' http://127.0.0.1:8080/event/search
```
2. Verify that the response contains events within the specified polygon
### 3.3 Test Search with Buffer
1. Make a POST request to `/event/search` with a Point geometry and a buffer:
```bash
curl -v -X POST -H "Content-Type: application/json" -d '{"geometry":{"type":"Point","coordinates":[2.3522, 48.8566]}}' "http://127.0.0.1:8080/event/search?buffer=5000"
```
2. Verify that the response contains events within 5km of the specified point
### 3.4 Test Search with Time Filter
1. Make a POST request to `/event/search` with a Point geometry and a time filter:
```bash
curl -v -X POST -H "Content-Type: application/json" -d '{"geometry":{"type":"Point","coordinates":[2.3522, 48.8566]}}' "http://127.0.0.1:8080/event/search?when=today"
```
2. Verify that the response contains events near the specified point that are active today
### 3.5 Test Search with Category Filter
1. Make a POST request to `/event/search` with a Point geometry and a category filter:
```bash
curl -v -X POST -H "Content-Type: application/json" -d '{"geometry":{"type":"Point","coordinates":[2.3522, 48.8566]}}' "http://127.0.0.1:8080/event/search?what=sport"
```
2. Verify that the response contains events near the specified point with a "what" value that starts with "sport"
### 3.6 Test Search with Type Filter
1. Make a POST request to `/event/search` with a Point geometry and a type filter:
```bash
curl -v -X POST -H "Content-Type: application/json" -d '{"geometry":{"type":"Point","coordinates":[2.3522, 48.8566]}}' "http://127.0.0.1:8080/event/search?type=scheduled"
```
2. Verify that the response contains events near the specified point with a "type" value of "scheduled"
### 3.7 Test Search with Limit
1. Make a POST request to `/event/search` with a Point geometry and a limit:
```bash
curl -v -X POST -H "Content-Type: application/json" -d '{"geometry":{"type":"Point","coordinates":[2.3522, 48.8566]}}' "http://127.0.0.1:8080/event/search?limit=5"
```
2. Verify that the response contains at most 5 events
### 3.8 Test Search with Full Geometry
1. Make a POST request to `/event/search` with a Point geometry and a request for full geometry:
```bash
curl -v -X POST -H "Content-Type: application/json" -d '{"geometry":{"type":"Point","coordinates":[2.3522, 48.8566]}}' "http://127.0.0.1:8080/event/search?geom=full"
```
2. Verify that the response contains events with their full geometry
### 3.9 Test Search with OSM ID
1. Make a POST request to `/event/search` with a Point geometry and an OSM ID filter:
```bash
curl -v -X POST -H "Content-Type: application/json" -d '{"geometry":{"type":"Point","coordinates":[2.3522, 48.8566]}}' "http://127.0.0.1:8080/event/search?where:osm=R12345"
```
2. Verify that the response contains events associated with the specified OSM ID
### 3.10 Test Search with Wikidata ID
1. Make a POST request to `/event/search` with a Point geometry and a Wikidata ID filter:
```bash
curl -v -X POST -H "Content-Type: application/json" -d '{"geometry":{"type":"Point","coordinates":[2.3522, 48.8566]}}' "http://127.0.0.1:8080/event/search?where:wikidata=Q90"
```
2. Verify that the response contains events associated with the specified Wikidata ID
## 4. Testing Error Handling
### 4.1 Test Invalid JSON
1. Make a POST request to `/event/search` with invalid JSON:
```bash
curl -v -X POST -H "Content-Type: application/json" -d '{invalid json}' http://127.0.0.1:8080/event/search
```
2. Verify that the response is a `400 Bad Request` with an error message
### 4.2 Test Missing Geometry
1. Make a POST request to `/event/search` without a geometry field:
```bash
curl -v -X POST -H "Content-Type: application/json" -d '{}' http://127.0.0.1:8080/event/search
```
2. Verify that the response is a `400 Bad Request` with an error message about the missing geometry field
## 5. Testing Demo Page
### 5.1 Test Demo Page with Real Events
1. Open `http://127.0.0.1:8080/demo` in a web browser
2. Verify that the map shows real events from the database
3. Click on an event marker and verify that the popup shows the event details
4. Verify that the map is centered on the events
### 5.2 Test Event Form
1. Open `http://127.0.0.1:8080/demo/add` in a web browser
2. Verify that the form has default values for the date fields (current date) and the map is centered on France
3. Fill out the form with test data and submit it
4. Verify that the event is created successfully
5. Go back to the demo page and verify that the new event is displayed on the map
## Conclusion
This test plan covers the key functionality of the caching, rate limiting, and search features implemented in the OpenEventDatabase API. By following these tests, you can verify that the implementations work as expected and meet the requirements specified in the issue description.

2
backend.py
View file

@ -59,6 +59,8 @@ def create_app():
app.add_route('/stats', stats) # Handle stats requests
app.add_route('/demo', demo) # Handle demo page requests
app.add_route('/demo/add', event_form) # Handle event submission form
app.add_route('/demo/by-what', demo, suffix='by_what') # Handle events by type page
app.add_route('/demo/map-by-what', demo, suffix='map_by_what') # Handle map by event type page
logger.success("Application initialized successfully")
return app

6
extractors/README.md Normal file
View file

@ -0,0 +1,6 @@
# Extractors
permettent de récupérer des évènements depuis de l'open data,
convertis dans le format attendu.
WIP.

749
oedb/resources/demo.py
View file

@ -3,6 +3,9 @@ Demo resource for the OpenEventDatabase.
"""
import falcon
import requests
import json
from collections import defaultdict
from oedb.utils.logging import logger
class DemoResource:
@ -67,6 +70,11 @@ class DemoResource:
<li><a href="/event" target="_blank">/event - Get Events</a></li>
<li><a href="/stats" target="_blank">/stats - Database Statistics</a></li>
</ul>
<h3>Demo Pages:</h3>
<ul>
<li><a href="/demo/by-what" target="_blank">/demo/by-what - Events by Type</a></li>
<li><a href="/demo/map-by-what" target="_blank">/demo/map-by-what - Map by Event Type</a></li>
</ul>
<p><a href="/demo/add" class="add-event-btn" style="display: block; text-align: center; margin-top: 15px; padding: 8px; background-color: #0078ff; color: white; border-radius: 4px; font-weight: bold;">+ Add New Event</a></p>
<p><a href="https://source.cipherbliss.com/tykayn/oedb-backend" target="_blank">Source Code on Cipherbliss</a></p>
</div>
@ -138,15 +146,41 @@ class DemoResource:
// Create popup content
let popupContent = '<div class="event-popup">';
popupContent += `<h3>${properties.label || 'Event'}</h3>`;
popupContent += `<p><strong>Date:</strong> ${properties.when || 'Unknown'}</p>`;
if (properties.what) {
popupContent += `<p><strong>Type:</strong> ${properties.what}</p>`;
// Display all properties
popupContent += '<div style="max-height: 300px; overflow-y: auto;">';
popupContent += '<table style="width: 100%; border-collapse: collapse;">';
// Sort properties alphabetically for better organization
const sortedKeys = Object.keys(properties).sort();
for (const key of sortedKeys) {
// Skip the label as it's already displayed as the title
if (key === 'label') continue;
const value = properties[key];
let displayValue;
// Format the value based on its type
if (value === null || value === undefined) {
displayValue = '<em>null</em>';
} else if (typeof value === 'object') {
displayValue = `<pre style="margin: 0; white-space: pre-wrap;">${JSON.stringify(value, null, 2)}</pre>`;
} else if (typeof value === 'string' && value.startsWith('http')) {
displayValue = `<a href="${value}" target="_blank">${value}</a>`;
} else {
displayValue = String(value);
}
popupContent += `
<tr style="border-bottom: 1px solid #eee;">
<td style="padding: 4px; font-weight: bold; vertical-align: top;">${key}:</td>
<td style="padding: 4px;">${displayValue}</td>
</tr>`;
}
if (properties.source) {
popupContent += `<p><a href="${properties.source}" target="_blank">More Information</a></p>`;
}
popupContent += '</table>';
popupContent += '</div>';
popupContent += '</div>';
@ -197,6 +231,709 @@ class DemoResource:
logger.error(f"Error processing GET request to /demo: {e}")
resp.status = falcon.HTTP_500
resp.text = f"Error: {str(e)}"
def on_get_by_what(self, req, resp):
"""
Handle GET requests to the /demo/by-what endpoint.
Returns an HTML page with links to events organized by their "what" type.
Args:
req: The request object.
resp: The response object.
"""
logger.info("Processing GET request to /demo/by-what")
try:
# Set content type to HTML
resp.content_type = 'text/html'
# Fetch events from the API
response = requests.get('http://localhost/event?limit=1000')
events_data = response.json()
# Group events by "what" type
events_by_what = defaultdict(list)
if events_data.get('features'):
for feature in events_data['features']:
properties = feature.get('properties', {})
what = properties.get('what', 'Unknown')
events_by_what[what].append({
'id': properties.get('id'),
'label': properties.get('label', 'Unnamed Event'),
'coordinates': feature.get('geometry', {}).get('coordinates', [0, 0])
})
# Create HTML response
html = """
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Events by Type - OpenEventDatabase</title>
<style>
body {
font-family: Arial, sans-serif;
line-height: 1.6;
max-width: 1200px;
margin: 0 auto;
padding: 20px;
}
h1 { color: #333; }
h2 {
color: #0078ff;
margin-top: 30px;
padding-bottom: 5px;
border-bottom: 1px solid #eee;
}
ul { padding-left: 20px; }
li { margin-bottom: 8px; }
a { color: #0078ff; text-decoration: none; }
a:hover { text-decoration: underline; }
.nav {
background-color: #f5f5f5;
padding: 10px;
border-radius: 5px;
margin-bottom: 20px;
}
.nav a {
margin-right: 15px;
}
.event-count {
color: #666;
font-size: 0.9em;
}
</style>
</head>
<body>
<div class="nav">
<a href="/">Home</a>
<a href="/demo">Demo Map</a>
<a href="/demo/map-by-what">Map by Event Type</a>
</div>
<h1>Events by Type</h1>
<p>This page lists all events from the OpenEventDatabase organized by their type.</p>
"""
# Add event types and their events
if events_by_what:
# Sort event types alphabetically
sorted_what_types = sorted(events_by_what.keys())
# Add quick navigation
html += "<h2>Quick Navigation</h2><ul>"
for what_type in sorted_what_types:
event_count = len(events_by_what[what_type])
html += f'<li><a href="#what-{what_type.replace(" ", "-")}">{what_type}</a> <span class="event-count">({event_count} events)</span></li>'
html += "</ul>"
# Add sections for each event type
for what_type in sorted_what_types:
events = events_by_what[what_type]
html += f'<h2 id="what-{what_type.replace(" ", "-")}">{what_type} <span class="event-count">({len(events)} events)</span></h2>'
html += "<ul>"
# Sort events by label
sorted_events = sorted(events, key=lambda x: x.get('label', ''))
for event in sorted_events:
event_id = event.get('id')
event_label = event.get('label', 'Unnamed Event')
coordinates = event.get('coordinates', [0, 0])
html += f'<li><a href="/event/{event_id}" target="_blank">{event_label}</a> '
html += f'<small>[<a href="https://www.openstreetmap.org/?mlat={coordinates[1]}&mlon={coordinates[0]}&zoom=15" target="_blank">map</a>]</small></li>'
html += "</ul>"
else:
html += "<p>No events found in the database.</p>"
html += """
</body>
</html>
"""
# Set the response body and status
resp.text = html
resp.status = falcon.HTTP_200
logger.success("Successfully processed GET request to /demo/by-what")
except Exception as e:
logger.error(f"Error processing GET request to /demo/by-what: {e}")
resp.status = falcon.HTTP_500
resp.text = f"Error: {str(e)}"
def on_get_map_by_what(self, req, resp):
"""
Handle GET requests to the /demo/map-by-what endpoint.
Returns an HTML page with a MapLibre map showing events filtered by "what" type.
Args:
req: The request object.
resp: The response object.
"""
logger.info("Processing GET request to /demo/map-by-what")
try:
# Set content type to HTML
resp.content_type = 'text/html'
# Create HTML response with MapLibre map and filtering controls
html = """
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Map by Event Type - OpenEventDatabase</title>
<script src="https://unpkg.com/maplibre-gl@3.0.0/dist/maplibre-gl.js"></script>
<link href="https://unpkg.com/maplibre-gl@3.0.0/dist/maplibre-gl.css" rel="stylesheet" />
<style>
body { margin: 0; padding: 0; font-family: Arial, sans-serif; }
#map { position: absolute; top: 0; bottom: 0; width: 100%; }
.map-overlay {
position: absolute;
top: 10px;
left: 10px;
background: rgba(255, 255, 255, 0.9);
padding: 10px;
border-radius: 5px;
box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
max-width: 300px;
max-height: 90vh;
overflow-y: auto;
}
.filter-overlay {
position: absolute;
top: 10px;
right: 10px;
background: rgba(255, 255, 255, 0.9);
padding: 10px;
border-radius: 5px;
box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
max-width: 300px;
max-height: 90vh;
overflow-y: auto;
}
h2, h3 { margin-top: 0; }
ul { padding-left: 20px; }
a { color: #0078ff; text-decoration: none; }
a:hover { text-decoration: underline; }
.event-popup { max-width: 300px; }
.filter-list {
list-style: none;
padding: 0;
margin: 0;
}
.filter-item {
margin-bottom: 8px;
display: flex;
align-items: center;
}
.filter-item input {
margin-right: 8px;
}
.filter-item label {
cursor: pointer;
flex-grow: 1;
}
.filter-count {
color: #666;
font-size: 0.8em;
margin-left: 5px;
}
.color-dot {
display: inline-block;
width: 12px;
height: 12px;
border-radius: 50%;
margin-right: 5px;
}
.nav {
margin-bottom: 15px;
}
.nav a {
margin-right: 15px;
}
button {
background-color: #0078ff;
color: white;
border: none;
padding: 5px 10px;
border-radius: 3px;
cursor: pointer;
margin-right: 5px;
margin-bottom: 5px;
}
button:hover {
background-color: #0056b3;
}
</style>
</head>
<body>
<div id="map"></div>
<div class="map-overlay">
<h2>Map by Event Type</h2>
<div class="nav">
<a href="/">Home</a>
<a href="/demo">Demo Map</a>
<a href="/demo/by-what">Events by Type</a>
</div>
<p>This map shows events from the OpenEventDatabase filtered by their type.</p>
<p>Use the filter panel on the right to show/hide different event types.</p>
<div id="event-info">
<p>Loading events...</p>
</div>
</div>
<div class="filter-overlay">
<h3>Filter by Event Type</h3>
<div>
<button id="select-all">Select All</button>
<button id="deselect-all">Deselect All</button>
</div>
<ul id="filter-list" class="filter-list">
<li>Loading event types...</li>
</ul>
</div>
<script>
// Initialize the map
const map = new maplibregl.Map({
container: 'map',
style: 'https://demotiles.maplibre.org/style.json',
center: [2.3522, 48.8566], // Default center (Paris)
zoom: 4
});
// Add navigation controls
map.addControl(new maplibregl.NavigationControl());
// Store all events and their types
let allEvents = null;
let eventTypes = new Set();
let eventsByType = {};
let markersByType = {};
let colorsByType = {};
// Generate a color for each event type
function getColorForType(type, index) {
// Predefined colors for better visual distinction
const colors = [
'#FF5722', '#E91E63', '#9C27B0', '#673AB7', '#3F51B5',
'#2196F3', '#03A9F4', '#00BCD4', '#009688', '#4CAF50',
'#8BC34A', '#CDDC39', '#FFEB3B', '#FFC107', '#FF9800'
];
return colors[index % colors.length];
}
// Fetch events when the map is loaded
map.on('load', function() {
fetchEvents();
});
// Function to fetch events from the API
function fetchEvents() {
// Update event info
document.getElementById('event-info').innerHTML = '<p>Loading events...</p>';
// Fetch events from the API - using limit=1000 to get more events
fetch('/event?limit=1000')
.then(response => response.json())
.then(data => {
if (data.features && data.features.length > 0) {
// Store all events
allEvents = data;
// Process events by type
processEventsByType(data);
// Create filter UI
createFilterUI();
// Add all events to the map initially
addAllEventsToMap();
// Fit map to events bounds
fitMapToBounds(data);
// Update event info
document.getElementById('event-info').innerHTML =
`<p>Found ${data.features.length} events across ${eventTypes.size} different types.</p>`;
} else {
document.getElementById('event-info').innerHTML = '<p>No events found.</p>';
document.getElementById('filter-list').innerHTML = '<li>No event types available.</li>';
}
})
.catch(error => {
console.error('Error fetching events:', error);
document.getElementById('event-info').innerHTML =
`<p>Error loading events: ${error.message}</p>`;
});
}
// Process events by their "what" type
function processEventsByType(data) {
eventTypes = new Set();
eventsByType = {};
// Group events by their "what" type
data.features.forEach(feature => {
const properties = feature.properties;
const what = properties.what || 'Unknown';
// Add to set of event types
eventTypes.add(what);
// Add to events by type
if (!eventsByType[what]) {
eventsByType[what] = [];
}
eventsByType[what].push(feature);
});
// Assign colors to each type
let index = 0;
eventTypes.forEach(type => {
colorsByType[type] = getColorForType(type, index);
index++;
});
}
// Create the filter UI
function createFilterUI() {
const filterList = document.getElementById('filter-list');
filterList.innerHTML = '';
// Sort event types alphabetically
const sortedTypes = Array.from(eventTypes).sort();
// Create a checkbox for each event type
sortedTypes.forEach(type => {
const count = eventsByType[type].length;
const color = colorsByType[type];
const li = document.createElement('li');
li.className = 'filter-item';
const checkbox = document.createElement('input');
checkbox.type = 'checkbox';
checkbox.id = `filter-${type}`;
checkbox.checked = true;
checkbox.addEventListener('change', () => {
toggleEventType(type, checkbox.checked);
});
const colorDot = document.createElement('span');
colorDot.className = 'color-dot';
colorDot.style.backgroundColor = color;
const label = document.createElement('label');
label.htmlFor = `filter-${type}`;
label.appendChild(colorDot);
label.appendChild(document.createTextNode(type));
const countSpan = document.createElement('span');
countSpan.className = 'filter-count';
countSpan.textContent = `(${count})`;
label.appendChild(countSpan);
li.appendChild(checkbox);
li.appendChild(label);
filterList.appendChild(li);
});
// Add event listeners for select/deselect all buttons
document.getElementById('select-all').addEventListener('click', selectAllEventTypes);
document.getElementById('deselect-all').addEventListener('click', deselectAllEventTypes);
}
// Add all events to the map
function addAllEventsToMap() {
// Clear existing markers
clearAllMarkers();
// Add markers for each event type
Object.keys(eventsByType).forEach(type => {
addEventsOfTypeToMap(type);
});
}
// Add events of a specific type to the map
function addEventsOfTypeToMap(type) {
if (!markersByType[type]) {
markersByType[type] = [];
}
const events = eventsByType[type];
const color = colorsByType[type];
events.forEach(feature => {
const coordinates = feature.geometry.coordinates.slice();
const properties = feature.properties;
// Create popup content
let popupContent = '<div class="event-popup">';
popupContent += `<h3>${properties.label || 'Event'}</h3>`;
popupContent += `<p><strong>Type:</strong> ${type}</p>`;
// Display all properties
popupContent += '<div style="max-height: 300px; overflow-y: auto;">';
popupContent += '<table style="width: 100%; border-collapse: collapse;">';
// Sort properties alphabetically for better organization
const sortedKeys = Object.keys(properties).sort();
for (const key of sortedKeys) {
// Skip the label as it's already displayed as the title
if (key === 'label') continue;
const value = properties[key];
let displayValue;
// Format the value based on its type
if (value === null || value === undefined) {
displayValue = '<em>null</em>';
} else if (typeof value === 'object') {
displayValue = `<pre style="margin: 0; white-space: pre-wrap;">${JSON.stringify(value, null, 2)}</pre>`;
} else if (typeof value === 'string' && value.startsWith('http')) {
displayValue = `<a href="${value}" target="_blank">${value}</a>`;
} else {
displayValue = String(value);
}
popupContent += `
<tr style="border-bottom: 1px solid #eee;">
<td style="padding: 4px; font-weight: bold; vertical-align: top;">${key}:</td>
<td style="padding: 4px;">${displayValue}</td>
</tr>`;
}
popupContent += '</table>';
popupContent += '</div>';
popupContent += '</div>';
// Create popup
const popup = new maplibregl.Popup({
closeButton: true,
closeOnClick: true
}).setHTML(popupContent);
// Add marker with popup
const marker = new maplibregl.Marker({
color: color
})
.setLngLat(coordinates)
.setPopup(popup)
.addTo(map);
// Store marker reference
markersByType[type].push(marker);
});
}
// Toggle visibility of events by type
function toggleEventType(type, visible) {
if (!markersByType[type]) return;
markersByType[type].forEach(marker => {
if (visible) {
marker.addTo(map);
} else {
marker.remove();
}
});
}
// Select all event types
function selectAllEventTypes() {
const checkboxes = document.querySelectorAll('#filter-list input[type="checkbox"]');
checkboxes.forEach(checkbox => {
checkbox.checked = true;
const type = checkbox.id.replace('filter-', '');
toggleEventType(type, true);
});
}
// Deselect all event types
function deselectAllEventTypes() {
const checkboxes = document.querySelectorAll('#filter-list input[type="checkbox"]');
checkboxes.forEach(checkbox => {
checkbox.checked = false;
const type = checkbox.id.replace('filter-', '');
toggleEventType(type, false);
});
}
// Clear all markers from the map
function clearAllMarkers() {
Object.keys(markersByType).forEach(type => {
if (markersByType[type]) {
markersByType[type].forEach(marker => marker.remove());
}
});
markersByType = {};
}
// Function to fit map to events bounds
function fitMapToBounds(geojson) {
if (geojson.features.length === 0) return;
// Create a bounds object
const bounds = new maplibregl.LngLatBounds();
// Extend bounds with each feature
geojson.features.forEach(feature => {
bounds.extend(feature.geometry.coordinates);
});
// Fit map to bounds with padding
map.fitBounds(bounds, {
padding: 50,
maxZoom: 12
});
}
</script>
</body>
</html>
"""
# Set the response body and status
resp.text = html
resp.status = falcon.HTTP_200
logger.success("Successfully processed GET request to /demo/map-by-what")
except Exception as e:
logger.error(f"Error processing GET request to /demo/map-by-what: {e}")
resp.status = falcon.HTTP_500
resp.text = f"Error: {str(e)}"
events_by_what = defaultdict(list)
if events_data.get('features'):
for feature in events_data['features']:
properties = feature.get('properties', {})
what = properties.get('what', 'Unknown')
events_by_what[what].append({
'id': properties.get('id'),
'label': properties.get('label', 'Unnamed Event'),
'coordinates': feature.get('geometry', {}).get('coordinates', [0, 0])
})
# Create HTML response
html = """
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Events by Type - OpenEventDatabase</title>
<style>
body {
font-family: Arial, sans-serif;
line-height: 1.6;
max-width: 1200px;
margin: 0 auto;
padding: 20px;
}
h1 { color: #333; }
h2 {
color: #0078ff;
margin-top: 30px;
padding-bottom: 5px;
border-bottom: 1px solid #eee;
}
ul { padding-left: 20px; }
li { margin-bottom: 8px; }
a { color: #0078ff; text-decoration: none; }
a:hover { text-decoration: underline; }
.nav {
background-color: #f5f5f5;
padding: 10px;
border-radius: 5px;
margin-bottom: 20px;
}
.nav a {
margin-right: 15px;
}
.event-count {
color: #666;
font-size: 0.9em;
}
</style>
</head>
<body>
<div class="nav">
<a href="/">Home</a>
<a href="/demo">Demo Map</a>
<a href="/demo/map-by-what">Map by Event Type</a>
</div>
<h1>Events by Type</h1>
<p>This page lists all events from the OpenEventDatabase organized by their type.</p>
"""
# Add event types and their events
if events_by_what:
# Sort event types alphabetically
sorted_what_types = sorted(events_by_what.keys())
# Add quick navigation
html += "<h2>Quick Navigation</h2><ul>"
for what_type in sorted_what_types:
event_count = len(events_by_what[what_type])
html += f'<li><a href="#what-{what_type.replace(" ", "-")}">{what_type}</a> <span class="event-count">({event_count} events)</span></li>'
html += "</ul>"
# Add sections for each event type
for what_type in sorted_what_types:
events = events_by_what[what_type]
html += f'<h2 id="what-{what_type.replace(" ", "-")}">{what_type} <span class="event-count">({len(events)} events)</span></h2>'
html += "<ul>"
# Sort events by label
sorted_events = sorted(events, key=lambda x: x.get('label', ''))
for event in sorted_events:
event_id = event.get('id')
event_label = event.get('label', 'Unnamed Event')
coordinates = event.get('coordinates', [0, 0])
html += f'<li><a href="/event/{event_id}" target="_blank">{event_label}</a> '
html += f'<small>[<a href="https://www.openstreetmap.org/?mlat={coordinates[1]}&mlon={coordinates[0]}&zoom=15" target="_blank">map</a>]</small></li>'
html += "</ul>"
else:
html += "<p>No events found in the database.</p>"
html += """
</body>
</html>
"""
# Set the response body and status
resp.text = html
resp.status = falcon.HTTP_200
logger.success("Successfully processed GET request to /demo/by-what")
except Exception as e:
logger.error(f"Error processing GET request to /demo/by-what: {e}")
resp.status = falcon.HTTP_500
resp.text = f"Error: {str(e)}"
# Create a global instance of DemoResource
demo = DemoResource()