-
{% endblock %}
\ No newline at end of file
diff --git a/templates/admin/wiki_recent_changes.html.twig b/templates/admin/wiki_recent_changes.html.twig
new file mode 100644
index 0000000..96772b1
--- /dev/null
+++ b/templates/admin/wiki_recent_changes.html.twig
@@ -0,0 +1,87 @@
+{% extends 'base.html.twig' %}
+
+{% block title %}Changements récents Wiki OSM{% endblock %}
+
+{% block body %}
+
+ {% include 'admin/_wiki_navigation.html.twig' %}
+
+
Changements récents Wiki OpenStreetMap
+
Liste des changements récents dans l'espace de noms français du wiki OpenStreetMap.
+
+ {% if last_updated %}
+
+ Dernière mise à jour : {{ last_updated|date('d/m/Y H:i') }}
+
+ {% endif %}
+
+
+
+
+ {% if recent_changes|length > 0 %}
+
+
+
+
+ | Page |
+ Date |
+ Utilisateur |
+ Commentaire |
+ Taille |
+ Actions |
+
+
+
+ {% for change in recent_changes %}
+
+ |
+ {{ change.page_name }}
+ |
+ {{ change.timestamp }} |
+ {{ change.user }} |
+ {{ change.comment }} |
+ {{ change.change_size }} |
+
+
+ Voir
+
+ |
+
+ {% endfor %}
+
+
+
+ {% else %}
+
+
Aucun changement récent n'a été trouvé.
+
+ {% endif %}
+
+
+
+
+
+
+
Cette page affiche les changements récents dans l'espace de noms français (FR:) du wiki OpenStreetMap.
+
Ces informations sont utiles pour suivre les traductions manquantes et les mises à jour des pages wiki.
+
Les données sont mises à jour automatiquement toutes les heures.
+
+
+
+
+
+{% endblock %}
\ No newline at end of file
diff --git a/wiki_compare/CHANGES.md b/wiki_compare/CHANGES.md
new file mode 100644
index 0000000..065c6fd
--- /dev/null
+++ b/wiki_compare/CHANGES.md
@@ -0,0 +1,66 @@
+# Changements implémentés
+
+Ce document résume les changements et nouvelles fonctionnalités implémentés dans le cadre de la mise à jour du système de gestion des pages wiki OSM.
+
+## 1. Suivi des changements récents du wiki OSM
+
+### Fonctionnalités ajoutées
+- Création d'un script `fetch_recent_changes.py` qui récupère les changements récents dans l'espace de noms français du wiki OSM
+- Ajout d'une nouvelle route `/wiki/recent-changes` dans le contrôleur WikiController
+- Création d'un template `wiki_recent_changes.html.twig` pour afficher les changements récents
+- Mise à jour de la navigation pour inclure un lien vers la page des changements récents
+
+### Utilisation
+- Les changements récents sont automatiquement récupérés toutes les heures
+- La page affiche la liste des pages modifiées récemment avec des liens vers ces pages
+
+## 2. Validation de la hiérarchie des titres
+
+### Fonctionnalités ajoutées
+- Implémentation d'une logique de détection des hiérarchies de titres incorrectes (par exemple, h4 directement sous h2 sans h3 intermédiaire)
+- Ajout d'indicateurs visuels (badges) pour signaler les hiérarchies incorrectes dans les listes de sections
+- Mise à jour du template `wiki_compare.html.twig` pour afficher ces indicateurs
+
+### Utilisation
+- Les hiérarchies incorrectes sont automatiquement détectées lors de la comparaison des pages wiki
+- Un badge rouge avec un point d'exclamation est affiché à côté des titres ayant une hiérarchie incorrecte
+
+## 3. Vérification des groupes locaux
+
+### Fonctionnalités ajoutées
+- Mise à jour du script `fetch_osm_fr_groups.py` pour récupérer les données des groupes locaux depuis Framacalc
+- Ajout d'une fonctionnalité de vérification de l'existence d'une page wiki pour chaque groupe
+- Mise à jour du template `wiki_osm_fr_groups.html.twig` pour afficher les résultats de vérification
+- Ajout de filtres pour faciliter la navigation parmi les groupes
+
+### Utilisation
+- Les groupes sont affichés avec des badges indiquant leur source (wiki ou Framacalc)
+- Les groupes sans page wiki sont mis en évidence avec un badge rouge
+- Les filtres permettent de voir uniquement les groupes d'une certaine catégorie (tous, wiki, Framacalc, avec page wiki, sans page wiki)
+
+## Limitations connues
+
+1. **Accès aux données externes** : Les scripts peuvent rencontrer des difficultés pour accéder aux données externes (wiki OSM, Framacalc) en fonction de l'environnement d'exécution.
+
+2. **Détection des hiérarchies** : La détection des hiérarchies incorrectes se base uniquement sur les niveaux des titres et ne prend pas en compte le contenu ou la sémantique.
+
+3. **Correspondance des groupes** : La correspondance entre les groupes Framacalc et les pages wiki se fait par une comparaison approximative des noms, ce qui peut parfois donner des résultats imprécis.
+
+## Maintenance future
+
+### Scripts Python
+- Les scripts Python sont situés dans le répertoire `wiki_compare/`
+- Ils peuvent être exécutés manuellement ou via des tâches cron
+- L'option `--dry-run` permet de tester les scripts sans modifier les fichiers
+- L'option `--force` permet de forcer la mise à jour même si le cache est récent
+
+### Templates Twig
+- Les templates sont situés dans le répertoire `templates/admin/`
+- `wiki_recent_changes.html.twig` : Affichage des changements récents
+- `wiki_compare.html.twig` : Comparaison des pages wiki avec validation de hiérarchie
+- `wiki_osm_fr_groups.html.twig` : Affichage des groupes locaux avec vérification des pages wiki
+
+### Contrôleur
+- Le contrôleur `WikiController.php` contient toutes les routes et la logique de traitement
+- La méthode `detectHeadingHierarchyErrors()` peut être ajustée pour modifier les règles de validation des hiérarchies
+- Les méthodes de rafraîchissement des données (`refreshRecentChangesData()`, etc.) peuvent être modifiées pour ajuster la fréquence de mise à jour
\ No newline at end of file
diff --git a/wiki_compare/fetch_osm_fr_groups.py b/wiki_compare/fetch_osm_fr_groups.py
index 0189730..565cdd3 100755
--- a/wiki_compare/fetch_osm_fr_groups.py
+++ b/wiki_compare/fetch_osm_fr_groups.py
@@ -4,9 +4,11 @@
"""
fetch_osm_fr_groups.py
-This script scrapes the OpenStreetMap wiki page for France/OSM-FR to extract
-information about local working groups. It specifically targets links in the
-#Pages_des_groupes_locaux section.
+This script fetches information about OSM-FR local groups from two sources:
+1. The OpenStreetMap wiki page for France/OSM-FR (specifically the #Pages_des_groupes_locaux section)
+2. The Framacalc spreadsheet at https://framacalc.org/osm-groupes-locaux
+
+It then verifies that each group from the Framacalc has a corresponding wiki page.
Usage:
python fetch_osm_fr_groups.py [--dry-run] [--force]
@@ -24,6 +26,8 @@ import json
import argparse
import logging
import os
+import csv
+import io
from datetime import datetime, timedelta
import requests
from bs4 import BeautifulSoup
@@ -40,6 +44,8 @@ logger = logging.getLogger(__name__)
OUTPUT_FILE = "osm_fr_groups.json"
BASE_URL = "https://wiki.openstreetmap.org/wiki/France/OSM-FR"
WIKI_BASE_URL = "https://wiki.openstreetmap.org"
+FRAMACALC_URL = "https://framacalc.org/osm-groupes-locaux/export/csv"
+WIKI_GROUPS_URL = "https://wiki.openstreetmap.org/wiki/France/OSM-FR#Groupes_locaux"
CACHE_DURATION = timedelta(hours=1) # Cache duration of 1 hour
def is_cache_fresh():
@@ -141,7 +147,7 @@ def extract_working_groups(html_content):
logger.info(f"Found {len(working_groups)} working groups")
return working_groups
-def extract_local_groups(html_content):
+def extract_local_groups_from_wiki(html_content):
"""
Extract local groups from the wiki page HTML
@@ -193,13 +199,170 @@ def extract_local_groups(html_content):
"name": name,
"url": url,
"description": description,
- "type": "local_group"
+ "type": "local_group",
+ "source": "wiki"
})
current = current.next_sibling
- logger.info(f"Found {len(local_groups)} local groups")
+ logger.info(f"Found {len(local_groups)} local groups from wiki")
return local_groups
+def fetch_framacalc_data():
+ """
+ Fetch local groups data from Framacalc
+
+ Returns:
+ list: List of local group dictionaries from Framacalc
+ """
+ try:
+ response = requests.get(FRAMACALC_URL)
+ response.raise_for_status()
+
+ # Parse CSV data
+ csv_data = csv.reader(io.StringIO(response.text))
+ rows = list(csv_data)
+
+ # Check if we have data
+ if len(rows) < 2:
+ logger.warning("No data found in Framacalc CSV")
+ return []
+
+ # Extract headers (first row)
+ headers = rows[0]
+
+ # Find the indices of important columns
+ name_idx = -1
+ contact_idx = -1
+ website_idx = -1
+
+ for i, header in enumerate(headers):
+ header_lower = header.lower()
+ if 'nom' in header_lower or 'groupe' in header_lower:
+ name_idx = i
+ elif 'contact' in header_lower or 'email' in header_lower:
+ contact_idx = i
+ elif 'site' in header_lower or 'web' in header_lower:
+ website_idx = i
+
+ if name_idx == -1:
+ logger.warning("Could not find name column in Framacalc CSV")
+ return []
+
+ # Process data rows
+ local_groups = []
+ for row in rows[1:]: # Skip header row
+ if len(row) <= name_idx or not row[name_idx].strip():
+ continue # Skip empty rows
+
+ name = row[name_idx].strip()
+ contact = row[contact_idx].strip() if contact_idx != -1 and contact_idx < len(row) else ""
+ website = row[website_idx].strip() if website_idx != -1 and website_idx < len(row) else ""
+
+ local_groups.append({
+ "name": name,
+ "contact": contact,
+ "website": website,
+ "type": "local_group",
+ "source": "framacalc",
+ "has_wiki_page": False, # Will be updated later
+ "wiki_url": "" # Will be updated later
+ })
+
+ logger.info(f"Found {len(local_groups)} local groups from Framacalc")
+ return local_groups
+
+ except requests.exceptions.RequestException as e:
+ logger.error(f"Error fetching Framacalc data: {e}")
+ return []
+ except Exception as e:
+ logger.error(f"Error processing Framacalc data: {e}")
+ return []
+
+def extract_wiki_group_links():
+ """
+ Extract links to local group wiki pages from the OSM-FR wiki page
+
+ Returns:
+ dict: Dictionary mapping group names to wiki URLs
+ """
+ try:
+ # Get the wiki page content
+ response = requests.get(WIKI_GROUPS_URL)
+ response.raise_for_status()
+
+ soup = BeautifulSoup(response.text, 'html.parser')
+ wiki_links = {}
+
+ # Find the "Pages des groupes locaux" section
+ pages_section = None
+ for heading in soup.find_all(['h2', 'h3', 'h4']):
+ if 'Pages des groupes locaux' in heading.get_text():
+ pages_section = heading
+ break
+
+ if not pages_section:
+ logger.warning("Could not find 'Pages des groupes locaux' section")
+ return {}
+
+ # Get the content following the heading until the next heading
+ current = pages_section.next_sibling
+ while current and not current.name in ['h2', 'h3', 'h4']:
+ if current.name == 'ul':
+ # Process list items
+ for li in current.find_all('li', recursive=False):
+ text = li.get_text().strip()
+ link = li.find('a')
+
+ if link and text:
+ # Extract group name (before the comma)
+ parts = text.split(',', 1)
+ group_name = parts[0].strip()
+
+ url = WIKI_BASE_URL + link.get('href') if link.get('href').startswith('/') else link.get('href')
+ wiki_links[group_name] = url
+
+ current = current.next_sibling
+
+ logger.info(f"Found {len(wiki_links)} wiki links for local groups")
+ return wiki_links
+
+ except requests.exceptions.RequestException as e:
+ logger.error(f"Error fetching wiki group links: {e}")
+ return {}
+ except Exception as e:
+ logger.error(f"Error processing wiki group links: {e}")
+ return {}
+
+def verify_framacalc_groups_have_wiki(framacalc_groups, wiki_links):
+ """
+ Verify that each group from Framacalc has a corresponding wiki page
+
+ Args:
+ framacalc_groups (list): List of local group dictionaries from Framacalc
+ wiki_links (dict): Dictionary mapping group names to wiki URLs
+
+ Returns:
+ list: Updated list of local group dictionaries with wiki verification
+ """
+ for group in framacalc_groups:
+ group_name = group['name']
+
+ # Try to find a matching wiki link
+ found = False
+ for wiki_name, wiki_url in wiki_links.items():
+ # Check if the group name is similar to the wiki name
+ if group_name.lower() in wiki_name.lower() or wiki_name.lower() in group_name.lower():
+ group['has_wiki_page'] = True
+ group['wiki_url'] = wiki_url
+ found = True
+ break
+
+ if not found:
+ group['has_wiki_page'] = False
+ group['wiki_url'] = ""
+
+ return framacalc_groups
+
def extract_umap_url(html_content):
"""
Extract the uMap URL for OSM-FR local groups
@@ -223,14 +386,16 @@ def extract_umap_url(html_content):
return None
-def save_results(local_groups, working_groups, umap_url, dry_run=False):
+def save_results(wiki_local_groups, framacalc_groups, working_groups, umap_url, wiki_links, dry_run=False):
"""
Save the results to a JSON file
Args:
- local_groups (list): List of local group dictionaries
+ wiki_local_groups (list): List of local group dictionaries from wiki
+ framacalc_groups (list): List of local group dictionaries from Framacalc
working_groups (list): List of working group dictionaries
umap_url (str): URL to the uMap for local groups
+ wiki_links (dict): Dictionary mapping group names to wiki URLs
dry_run (bool): If True, don't actually save to file
Returns:
@@ -238,28 +403,41 @@ def save_results(local_groups, working_groups, umap_url, dry_run=False):
"""
if dry_run:
logger.info("DRY RUN: Would have saved results to file")
- logger.info(f"Local groups: {len(local_groups)}")
- for group in local_groups:
+ logger.info(f"Wiki local groups: {len(wiki_local_groups)}")
+ for group in wiki_local_groups[:5]: # Show only first 5 for brevity
logger.info(f" - {group['name']}: {group['url']}")
+
+ logger.info(f"Framacalc groups: {len(framacalc_groups)}")
+ for group in framacalc_groups[:5]: # Show only first 5 for brevity
+ wiki_status = "Has wiki page" if group.get('has_wiki_page') else "No wiki page"
+ logger.info(f" - {group['name']}: {wiki_status}")
+
logger.info(f"Working groups: {len(working_groups)}")
- for group in working_groups:
+ for group in working_groups[:5]: # Show only first 5 for brevity
logger.info(f" - {group['name']}: {group['url']}")
+
if umap_url:
logger.info(f"uMap URL: {umap_url}")
+
+ logger.info(f"Wiki links: {len(wiki_links)}")
return True
+ # Combine all local groups
+ all_local_groups = wiki_local_groups + framacalc_groups
+
# Prepare the data structure
data = {
"last_updated": datetime.now().isoformat(),
- "local_groups": local_groups,
+ "local_groups": all_local_groups,
"working_groups": working_groups,
- "umap_url": umap_url
+ "umap_url": umap_url,
+ "wiki_links": wiki_links
}
try:
with open(OUTPUT_FILE, 'w', encoding='utf-8') as f:
json.dump(data, f, indent=2, ensure_ascii=False)
- logger.info(f"Successfully saved {len(local_groups)} local groups and {len(working_groups)} working groups to {OUTPUT_FILE}")
+ logger.info(f"Successfully saved {len(all_local_groups)} local groups and {len(working_groups)} working groups to {OUTPUT_FILE}")
return True
except IOError as e:
logger.error(f"Error saving results to {OUTPUT_FILE}: {e}")
@@ -267,7 +445,7 @@ def save_results(local_groups, working_groups, umap_url, dry_run=False):
def main():
"""Main function to execute the script"""
- parser = argparse.ArgumentParser(description="Scrape OSM-FR local groups from the wiki")
+ parser = argparse.ArgumentParser(description="Fetch OSM-FR local groups from wiki and Framacalc")
parser.add_argument("--dry-run", action="store_true", help="Run without saving results to file")
parser.add_argument("--force", action="store_true", help="Force update even if cache is fresh")
args = parser.parse_args()
@@ -287,11 +465,11 @@ def main():
logger.error("Failed to get wiki page content")
return
- # Extract local groups
- local_groups = extract_local_groups(html_content)
+ # Extract local groups from wiki
+ wiki_local_groups = extract_local_groups_from_wiki(html_content)
- if not local_groups:
- logger.warning("No local groups found")
+ if not wiki_local_groups:
+ logger.warning("No local groups found in wiki")
# Extract working groups
working_groups = extract_working_groups(html_content)
@@ -304,8 +482,31 @@ def main():
# Extract uMap URL
umap_url = extract_umap_url(html_content)
+ # Fetch local groups from Framacalc
+ framacalc_groups = fetch_framacalc_data()
+
+ if not framacalc_groups:
+ logger.warning("No local groups found in Framacalc")
+
+ # Extract wiki group links
+ wiki_links = extract_wiki_group_links()
+
+ if not wiki_links:
+ logger.warning("No wiki links found for local groups")
+
+ # Verify Framacalc groups have wiki pages
+ if framacalc_groups and wiki_links:
+ framacalc_groups = verify_framacalc_groups_have_wiki(framacalc_groups, wiki_links)
+
+ # Count groups with and without wiki pages
+ groups_with_wiki = sum(1 for group in framacalc_groups if group.get('has_wiki_page'))
+ groups_without_wiki = sum(1 for group in framacalc_groups if not group.get('has_wiki_page'))
+
+ logger.info(f"Framacalc groups with wiki pages: {groups_with_wiki}")
+ logger.info(f"Framacalc groups without wiki pages: {groups_without_wiki}")
+
# Save results
- success = save_results(local_groups, working_groups, umap_url, args.dry_run)
+ success = save_results(wiki_local_groups, framacalc_groups, working_groups, umap_url, wiki_links, args.dry_run)
if success:
logger.info("Script completed successfully")
diff --git a/wiki_compare/fetch_recent_changes.py b/wiki_compare/fetch_recent_changes.py
new file mode 100644
index 0000000..4147ad7
--- /dev/null
+++ b/wiki_compare/fetch_recent_changes.py
@@ -0,0 +1,216 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+"""
+fetch_recent_changes.py
+
+This script fetches recent changes from the OpenStreetMap wiki for the French namespace
+and stores the URLs of these pages. It specifically targets the recent changes page:
+https://wiki.openstreetmap.org/w/index.php?hidebots=1&hidepreviousrevisions=1&hidecategorization=1&hideWikibase=1&hidelog=1&hidenewuserlog=1&namespace=202&limit=500&days=30&enhanced=1&title=Special:RecentChanges&urlversion=2
+
+Usage:
+ python fetch_recent_changes.py [--dry-run] [--force]
+
+Options:
+ --dry-run Run the script without saving the results to a file
+ --force Force update even if the cache is still fresh (less than 1 hour old)
+
+Output:
+ - recent_changes.json: JSON file with information about recent changes in the French namespace
+ - Log messages about the scraping process and results
+"""
+
+import json
+import argparse
+import logging
+import os
+from datetime import datetime, timedelta
+import requests
+from bs4 import BeautifulSoup
+
+# Configure logging
+logging.basicConfig(
+ level=logging.INFO,
+ format='%(asctime)s - %(levelname)s - %(message)s',
+ datefmt='%Y-%m-%d %H:%M:%S'
+)
+logger = logging.getLogger(__name__)
+
+# Constants
+OUTPUT_FILE = "recent_changes.json"
+RECENT_CHANGES_URL = "https://wiki.openstreetmap.org/w/index.php?hidebots=1&hidepreviousrevisions=1&hidecategorization=1&hideWikibase=1&hidelog=1&hidenewuserlog=1&namespace=202&limit=500&days=30&enhanced=1&title=Special:RecentChanges&urlversion=2"
+WIKI_BASE_URL = "https://wiki.openstreetmap.org"
+CACHE_DURATION = timedelta(hours=1) # Cache duration of 1 hour
+
+def is_cache_fresh():
+ """
+ Check if the cache file exists and is less than CACHE_DURATION old
+
+ Returns:
+ bool: True if cache is fresh, False otherwise
+ """
+ if not os.path.exists(OUTPUT_FILE):
+ return False
+
+ try:
+ with open(OUTPUT_FILE, 'r', encoding='utf-8') as f:
+ data = json.load(f)
+ last_updated = datetime.fromisoformat(data.get('last_updated', '2000-01-01T00:00:00'))
+ now = datetime.now()
+ return (now - last_updated) < CACHE_DURATION
+ except (IOError, json.JSONDecodeError, ValueError) as e:
+ logger.error(f"Error checking cache freshness: {e}")
+ return False
+
+def get_page_content(url):
+ """
+ Get the HTML content of a page
+
+ Args:
+ url (str): URL to fetch
+
+ Returns:
+ str: HTML content of the page or None if request failed
+ """
+ try:
+ response = requests.get(url)
+ response.raise_for_status()
+ return response.text
+ except requests.exceptions.RequestException as e:
+ logger.error(f"Error fetching {url}: {e}")
+ return None
+
+def extract_recent_changes(html_content):
+ """
+ Extract recent changes from the wiki page HTML
+
+ Args:
+ html_content (str): HTML content of the recent changes page
+
+ Returns:
+ list: List of recent change dictionaries
+ """
+ if not html_content:
+ return []
+
+ soup = BeautifulSoup(html_content, 'html.parser')
+ recent_changes = []
+
+ # Find the changes list
+ changes_list = soup.find('ul', class_='special')
+
+ if not changes_list:
+ logger.warning("Could not find recent changes list")
+ return []
+
+ # Process each list item (each change)
+ for li in changes_list.find_all('li'):
+ # Extract the page link
+ page_link = li.find('a', class_='mw-changeslist-title')
+ if not page_link:
+ continue
+
+ page_name = page_link.get_text().strip()
+ page_url = WIKI_BASE_URL + page_link.get('href')
+
+ # Extract the timestamp
+ timestamp_span = li.find('span', class_='mw-changeslist-date')
+ timestamp = timestamp_span.get_text().strip() if timestamp_span else "Unknown"
+
+ # Extract the user
+ user_link = li.find('a', class_='mw-userlink')
+ user = user_link.get_text().strip() if user_link else "Unknown"
+
+ # Extract the comment
+ comment_span = li.find('span', class_='comment')
+ comment = comment_span.get_text().strip() if comment_span else ""
+
+ # Extract the change size
+ change_size_span = li.find('span', class_='mw-changeslist-separator').next_sibling
+ change_size = change_size_span.get_text().strip() if change_size_span else "0"
+
+ recent_changes.append({
+ "page_name": page_name,
+ "page_url": page_url,
+ "timestamp": timestamp,
+ "user": user,
+ "comment": comment,
+ "change_size": change_size
+ })
+
+ logger.info(f"Found {len(recent_changes)} recent changes")
+ return recent_changes
+
+def save_results(recent_changes, dry_run=False):
+ """
+ Save the results to a JSON file
+
+ Args:
+ recent_changes (list): List of recent change dictionaries
+ dry_run (bool): If True, don't actually save to file
+
+ Returns:
+ bool: True if saving was successful or dry run, False otherwise
+ """
+ if dry_run:
+ logger.info("DRY RUN: Would have saved results to file")
+ logger.info(f"Recent changes: {len(recent_changes)}")
+ for change in recent_changes[:5]: # Show only first 5 for brevity
+ logger.info(f" - {change['page_name']}: {change['page_url']} ({change['timestamp']})")
+ if len(recent_changes) > 5:
+ logger.info(f" ... and {len(recent_changes) - 5} more")
+ return True
+
+ # Prepare the data structure
+ data = {
+ "last_updated": datetime.now().isoformat(),
+ "recent_changes": recent_changes
+ }
+
+ try:
+ with open(OUTPUT_FILE, 'w', encoding='utf-8') as f:
+ json.dump(data, f, indent=2, ensure_ascii=False)
+ logger.info(f"Successfully saved {len(recent_changes)} recent changes to {OUTPUT_FILE}")
+ return True
+ except IOError as e:
+ logger.error(f"Error saving results to {OUTPUT_FILE}: {e}")
+ return False
+
+def main():
+ """Main function to execute the script"""
+ parser = argparse.ArgumentParser(description="Fetch recent changes from the OSM wiki French namespace")
+ parser.add_argument("--dry-run", action="store_true", help="Run without saving results to file")
+ parser.add_argument("--force", action="store_true", help="Force update even if cache is fresh")
+ args = parser.parse_args()
+
+ logger.info("Starting fetch_recent_changes.py")
+
+ # Check if cache is fresh
+ if is_cache_fresh() and not args.force:
+ logger.info(f"Cache is still fresh (less than {CACHE_DURATION.total_seconds()/3600} hours old)")
+ logger.info(f"Use --force to update anyway")
+ return
+
+ # Get the recent changes page content
+ html_content = get_page_content(RECENT_CHANGES_URL)
+
+ if not html_content:
+ logger.error("Failed to get recent changes page content")
+ return
+
+ # Extract recent changes
+ recent_changes = extract_recent_changes(html_content)
+
+ if not recent_changes:
+ logger.warning("No recent changes found")
+
+ # Save results
+ success = save_results(recent_changes, args.dry_run)
+
+ if success:
+ logger.info("Script completed successfully")
+ else:
+ logger.error("Script completed with errors")
+
+if __name__ == "__main__":
+ main()
\ No newline at end of file