Mercurial > hg > Feedworm
annotate backend/sqlalchemy/FeedUpdater.py @ 207:c694bfb732bc
implement zooming the web view
author | dirk |
---|---|
date | Wed, 25 Jul 2012 01:49:28 +0200 |
parents | f4708d38419c |
children | bb3c851b18b1 |
rev | line source |
---|---|
4
e0199f383442
retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff
changeset
|
1 |
167
a3c945ce434c
adjust the sqlalchemy backend to the changes in AbstractFeedUpdater
dirk
parents:
160
diff
changeset
|
2 from Feed import Feed |
a3c945ce434c
adjust the sqlalchemy backend to the changes in AbstractFeedUpdater
dirk
parents:
160
diff
changeset
|
3 from FeedEntry import FeedEntry |
141
6ea813cfac33
pull out common code for updating a feed into an abstract class, have the sqlalchemy backend use that class.
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
123
diff
changeset
|
4 from backend.AbstractFeedUpdater import AbstractFeedUpdater, FeedUpdateException |
5
bfd47f55d85b
add the updated date of the feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
4
diff
changeset
|
5 from datetime import datetime |
11
e87c54b3a216
use the logging framework for printing messages
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
10
diff
changeset
|
6 import logging |
4
e0199f383442
retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff
changeset
|
7 |
28
72dfae865899
better logging when updating feeds, handle entries that have no id
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
27
diff
changeset
|
8 log = logging.getLogger("FeedUpdater") |
9
fd4c8bfa62d6
FeedUpdater throws an exception if the URL could not be retrieved successfully. Includes unit tests.
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
7
diff
changeset
|
9 |
168
f4708d38419c
move methods around so the SqlAlchemyBackend only needs to import the class
dirk
parents:
167
diff
changeset
|
10 def _findFeedsToUpdate(session): |
35
aaec263f07ca
Feeds manage the point in time when the next update should happen. FeedUpdater only updates feeds that are due.
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
28
diff
changeset
|
11 return session.query(Feed).filter(Feed.next_update < datetime.now()) |
aaec263f07ca
Feeds manage the point in time when the next update should happen. FeedUpdater only updates feeds that are due.
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
28
diff
changeset
|
12 |
168
f4708d38419c
move methods around so the SqlAlchemyBackend only needs to import the class
dirk
parents:
167
diff
changeset
|
13 class FeedUpdater(AbstractFeedUpdater): |
f4708d38419c
move methods around so the SqlAlchemyBackend only needs to import the class
dirk
parents:
167
diff
changeset
|
14 @staticmethod |
f4708d38419c
move methods around so the SqlAlchemyBackend only needs to import the class
dirk
parents:
167
diff
changeset
|
15 def updateAllFeeds(preferences, session): |
f4708d38419c
move methods around so the SqlAlchemyBackend only needs to import the class
dirk
parents:
167
diff
changeset
|
16 allFeeds = _findFeedsToUpdate(session) |
f4708d38419c
move methods around so the SqlAlchemyBackend only needs to import the class
dirk
parents:
167
diff
changeset
|
17 for feed in allFeeds: |
f4708d38419c
move methods around so the SqlAlchemyBackend only needs to import the class
dirk
parents:
167
diff
changeset
|
18 try: |
f4708d38419c
move methods around so the SqlAlchemyBackend only needs to import the class
dirk
parents:
167
diff
changeset
|
19 FeedUpdater(preferences, session).update(feed) |
f4708d38419c
move methods around so the SqlAlchemyBackend only needs to import the class
dirk
parents:
167
diff
changeset
|
20 except FeedUpdateException, fue: |
f4708d38419c
move methods around so the SqlAlchemyBackend only needs to import the class
dirk
parents:
167
diff
changeset
|
21 log.warn("problems while updating feed " + feed.rss_url + ": " + str(fue)) |
f4708d38419c
move methods around so the SqlAlchemyBackend only needs to import the class
dirk
parents:
167
diff
changeset
|
22 session.commit() |
123
862760b161b4
restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
121
diff
changeset
|
23 |
167
a3c945ce434c
adjust the sqlalchemy backend to the changes in AbstractFeedUpdater
dirk
parents:
160
diff
changeset
|
24 def __init__(self, preferences, session): |
a3c945ce434c
adjust the sqlalchemy backend to the changes in AbstractFeedUpdater
dirk
parents:
160
diff
changeset
|
25 AbstractFeedUpdater.__init__(self, preferences) |
4
e0199f383442
retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff
changeset
|
26 self.session = session |
e0199f383442
retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff
changeset
|
27 |
141
6ea813cfac33
pull out common code for updating a feed into an abstract class, have the sqlalchemy backend use that class.
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
123
diff
changeset
|
28 def _processEntry(self, entry): |
4
e0199f383442
retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff
changeset
|
29 feedEntry = FeedEntry.findById(entry.id, self.session) |
e0199f383442
retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff
changeset
|
30 if feedEntry is None: |
141
6ea813cfac33
pull out common code for updating a feed into an abstract class, have the sqlalchemy backend use that class.
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
123
diff
changeset
|
31 self._createFeedEntry(entry) |
100
99807963d9e0
use the URL as feed title if the feed itself does not come with a title
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
85
diff
changeset
|
32 |
141
6ea813cfac33
pull out common code for updating a feed into an abstract class, have the sqlalchemy backend use that class.
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
123
diff
changeset
|
33 def _createFeedEntry(self, entry): |
62
abc0516a1c0c
FeedEntry provides a static method for creating new entries: better modularization and support for working with the class in interactive mode. FeedUpdater's normalize method is a module function now, again for ease of use in interactive scenarios
dirk@xanthippe.ping.de
parents:
58
diff
changeset
|
34 new = FeedEntry.create(entry) |
5
bfd47f55d85b
add the updated date of the feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
4
diff
changeset
|
35 new.feed = self.feed |
bfd47f55d85b
add the updated date of the feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
4
diff
changeset
|
36 self.session.add(new) |
66 | 37 log.info("new feed entry: " + entry.title) |
144
74217db92993
updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
143
diff
changeset
|
38 |
74217db92993
updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
143
diff
changeset
|
39 def _incrementFeedUpdateDate(self): |
74217db92993
updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
143
diff
changeset
|
40 self.feed.incrementNextUpdateDate() |