Mercurial > hg > Feedworm

annotate feedupdate-main.py @ 110:43c234c8fe87

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
store the number of days to keep feed entries as preference setting in the database. The feed update process reads and uses that value.
author Dirk Olmes <dirk@xanthippe.ping.de>
date Wed, 30 Mar 2011 02:59:22 +0200
parents bf22b32ae9f7
children 32a173cb081c
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
1 #!/usr/bin/env python
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
2
34
5813e3c10f14 move the database logic out into its own module. Make everything reload safe so that multiple sessions can be created from interactive sessions
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 14
diff changeset
3 import Database
68
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
4 from datetime import datetime, timedelta
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
5 from Feed import Feed
35
aaec263f07ca Feeds manage the point in time when the next update should happen. FeedUpdater only updates feeds that are due.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 34
diff changeset
6 import FeedUpdater
11
e87c54b3a216 use the logging framework for printing messages
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 6
diff changeset
7 import logging
68
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
8 import Mapping
110
43c234c8fe87 store the number of days to keep feed entries as preference setting in the database. The feed update process reads and uses that value.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 109
diff changeset
9 from Preferences import Preferences
68
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
10 from sqlalchemy.sql import and_
34
5813e3c10f14 move the database logic out into its own module. Make everything reload safe so that multiple sessions can be created from interactive sessions
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 14
diff changeset
11 import util
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
12
11
e87c54b3a216 use the logging framework for printing messages
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 6
diff changeset
13 logger = logging.getLogger("feedupdater")
e87c54b3a216 use the logging framework for printing messages
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 6
diff changeset
14
4
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
15 def listFeeds(session):
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
16 allFeeds = session.query(Feed)
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
17 for feed in allFeeds:
11
e87c54b3a216 use the logging framework for printing messages
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 6
diff changeset
18 logger.info("feed: " + feed.name)
4
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
19 for entry in feed.entries:
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
20 print entry.title
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
21
68
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
22 def expireFeedEntries(session):
110
43c234c8fe87 store the number of days to keep feed entries as preference setting in the database. The feed update process reads and uses that value.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 109
diff changeset
23 expireDate = _calculateExpireDate(session)
68
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
24 logger.info("expiring entries older than " + str(expireDate))
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
25 feedEntry = Mapping.feedEntryTable
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
26 deleteStatement = feedEntry.delete().where(
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
27 and_(feedEntry.c.create_timestamp < expireDate, feedEntry.c.read == 1)
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
28 )
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
29 deleteStatement.execute()
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
30
110
43c234c8fe87 store the number of days to keep feed entries as preference setting in the database. The feed update process reads and uses that value.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 109
diff changeset
31 def _calculateExpireDate(session):
68
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
32 now = datetime.now()
110
43c234c8fe87 store the number of days to keep feed entries as preference setting in the database. The feed update process reads and uses that value.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 109
diff changeset
33 daysToKeepFeedEntries = Preferences(session).daysToKeepFeedEntries()
43c234c8fe87 store the number of days to keep feed entries as preference setting in the database. The feed update process reads and uses that value.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 109
diff changeset
34 delta = timedelta(days=daysToKeepFeedEntries)
68
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
35 return now - delta
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
36
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
37 if __name__ == "__main__":
34
5813e3c10f14 move the database logic out into its own module. Make everything reload safe so that multiple sessions can be created from interactive sessions
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 14
diff changeset
38 util.configureLogging()
5813e3c10f14 move the database logic out into its own module. Make everything reload safe so that multiple sessions can be created from interactive sessions
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 14
diff changeset
39 session = Database.createSession()
110
43c234c8fe87 store the number of days to keep feed entries as preference setting in the database. The feed update process reads and uses that value.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 109
diff changeset
40
35
aaec263f07ca Feeds manage the point in time when the next update should happen. FeedUpdater only updates feeds that are due.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 34
diff changeset
41 #util.loadFeeds(session)
aaec263f07ca Feeds manage the point in time when the next update should happen. FeedUpdater only updates feeds that are due.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 34
diff changeset
42 #util.forceUpdateAllFeeds(session)
6
87317ba41816 add a creation date for the feed entry
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 5
diff changeset
43 #listFeeds(session)
64
d21f5025034d feedupdate-main updates all feeds per default
dirk@xanthippe.ping.de
parents: 35
diff changeset
44 #OpmlImport.createFeedsFromOPML(session, "feed-data/feeds.opml.xml")
69
d20e99d46d78 grr, need to re-enable the actual updating of entries
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 68
diff changeset
45 FeedUpdater.updateAllFeeds(session)
68
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
46 expireFeedEntries(session)
110
43c234c8fe87 store the number of days to keep feed entries as preference setting in the database. The feed update process reads and uses that value.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 109
diff changeset
47
35
aaec263f07ca Feeds manage the point in time when the next update should happen. FeedUpdater only updates feeds that are due.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 34
diff changeset
48 session.close()