Mercurial > hg > Feedworm

annotate feedupdate-main.py @ 108:e50d446f9942 python3

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
updates fro python3
author Dirk Olmes <dirk@xanthippe.ping.de>
date Tue, 29 Mar 2011 03:17:30 +0200
parents d20e99d46d78
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
108
e50d446f9942 updates fro python3
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 69
diff changeset
1 #!/usr/bin/env python3
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
2
34
5813e3c10f14 move the database logic out into its own module. Make everything reload safe so that multiple sessions can be created from interactive sessions
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 14
diff changeset
3 import Database
68
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
4 from datetime import datetime, timedelta
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
5 from Feed import Feed
35
aaec263f07ca Feeds manage the point in time when the next update should happen. FeedUpdater only updates feeds that are due.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 34
diff changeset
6 import FeedUpdater
11
e87c54b3a216 use the logging framework for printing messages
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 6
diff changeset
7 import logging
68
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
8 import Mapping
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
9 from sqlalchemy.sql import and_
108
e50d446f9942 updates fro python3
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 69
diff changeset
10 import sys
34
5813e3c10f14 move the database logic out into its own module. Make everything reload safe so that multiple sessions can be created from interactive sessions
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 14
diff changeset
11 import util
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
12
11
e87c54b3a216 use the logging framework for printing messages
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 6
diff changeset
13 logger = logging.getLogger("feedupdater")
e87c54b3a216 use the logging framework for printing messages
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 6
diff changeset
14
4
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
15 def listFeeds(session):
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
16 allFeeds = session.query(Feed)
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
17 for feed in allFeeds:
11
e87c54b3a216 use the logging framework for printing messages
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 6
diff changeset
18 logger.info("feed: " + feed.name)
4
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
19 for entry in feed.entries:
108
e50d446f9942 updates fro python3
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 69
diff changeset
20 print(entry.title)
4
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
21
68
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
22 def expireFeedEntries(session):
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
23 expireDate = _calculateExpireDate()
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
24 logger.info("expiring entries older than " + str(expireDate))
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
25 feedEntry = Mapping.feedEntryTable
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
26 deleteStatement = feedEntry.delete().where(
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
27 and_(feedEntry.c.create_timestamp < expireDate, feedEntry.c.read == 1)
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
28 )
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
29 deleteStatement.execute()
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
30
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
31 def _calculateExpireDate():
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
32 now = datetime.now()
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
33 delta = timedelta(days=30)
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
34 return now - delta
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
35
108
e50d446f9942 updates fro python3
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 69
diff changeset
36 def checkPythonVersion():
e50d446f9942 updates fro python3
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 69
diff changeset
37 version = sys.version_info
e50d446f9942 updates fro python3
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 69
diff changeset
38 if version[0] < 3:
e50d446f9942 updates fro python3
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 69
diff changeset
39 print("feedupdate-main must run on Python 3")
e50d446f9942 updates fro python3
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 69
diff changeset
40 sys.exit(1)
e50d446f9942 updates fro python3
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 69
diff changeset
41
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
42 if __name__ == "__main__":
108
e50d446f9942 updates fro python3
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 69
diff changeset
43 checkPythonVersion()
34
5813e3c10f14 move the database logic out into its own module. Make everything reload safe so that multiple sessions can be created from interactive sessions
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 14
diff changeset
44 util.configureLogging()
5813e3c10f14 move the database logic out into its own module. Make everything reload safe so that multiple sessions can be created from interactive sessions
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 14
diff changeset
45 session = Database.createSession()
108
e50d446f9942 updates fro python3
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 69
diff changeset
46
35
aaec263f07ca Feeds manage the point in time when the next update should happen. FeedUpdater only updates feeds that are due.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 34
diff changeset
47 #util.loadFeeds(session)
aaec263f07ca Feeds manage the point in time when the next update should happen. FeedUpdater only updates feeds that are due.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 34
diff changeset
48 #util.forceUpdateAllFeeds(session)
6
87317ba41816 add a creation date for the feed entry
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 5
diff changeset
49 #listFeeds(session)
64
d21f5025034d feedupdate-main updates all feeds per default
dirk@xanthippe.ping.de
parents: 35
diff changeset
50 #OpmlImport.createFeedsFromOPML(session, "feed-data/feeds.opml.xml")
69
d20e99d46d78 grr, need to re-enable the actual updating of entries
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 68
diff changeset
51 FeedUpdater.updateAllFeeds(session)
68
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 64
diff changeset
52 expireFeedEntries(session)
108
e50d446f9942 updates fro python3
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 69
diff changeset
53
35
aaec263f07ca Feeds manage the point in time when the next update should happen. FeedUpdater only updates feeds that are due.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 34
diff changeset
54 session.close()