Mercurial > hg > Feedworm

annotate backend/sqlalchemy/SqlAlchemyBackend.py @ 134:40163eac0d27

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
rename feedupdate-main to feedupdate, adjust the launch script
author Dirk Olmes <dirk@xanthippe.ping.de>
date Tue, 23 Aug 2011 03:49:29 +0200
parents 32a173cb081c
children 71c5dc02ff87
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
1
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
2 from Preferences import Preferences
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
3 from datetime import datetime, timedelta
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
4 from sqlalchemy.sql import and_
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
5 import Database
123
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
6 import FeedList
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
7 import FeedUpdater
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
8 import Mapping
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
9 import logging
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
10 import util
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
11
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
12 class SqlAlchemyBackend(object):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
13 '''
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
14 Backend that uses sqlalchemy for persistence
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
15 '''
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
16
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
17 def __init__(self):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
18 self._initLogging()
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
19 self.session = Database.createSession()
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
20 self.prefs = Preferences(self.session)
121
510a5d00e98a re-enabled AddFeed - does not work yet
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 119
diff changeset
21 self.updater = None
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
22
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
23 def _initLogging(self):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
24 logging.getLogger("sqlalchemy.orm").setLevel(logging.WARN)
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
25
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
26 sqlalchemyLogLevel = logging.ERROR
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
27 if util.databaseLoggingEnabled():
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
28 sqlalchemyLogLevel = logging.INFO
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
29 logging.getLogger("sqlalchemy").setLevel(sqlalchemyLogLevel)
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
30
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
31 def preferences(self):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
32 return self.prefs
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
33
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
34 def getFeeds(self):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
35 return FeedList.getFeeds(self.session)
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
36
122
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
37 def toggleRead(self, feedEntry):
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
38 feedEntry.toggleRead()
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
39 self.session.commit()
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
40
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
41 def markAllEntriesRead(self, feed):
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
42 feed.markAllEntriesRead()
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
43 self.session.commit()
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
44
123
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
45 def createFeed(self, url):
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
46 try:
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
47 FeedUpdater.createNewFeed(url, self.session)
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
48 self.session.commit()
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
49 except AttributeError as ae:
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
50 self.session.rollback()
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
51 raise ae
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
52
124
a4b2077c9603 editing a feed's properties is implemented through the backend now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 123
diff changeset
53 def updateFeed(self, feed, changes):
a4b2077c9603 editing a feed's properties is implemented through the backend now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 123
diff changeset
54 feed.takeChangesFrom(changes)
a4b2077c9603 editing a feed's properties is implemented through the backend now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 123
diff changeset
55 feed.incrementNextUpdateDate()
a4b2077c9603 editing a feed's properties is implemented through the backend now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 123
diff changeset
56 self.session.commit()
a4b2077c9603 editing a feed's properties is implemented through the backend now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 123
diff changeset
57
125
514e5d7dca98 deleting a feed is now done via the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 124
diff changeset
58 def deleteFeed(self, feed):
514e5d7dca98 deleting a feed is now done via the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 124
diff changeset
59 self.session.delete(feed)
514e5d7dca98 deleting a feed is now done via the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 124
diff changeset
60 self.session.commit()
514e5d7dca98 deleting a feed is now done via the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 124
diff changeset
61
126
089ee60b28fb mark feed entries as read in the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 125
diff changeset
62 def markFeedEntriesAsRead(self, entries):
089ee60b28fb mark feed entries as read in the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 125
diff changeset
63 for entry in entries:
089ee60b28fb mark feed entries as read in the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 125
diff changeset
64 entry.markRead()
089ee60b28fb mark feed entries as read in the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 125
diff changeset
65 self.session.commit()
089ee60b28fb mark feed entries as read in the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 125
diff changeset
66
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
67 def updateAllFeeds(self):
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
68 FeedUpdater.updateAllFeeds(self.session)
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
69 self.session.commit()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
70
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
71 def expireFeedEntries(self):
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
72 logger = logging.getLogger("feedupdater")
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
73 expireDate = self._calculateExpireDate()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
74 logger.info("expiring entries older than " + str(expireDate))
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
75 feedEntry = Mapping.feedEntryTable
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
76 deleteStatement = feedEntry.delete().where(
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
77 and_(feedEntry.c.create_timestamp < expireDate, feedEntry.c.read == 1)
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
78 )
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
79 deleteStatement.execute()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
80 self.session.commit()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
81
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
82 def _calculateExpireDate(self):
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
83 now = datetime.now()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
84 daysToKeepFeedEntries = self.prefs.daysToKeepFeedEntries()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
85 delta = timedelta(days=daysToKeepFeedEntries)
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
86 return now - delta
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
87
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
88 def dispose(self):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
89 # save all uncommitted state, just in case
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
90 self.session.commit()
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
91 self.session.close()