Mercurial > hg > Feedworm

annotate backend/sqlalchemy/SqlAlchemyBackend.py @ 173:3bcf39181f6e

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
pull the calculation of the epxire date up into AbstractBackend
author dirk
date Fri, 09 Sep 2011 17:19:16 +0200
parents f4708d38419c
children 7157bfd3f4d4
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
1
145
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
2 from Feed import Feed
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
3 from FeedEntry import FeedEntry
168
f4708d38419c move methods around so the SqlAlchemyBackend only needs to import the class
dirk
parents: 167
diff changeset
4 from FeedUpdater import FeedUpdater
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
5 from Preferences import Preferences
155
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 154
diff changeset
6 from backend.AbstractBackend import AbstractBackend
145
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
7 from sqlalchemy.orm import joinedload
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
8 from sqlalchemy.sql import and_
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
9 import Database
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
10 import Mapping
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
11 import logging
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
12 import util
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
13
155
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 154
diff changeset
14 class SqlAlchemyBackend(AbstractBackend):
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
15 '''
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
16 Backend that uses sqlalchemy for persistence
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
17 '''
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
18
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
19 def __init__(self):
155
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 154
diff changeset
20 AbstractBackend.__init__(self)
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
21 self._initLogging()
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
22 self.session = Database.createSession()
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
23 self.prefs = Preferences(self.session)
121
510a5d00e98a re-enabled AddFeed - does not work yet
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 119
diff changeset
24 self.updater = None
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
25
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
26 def _initLogging(self):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
27 logging.getLogger("sqlalchemy.orm").setLevel(logging.WARN)
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
28
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
29 sqlalchemyLogLevel = logging.ERROR
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
30 if util.databaseLoggingEnabled():
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
31 sqlalchemyLogLevel = logging.INFO
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
32 logging.getLogger("sqlalchemy").setLevel(sqlalchemyLogLevel)
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
33
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
34 def preferences(self):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
35 return self.prefs
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
36
154
2682db280be0 reorder methods
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 153
diff changeset
37 def dispose(self):
2682db280be0 reorder methods
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 153
diff changeset
38 # save all uncommitted state, just in case
2682db280be0 reorder methods
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 153
diff changeset
39 self.session.commit()
2682db280be0 reorder methods
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 153
diff changeset
40 self.session.close()
2682db280be0 reorder methods
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 153
diff changeset
41
153
65c4bb6d5add move management of the selected feed entry into the backend - sqlachemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 151
diff changeset
42 #
65c4bb6d5add move management of the selected feed entry into the backend - sqlachemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 151
diff changeset
43 # handling of feeds
65c4bb6d5add move management of the selected feed entry into the backend - sqlachemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 151
diff changeset
44 #
65c4bb6d5add move management of the selected feed entry into the backend - sqlachemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 151
diff changeset
45
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
46 def getFeeds(self):
145
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
47 if self.preferences().showOnlyUnreadFeeds():
151
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
48 self.feeds = self._getUnreadFeeds()
145
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
49 else:
151
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
50 self.feeds = Feed.all(self.session)
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
51 return self.feeds
145
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
52
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
53 def _getUnreadFeeds(self):
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
54 query = self.session.query(FeedEntry).filter(FeedEntry.read == 0)
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
55 queryWithOptions = query.options(joinedload("feed"))
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
56 result = queryWithOptions.all()
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
57 return self._collectFeeds(result)
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
58
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
59 def _collectFeeds(self, feedEntries):
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
60 feeds = [entry.feed for entry in feedEntries]
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
61 uniqueFeeds = set(feeds)
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
62 return list(uniqueFeeds)
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
63
155
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 154
diff changeset
64 def _retrieveEntriesForSelectedFeed(self, hideReadEntries):
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 154
diff changeset
65 return self.selectedFeed.entriesSortedByUpdateDate(hideReadEntries)
151
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
66
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
67 def markSelectedFeedAsRead(self):
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
68 self.selectedFeed.markAllEntriesRead()
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
69 self.session.commit()
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
70
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
71 def deleteSelectedFeed(self):
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
72 self.session.delete(self.selectedFeed)
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
73 self.session.commit()
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
74
154
2682db280be0 reorder methods
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 153
diff changeset
75 def createFeed(self, url):
2682db280be0 reorder methods
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 153
diff changeset
76 try:
167
a3c945ce434c adjust the sqlalchemy backend to the changes in AbstractFeedUpdater
dirk
parents: 155
diff changeset
77 newFeed = Feed(url)
a3c945ce434c adjust the sqlalchemy backend to the changes in AbstractFeedUpdater
dirk
parents: 155
diff changeset
78 self.session.add(newFeed)
a3c945ce434c adjust the sqlalchemy backend to the changes in AbstractFeedUpdater
dirk
parents: 155
diff changeset
79
168
f4708d38419c move methods around so the SqlAlchemyBackend only needs to import the class
dirk
parents: 167
diff changeset
80 FeedUpdater(self.preferences(), self.session).update(newFeed)
154
2682db280be0 reorder methods
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 153
diff changeset
81 self.session.commit()
2682db280be0 reorder methods
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 153
diff changeset
82 except AttributeError as ae:
2682db280be0 reorder methods
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 153
diff changeset
83 self.session.rollback()
2682db280be0 reorder methods
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 153
diff changeset
84 raise ae
2682db280be0 reorder methods
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 153
diff changeset
85
2682db280be0 reorder methods
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 153
diff changeset
86 def updateAllFeeds(self):
167
a3c945ce434c adjust the sqlalchemy backend to the changes in AbstractFeedUpdater
dirk
parents: 155
diff changeset
87 FeedUpdater.updateAllFeeds(self.preferences(), self.session)
154
2682db280be0 reorder methods
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 153
diff changeset
88 self.session.commit()
2682db280be0 reorder methods
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 153
diff changeset
89
2682db280be0 reorder methods
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 153
diff changeset
90 def updateFeed(self, feed, changes):
2682db280be0 reorder methods
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 153
diff changeset
91 feed.takeChangesFrom(changes)
2682db280be0 reorder methods
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 153
diff changeset
92 feed.incrementNextUpdateDate()
2682db280be0 reorder methods
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 153
diff changeset
93 self.session.commit()
2682db280be0 reorder methods
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 153
diff changeset
94
153
65c4bb6d5add move management of the selected feed entry into the backend - sqlachemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 151
diff changeset
95 #
65c4bb6d5add move management of the selected feed entry into the backend - sqlachemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 151
diff changeset
96 # handling of the selected feed entry
65c4bb6d5add move management of the selected feed entry into the backend - sqlachemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 151
diff changeset
97 #
155
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 154
diff changeset
98
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 154
diff changeset
99 def _markSelectedFeedEntryRead(self):
153
65c4bb6d5add move management of the selected feed entry into the backend - sqlachemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 151
diff changeset
100 self.selectedFeedEntry.markRead()
65c4bb6d5add move management of the selected feed entry into the backend - sqlachemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 151
diff changeset
101
65c4bb6d5add move management of the selected feed entry into the backend - sqlachemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 151
diff changeset
102 def markFeedEntriesAsRead(self, indices):
65c4bb6d5add move management of the selected feed entry into the backend - sqlachemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 151
diff changeset
103 for index in indices:
65c4bb6d5add move management of the selected feed entry into the backend - sqlachemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 151
diff changeset
104 self.feedEntries[index].markRead()
65c4bb6d5add move management of the selected feed entry into the backend - sqlachemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 151
diff changeset
105 self.session.commit()
65c4bb6d5add move management of the selected feed entry into the backend - sqlachemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 151
diff changeset
106
65c4bb6d5add move management of the selected feed entry into the backend - sqlachemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 151
diff changeset
107 def toggleSelectedFeedEntryRead(self):
65c4bb6d5add move management of the selected feed entry into the backend - sqlachemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 151
diff changeset
108 self.selectedFeedEntry.toggleRead()
65c4bb6d5add move management of the selected feed entry into the backend - sqlachemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 151
diff changeset
109 self.session.commit()
151
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
110
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
111 def expireFeedEntries(self):
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
112 logger = logging.getLogger("feedupdater")
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
113 expireDate = self._calculateExpireDate()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
114 logger.info("expiring entries older than " + str(expireDate))
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
115 feedEntry = Mapping.feedEntryTable
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
116 deleteStatement = feedEntry.delete().where(
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
117 and_(feedEntry.c.create_timestamp < expireDate, feedEntry.c.read == 1)
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
118 )
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
119 deleteStatement.execute()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
120 self.session.commit()