Mercurial > hg > Feedworm

annotate backend/sqlalchemy/SqlAlchemyBackend.py @ 146:8ec20377bcb0

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
move getting the entries for a feed to the backend so that the couchdb backend can use a custom view for feed entries
author Dirk Olmes <dirk@xanthippe.ping.de>
date Thu, 25 Aug 2011 07:01:45 +0200
parents 71c5dc02ff87
children bca9341dc67f
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
1
145
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
2 from Feed import Feed
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
3 from FeedEntry import FeedEntry
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
4 from Preferences import Preferences
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
5 from datetime import datetime, timedelta
145
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
6 from sqlalchemy.orm import joinedload
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
7 from sqlalchemy.sql import and_
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
8 import Database
123
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
9 import FeedUpdater
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
10 import Mapping
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
11 import logging
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
12 import util
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
13
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
14 class SqlAlchemyBackend(object):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
15 '''
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
16 Backend that uses sqlalchemy for persistence
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
17 '''
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
18
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
19 def __init__(self):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
20 self._initLogging()
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
21 self.session = Database.createSession()
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
22 self.prefs = Preferences(self.session)
121
510a5d00e98a re-enabled AddFeed - does not work yet
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 119
diff changeset
23 self.updater = None
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
24
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
25 def _initLogging(self):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
26 logging.getLogger("sqlalchemy.orm").setLevel(logging.WARN)
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
27
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
28 sqlalchemyLogLevel = logging.ERROR
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
29 if util.databaseLoggingEnabled():
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
30 sqlalchemyLogLevel = logging.INFO
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
31 logging.getLogger("sqlalchemy").setLevel(sqlalchemyLogLevel)
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
32
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
33 def preferences(self):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
34 return self.prefs
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
35
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
36 def getFeeds(self):
145
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
37 if self.preferences().showOnlyUnreadFeeds():
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
38 return self._getUnreadFeeds()
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
39 else:
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
40 return Feed.all(self.session)
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
41
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
42 def _getUnreadFeeds(self):
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
43 query = self.session.query(FeedEntry).filter(FeedEntry.read == 0)
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
44 queryWithOptions = query.options(joinedload("feed"))
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
45 result = queryWithOptions.all()
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
46 return self._collectFeeds(result)
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
47
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
48 def _collectFeeds(self, feedEntries):
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
49 feeds = [entry.feed for entry in feedEntries]
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
50 uniqueFeeds = set(feeds)
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
51 return list(uniqueFeeds)
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
52
122
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
53 def toggleRead(self, feedEntry):
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
54 feedEntry.toggleRead()
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
55 self.session.commit()
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
56
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
57 def markAllEntriesRead(self, feed):
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
58 feed.markAllEntriesRead()
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
59 self.session.commit()
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
60
123
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
61 def createFeed(self, url):
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
62 try:
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
63 FeedUpdater.createNewFeed(url, self.session)
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
64 self.session.commit()
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
65 except AttributeError as ae:
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
66 self.session.rollback()
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
67 raise ae
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
68
124
a4b2077c9603 editing a feed's properties is implemented through the backend now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 123
diff changeset
69 def updateFeed(self, feed, changes):
a4b2077c9603 editing a feed's properties is implemented through the backend now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 123
diff changeset
70 feed.takeChangesFrom(changes)
a4b2077c9603 editing a feed's properties is implemented through the backend now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 123
diff changeset
71 feed.incrementNextUpdateDate()
a4b2077c9603 editing a feed's properties is implemented through the backend now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 123
diff changeset
72 self.session.commit()
a4b2077c9603 editing a feed's properties is implemented through the backend now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 123
diff changeset
73
125
514e5d7dca98 deleting a feed is now done via the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 124
diff changeset
74 def deleteFeed(self, feed):
514e5d7dca98 deleting a feed is now done via the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 124
diff changeset
75 self.session.delete(feed)
514e5d7dca98 deleting a feed is now done via the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 124
diff changeset
76 self.session.commit()
514e5d7dca98 deleting a feed is now done via the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 124
diff changeset
77
146
8ec20377bcb0 move getting the entries for a feed to the backend so that the couchdb backend can use a custom view for feed entries
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 145
diff changeset
78 def entriesForFeed(self, feed, hideReadEntries):
8ec20377bcb0 move getting the entries for a feed to the backend so that the couchdb backend can use a custom view for feed entries
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 145
diff changeset
79 return feed.entriesSortedByUpdateDate(hideReadEntries)
8ec20377bcb0 move getting the entries for a feed to the backend so that the couchdb backend can use a custom view for feed entries
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 145
diff changeset
80
126
089ee60b28fb mark feed entries as read in the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 125
diff changeset
81 def markFeedEntriesAsRead(self, entries):
089ee60b28fb mark feed entries as read in the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 125
diff changeset
82 for entry in entries:
089ee60b28fb mark feed entries as read in the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 125
diff changeset
83 entry.markRead()
089ee60b28fb mark feed entries as read in the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 125
diff changeset
84 self.session.commit()
089ee60b28fb mark feed entries as read in the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 125
diff changeset
85
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
86 def updateAllFeeds(self):
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
87 FeedUpdater.updateAllFeeds(self.session)
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
88 self.session.commit()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
89
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
90 def expireFeedEntries(self):
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
91 logger = logging.getLogger("feedupdater")
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
92 expireDate = self._calculateExpireDate()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
93 logger.info("expiring entries older than " + str(expireDate))
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
94 feedEntry = Mapping.feedEntryTable
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
95 deleteStatement = feedEntry.delete().where(
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
96 and_(feedEntry.c.create_timestamp < expireDate, feedEntry.c.read == 1)
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
97 )
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
98 deleteStatement.execute()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
99 self.session.commit()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
100
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
101 def _calculateExpireDate(self):
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
102 now = datetime.now()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
103 daysToKeepFeedEntries = self.prefs.daysToKeepFeedEntries()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
104 delta = timedelta(days=daysToKeepFeedEntries)
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
105 return now - delta
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
106
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
107 def dispose(self):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
108 # save all uncommitted state, just in case
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
109 self.session.commit()
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
110 self.session.close()