Mercurial > hg > Feedworm

annotate backend/sqlalchemy/SqlAlchemyBackend.py @ 150:babe14449162

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
the entries for the selected feeds had to be set onto the item delegate so had access to the selected feed entry. Keep the list in one place only (the model) and access it from the item delegate.
author Dirk Olmes <dirk@xanthippe.ping.de>
date Sat, 27 Aug 2011 06:43:28 +0200
parents 8ec20377bcb0
children bca9341dc67f
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
1
145
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
2 from Feed import Feed
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
3 from FeedEntry import FeedEntry
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
4 from Preferences import Preferences
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
5 from datetime import datetime, timedelta
145
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
6 from sqlalchemy.orm import joinedload
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
7 from sqlalchemy.sql import and_
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
8 import Database
123
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
9 import FeedUpdater
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
10 import Mapping
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
11 import logging
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
12 import util
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
13
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
14 class SqlAlchemyBackend(object):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
15 '''
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
16 Backend that uses sqlalchemy for persistence
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
17 '''
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
18
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
19 def __init__(self):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
20 self._initLogging()
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
21 self.session = Database.createSession()
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
22 self.prefs = Preferences(self.session)
121
510a5d00e98a re-enabled AddFeed - does not work yet
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 119
diff changeset
23 self.updater = None
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
24
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
25 def _initLogging(self):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
26 logging.getLogger("sqlalchemy.orm").setLevel(logging.WARN)
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
27
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
28 sqlalchemyLogLevel = logging.ERROR
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
29 if util.databaseLoggingEnabled():
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
30 sqlalchemyLogLevel = logging.INFO
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
31 logging.getLogger("sqlalchemy").setLevel(sqlalchemyLogLevel)
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
32
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
33 def preferences(self):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
34 return self.prefs
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
35
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
36 def getFeeds(self):
145
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
37 if self.preferences().showOnlyUnreadFeeds():
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
38 return self._getUnreadFeeds()
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
39 else:
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
40 return Feed.all(self.session)
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
41
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
42 def _getUnreadFeeds(self):
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
43 query = self.session.query(FeedEntry).filter(FeedEntry.read == 0)
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
44 queryWithOptions = query.options(joinedload("feed"))
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
45 result = queryWithOptions.all()
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
46 return self._collectFeeds(result)
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
47
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
48 def _collectFeeds(self, feedEntries):
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
49 feeds = [entry.feed for entry in feedEntries]
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
50 uniqueFeeds = set(feeds)
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
51 return list(uniqueFeeds)
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
52
122
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
53 def toggleRead(self, feedEntry):
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
54 feedEntry.toggleRead()
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
55 self.session.commit()
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
56
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
57 def markAllEntriesRead(self, feed):
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
58 feed.markAllEntriesRead()
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
59 self.session.commit()
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
60
123
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
61 def createFeed(self, url):
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
62 try:
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
63 FeedUpdater.createNewFeed(url, self.session)
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
64 self.session.commit()
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
65 except AttributeError as ae:
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
66 self.session.rollback()
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
67 raise ae
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
68
124
a4b2077c9603 editing a feed's properties is implemented through the backend now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 123
diff changeset
69 def updateFeed(self, feed, changes):
a4b2077c9603 editing a feed's properties is implemented through the backend now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 123
diff changeset
70 feed.takeChangesFrom(changes)
a4b2077c9603 editing a feed's properties is implemented through the backend now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 123
diff changeset
71 feed.incrementNextUpdateDate()
a4b2077c9603 editing a feed's properties is implemented through the backend now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 123
diff changeset
72 self.session.commit()
a4b2077c9603 editing a feed's properties is implemented through the backend now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 123
diff changeset
73
125
514e5d7dca98 deleting a feed is now done via the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 124
diff changeset
74 def deleteFeed(self, feed):
514e5d7dca98 deleting a feed is now done via the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 124
diff changeset
75 self.session.delete(feed)
514e5d7dca98 deleting a feed is now done via the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 124
diff changeset
76 self.session.commit()
514e5d7dca98 deleting a feed is now done via the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 124
diff changeset
77
146
8ec20377bcb0 move getting the entries for a feed to the backend so that the couchdb backend can use a custom view for feed entries
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 145
diff changeset
78 def entriesForFeed(self, feed, hideReadEntries):
8ec20377bcb0 move getting the entries for a feed to the backend so that the couchdb backend can use a custom view for feed entries
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 145
diff changeset
79 return feed.entriesSortedByUpdateDate(hideReadEntries)
8ec20377bcb0 move getting the entries for a feed to the backend so that the couchdb backend can use a custom view for feed entries
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 145
diff changeset
80
126
089ee60b28fb mark feed entries as read in the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 125
diff changeset
81 def markFeedEntriesAsRead(self, entries):
089ee60b28fb mark feed entries as read in the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 125
diff changeset
82 for entry in entries:
089ee60b28fb mark feed entries as read in the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 125
diff changeset
83 entry.markRead()
089ee60b28fb mark feed entries as read in the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 125
diff changeset
84 self.session.commit()
089ee60b28fb mark feed entries as read in the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 125
diff changeset
85
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
86 def updateAllFeeds(self):
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
87 FeedUpdater.updateAllFeeds(self.session)
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
88 self.session.commit()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
89
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
90 def expireFeedEntries(self):
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
91 logger = logging.getLogger("feedupdater")
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
92 expireDate = self._calculateExpireDate()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
93 logger.info("expiring entries older than " + str(expireDate))
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
94 feedEntry = Mapping.feedEntryTable
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
95 deleteStatement = feedEntry.delete().where(
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
96 and_(feedEntry.c.create_timestamp < expireDate, feedEntry.c.read == 1)
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
97 )
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
98 deleteStatement.execute()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
99 self.session.commit()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
100
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
101 def _calculateExpireDate(self):
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
102 now = datetime.now()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
103 daysToKeepFeedEntries = self.prefs.daysToKeepFeedEntries()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
104 delta = timedelta(days=daysToKeepFeedEntries)
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
105 return now - delta
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
106
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
107 def dispose(self):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
108 # save all uncommitted state, just in case
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
109 self.session.commit()
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
110 self.session.close()