Mercurial > hg > Feedworm

annotate backend/sqlalchemy/SqlAlchemyBackend.py @ 151:bca9341dc67f

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
author Dirk Olmes <dirk@xanthippe.ping.de>
date Sat, 27 Aug 2011 07:07:50 +0200
parents 8ec20377bcb0
children 65c4bb6d5add
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
1
145
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
2 from Feed import Feed
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
3 from FeedEntry import FeedEntry
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
4 from Preferences import Preferences
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
5 from datetime import datetime, timedelta
145
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
6 from sqlalchemy.orm import joinedload
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
7 from sqlalchemy.sql import and_
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
8 import Database
123
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
9 import FeedUpdater
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
10 import Mapping
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
11 import logging
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
12 import util
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
13
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
14 class SqlAlchemyBackend(object):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
15 '''
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
16 Backend that uses sqlalchemy for persistence
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
17 '''
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
18
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
19 def __init__(self):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
20 self._initLogging()
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
21 self.session = Database.createSession()
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
22 self.prefs = Preferences(self.session)
121
510a5d00e98a re-enabled AddFeed - does not work yet
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 119
diff changeset
23 self.updater = None
151
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
24 self.feeds = []
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
25 self.selectedFeed = None
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
26
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
27 def _initLogging(self):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
28 logging.getLogger("sqlalchemy.orm").setLevel(logging.WARN)
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
29
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
30 sqlalchemyLogLevel = logging.ERROR
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
31 if util.databaseLoggingEnabled():
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
32 sqlalchemyLogLevel = logging.INFO
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
33 logging.getLogger("sqlalchemy").setLevel(sqlalchemyLogLevel)
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
34
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
35 def preferences(self):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
36 return self.prefs
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
37
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
38 def getFeeds(self):
145
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
39 if self.preferences().showOnlyUnreadFeeds():
151
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
40 self.feeds = self._getUnreadFeeds()
145
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
41 else:
151
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
42 self.feeds = Feed.all(self.session)
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
43 return self.feeds
145
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
44
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
45 def _getUnreadFeeds(self):
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
46 query = self.session.query(FeedEntry).filter(FeedEntry.read == 0)
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
47 queryWithOptions = query.options(joinedload("feed"))
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
48 result = queryWithOptions.all()
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
49 return self._collectFeeds(result)
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
50
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
51 def _collectFeeds(self, feedEntries):
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
52 feeds = [entry.feed for entry in feedEntries]
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
53 uniqueFeeds = set(feeds)
71c5dc02ff87 move the code from FeedList into the backend class
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 128
diff changeset
54 return list(uniqueFeeds)
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
55
151
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
56 def selectFeed(self, index):
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
57 self.selectedFeed = self.feeds[index]
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
58 return self.selectedFeed
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
59
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
60 def entriesForSelectedFeed(self, hideReadEntries):
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
61 return self.selectedFeed.entriesSortedByUpdateDate(hideReadEntries)
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
62
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
63 def markSelectedFeedAsRead(self):
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
64 self.selectedFeed.markAllEntriesRead()
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
65 self.session.commit()
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
66
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
67 def deleteSelectedFeed(self):
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
68 self.session.delete(self.selectedFeed)
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
69 self.session.commit()
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
70
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
71
bca9341dc67f move the selected feed into the backend - sqlalchemy backend works, couchdb backend currently broken
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
72
122
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
73 def toggleRead(self, feedEntry):
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
74 feedEntry.toggleRead()
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
75 self.session.commit()
f5afe0c1f4d2 move more operations to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 121
diff changeset
76
123
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
77 def createFeed(self, url):
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
78 try:
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
79 FeedUpdater.createNewFeed(url, self.session)
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
80 self.session.commit()
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
81 except AttributeError as ae:
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
82 self.session.rollback()
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
83 raise ae
862760b161b4 restructured adding a feed so that only the URL is passed into the backend - the rest of the operation is backend-internal
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 122
diff changeset
84
124
a4b2077c9603 editing a feed's properties is implemented through the backend now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 123
diff changeset
85 def updateFeed(self, feed, changes):
a4b2077c9603 editing a feed's properties is implemented through the backend now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 123
diff changeset
86 feed.takeChangesFrom(changes)
a4b2077c9603 editing a feed's properties is implemented through the backend now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 123
diff changeset
87 feed.incrementNextUpdateDate()
a4b2077c9603 editing a feed's properties is implemented through the backend now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 123
diff changeset
88 self.session.commit()
a4b2077c9603 editing a feed's properties is implemented through the backend now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 123
diff changeset
89
146
8ec20377bcb0 move getting the entries for a feed to the backend so that the couchdb backend can use a custom view for feed entries
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 145
diff changeset
90
126
089ee60b28fb mark feed entries as read in the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 125
diff changeset
91 def markFeedEntriesAsRead(self, entries):
089ee60b28fb mark feed entries as read in the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 125
diff changeset
92 for entry in entries:
089ee60b28fb mark feed entries as read in the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 125
diff changeset
93 entry.markRead()
089ee60b28fb mark feed entries as read in the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 125
diff changeset
94 self.session.commit()
089ee60b28fb mark feed entries as read in the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 125
diff changeset
95
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
96 def updateAllFeeds(self):
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
97 FeedUpdater.updateAllFeeds(self.session)
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
98 self.session.commit()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
99
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
100 def expireFeedEntries(self):
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
101 logger = logging.getLogger("feedupdater")
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
102 expireDate = self._calculateExpireDate()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
103 logger.info("expiring entries older than " + str(expireDate))
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
104 feedEntry = Mapping.feedEntryTable
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
105 deleteStatement = feedEntry.delete().where(
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
106 and_(feedEntry.c.create_timestamp < expireDate, feedEntry.c.read == 1)
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
107 )
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
108 deleteStatement.execute()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
109 self.session.commit()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
110
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
111 def _calculateExpireDate(self):
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
112 now = datetime.now()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
113 daysToKeepFeedEntries = self.prefs.daysToKeepFeedEntries()
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
114 delta = timedelta(days=daysToKeepFeedEntries)
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
115 return now - delta
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
116
119
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
117 def dispose(self):
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
118 # save all uncommitted state, just in case
04a730f9d07d move all sqlalchemy related classes to the respective sub-package. use a backend to abstract from access to the data
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
119 self.session.commit()
128
32a173cb081c move updating the feeds to the backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 126
diff changeset
120 self.session.close()