Mercurial > hg > Feedworm

annotate backend/couchdb/CouchDbBackend.py @ 174:d0ced79b5030

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
implement expiring read feed entries
author dirk
date Fri, 09 Sep 2011 17:21:34 +0200
parents 91a24f499318
children 57e324fa4350
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
132
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
1
144
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 139
diff changeset
2 from FeedUpdater import FeedUpdater
133
9e1e6b96d8b0 implement proxyHost/proxyPort in Preferences
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 132
diff changeset
3 from Preferences import Preferences
155
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
4 from backend.AbstractBackend import AbstractBackend
174
d0ced79b5030 implement expiring read feed entries
dirk
parents: 169
diff changeset
5 from backend.couchdb import CouchDb
139
2cd30af937fa add the required methods for determining if a feed needs to be updated
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 133
diff changeset
6 from backend.couchdb.Feed import Feed
146
8ec20377bcb0 move getting the entries for a feed to the backend so that the couchdb backend can use a custom view for feed entries
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
7 from backend.couchdb.FeedEntry import FeedEntry
132
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
8 import couchdb
174
d0ced79b5030 implement expiring read feed entries
dirk
parents: 169
diff changeset
9 import logging
133
9e1e6b96d8b0 implement proxyHost/proxyPort in Preferences
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 132
diff changeset
10
155
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
11 class CouchDbBackend(AbstractBackend):
132
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
12 '''
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
13 Backend that uses CouchDB for persistence
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
14 '''
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
15
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
16 def __init__(self):
169
91a24f499318 introdue an abstraction for the name of the database so it can be changed via commandline parameter
dirk
parents: 166
diff changeset
17 CouchDb.init()
132
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
18 server = couchdb.Server()
169
91a24f499318 introdue an abstraction for the name of the database so it can be changed via commandline parameter
dirk
parents: 166
diff changeset
19 self.database = server[CouchDb.database]
158
e8bb107a74e1 all preferences are stored in a single JSON document in the couchdb backend. PreferencesDialog converts QString to a proper Python datatybe before sending it to the backend.
dirk
parents: 155
diff changeset
20 self.prefs = None
132
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
21
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
22 def preferences(self):
158
e8bb107a74e1 all preferences are stored in a single JSON document in the couchdb backend. PreferencesDialog converts QString to a proper Python datatybe before sending it to the backend.
dirk
parents: 155
diff changeset
23 if self.prefs is None:
e8bb107a74e1 all preferences are stored in a single JSON document in the couchdb backend. PreferencesDialog converts QString to a proper Python datatybe before sending it to the backend.
dirk
parents: 155
diff changeset
24 self.prefs = Preferences(self.database)
e8bb107a74e1 all preferences are stored in a single JSON document in the couchdb backend. PreferencesDialog converts QString to a proper Python datatybe before sending it to the backend.
dirk
parents: 155
diff changeset
25 return self.prefs
132
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
26
155
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
27 #
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
28 # handling of feeds
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
29 #
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
30
132
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
31 def getFeeds(self):
146
8ec20377bcb0 move getting the entries for a feed to the backend so that the couchdb backend can use a custom view for feed entries
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
32 if self.preferences().showOnlyUnreadFeeds():
155
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
33 self.feeds = self._getUnreadFeeds()
146
8ec20377bcb0 move getting the entries for a feed to the backend so that the couchdb backend can use a custom view for feed entries
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
34 else:
148
c5a427d46703 displaying entries for a feed works now with the couchdb backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
35 # make sure that the results are actually fetched into memory, otherwise we'll pass
c5a427d46703 displaying entries for a feed works now with the couchdb backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
36 # a ViewResults instance around which is not what we want
155
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
37 self.feeds = list(Feed.all(self.database))
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
38 return self.feeds
146
8ec20377bcb0 move getting the entries for a feed to the backend so that the couchdb backend can use a custom view for feed entries
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
39
8ec20377bcb0 move getting the entries for a feed to the backend so that the couchdb backend can use a custom view for feed entries
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
40 def _getUnreadFeeds(self):
132
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
41 raise Exception("not yet implemented")
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
42
155
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
43 def _retrieveEntriesForSelectedFeed(self, hideReadEntries):
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
44 # TODO how to hide read entries if requested?
169
91a24f499318 introdue an abstraction for the name of the database so it can be changed via commandline parameter
dirk
parents: 166
diff changeset
45 viewResults = FeedEntry.view(self.database, CouchDb.feedEntriesByFeed(),
155
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
46 key=self.selectedFeed.id)
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
47 return list(viewResults)
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
48
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
49 def markSelectedFeedAsRead(self):
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
50 for feedEntry in self.entriesForSelectedFeed():
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
51 feedEntry.markRead(self.database)
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
52
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
53 #
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
54 # handling of the selected feed entry
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
55 #
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
56
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
57 def _markSelectedFeedEntryRead(self):
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
58 self.selectedFeedEntry.markRead(self.database)
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
59
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
60 def toggleSelectedFeedEntryRead(self):
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
61 self.selectedFeedEntry.toggleRead(self.database)
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
62
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
63
132
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
64 def toggleRead(self, feedEntry):
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
65 raise Exception("not yet implemented")
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
66
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
67 def markAllEntriesRead(self, feed):
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
68 raise Exception("not yet implemented")
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
69
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
70 def createFeed(self, url):
166
04c3b9796b89 feedparser uses the proxy now if one is configured. To implement this the FeedUpdater had to change a bit - sqlalchemy backend is not yet refactored.
dirk
parents: 164
diff changeset
71 feed = Feed.create(url)
04c3b9796b89 feedparser uses the proxy now if one is configured. To implement this the FeedUpdater had to change a bit - sqlalchemy backend is not yet refactored.
dirk
parents: 164
diff changeset
72 feed.store(self.database)
04c3b9796b89 feedparser uses the proxy now if one is configured. To implement this the FeedUpdater had to change a bit - sqlalchemy backend is not yet refactored.
dirk
parents: 164
diff changeset
73 FeedUpdater(self.database, self.preferences()).update(feed)
132
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
74
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
75 def updateFeed(self, feed, changes):
162
ab2b26412b77 setting a feed's properties via the GUI works now
dirk
parents: 161
diff changeset
76 for key in changes.keys():
ab2b26412b77 setting a feed's properties via the GUI works now
dirk
parents: 161
diff changeset
77 feed[key] = changes[key]
ab2b26412b77 setting a feed's properties via the GUI works now
dirk
parents: 161
diff changeset
78 feed.store(self.database)
132
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
79
164
3eeda7cec39b delete a feed via the GUI works now
dirk
parents: 162
diff changeset
80 def deleteSelectedFeed(self):
169
91a24f499318 introdue an abstraction for the name of the database so it can be changed via commandline parameter
dirk
parents: 166
diff changeset
81 viewResults = self.database.view(CouchDb.feedEntriesByFeed(), key=self.selectedFeed.id)
164
3eeda7cec39b delete a feed via the GUI works now
dirk
parents: 162
diff changeset
82 for row in viewResults:
3eeda7cec39b delete a feed via the GUI works now
dirk
parents: 162
diff changeset
83 del self.database[row.id]
3eeda7cec39b delete a feed via the GUI works now
dirk
parents: 162
diff changeset
84 del self.database[self.selectedFeed.id]
132
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
85
146
8ec20377bcb0 move getting the entries for a feed to the backend so that the couchdb backend can use a custom view for feed entries
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
86 def entriesForFeed(self, feed, hideReadEntries):
169
91a24f499318 introdue an abstraction for the name of the database so it can be changed via commandline parameter
dirk
parents: 166
diff changeset
87 viewName = CouchDb.feedEntriesByFeed()
146
8ec20377bcb0 move getting the entries for a feed to the backend so that the couchdb backend can use a custom view for feed entries
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
88 if hideReadEntries:
169
91a24f499318 introdue an abstraction for the name of the database so it can be changed via commandline parameter
dirk
parents: 166
diff changeset
89 viewName = CouchDb.unreadFeedEntriesByFeed()
148
c5a427d46703 displaying entries for a feed works now with the couchdb backend
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 146
diff changeset
90 return list(FeedEntry.view(self.database, viewName))
146
8ec20377bcb0 move getting the entries for a feed to the backend so that the couchdb backend can use a custom view for feed entries
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
91
155
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
92 def markFeedEntriesAsRead(self, indices):
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
93 for index in indices:
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
94 feedEntry = self.entriesForSelectedFeed()[index]
a05719a6175e move common functionality into an abstract backend class, have both backends inherit from it. Implement enough of the couchdb backend that reading feeds (and marking feed entries as read) is possible
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 148
diff changeset
95 feedEntry.markRead(self.database)
132
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
96
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
97 def updateAllFeeds(self):
139
2cd30af937fa add the required methods for determining if a feed needs to be updated
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 133
diff changeset
98 # TODO use a view instead of iterating all feeds
2cd30af937fa add the required methods for determining if a feed needs to be updated
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 133
diff changeset
99 allFeeds = Feed.all(self.database)
2cd30af937fa add the required methods for determining if a feed needs to be updated
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 133
diff changeset
100 for feed in allFeeds:
2cd30af937fa add the required methods for determining if a feed needs to be updated
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 133
diff changeset
101 if feed.needsUpdate():
166
04c3b9796b89 feedparser uses the proxy now if one is configured. To implement this the FeedUpdater had to change a bit - sqlalchemy backend is not yet refactored.
dirk
parents: 164
diff changeset
102 FeedUpdater(self.database, self.preferences()).update(feed)
132
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
103
63d6d60d37ff new backend for using CouchDB as persistence mechanism ... currently all methods are unimplemented
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
104 def expireFeedEntries(self):
174
d0ced79b5030 implement expiring read feed entries
dirk
parents: 169
diff changeset
105 logger = logging.getLogger("expiry")
d0ced79b5030 implement expiring read feed entries
dirk
parents: 169
diff changeset
106 expireDate = self._calculateExpireDate()
d0ced79b5030 implement expiring read feed entries
dirk
parents: 169
diff changeset
107 logger.info("expiring entries older than " + str(expireDate))
d0ced79b5030 implement expiring read feed entries
dirk
parents: 169
diff changeset
108 for entry in FeedEntry.getReadFeedEntriesOlderThan(expireDate, self.database):
d0ced79b5030 implement expiring read feed entries
dirk
parents: 169
diff changeset
109 del self.database[entry.id]