Mercurial > hg > Feedworm

annotate backend/couchdb/FeedUpdater.py @ 205:adf7f617bda9

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
make the name of the design document configurable via command line switch. When cloning the feedworm db, the design document is no longer the same as the database name
author dirk
date Sat, 02 Jun 2012 04:24:49 +0200
parents 95445513cc34
children bb3c851b18b1
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
144
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
1
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
2 from FeedEntry import FeedEntry
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
3 from backend.AbstractFeedUpdater import AbstractFeedUpdater
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
4 import logging
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
5
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
6 log = logging.getLogger("FeedUpdater")
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
7
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
8 class FeedUpdater(AbstractFeedUpdater):
166
04c3b9796b89 feedparser uses the proxy now if one is configured. To implement this the FeedUpdater had to change a bit - sqlalchemy backend is not yet refactored.
dirk
parents: 161
diff changeset
9 def __init__(self, database, preferences):
04c3b9796b89 feedparser uses the proxy now if one is configured. To implement this the FeedUpdater had to change a bit - sqlalchemy backend is not yet refactored.
dirk
parents: 161
diff changeset
10 AbstractFeedUpdater.__init__(self, preferences)
144
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
11 self.database = database
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
12
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
13 def _processEntry(self, entry):
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
14 feedEntry = FeedEntry.findByLink(entry.link, self.database)
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
15 if feedEntry is None:
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
16 self._createFeedEntry(entry)
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
17
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
18 def _createFeedEntry(self, entry):
202
95445513cc34 adjust logging if feed entry does not have a published date
dirk
parents: 196
diff changeset
19 if hasattr(entry, "published"):
95445513cc34 adjust logging if feed entry does not have a published date
dirk
parents: 196
diff changeset
20 log.info("new feed entry: %s (%s)" % (entry.title, entry.published))
95445513cc34 adjust logging if feed entry does not have a published date
dirk
parents: 196
diff changeset
21 else:
95445513cc34 adjust logging if feed entry does not have a published date
dirk
parents: 196
diff changeset
22 log.info("new feed entry: %s" % (entry.title))
147
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
23 feedEntry = FeedEntry()
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
24 feedEntry.feed = self.feed.id
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
25 feedEntry.link = entry.link
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
26 feedEntry.title = entry.title
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
27 feedEntry.summary = entry.summary
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
28 feedEntry.updated = entry.updated_parsed
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
29 feedEntry.store(self.database)
144
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
30
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
31 def _incrementFeedUpdateDate(self):
147
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
32 self.feed.incrementNextUpdateDate()
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
33 self.feed.store(self.database)
166
04c3b9796b89 feedparser uses the proxy now if one is configured. To implement this the FeedUpdater had to change a bit - sqlalchemy backend is not yet refactored.
dirk
parents: 161
diff changeset
34
04c3b9796b89 feedparser uses the proxy now if one is configured. To implement this the FeedUpdater had to change a bit - sqlalchemy backend is not yet refactored.
dirk
parents: 161
diff changeset
35 def _setFeedTitle(self, feedDict):
04c3b9796b89 feedparser uses the proxy now if one is configured. To implement this the FeedUpdater had to change a bit - sqlalchemy backend is not yet refactored.
dirk
parents: 161
diff changeset
36 oldTitle = self.feed.title
04c3b9796b89 feedparser uses the proxy now if one is configured. To implement this the FeedUpdater had to change a bit - sqlalchemy backend is not yet refactored.
dirk
parents: 161
diff changeset
37 AbstractFeedUpdater._setFeedTitle(self, feedDict)
04c3b9796b89 feedparser uses the proxy now if one is configured. To implement this the FeedUpdater had to change a bit - sqlalchemy backend is not yet refactored.
dirk
parents: 161
diff changeset
38 if oldTitle != self.feed.title:
04c3b9796b89 feedparser uses the proxy now if one is configured. To implement this the FeedUpdater had to change a bit - sqlalchemy backend is not yet refactored.
dirk
parents: 161
diff changeset
39 self.feed.store(self.database)