Mercurial > hg > Feedworm

annotate backend/couchdb/FeedUpdater.py @ 161:2940e74c9663

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Creating a feed via the GUI works now.
author dirk
date Mon, 29 Aug 2011 03:08:05 +0200
parents b290e29a94b5
children 04c3b9796b89
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
144
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
1
161
2940e74c9663 Creating a feed via the GUI works now.
dirk
parents: 147
diff changeset
2 from Feed import Feed
144
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
3 from FeedEntry import FeedEntry
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
4 from backend.AbstractFeedUpdater import AbstractFeedUpdater
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
5 import logging
161
2940e74c9663 Creating a feed via the GUI works now.
dirk
parents: 147
diff changeset
6 import feedparser
144
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
7
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
8 log = logging.getLogger("FeedUpdater")
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
9
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
10 class FeedUpdater(AbstractFeedUpdater):
161
2940e74c9663 Creating a feed via the GUI works now.
dirk
parents: 147
diff changeset
11 @staticmethod
2940e74c9663 Creating a feed via the GUI works now.
dirk
parents: 147
diff changeset
12 def createFeed(url, database):
2940e74c9663 Creating a feed via the GUI works now.
dirk
parents: 147
diff changeset
13 feedDict = feedparser.parse(url)
2940e74c9663 Creating a feed via the GUI works now.
dirk
parents: 147
diff changeset
14 if feedDict.has_key("title"):
2940e74c9663 Creating a feed via the GUI works now.
dirk
parents: 147
diff changeset
15 title = feedDict["feed"].title
2940e74c9663 Creating a feed via the GUI works now.
dirk
parents: 147
diff changeset
16 else:
2940e74c9663 Creating a feed via the GUI works now.
dirk
parents: 147
diff changeset
17 title = url
2940e74c9663 Creating a feed via the GUI works now.
dirk
parents: 147
diff changeset
18 feed = Feed.create(url, title)
2940e74c9663 Creating a feed via the GUI works now.
dirk
parents: 147
diff changeset
19 feed.store(database)
2940e74c9663 Creating a feed via the GUI works now.
dirk
parents: 147
diff changeset
20 FeedUpdater(feed, database).update(feedDict)
2940e74c9663 Creating a feed via the GUI works now.
dirk
parents: 147
diff changeset
21
144
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
22 def __init__(self, feed, database):
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
23 AbstractFeedUpdater.__init__(self, feed)
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
24 self.database = database
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
25
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
26 def _processEntry(self, entry):
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
27 feedEntry = FeedEntry.findByLink(entry.link, self.database)
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
28 if feedEntry is None:
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
29 self._createFeedEntry(entry)
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
30
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
31 def _createFeedEntry(self, entry):
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
32 log.info("new feed entry: " + entry.title)
147
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
33 feedEntry = FeedEntry()
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
34 feedEntry.feed = self.feed.id
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
35 feedEntry.link = entry.link
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
36 feedEntry.title = entry.title
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
37 feedEntry.summary = entry.summary
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
38 feedEntry.updated = entry.updated_parsed
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
39 feedEntry.store(self.database)
144
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
40
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
41 def _incrementFeedUpdateDate(self):
147
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
42 self.feed.incrementNextUpdateDate()
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
43 self.feed.store(self.database)