Mercurial > hg > Feedworm

annotate backend/couchdb/FeedUpdater.py @ 259:304917762618 default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
implementation of feed updates
author Dirk Olmes <dirk@xanthippe.ping.de>
date Tue, 12 Mar 2019 02:41:22 +0100
parents bb3c851b18b1
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
217
bb3c851b18b1 add source file endcoding header
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 202
diff changeset
1 # -*- coding: utf-8 -*-
144
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
2 from FeedEntry import FeedEntry
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
3 from backend.AbstractFeedUpdater import AbstractFeedUpdater
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
4 import logging
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
5
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
6 log = logging.getLogger("FeedUpdater")
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
7
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
8 class FeedUpdater(AbstractFeedUpdater):
166
04c3b9796b89 feedparser uses the proxy now if one is configured. To implement this the FeedUpdater had to change a bit - sqlalchemy backend is not yet refactored.
dirk
parents: 161
diff changeset
9 def __init__(self, database, preferences):
04c3b9796b89 feedparser uses the proxy now if one is configured. To implement this the FeedUpdater had to change a bit - sqlalchemy backend is not yet refactored.
dirk
parents: 161
diff changeset
10 AbstractFeedUpdater.__init__(self, preferences)
144
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
11 self.database = database
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
12
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
13 def _processEntry(self, entry):
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
14 feedEntry = FeedEntry.findByLink(entry.link, self.database)
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
15 if feedEntry is None:
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
16 self._createFeedEntry(entry)
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
17
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
18 def _createFeedEntry(self, entry):
202
95445513cc34 adjust logging if feed entry does not have a published date
dirk
parents: 196
diff changeset
19 if hasattr(entry, "published"):
95445513cc34 adjust logging if feed entry does not have a published date
dirk
parents: 196
diff changeset
20 log.info("new feed entry: %s (%s)" % (entry.title, entry.published))
95445513cc34 adjust logging if feed entry does not have a published date
dirk
parents: 196
diff changeset
21 else:
95445513cc34 adjust logging if feed entry does not have a published date
dirk
parents: 196
diff changeset
22 log.info("new feed entry: %s" % (entry.title))
147
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
23 feedEntry = FeedEntry()
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
24 feedEntry.feed = self.feed.id
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
25 feedEntry.link = entry.link
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
26 feedEntry.title = entry.title
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
27 feedEntry.summary = entry.summary
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
28 feedEntry.updated = entry.updated_parsed
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
29 feedEntry.store(self.database)
144
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
30
74217db92993 updating feeds on the couchdb backend works now
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
31 def _incrementFeedUpdateDate(self):
147
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
32 self.feed.incrementNextUpdateDate()
b290e29a94b5 use couchdb's mapping API instead of manually coding around Rows - much leaner code :-)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 144
diff changeset
33 self.feed.store(self.database)
166
04c3b9796b89 feedparser uses the proxy now if one is configured. To implement this the FeedUpdater had to change a bit - sqlalchemy backend is not yet refactored.
dirk
parents: 161
diff changeset
34
04c3b9796b89 feedparser uses the proxy now if one is configured. To implement this the FeedUpdater had to change a bit - sqlalchemy backend is not yet refactored.
dirk
parents: 161
diff changeset
35 def _setFeedTitle(self, feedDict):
04c3b9796b89 feedparser uses the proxy now if one is configured. To implement this the FeedUpdater had to change a bit - sqlalchemy backend is not yet refactored.
dirk
parents: 161
diff changeset
36 oldTitle = self.feed.title
04c3b9796b89 feedparser uses the proxy now if one is configured. To implement this the FeedUpdater had to change a bit - sqlalchemy backend is not yet refactored.
dirk
parents: 161
diff changeset
37 AbstractFeedUpdater._setFeedTitle(self, feedDict)
04c3b9796b89 feedparser uses the proxy now if one is configured. To implement this the FeedUpdater had to change a bit - sqlalchemy backend is not yet refactored.
dirk
parents: 161
diff changeset
38 if oldTitle != self.feed.title:
04c3b9796b89 feedparser uses the proxy now if one is configured. To implement this the FeedUpdater had to change a bit - sqlalchemy backend is not yet refactored.
dirk
parents: 161
diff changeset
39 self.feed.store(self.database)