Mercurial > hg > Feedworm

annotate feedupdate-main.py @ 11:e87c54b3a216

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
use the logging framework for printing messages
author Dirk Olmes <dirk@xanthippe.ping.de>
date Tue, 27 Apr 2010 20:18:22 +0200
parents 87317ba41816
children 9ede118b93ef
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
1 #!/usr/bin/env python
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
3 from ConfigParser import ConfigParser
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
4 from Feed import Feed
4
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
5 from FeedUpdater import updateAllFeeds
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
6 import Mapping
11
e87c54b3a216 use the logging framework for printing messages
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 6
diff changeset
7 import logging
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
8 import socket
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
9 from sqlalchemy import create_engine
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
10 from sqlalchemy.orm import sessionmaker
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
11
11
e87c54b3a216 use the logging framework for printing messages
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 6
diff changeset
12 logger = logging.getLogger("feedupdater")
e87c54b3a216 use the logging framework for printing messages
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 6
diff changeset
13
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
14 def loadDatabaseProperties():
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
15 hostname = socket.gethostname()
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
16 filename = "database-%s.ini" % hostname
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
17
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
18 parser = ConfigParser();
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
19 parser.read(filename)
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
20 return parser.get("database", "url")
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
21
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
22 def createFeed(session):
5
bfd47f55d85b add the updated date of the feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 4
diff changeset
23 if len(session.query(Feed).all()) == 0:
bfd47f55d85b add the updated date of the feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 4
diff changeset
24 newFeed = Feed("Joel on Software", "http://www.joelonsoftware.com/rss.xml")
bfd47f55d85b add the updated date of the feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 4
diff changeset
25 session.add(newFeed)
bfd47f55d85b add the updated date of the feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 4
diff changeset
26 session.commit()
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
27
4
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
28 def listFeeds(session):
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
29 allFeeds = session.query(Feed)
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
30 for feed in allFeeds:
11
e87c54b3a216 use the logging framework for printing messages
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 6
diff changeset
31 logger.info("feed: " + feed.name)
4
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
32 for entry in feed.entries:
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
33 print entry.title
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
34
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
35 if __name__ == "__main__":
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
36 databaseUrl = loadDatabaseProperties()
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
37 engine = create_engine(databaseUrl,echo=True)
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
38 Mapping.createMapping(engine)
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
39 SessionMaker = sessionmaker(bind = engine)
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
40 session = SessionMaker()
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
41
6
87317ba41816 add a creation date for the feed entry
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 5
diff changeset
42 createFeed(session)
87317ba41816 add a creation date for the feed entry
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 5
diff changeset
43 updateAllFeeds(session)
87317ba41816 add a creation date for the feed entry
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 5
diff changeset
44 #listFeeds(session)