Mercurial > hg > Feedworm

annotate backend/sqlalchemy/Mapping.py @ 259:304917762618 default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
implementation of feed updates
author Dirk Olmes <dirk@xanthippe.ping.de>
date Tue, 12 Mar 2019 02:41:22 +0100
parents 7c719c4f5655
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
217
bb3c851b18b1 add source file endcoding header
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 119
diff changeset
1 # -*- coding: utf-8 -*-
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
2 from Feed import Feed
4
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
3 from FeedEntry import FeedEntry
43
12ed8b5fa08c first system preference: configure app to stat maximized.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 35
diff changeset
4 from Preference import Preference
47
a8442c3487b5 add an option to Feed that allows loading an entry's link right away instead of displaying a feed's summary
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 43
diff changeset
5 from sqlalchemy import Boolean
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
6 from sqlalchemy import Column
5
bfd47f55d85b add the updated date of the feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 4
diff changeset
7 from sqlalchemy import DateTime
4
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
8 from sqlalchemy import ForeignKey
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
9 from sqlalchemy import Integer
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
10 from sqlalchemy import MetaData
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
11 from sqlalchemy import String
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
12 from sqlalchemy import Table
30
ee1432a91141 map the potentially long columns as Text (CLOB) not as String (VARCHAR)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 29
diff changeset
13 from sqlalchemy import Text
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
14 from sqlalchemy.orm import mapper
4
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
15 from sqlalchemy.orm import relation
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
16
34
5813e3c10f14 move the database logic out into its own module. Make everything reload safe so that multiple sessions can be created from interactive sessions
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 30
diff changeset
17 mappingDefined = False
68
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 54
diff changeset
18 feedEntryTable = None
34
5813e3c10f14 move the database logic out into its own module. Make everything reload safe so that multiple sessions can be created from interactive sessions
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 30
diff changeset
19
245
8e73a8ae863f Fix the docstrings
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 233
diff changeset
20 """
8e73a8ae863f Fix the docstrings
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 233
diff changeset
21 Make sure the mapping is defined only once. This is not really needed for the feed updater
8e73a8ae863f Fix the docstrings
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 233
diff changeset
22 or the GUI app but comes in handy when working interactively with the system.
8e73a8ae863f Fix the docstrings
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 233
diff changeset
23 """
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
24 def createMapping(engine):
34
5813e3c10f14 move the database logic out into its own module. Make everything reload safe so that multiple sessions can be created from interactive sessions
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 30
diff changeset
25 global mappingDefined
5813e3c10f14 move the database logic out into its own module. Make everything reload safe so that multiple sessions can be created from interactive sessions
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 30
diff changeset
26 if not mappingDefined:
5813e3c10f14 move the database logic out into its own module. Make everything reload safe so that multiple sessions can be created from interactive sessions
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 30
diff changeset
27 _createMapping(engine)
5813e3c10f14 move the database logic out into its own module. Make everything reload safe so that multiple sessions can be created from interactive sessions
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 30
diff changeset
28 mappingDefined = True
5813e3c10f14 move the database logic out into its own module. Make everything reload safe so that multiple sessions can be created from interactive sessions
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 30
diff changeset
29
5813e3c10f14 move the database logic out into its own module. Make everything reload safe so that multiple sessions can be created from interactive sessions
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 30
diff changeset
30 def _createMapping(engine):
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
31 metadata = MetaData(engine)
68
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 54
diff changeset
32 metadata.bind = engine
102
25fef7c29c5b remove all feed entries when removing a feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 79
diff changeset
33
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
34 feedTable = Table("feed", metadata,
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
35 Column("pk", Integer, primary_key=True),
7
215c34f61e95 Feed.url -> Feed.rss_url
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 6
diff changeset
36 Column("title", String(255), nullable=False),
35
aaec263f07ca Feeds manage the point in time when the next update should happen. FeedUpdater only updates feeds that are due.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 34
diff changeset
37 Column("rss_url", String(255), nullable=False),
79
d11c3f71ac40 Make update interval editable via the feed's settings dialog.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 77
diff changeset
38 # update interval is specified in minutes
35
aaec263f07ca Feeds manage the point in time when the next update should happen. FeedUpdater only updates feeds that are due.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 34
diff changeset
39 Column("update_interval", Integer, nullable=False),
47
a8442c3487b5 add an option to Feed that allows loading an entry's link right away instead of displaying a feed's summary
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 43
diff changeset
40 Column("next_update", DateTime, nullable=False),
102
25fef7c29c5b remove all feed entries when removing a feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 79
diff changeset
41 # when displaying an entry of this feed, do not display the summary but rather load
47
a8442c3487b5 add an option to Feed that allows loading an entry's link right away instead of displaying a feed's summary
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 43
diff changeset
42 # the link directly
77
d292ab61ed6f Add another setting to feed: when opening a feed entry in browser, you can force opening it in an external browser now. This is because some sites just crash the QWebView.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 76
diff changeset
43 Column("auto_load_entry_link", Boolean, nullable=False),
d292ab61ed6f Add another setting to feed: when opening a feed entry in browser, you can force opening it in an external browser now. This is because some sites just crash the QWebView.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 76
diff changeset
44 # this is actually a hack: when opening some sites in the QWebView it just crashes.
d292ab61ed6f Add another setting to feed: when opening a feed entry in browser, you can force opening it in an external browser now. This is because some sites just crash the QWebView.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 76
diff changeset
45 # This setting forces to open an entry's link in the external browser
d292ab61ed6f Add another setting to feed: when opening a feed entry in browser, you can force opening it in an external browser now. This is because some sites just crash the QWebView.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 76
diff changeset
46 Column("always_open_in_browser", Boolean, nullable=False)
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
47 )
102
25fef7c29c5b remove all feed entries when removing a feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 79
diff changeset
48
68
525a52169f60 Make FeedEntry table object available in Mapping to use it to expire old, read feed entries.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 54
diff changeset
49 global feedEntryTable
4
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
50 feedEntryTable = Table("feed_entry", metadata,
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
51 Column("pk", Integer, primary_key=True),
6
87317ba41816 add a creation date for the feed entry
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 5
diff changeset
52 Column("create_timestamp", DateTime, nullable=False),
29
74481aa49974 add a read flag for feed entries so they can be marked read
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 7
diff changeset
53 Column("read", Integer, nullable=False),
6
87317ba41816 add a creation date for the feed entry
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 5
diff changeset
54
76
f909550683b0 some links/ids can be very long. give the link/id column more space
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 68
diff changeset
55 Column("id", String(512), nullable=False),
f909550683b0 some links/ids can be very long. give the link/id column more space
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 68
diff changeset
56 Column("link", String(512), nullable=False),
30
ee1432a91141 map the potentially long columns as Text (CLOB) not as String (VARCHAR)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 29
diff changeset
57 Column("title", Text, nullable=False),
ee1432a91141 map the potentially long columns as Text (CLOB) not as String (VARCHAR)
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 29
diff changeset
58 Column("summary", Text, nullable=False),
5
bfd47f55d85b add the updated date of the feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 4
diff changeset
59 Column("updated", DateTime),
4
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
60 Column("feed_id", Integer, ForeignKey("feed.pk"))
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
61 )
102
25fef7c29c5b remove all feed entries when removing a feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 79
diff changeset
62
43
12ed8b5fa08c first system preference: configure app to stat maximized.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 35
diff changeset
63 preferencesTable = Table("preference", metadata,
12ed8b5fa08c first system preference: configure app to stat maximized.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 35
diff changeset
64 Column("pk", Integer, primary_key=True),
12ed8b5fa08c first system preference: configure app to stat maximized.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 35
diff changeset
65 Column("key", String(255), nullable=False),
12ed8b5fa08c first system preference: configure app to stat maximized.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 35
diff changeset
66 Column("value", String(255), nullable=False)
12ed8b5fa08c first system preference: configure app to stat maximized.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 35
diff changeset
67 )
2
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
68
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
69 metadata.create_all()
8a624ee48a74 First skeleton for sqlalchemy: define the mapping and create the first feed
Dirk Olmes <dirk@xanthippe.ping.de>
parents:
diff changeset
70
4
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
71 mapper(FeedEntry, feedEntryTable)
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
72 mapper(Feed, feedTable,
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
73 properties = {
246
7c719c4f5655 Fix all remaining code style bugs
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 245
diff changeset
74 'entries': relation(FeedEntry, backref="feed", lazy=True, cascade="delete, delete-orphan")
4
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
75 }
e0199f383442 retrieve a feed for the given URL, store entries as feed_entry rows into the database
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 2
diff changeset
76 )
43
12ed8b5fa08c first system preference: configure app to stat maximized.
Dirk Olmes <dirk@xanthippe.ping.de>
parents: 35
diff changeset
77 mapper(Preference, preferencesTable)