Start rewriting backend with pyotherside+asyncio

This commit is contained in:
miruka 2019-06-27 02:31:03 -04:00
parent f530f51937
commit 3344debbbf
128 changed files with 715 additions and 2941 deletions

12
.gitignore vendored
View File

@ -1,12 +0,0 @@
__pycache__
.mypy_cache
build
dist
*.egg-info
*.pyc
*.qmlc
*.jsc
.pylintrc
tmp-*

165
COPYING.LESSER Normal file
View File

@ -0,0 +1,165 @@
GNU LESSER GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
This version of the GNU Lesser General Public License incorporates
the terms and conditions of version 3 of the GNU General Public
License, supplemented by the additional permissions listed below.
0. Additional Definitions.
As used herein, "this License" refers to version 3 of the GNU Lesser
General Public License, and the "GNU GPL" refers to version 3 of the GNU
General Public License.
"The Library" refers to a covered work governed by this License,
other than an Application or a Combined Work as defined below.
An "Application" is any work that makes use of an interface provided
by the Library, but which is not otherwise based on the Library.
Defining a subclass of a class defined by the Library is deemed a mode
of using an interface provided by the Library.
A "Combined Work" is a work produced by combining or linking an
Application with the Library. The particular version of the Library
with which the Combined Work was made is also called the "Linked
Version".
The "Minimal Corresponding Source" for a Combined Work means the
Corresponding Source for the Combined Work, excluding any source code
for portions of the Combined Work that, considered in isolation, are
based on the Application, and not on the Linked Version.
The "Corresponding Application Code" for a Combined Work means the
object code and/or source code for the Application, including any data
and utility programs needed for reproducing the Combined Work from the
Application, but excluding the System Libraries of the Combined Work.
1. Exception to Section 3 of the GNU GPL.
You may convey a covered work under sections 3 and 4 of this License
without being bound by section 3 of the GNU GPL.
2. Conveying Modified Versions.
If you modify a copy of the Library, and, in your modifications, a
facility refers to a function or data to be supplied by an Application
that uses the facility (other than as an argument passed when the
facility is invoked), then you may convey a copy of the modified
version:
a) under this License, provided that you make a good faith effort to
ensure that, in the event an Application does not supply the
function or data, the facility still operates, and performs
whatever part of its purpose remains meaningful, or
b) under the GNU GPL, with none of the additional permissions of
this License applicable to that copy.
3. Object Code Incorporating Material from Library Header Files.
The object code form of an Application may incorporate material from
a header file that is part of the Library. You may convey such object
code under terms of your choice, provided that, if the incorporated
material is not limited to numerical parameters, data structure
layouts and accessors, or small macros, inline functions and templates
(ten or fewer lines in length), you do both of the following:
a) Give prominent notice with each copy of the object code that the
Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the object code with a copy of the GNU GPL and this license
document.
4. Combined Works.
You may convey a Combined Work under terms of your choice that,
taken together, effectively do not restrict modification of the
portions of the Library contained in the Combined Work and reverse
engineering for debugging such modifications, if you also do each of
the following:
a) Give prominent notice with each copy of the Combined Work that
the Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the Combined Work with a copy of the GNU GPL and this license
document.
c) For a Combined Work that displays copyright notices during
execution, include the copyright notice for the Library among
these notices, as well as a reference directing the user to the
copies of the GNU GPL and this license document.
d) Do one of the following:
0) Convey the Minimal Corresponding Source under the terms of this
License, and the Corresponding Application Code in a form
suitable for, and under terms that permit, the user to
recombine or relink the Application with a modified version of
the Linked Version to produce a modified Combined Work, in the
manner specified by section 6 of the GNU GPL for conveying
Corresponding Source.
1) Use a suitable shared library mechanism for linking with the
Library. A suitable mechanism is one that (a) uses at run time
a copy of the Library already present on the user's computer
system, and (b) will operate properly with a modified version
of the Library that is interface-compatible with the Linked
Version.
e) Provide Installation Information, but only if you would otherwise
be required to provide such information under section 6 of the
GNU GPL, and only to the extent that such information is
necessary to install and execute a modified version of the
Combined Work produced by recombining or relinking the
Application with a modified version of the Linked Version. (If
you use option 4d0, the Installation Information must accompany
the Minimal Corresponding Source and Corresponding Application
Code. If you use option 4d1, you must provide the Installation
Information in the manner specified by section 6 of the GNU GPL
for conveying Corresponding Source.)
5. Combined Libraries.
You may place library facilities that are a work based on the
Library side by side in a single library together with other library
facilities that are not Applications and are not covered by this
License, and convey such a combined library under terms of your
choice, if you do both of the following:
a) Accompany the combined library with a copy of the same work based
on the Library, uncombined with any other library facilities,
conveyed under the terms of this License.
b) Give prominent notice with the combined library that part of it
is a work based on the Library, and explaining where to find the
accompanying uncombined form of the same work.
6. Revised Versions of the GNU Lesser General Public License.
The Free Software Foundation may publish revised and/or new versions
of the GNU Lesser General Public License from time to time. Such new
versions will be similar in spirit to the present version, but may
differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the
Library as you received it specifies that a certain numbered version
of the GNU Lesser General Public License "or any later version"
applies to it, you have the option of following the terms and
conditions either of that published version or of any later version
published by the Free Software Foundation. If the Library as you
received it does not specify a version number of the GNU Lesser
General Public License, you may choose any version of the GNU Lesser
General Public License ever published by the Free Software Foundation.
If the Library as you received it specifies that a proxy can decide
whether future versions of the GNU Lesser General Public License shall
apply, that proxy's public statement of acceptance of any version is
permanent authorization for you to choose that version for the
Library.

View File

@ -1,10 +1,6 @@
# Copyright 2019 miruka PKG_DIR = src
# This file is part of harmonyqml, licensed under GPLv3.
PKG_DIR = harmonyqml
PYTHON = python3 PYTHON = python3
PIP = pip3
PYLINT = pylint PYLINT = pylint
MYPY = mypy MYPY = mypy
VULTURE = vulture VULTURE = vulture
@ -12,8 +8,6 @@ BANDIT = bandit
PYCYLE = pycycle PYCYLE = pycycle
CLOC = cloc CLOC = cloc
ARCHIVE_FORMATS = gztar
INSTALL_FLAGS = --user --editable
PYLINT_FLAGS = --output-format colorized PYLINT_FLAGS = --output-format colorized
MYPY_FLAGS = --ignore-missing-imports MYPY_FLAGS = --ignore-missing-imports
VULTURE_FLAGS = --min-confidence 70 VULTURE_FLAGS = --min-confidence 70
@ -23,11 +17,7 @@ CLOC_FLAGS = --ignore-whitespace
LINE = "\033[35m―――――――――――――――――――――――――――――――――――――――――――――――――――――――\033[0m" LINE = "\033[35m―――――――――――――――――――――――――――――――――――――――――――――――――――――――\033[0m"
.PHONY: clean test
.PHONY: all clean dist install upload test
all: clean dist install
clean: clean:
find . -name '__pycache__' -exec rm -Rfv {} + find . -name '__pycache__' -exec rm -Rfv {} +
@ -35,23 +25,6 @@ clean:
find . -name '*.qmlc' -exec rm -Rfv {} + find . -name '*.qmlc' -exec rm -Rfv {} +
find . -name '*.jsc' -exec rm -Rfv {} + find . -name '*.jsc' -exec rm -Rfv {} +
find . -name '*.egg-info' -exec rm -Rfv {} + find . -name '*.egg-info' -exec rm -Rfv {} +
rm -Rfv build dist
dist: clean
@echo
${PYTHON} setup.py sdist --format ${ARCHIVE_FORMATS}
@echo
${PYTHON} setup.py bdist_wheel
install: clean
@echo
${PIP} install ${INSTALL_FLAGS} .
upload: dist
@echo
twine upload dist/*
test: test:
@echo @echo
@ -73,7 +46,7 @@ test:
@echo @echo
@echo pylint ${LINE} @echo pylint ${LINE}
@echo @echo
- ${PYLINT} ${PYLINT_FLAGS} ${PKG_DIR} *.py - ${PYLINT} ${PYLINT_FLAGS} ${PKG_DIR}
@echo @echo
@echo cloc ${LINE} @echo cloc ${LINE}
@echo @echo

92
TODO.md
View File

@ -1,92 +0,0 @@
- Refactoring
- Migrate more JS functions to their own files / Implement in Python instead
- Don't bake in size properties for components
- Bug fixes
- dataclass-like `default_factory` for ListItem
- Prevent briefly seeing login screen if there are accounts to
resumeSession for but they take time to appear
- 100% CPU usage when hitting top edge to trigger messages loading
- Sending `![A picture](https://picsum.photos/256/256)` → not clickable?
- Icons, images and HStyle singleton aren't reloaded
- `MessageDelegate.qml:63: TypeError: 'reloadPreviousItem' not a function`
- RoomEventsList scrolling when resizing the window
- UI
- Invite to room
- Accounts delegates background
- SidePane delegates hover effect
- Server selection
- Register/Forgot? for SignIn dialog
- Scaling
- See [Text.fontSizeMode](https://doc.qt.io/qt-5/qml-qtquick-text.html#fontSizeMode-prop)
- Add room
- Leave room
- Forget room warning popup
- Prevent using the SendBox if no permission (power levels)
- Spinner when loading past room events, images or clicking buttons
- Better theming/styling system
- See about <https://doc.qt.io/qt-5/qtquickcontrols2-configuration.html>
- Settings page
- Multiaccount aliases
- Message/text selection
- Major features
- E2E
- Device verification
- Edit/delete own devices
- Request room keys from own other devices
- Auto-trust accounts within the same client
- Import/export keys
- Uploads
- QQuickImageProvider
- Read receipts
- Status message and presence
- Links preview
- Client improvements
- Filtering rooms: search more than display names?
- nio.MatrixRoom has `typing_users`, no need to handle it on our own
- Initial sync filter and lazy load, see weechat-matrix `_handle_login()`
- See also `handle_response()`'s `keys_query` request
- HTTP/2
- `retry_after_ms` when rate-limited
- Direct chats category
- On sync, check messages API, if a limited sync timeline was received
- Markdown: don't turn #things (no space) and `thing\n---` into title,
disable `__` syntax for bold/italic
- Push instead of replacing in stack view (remove getMemberFilter when done)
- Make links in room subtitle clickable, formatting?
- `<pre>` scrollbar on overflow
- Handle cases where an avatar char is # or @ (#alias room, @user\_id)
- When inviting someone to direct chat, room is "Empty room" until accepted,
it should be the peer's display name instead.
- Keep an accounts order
- See `Qt.callLater()` potential usages
- Banner name color instead of bold
- Animate RoomEventDelegate DayBreak apparition
- Missing nio support
- MatrixRoom invited members list
- Invite events are missing their timestamps (needed for sorting)
- Left room events after client reboot
- `org.matrix.room.preview_urls` event
- `m.room.aliases` event
- Support "Empty room (was ...)" after peer left
- Waiting for approval/release
- nio avatars
- olm/olm-devel 0.3.1 in void repos
- Distribution
- Review setup.py, add dependencies
- README.md
- Use PyInstaller or pyqtdeploy
- Test command:
```
pyinstaller --onefile --windowed --name harmonyqml \
--add-data 'harmonyqml/components:harmonyqml/components' \
--additional-hooks-dir . \
--upx-dir ~/opt/upx-3.95-amd64_linux \
run.py
```

View File

@ -1,21 +0,0 @@
# Copyright 2019 miruka
# This file is part of harmonyqml, licensed under GPLv3.
import os
import sys
# The disk cache is responsible for multiple display bugs when running
# the app for the first time/when cache needs to be recompiled, on top
# of litering the source folders with .qmlc files.
os.environ["QML_DISABLE_DISK_CACHE"] = "1"
def run() -> None:
from .app import Application
app = Application(sys.argv)
from .engine import Engine
engine = Engine(debug=app.debug)
engine.showWindow()
sys.exit(app.exec_())

View File

@ -1,8 +0,0 @@
# Copyright 2019 miruka
# This file is part of harmonyqml, licensed under GPLv3.
"Run app when this package is executed from 'python -m <pkgname>'."
from . import run
run()

View File

@ -1,22 +0,0 @@
# Copyright 2019 miruka
# This file is part of harmonyqml, licensed under GPLv3.
from typing import List, Optional
from PyQt5.QtGui import QGuiApplication
from . import __about__
class Application(QGuiApplication):
def __init__(self, args: Optional[List[str]] = None) -> None:
self.debug = False
if args and "--debug" in args:
del args[args.index("--debug")]
self.debug = True
super().__init__(args or [])
self.setApplicationName(__about__.__pkg_name__)
self.setApplicationDisplayName(__about__.__pretty_name__)

View File

@ -1,4 +0,0 @@
# Copyright 2019 miruka
# This file is part of harmonyqml, licensed under GPLv3.
from .backend import Backend

View File

@ -1,168 +0,0 @@
# Copyright 2019 miruka
# This file is part of harmonyqml, licensed under GPLv3.
import os
import random
from concurrent.futures import ThreadPoolExecutor
from typing import Deque, Dict, Optional, Sequence, Set, Tuple
from atomicfile import AtomicFile
from PyQt5.QtCore import QObject, QStandardPaths, pyqtProperty, pyqtSlot
from .html_filter import HtmlFilter
from .model import ListModel, ListModelMap
from .model.items import User
from .network_manager import NioErrorResponse
from .pyqt_future import futurize
class Backend(QObject):
def __init__(self, parent: QObject) -> None:
super().__init__(parent)
self.pool: ThreadPoolExecutor = ThreadPoolExecutor(max_workers=6)
self.past_tokens: Dict[str, str] = {}
self.fully_loaded_rooms: Set[str] = set()
self._html_filter: HtmlFilter = HtmlFilter(self)
from .client_manager import ClientManager
self._client_manager: ClientManager = ClientManager(self)
self._accounts: ListModel = ListModel(parent=parent)
self._room_events: ListModelMap = ListModelMap(
container = Deque,
parent = self
)
self._users: ListModel = ListModel(
default_factory = self._query_user,
parent = self
)
from .signal_manager import SignalManager
self._signal_manager: SignalManager = SignalManager(self)
self.clients.configLoad()
@pyqtProperty("QVariant", constant=True)
def htmlFilter(self):
return self._html_filter
@pyqtProperty("QVariant", constant=True)
def clients(self):
return self._client_manager
@pyqtProperty("QVariant", constant=True)
def accounts(self):
return self._accounts
@pyqtProperty("QVariant", constant=True)
def roomEvents(self):
return self._room_events
@pyqtProperty("QVariant", constant=True)
def users(self):
return self._users
@pyqtProperty("QVariant", constant=True)
def signals(self):
return self._signal_manager
def _query_user(self, user_id: str) -> User:
client = random.choice(tuple(self.clients.values())) # nosec
@futurize(running_value=user_id)
def get_displayname(self) -> str:
try:
response = client.net.talk(client.nio.get_displayname, user_id)
return response.displayname or user_id
except NioErrorResponse:
return user_id
return User(
userId = user_id,
displayName = get_displayname(self),
devices = ListModel(),
)
@pyqtSlot(str, result=float)
def hueFromString(self, string: str) -> float:
# pylint:disable=no-self-use
return sum((ord(char) * 99 for char in string)) % 360 / 360
@pyqtSlot(str)
@pyqtSlot(str, int)
def loadPastEvents(self, room_id: str, limit: int = 100) -> None:
if not room_id in self.past_tokens:
return # Initial sync not done yet
if room_id in self.fully_loaded_rooms:
return
for client in self.clients.values():
if room_id in client.nio.rooms:
client.loadPastEvents(
room_id, self.past_tokens[room_id], limit
)
break
@pyqtSlot(str)
def setRoomFilter(self, pattern: str) -> None:
for account in self.accounts:
for categ in account.roomCategories:
categ.sortedRooms.filter = pattern
@staticmethod
def getDir(standard_dir: QStandardPaths.StandardLocation) -> str:
path = QStandardPaths.writableLocation(standard_dir)
os.makedirs(path, exist_ok=True)
return path
def getFile(self,
standard_dir: QStandardPaths.StandardLocation,
relative_file_path: str,
initial_content: Optional[str] = None) -> str:
relative_file_path = relative_file_path.replace("/", os.sep)
path = QStandardPaths.locate(standard_dir, relative_file_path)
if path:
return path
path = os.path.join(self.getDir(standard_dir), relative_file_path)
if initial_content is not None:
with AtomicFile(path, "w") as new:
new.write(initial_content)
return path
@pyqtSlot()
@pyqtSlot(list)
def pdb(self, additional_data: Sequence = ()) -> None:
# pylint: disable=all
ad = additional_data
cl = self.clients
ac = self.accounts
re = self.roomEvents
us = self.users
tcl = lambda user: cl[f"@test_{user}:matrix.org"]
import json
jd = lambda obj: print(json.dumps(obj, indent=4, ensure_ascii=False))
import pdb
from PyQt5.QtCore import pyqtRemoveInputHook
pyqtRemoveInputHook()
pdb.set_trace()

View File

@ -1,357 +0,0 @@
# Copyright 2019 miruka
# This file is part of harmonyqml, licensed under GPLv3.
import logging as log
import time
from concurrent.futures import ThreadPoolExecutor
from threading import Event
from typing import DefaultDict, Tuple
from PyQt5.QtCore import (
QObject, QStandardPaths, pyqtProperty, pyqtSignal, pyqtSlot
)
import nio
from .model.items import Trust
from .network_manager import NetworkManager
from .pyqt_future import PyQtFuture, futurize
class Client(QObject):
roomInvited = pyqtSignal([str, dict], [str])
roomJoined = pyqtSignal(str)
roomLeft = pyqtSignal([str, dict], [str])
roomAboutToBeForgotten = pyqtSignal(str)
roomSyncPrevBatchTokenReceived = pyqtSignal(str, str)
roomPastPrevBatchTokenReceived = pyqtSignal(str, str)
roomEventReceived = pyqtSignal(str, str, dict)
roomTypingMembersUpdated = pyqtSignal(str, list)
messageAboutToBeSent = pyqtSignal(str, dict)
deviceIsPresent = pyqtSignal(str, str, str)
deviceIsDeleted = pyqtSignal(str, str)
def __init__(self,
manager,
hostname: str,
username: str,
device_id: str = "") -> None:
super().__init__(manager)
self.manager = manager
host, *port = hostname.split(":")
self.host: str = host
self.port: int = int(port[0]) if port else 443
self.pool: ThreadPoolExecutor = ThreadPoolExecutor(6)
store_path = self.manager.backend.getDir(
QStandardPaths.AppDataLocation
)
self.nio: nio.client.HttpClient = nio.client.HttpClient(
self.host, username, device_id, store_path=store_path
)
# Since nio clients can't handle more than one talk operation
# at a time, this one is used exclusively to poll the sync API
self.nio_sync: nio.client.HttpClient = nio.client.HttpClient(
self.host, username, device_id, store_path=store_path
)
self.net = NetworkManager(self.host, self.port, self.nio)
self.net_sync = NetworkManager(self.host, self.port, self.nio_sync)
self._stop_sync: Event = Event()
# {room_id: (was_typing, at_timestamp_secs)}
self._last_typing_set: DefaultDict[str, Tuple[bool, float]] = \
DefaultDict(lambda: (False, 0))
def __repr__(self) -> str:
return "%s(host=%r, port=%r, user_id=%r)" % \
(type(self).__name__, self.host, self.port, self.userId)
@pyqtProperty(str, constant=True)
def userId(self) -> str:
return self.nio.user_id
@futurize(max_running=1, discard_if_max_running=True, pyqt=False)
def uploadE2EKeys(self) -> None:
self.net.talk(self.nio.keys_upload)
def queryE2EKeys(self) -> None:
self._on_query_e2e_keys(self.net.talk(self.nio.keys_query))
def _on_query_e2e_keys(self, response: nio.KeysQueryResponse) -> None:
for user_id, device_dict in response.device_keys.items():
for device_id, payload in device_dict.items():
if device_id == self.nio.device_id:
continue
ed25519_key = payload["keys"][f"ed25519:{device_id}"]
self.deviceIsPresent.emit(user_id, device_id, ed25519_key)
for device_id, device in self.nio.device_store[user_id].items():
if device.deleted:
self.deviceIsDeleted.emit(user_id, device_id)
def claimE2EKeysForRoom(self, room_id: str) -> None:
self.net.talk(self.nio.keys_claim, room_id)
def shareRoomE2ESession(self,
room_id: str,
ignore_missing_sessions: bool = False) -> None:
self.net.talk(
self.nio.share_group_session,
room_id = room_id,
ignore_missing_sessions = ignore_missing_sessions,
)
def getDeviceTrust(self, device: nio.crypto.OlmDevice) -> Trust:
olm = self.nio.olm
return (
Trust.trusted if olm.is_device_verified(device) else
Trust.blacklisted if olm.is_device_blacklisted(device) else
Trust.undecided
)
@pyqtSlot(str, result="QVariant")
@pyqtSlot(str, str, result="QVariant")
@futurize()
def login(self, password: str, device_name: str = "") -> "Client":
# Main nio client will receive the response here
response = self.net.talk(self.nio.login, password, device_name)
# Now, receive it with the sync nio client too:
self.nio_sync.receive_response(response)
return self
@pyqtSlot(str, str, str, result="QVariant")
@futurize()
def resumeSession(self, user_id: str, token: str, device_id: str
) -> "Client":
response = nio.LoginResponse(user_id, device_id, token)
self.nio.receive_response(response)
self.nio_sync.receive_response(response)
return self
@pyqtSlot(result="QVariant")
@futurize()
def logout(self) -> "Client":
self._stop_sync.set()
self.net.http_disconnect()
self.net_sync.http_disconnect()
return self
@futurize(pyqt=False)
def startSyncing(self) -> None:
while True:
try:
response = self.net_sync.talk(self.nio_sync.sync, timeout=8000)
except nio.LocalProtocolError: # logout occured
pass
else:
self._on_sync(response)
if self._stop_sync.is_set():
self._stop_sync.clear()
break
def _on_sync(self, response: nio.SyncResponse) -> None:
self.nio.receive_response(response)
if self.nio.should_upload_keys:
self.uploadE2EKeys()
if self.nio.should_query_keys:
self.queryE2EKeys()
for room_id, room_info in response.rooms.invite.items():
for ev in room_info.invite_state:
member_ev = isinstance(ev, nio.InviteMemberEvent)
if member_ev and ev.content["membership"] == "join":
self.roomInvited.emit(room_id, ev.content)
break
else:
self.roomInvited[str].emit(room_id)
for room_id, room_info in response.rooms.join.items():
self.roomJoined.emit(room_id)
self.roomSyncPrevBatchTokenReceived.emit(
room_id, room_info.timeline.prev_batch
)
for ev in room_info.timeline.events:
self.roomEventReceived.emit(
room_id, type(ev).__name__, ev.__dict__
)
for ev in room_info.ephemeral:
if isinstance(ev, nio.TypingNoticeEvent):
self.roomTypingMembersUpdated.emit(room_id, ev.users)
else:
print("ephemeral event: ", ev)
for room_id, room_info in response.rooms.leave.items():
for ev in room_info.timeline.events:
member_ev = isinstance(ev, nio.RoomMemberEvent)
if member_ev and ev.content["membership"] in ("leave", "ban"):
self.roomLeft.emit(room_id, ev.__dict__)
break
else:
self.roomLeft[str].emit(room_id)
@futurize(max_running=1, discard_if_max_running=True)
def loadPastEvents(self, room_id: str, start_token: str, limit: int = 100
) -> None:
# From QML, use Backend.loastPastEvents instead
self._on_past_events(
room_id,
self.net.talk(
self.nio.room_messages, room_id, start=start_token, limit=limit
)
)
def _on_past_events(self, room_id: str, response: nio.RoomMessagesResponse
) -> None:
self.roomPastPrevBatchTokenReceived.emit(room_id, response.end)
for ev in response.chunk:
self.roomEventReceived.emit(
room_id, type(ev).__name__, ev.__dict__
)
@pyqtSlot(str, bool)
@futurize(max_running=1, discard_if_max_running=True)
def setTypingState(self, room_id: str, typing: bool) -> None:
set_for_secs = 5
last_set, last_time = self._last_typing_set[room_id]
if not typing and last_set is False:
return
if typing and time.time() - last_time < set_for_secs - 1:
return
self._last_typing_set[room_id] = (typing, time.time())
self.net.talk(
self.nio.room_typing,
room_id = room_id,
typing_state = typing,
timeout = set_for_secs * 1000,
)
@pyqtSlot(str, str)
def sendMarkdown(self, room_id: str, text: str) -> PyQtFuture:
html = self.manager.backend.htmlFilter.fromMarkdown(text)
content = {
"body": text,
"formatted_body": html,
"format": "org.matrix.custom.html",
"msgtype": "m.text",
}
self.messageAboutToBeSent.emit(room_id, content)
# If the thread pool workers are all occupied, and @futurize
# wrapped sendMarkdown, the messageAboutToBeSent signal neccessary
# for local echoes would not be sent until a thread is free.
#
# send() only takes the room_id argument explicitely because
# of consider_args=True: This means the max number of messages being
# sent at a time is one per room at a time.
@futurize(max_running=1, consider_args=True)
def send(self, room_id: str) -> PyQtFuture:
talk = lambda: self.net.talk(
self.nio.room_send,
room_id = room_id,
message_type = "m.room.message",
content = content,
)
try:
log.debug("Try sending message %r to %r", content, room_id)
return talk()
except nio.GroupEncryptionError as err:
log.warning(err)
try:
self.shareRoomE2ESession(room_id)
except nio.EncryptionError as err:
log.warning(err)
self.claimE2EKeysForRoom(room_id)
self.shareRoomE2ESession(room_id,
ignore_missing_sessions=True)
log.debug("Final try to send %r to %r", content, room_id)
return talk()
return send(self, room_id)
@pyqtSlot(str, result="QVariant")
@futurize()
def joinRoom(self, room_id: str) -> None:
return self.net.talk(self.nio.join, room_id=room_id)
@pyqtSlot(str, result="QVariant")
@futurize()
def leaveRoom(self, room_id: str) -> None:
return self.net.talk(self.nio.room_leave, room_id=room_id)
@pyqtSlot(str, result="QVariant")
@futurize()
def forgetRoom(self, room_id: str) -> None:
self.roomAboutToBeForgotten.emit(room_id)
response = self.net.talk(self.nio.room_forget, room_id=room_id)
self.nio.invalidate_outbound_session(room_id)
return response
@pyqtSlot(str, result=bool)
def roomHasUnknownDevices(self, room_id: str) -> bool:
return self.nio.room_contains_unverified(room_id)
@pyqtSlot(str, str, result=str)
def getMemberFilter(self, room_category: str, room_id: str) -> str:
return self.manager.backend.accounts[self.userId]\
.roomCategories[room_category]\
.rooms[room_id]\
.sortedMembers.filter
@pyqtSlot(str, str, str)
def setMemberFilter(self, room_category: str, room_id: str, pattern: str
) -> None:
self.manager.backend.accounts[self.userId]\
.roomCategories[room_category]\
.rooms[room_id]\
.sortedMembers.filter = pattern

View File

@ -1,155 +0,0 @@
# Copyright 2018 miruka
# This file is part of harmonyqt, licensed under GPLv3.
import json
import platform
import threading
from collections.abc import Mapping
from typing import Dict
from atomicfile import AtomicFile
from PyQt5.QtCore import (
QObject, QStandardPaths, pyqtProperty, pyqtSignal, pyqtSlot
)
from harmonyqml import __about__
from .backend import Backend
from .client import Client
AccountConfig = Dict[str, Dict[str, str]]
_CONFIG_LOCK = threading.Lock()
class _ClientManagerMeta(type(QObject), type(Mapping)): # type: ignore
pass
class ClientManager(QObject, Mapping, metaclass=_ClientManagerMeta):
clientAdded = pyqtSignal(Client)
clientDeleted = pyqtSignal(str)
clientCountChanged = pyqtSignal(int)
def __init__(self, backend: Backend) -> None:
super().__init__(backend)
self.backend = backend
self._clients: Dict[str, Client] = {}
func = lambda: self.clientCountChanged.emit(len(self))
self.clientAdded.connect(func)
self.clientDeleted.connect(func)
def __repr__(self) -> str:
return f"{type(self).__name__}(clients={self._clients!r})"
def __getitem__(self, user_id: str) -> Client:
return self.get(user_id)
def __len__(self) -> int:
return self.count
def __iter__(self):
return iter(self._clients)
@pyqtSlot(str, result="QVariant")
def get(self, key: str) -> Client:
return self._clients[key]
@pyqtProperty(int, notify=clientCountChanged)
def count(self):
return len(self._clients)
@pyqtSlot()
def configLoad(self) -> None:
for user_id, info in self.configAccounts().items():
client = Client(self, info["hostname"], user_id)
client.resumeSession(user_id, info["token"], info["device_id"])\
.add_done_callback(lambda _, c=client: self._on_connected(c))
@pyqtSlot(str, str, str, result="QVariant")
@pyqtSlot(str, str, str, str, result="QVariant")
def new(self, hostname: str, username: str, password: str,
device_id: str = "") -> None:
client = Client(self, hostname, username, device_id)
future = client.login(password, self.defaultDeviceName)
future.add_done_callback(lambda _: self._on_connected(client))
return future
def _on_connected(self, client: Client) -> None:
self._clients[client.userId] = client
self.clientAdded.emit(client)
client.startSyncing()
@pyqtSlot(str)
def remove(self, user_id: str) -> None:
client = self._clients.pop(user_id, None)
if client:
self.clientDeleted.emit(user_id)
client.logout()
@pyqtSlot()
def removeAll(self) -> None:
for user_id in self._clients.copy():
self.remove(user_id)
@pyqtProperty(str, constant=True)
def defaultDeviceName(self) -> str: # pylint: disable=no-self-use
os_ = f" on {platform.system()}".rstrip()
os_ = f"{os_} {platform.release()}".rstrip() if os_ != " on" else ""
return f"{__about__.__pretty_name__}{os_}"
# Config file operations
def getAccountConfigPath(self) -> str:
return self.backend.getFile(
QStandardPaths.AppConfigLocation, "accounts.json", "[]"
)
def configAccounts(self) -> AccountConfig:
with open(self.getAccountConfigPath(), "r") as file:
return json.loads(file.read().strip()) or {}
@pyqtSlot("QVariant")
def remember(self, client: Client) -> None:
self._write_config({
**self.configAccounts(),
**{client.userId: {
"hostname": client.nio.host,
"token": client.nio.access_token,
"device_id": client.nio.device_id,
}}
})
@pyqtSlot(str)
def forget(self, user_id: str) -> None:
self._write_config({
uid: info
for uid, info in self.configAccounts().items() if uid != user_id
})
def _write_config(self, accounts: AccountConfig) -> None:
js = json.dumps(accounts, indent=4, ensure_ascii=False, sort_keys=True)
with _CONFIG_LOCK:
with AtomicFile(self.getAccountConfigPath(), "w") as new:
new.write(js)

View File

@ -1,154 +0,0 @@
# Copyright 2019 miruka
# This file is part of harmonyqml, licensed under GPLv3.
import re
import mistune
from lxml.html import HtmlElement, etree # nosec
from PyQt5.QtCore import QObject, pyqtProperty, pyqtSlot
import html_sanitizer.sanitizer as sanitizer
class HtmlFilter(QObject):
link_regexes = [re.compile(r, re.IGNORECASE) for r in [
(r"(?P<body>.+://(?P<host>[a-z0-9._-]+)(?:/[/\-_.,a-z0-9%&?;=~]*)?"
r"(?:\([/\-_.,a-z0-9%&?;=~]*\))?)"),
r"mailto:(?P<body>[a-z0-9._-]+@(?P<host>[a-z0-9_.-]+[a-z]))",
r"tel:(?P<body>[0-9+-]+)(?P<host>)",
r"(?P<body>magnet:\?xt=urn:[a-z0-9]+:.+)(?P<host>)",
]]
def __init__(self, parent: QObject) -> None:
super().__init__(parent)
self._sanitizer = sanitizer.Sanitizer(self.sanitizer_settings)
# The whitespace remover doesn't take <pre> into account
sanitizer.normalize_overall_whitespace = lambda html: html
sanitizer.normalize_whitespace_in_text_or_tail = lambda el: el
# hard_wrap: convert all \n to <br> without required two spaces
self._markdown_to_html = mistune.Markdown(hard_wrap=True)
@pyqtSlot(str, result=str)
def fromMarkdown(self, text: str) -> str:
return self.filter(self._markdown_to_html(text))
@pyqtSlot(str, result=str)
def filter(self, html: str) -> str:
html = self._sanitizer.sanitize(html)
tree = etree.fromstring(html, parser=etree.HTMLParser())
if tree is None:
return ""
for el in tree.iter("img"):
el = self._wrap_img_in_a(el)
for el in tree.iter("a"):
el = self._append_img_to_a(el)
result = b"".join((etree.tostring(el, encoding="utf-8")
for el in tree[0].iterchildren()))
return str(result, "utf-8")
@pyqtProperty("QVariantMap")
def sanitizer_settings(self) -> dict:
# https://matrix.org/docs/spec/client_server/latest.html#m-room-message-msgtypes
return {
"tags": {
# TODO: mx-reply, audio, video
"font", "h1", "h2", "h3", "h4", "h5", "h6",
"blockquote", "p", "a", "ul", "ol", "sup", "sub", "li",
"b", "i", "s", "u", "code", "hr", "br",
"table", "thead", "tbody", "tr", "th", "td",
"pre", "img",
},
"attributes": {
# TODO: translate font attrs to qt html subset
"font": {"data-mx-bg-color", "data-mx-color"},
"a": {"href"},
"img": {"width", "height", "alt", "title", "src"},
"ol": {"start"},
"code": {"class"},
},
"empty": {"hr", "br", "img"},
"separate": {
"a", "p", "li", "table", "tr", "th", "td", "br", "hr"
},
"whitespace": {},
"add_nofollow": False,
"autolink": { # FIXME: arg dict not working
"link_regexes": self.link_regexes,
"avoid_hosts": [],
},
"sanitize_href": lambda href: href,
"element_preprocessors": [
sanitizer.bold_span_to_strong,
sanitizer.italic_span_to_em,
sanitizer.tag_replacer("strong", "b"),
sanitizer.tag_replacer("em", "i"),
sanitizer.tag_replacer("strike", "s"),
sanitizer.tag_replacer("del", "s"),
sanitizer.tag_replacer("span", "font"),
self._remove_empty_font,
sanitizer.tag_replacer("form", "p"),
sanitizer.tag_replacer("div", "p"),
sanitizer.tag_replacer("caption", "p"),
sanitizer.target_blank_noopener,
],
"element_postprocessors": [],
"is_mergeable": lambda e1, e2: e1.attrib == e2.attrib,
}
def _remove_empty_font(self, el: HtmlElement) -> HtmlElement:
if el.tag != "font":
return el
if not self.sanitizer_settings["attributes"]["font"] & set(el.keys()):
el.clear()
return el
def _wrap_img_in_a(self, el: HtmlElement) -> HtmlElement:
link = el.attrib.get("src", "")
width = el.attrib.get("width", "256")
height = el.attrib.get("height", "256")
if el.getparent().tag == "a" or el.tag != "img" or \
not self._is_image_path(link):
return el
el.tag = "a"
el.attrib.clear()
el.attrib["href"] = link
el.append(etree.Element("img", src=link, width=width, height=height))
return el
def _append_img_to_a(self, el: HtmlElement) -> HtmlElement:
link = el.attrib.get("href", "")
if not (el.tag == "a" and self._is_image_path(link)):
return el
for _ in el.iter("img"): # if the <a> already has an <img> child
return el
el.append(etree.Element("br"))
el.append(etree.Element("img", src=link, width="256", height="256"))
return el
@staticmethod
def _is_image_path(link: str) -> bool:
return bool(re.match(
r".+\.(jpg|jpeg|png|gif|bmp|webp|tiff|svg)$", link, re.IGNORECASE
))

View File

@ -1,6 +0,0 @@
# Copyright 2019 miruka
# This file is part of harmonyqml, licensed under GPLv3.
from . import items
from .list_model import ListModel
from .list_model_map import ListModelMap

View File

@ -1,96 +0,0 @@
from enum import Enum
from typing import Any, Dict, List, Optional
from PyQt5.QtCore import QDateTime
from ..pyqt_future import PyQtFuture
from .list_item import ListItem
from .list_model import ListModel
from .sort_filter_proxy import SortFilterProxy
class Account(ListItem):
_required_init_values = {"userId", "roomCategories"}
_constant = {"userId", "roomCategories"}
userId: str = ""
roomCategories: ListModel = ListModel()
class RoomCategory(ListItem):
_required_init_values = {"name", "rooms", "sortedRooms"}
_constant = {"name", "rooms", "sortedRooms"}
name: str = ""
rooms: ListModel = ListModel()
sortedRooms: SortFilterProxy = SortFilterProxy(ListModel(), "", "")
class Room(ListItem):
_required_init_values = {"roomId", "displayName", "members",
"sortedMembers"}
_constant = {"roomId", "members", "sortedMembers"}
roomId: str = ""
displayName: str = ""
topic: Optional[str] = None
lastEventDateTime: Optional[QDateTime] = None
typingMembers: List[str] = []
members: ListModel = ListModel()
sortedMembers: SortFilterProxy = SortFilterProxy(ListModel(), "", "")
inviter: Optional[Dict[str, str]] = None
leftEvent: Optional[Dict[str, str]] = None
class RoomMember(ListItem):
_required_init_values = {"userId"}
_constant = {"userId"}
userId: str = ""
class RoomEvent(ListItem):
_required_init_values = {"eventId", "type", "dict", "dateTime"}
_constant = {"type"}
eventId: str = ""
type: str = ""
dict: Dict[str, Any] = {}
dateTime: QDateTime = QDateTime()
isLocalEcho: bool = False
# ----------
class User(ListItem):
_required_init_values = {"userId", "devices"}
_constant = {"userId", "devices"}
# Use PyQtFutures because the info might or might not need a request
# to be fetched, and we don't want to block the UI in any case.
# QML's property binding ability is used on the PyQtFuture.value
userId: str = ""
displayName: Optional[PyQtFuture] = None
avatarUrl: Optional[PyQtFuture] = None
statusMessage: Optional[PyQtFuture] = None
devices: ListModel = ListModel()
class Trust(Enum):
blacklisted = -1
undecided = 0
trusted = 1
class Device(ListItem):
_required_init_values = {"deviceId", "ed25519Key"}
_constant = {"deviceId", "ed25519Key"}
deviceId: str = ""
ed25519Key: str = ""
displayName: Optional[str] = None
trust: Trust = Trust.undecided
lastSeenIp: Optional[str] = None
lastSeenDate: Optional[QDateTime] = None

View File

@ -1,183 +0,0 @@
# Copyright 2019 miruka
# This file is part of harmonyqml, licensed under GPLv3.
import textwrap
from typing import Any, Dict, List, Mapping, Set, Tuple, Union
from PyQt5.QtCore import QObject, pyqtProperty, pyqtSignal, pyqtSlot
PyqtType = Union[str, type]
class _ListItemMeta(type(QObject)): # type: ignore
__slots__ = ()
def __new__(mcs, name: str, bases: Tuple[type], attrs: Dict[str, Any]):
def to_pyqt_type(type_) -> PyqtType:
"Return an appropriate pyqtProperty type from an annotation."
try:
if issubclass(type_, (bool, int, float, str, type(None))):
return type_
if issubclass(type_, Mapping):
return "QVariantMap"
return "QVariant"
except TypeError: # e.g. None passed
return to_pyqt_type(type(type_))
# These special attributes must not be processed like properties
special = {"_main_key", "_required_init_values", "_constant"}
# These properties won't be settable and will not have a notify signal
constant: Set[str] = set(attrs.get("_constant") or set())
# pyqtProperty objects that were directly defined in the class
direct_pyqt_props: Dict[str, pyqtProperty] = {
name: obj for name, obj in attrs.items()
if isinstance(obj, pyqtProperty)
}
# {property_name: (its_pyqt_type, its_default_value)}
props: Dict[str, Tuple[PyqtType, Any]] = {
name: (to_pyqt_type(attrs.get("__annotations__", {}).get(name)),
value)
for name, value in attrs.items()
if not (name.startswith("__") or callable(value) or
name in special)
}
# Signals for the pyqtProperty notify arguments
signals: Dict[str, pyqtSignal] = {
f"{name}Changed": pyqtSignal(type_)
for name, (type_, _) in props.items() if name not in constant
}
# pyqtProperty() won't take None, so we make dicts of extra kwargs
# to pass for each property
pyqt_props_kwargs: Dict[str, Dict[str, Any]] = {
name: {"constant": True} if name in constant else
{"notify": signals[f"{name}Changed"],
"fset": lambda self, value, n=name: (
setattr(self, f"_{n}", value) or # type: ignore
getattr(self, f"{n}Changed").emit(value),
)}
for name in props
}
# The final pyqtProperty objects we create
pyqt_props: Dict[str, pyqtProperty] = {
name: pyqtProperty(
type_,
fget=lambda self, n=name: getattr(self, f"_{n}"),
**pyqt_props_kwargs.get(name, {}),
)
for name, (type_, _) in props.items()
}
attrs = {
**attrs, # Original class attributes
**signals,
**direct_pyqt_props,
**pyqt_props,
# Set the internal _properties as slots for memory savings
"__slots__": tuple({f"_{prop}" for prop in props} & {"_main_key"}),
"_direct_props": list(direct_pyqt_props.keys()),
"_props": props,
# The main key is either the attribute _main_key,
# or the first defined property
"_main_key": attrs.get("_main_key") or
list(props.keys())[0] if props else None,
"_required_init_values": attrs.get("_required_init_values") or (),
"_constant": constant,
}
return type.__new__(mcs, name, bases, attrs)
class ListItem(QObject, metaclass=_ListItemMeta):
def __init__(self, *args, **kwargs) -> None:
super().__init__()
method: str = "%s.__init__()" % type(self).__name__
already_set: Set[str] = set()
required: Set[str] = set(self._required_init_values)
required_num: int = len(required) + 1 # + 1 = self
args_num: int = len(self._props) + 1
from_to: str = str(args_num) if required_num == args_num else \
f"from {required_num} to {args_num}"
# Check that not too many positional arguments were passed
if len(args) > len(self._props):
raise TypeError(
f"{method} takes {from_to} positional arguments but "
f"{len(args) + 1} were given"
)
# Set properties from provided positional arguments
for prop, value in zip(self._props, args):
setattr(self, f"_{prop}", self._set_parent(value))
already_set.add(prop)
# Set properties from provided keyword arguments
for prop, value in kwargs.items():
if prop in already_set:
raise TypeError(f"{method} got multiple values for "
f"argument {prop!r}")
if prop not in self._props:
raise TypeError(f"{method} got an unexpected keyword "
f"argument {prop!r}")
setattr(self, f"_{prop}", self._set_parent(value))
already_set.add(prop)
# Check for required init arguments not provided
missing: Set[str] = required - already_set
if missing:
raise TypeError("%s missing %d required argument: %s" % (
method, len(missing), ", ".join((repr(m) for m in missing))))
# Set default values for properties not provided in arguments
for prop in set(self._props) - already_set:
setattr(self, f"_{prop}", self._set_parent(self._props[prop][1]))
def _set_parent(self, value: Any) -> Any:
if isinstance(value, QObject):
value.setParent(self)
return value
def __repr__(self) -> str:
prop_strings = (
"\033[{0};34m{1}\033[0,{0}m = \033[{0};32m{2}\033[0m".format(
1 if p == self.mainKey else 0, # 1 = term bold
p,
repr(getattr(self, p))
) for p in list(self._props.keys()) + self._direct_props
)
return "\033[35m%s\033[0m(\n%s\n)" % (
type(self).__name__,
textwrap.indent(",\n".join(prop_strings), prefix=" " * 4)
)
@pyqtSlot(result=str)
def repr(self) -> str:
return self.__repr__()
@pyqtProperty("QStringList", constant=True)
def roles(self) -> List[str]:
return list(self._props.keys()) + self._direct_props
@pyqtProperty(str, constant=True)
def mainKey(self) -> str:
return self._main_key

View File

@ -1,443 +0,0 @@
import logging
import textwrap
from typing import (
Any, Callable, Dict, Iterable, List, Mapping, MutableSequence, Optional,
Sequence, Set, Tuple, Union
)
from PyQt5.QtCore import (
QAbstractListModel, QModelIndex, QObject, Qt, pyqtProperty, pyqtSignal,
pyqtSlot
)
from .list_item import ListItem
Index = Union[int, str]
NewItem = Union[ListItem, Mapping[str, Any], Sequence]
class _GetFail:
pass
class _PopFail:
pass
class ListModel(QAbstractListModel):
rolesSet = pyqtSignal()
changed = pyqtSignal()
countChanged = pyqtSignal(int)
def __init__(self,
initial_data: Optional[List[NewItem]] = None,
container: Callable[..., MutableSequence] = list,
default_factory: Optional[Callable[[str], ListItem]] = None,
parent: QObject = None) -> None:
super().__init__(parent)
self._data: MutableSequence[ListItem] = container()
self.default_factory = default_factory
if initial_data:
self.extend(initial_data)
def __repr__(self) -> str:
if not self._data:
return "\033[35m%s\033[0m()" % type(self).__name__
return "\033[35m%s\033[0m(\n%s\n)" % (
type(self).__name__,
textwrap.indent(
",\n".join((repr(item) for item in self._data)),
prefix = " " * 4,
)
)
def __contains__(self, index: Index) -> bool:
if isinstance(index, str):
try:
self.indexWhere(index)
return True
except ValueError:
return False
return index in self._data
def __getitem__(self, index: Index) -> ListItem:
return self.get(index)
def __setitem__(self, index: Index, value: NewItem) -> None:
self.set(index, value)
def __delitem__(self, index: Index) -> None:
self.remove(index)
def __len__(self) -> int:
return len(self._data)
def __iter__(self) -> Iterable[NewItem]:
return iter(self._data)
def __bool__(self) -> bool:
return bool(self._data)
@pyqtSlot(result=str)
def repr(self) -> str:
return self.__repr__()
@pyqtProperty("QStringList", notify=rolesSet)
def roles(self) -> Tuple[str, ...]:
return self._data[0].roles if self._data else () # type: ignore
@pyqtProperty("QVariant", notify=rolesSet)
def mainKey(self) -> Optional[str]:
return self._data[0].mainKey if self._data else None
def roleNumbers(self) -> Dict[str, int]:
return {name: Qt.UserRole + i
for i, name in enumerate(self.roles, 1)} \
if self._data else {}
def roleNames(self) -> Dict[int, bytes]:
return {Qt.UserRole + i: bytes(name, "utf-8")
for i, name in enumerate(self.roles, 1)} \
if self._data else {}
def data(self, index: QModelIndex, role: int = Qt.DisplayRole) -> Any:
if role <= Qt.UserRole:
return None
return getattr(self._data[index.row()],
str(self.roleNames()[role], "utf8"))
def rowCount(self, _: QModelIndex = QModelIndex()) -> int:
return len(self)
def _convert_new_value(self, value: NewItem) -> ListItem:
def convert() -> ListItem:
if self._data and isinstance(value, Mapping):
if not set(value.keys()) <= set(self.roles):
raise ValueError(
f"{value}: must have all these keys: {self.roles}"
)
return type(self._data[0])(**value)
if not self._data and isinstance(value, Mapping):
raise NotImplementedError("First item must be set from Python")
if self._data and isinstance(value, type(self._data[0])):
return value
if not self._data and isinstance(value, ListItem):
return value
raise TypeError("%r: must be mapping or %s" % (
value,
type(self._data[0]).__name__ if self._data else "ListItem"
))
value = convert()
value.setParent(self)
return value
@pyqtProperty(int, notify=countChanged)
def count(self) -> int:
return len(self)
@pyqtSlot("QVariant", result=int)
def indexWhere(self,
main_key_is_value: Any,
_can_use_default_factory: bool = True) -> int:
for i, item in enumerate(self._data):
if getattr(item, self.mainKey) == main_key_is_value:
return i
if _can_use_default_factory and self.default_factory:
return self.append(self.default_factory(main_key_is_value))
raise ValueError(
f"No item in model data with "
f"property {self.mainKey} is set to {main_key_is_value!r}."
)
@pyqtSlot(int, result="QVariant")
@pyqtSlot(str, result="QVariant")
@pyqtSlot(int, "QVariant", result="QVariant")
@pyqtSlot(str, "QVariant", result="QVariant")
def get(self, index: Index, default: Any = _GetFail()) -> ListItem:
try:
i_index: int = \
self.indexWhere(index, _can_use_default_factory=False) \
if isinstance(index, str) else index
return self._data[i_index]
except (ValueError, IndexError):
if isinstance(default, _GetFail):
if self.default_factory and isinstance(index, str):
item = self.default_factory(index)
self.append(item)
return item
raise
return default
@pyqtSlot(int, "QVariantMap", result=int)
def insert(self, index: int, value: NewItem) -> int:
value = self._convert_new_value(value)
try:
present_index = self.indexWhere(
main_key_is_value = getattr(value, self.mainKey),
_can_use_default_factory = False
)
except (TypeError, ValueError): # TypeError = no items in model
pass
else:
logging.warning(
"Duplicate mainKey %r in model - present: %r, inserting: %r",
self.mainKey,
self[present_index],
value
)
self.beginInsertRows(QModelIndex(), index, index)
had_data = bool(self._data)
self._data.insert(index, value)
if not had_data:
self.rolesSet.emit()
self.endInsertRows()
self.countChanged.emit(len(self))
self.changed.emit()
return index
@pyqtSlot("QVariantMap", result=int)
def append(self, value: NewItem) -> int:
return self.insert(len(self), value)
@pyqtSlot(list)
def extend(self, values: Iterable[NewItem]) -> None:
for val in values:
self.append(val)
@pyqtSlot(list)
@pyqtSlot(list, bool)
def updateAll(self, items: Sequence[NewItem], delete: bool = False
) -> None:
items_: List[ListItem] = [self._convert_new_value(i) for i in items]
if delete:
present_item: ListItem
for i, present_item in enumerate(self):
present_item_key = getattr(present_item, self.mainKey)
# If this present item is in the update items, based on mainKey
for update_item in items_:
if present_item_key == getattr(update_item, self.mainKey):
break
else:
del self[i]
for item in items_:
self.upsert(
where_main_key_is = getattr(item, item.mainKey),
update_with = item
)
@pyqtSlot(int, "QVariantMap", result=int)
@pyqtSlot(int, "QVariantMap", "QStringList", result=int)
@pyqtSlot(str, "QVariantMap", result=int)
@pyqtSlot(str, "QVariantMap", "QStringList", result=int)
def updateItem(self,
index: Index,
value: NewItem,
no_update: Sequence[str] = ()) -> int:
value = self._convert_new_value(value)
i_index: int = self.indexWhere(index, _can_use_default_factory=False) \
if isinstance(index, str) else index
to_update = self[i_index]
updated_roles: Set[int] = set()
for role_name, role_num in self.roleNumbers().items():
if role_name not in no_update:
old_value = getattr(to_update, role_name)
new_value = getattr(value, role_name)
if old_value != new_value:
try:
setattr(to_update, role_name, new_value)
except AttributeError: # constant/not settable
pass
else:
updated_roles.add(role_num)
if updated_roles:
qidx = QAbstractListModel.index(self, i_index, 0)
self.dataChanged.emit(qidx, qidx, updated_roles)
self.changed.emit()
return i_index
@pyqtSlot(str, "QVariantMap")
@pyqtSlot(str, "QVariantMap", int)
@pyqtSlot(str, "QVariantMap", int, int)
@pyqtSlot(str, "QVariantMap", int, int, "QStringList")
def upsert(self,
where_main_key_is: Any,
update_with: NewItem,
new_index_if_insert: Optional[int] = None,
new_index_if_update: Optional[int] = None,
no_update: Sequence[str] = ()) -> None:
try:
index = self.updateItem(
where_main_key_is, update_with, no_update
)
except (IndexError, ValueError):
self.insert(new_index_if_insert or len(self), update_with)
else:
if new_index_if_update:
self.move(index, new_index_if_update)
@pyqtSlot(int, list)
@pyqtSlot(str, list)
def set(self, index: Index, value: NewItem) -> None:
i_index: int = self.indexWhere(index) \
if isinstance(index, str) else index
qidx = QAbstractListModel.index(self, i_index, 0)
value = self._convert_new_value(value)
self._data[i_index] = value
self.dataChanged.emit(qidx, qidx, self.roleNames())
self.changed.emit()
@pyqtSlot(int, str, "QVariant")
@pyqtSlot(str, str, "QVariant")
def setProperty(self, index: Index, prop: str, value: Any) -> None:
i_index: int = self.indexWhere(index) \
if isinstance(index, str) else index
if getattr(self[i_index], prop) != value:
setattr(self[i_index], prop, value)
qidx = QAbstractListModel.index(self, i_index, 0)
self.dataChanged.emit(qidx, qidx, (self.roleNumbers()[prop],))
self.changed.emit()
@pyqtSlot(int, int)
@pyqtSlot(int, int, int)
@pyqtSlot(str, int)
@pyqtSlot(str, int, int)
def move(self, from_: Index, to: int, n: int = 1) -> None:
# pylint: disable=invalid-name
i_from: int = self.indexWhere(from_) \
if isinstance(from_, str) else from_
qlast = i_from + n - 1
if (n <= 0) or (i_from == to) or (qlast == to) or \
not (len(self) > qlast >= 0) or \
not len(self) >= to >= 0:
return
qidx = QModelIndex()
qto = min(len(self), to + n if to > i_from else to)
# print(f"self.beginMoveRows(qidx, {i_from}, {qlast}, qidx, {qto})")
valid = self.beginMoveRows(qidx, i_from, qlast, qidx, qto)
if not valid:
logging.warning("Invalid move operation - %r", locals())
return
last = i_from + n
cut = self._data[i_from:last]
del self._data[i_from:last]
self._data[to:to] = cut
self.endMoveRows()
self.changed.emit()
@pyqtSlot(int)
@pyqtSlot(str)
def remove(self, index: Index) -> None:
i_index: int = self.indexWhere(index) \
if isinstance(index, str) else index
self.beginRemoveRows(QModelIndex(), i_index, i_index)
del self._data[i_index]
self.endRemoveRows()
self.countChanged.emit(len(self))
self.changed.emit()
@pyqtSlot(int, result="QVariant")
@pyqtSlot(str, result="QVariant")
def pop(self, index: Index, default: Any = _PopFail()) -> ListItem:
try:
i_index: int = self.indexWhere(index) \
if isinstance(index, str) else index
item = self[i_index]
except (ValueError, IndexError):
if isinstance(default, _PopFail):
raise
return default
self.beginRemoveRows(QModelIndex(), i_index, i_index)
del self._data[i_index]
self.endRemoveRows()
self.countChanged.emit(len(self))
self.changed.emit()
return item
@pyqtSlot()
def clear(self) -> None:
if not self._data:
return
# Reimplemented for performance reasons (begin/endRemoveRows)
self.beginRemoveRows(QModelIndex(), 0, len(self))
self._data.clear()
self.endRemoveRows()
self.countChanged.emit(len(self))
self.changed.emit()

View File

@ -1,58 +0,0 @@
from typing import Any, DefaultDict
from PyQt5.QtCore import QObject, pyqtSlot
from .list_model import ListModel
class ListModelMap(QObject):
def __init__(self, *models_args, parent: QObject = None, **models_kwargs
) -> None:
super().__init__(parent)
models_kwargs["parent"] = self
# Set the parent to prevent item garbage-collection on the C++ side
self.dict: DefaultDict[Any, ListModel] = \
DefaultDict(
lambda: ListModel(*models_args, **models_kwargs)
)
def __repr__(self) -> str:
return "%s(%r)" % (type(self).__name__, self.dict)
def __getitem__(self, key) -> ListModel:
return self.dict[key]
def __setitem__(self, key, value: ListModel) -> None:
value.setParent(self)
self.dict[key] = value
def __detitem__(self, key) -> None:
del self.dict[key]
def __iter__(self):
return iter(self.dict)
def __len__(self) -> int:
return len(self.dict)
@pyqtSlot(result=str)
def repr(self) -> str:
return self.__repr__()
@pyqtSlot(str, result="QVariant")
def get(self, key) -> ListModel:
return self.dict[key]
@pyqtSlot(str, result=bool)
def has(self, key) -> bool:
return key in self.dict

View File

@ -1,134 +0,0 @@
from typing import Callable, Dict, Optional
from PyQt5.QtCore import (
QModelIndex, QObject, QSortFilterProxyModel, Qt, pyqtProperty, pyqtSignal,
pyqtSlot
)
from .list_model import ListModel
from .list_item import ListItem
SortCallable = Callable[["SortFilterProxy", ListItem, ListItem], bool]
FilterCallable = Callable[["SortFilterProxy", ListItem], bool]
class SortFilterProxy(QSortFilterProxyModel):
sortByRoleChanged = pyqtSignal()
filterByRoleChanged = pyqtSignal()
filterChanged = pyqtSignal()
countChanged = pyqtSignal(int)
def __init__(self,
source_model: ListModel,
sort_by_role: str = "",
filter_by_role: str = "",
sort_func: Optional[SortCallable] = None,
filter_func: Optional[FilterCallable] = None,
reverse: bool = False,
parent: QObject = None) -> None:
error = "{} and {}: only one can be set"
if (sort_by_role and sort_func):
raise TypeError(error.format("sort_by_role", "sort_func"))
if (filter_by_role and filter_func):
raise TypeError(error.format("filter_by_role", "filter_func"))
super().__init__(parent)
self.setDynamicSortFilter(False)
self.setSourceModel(source_model)
source_model.countChanged.connect(self.countChanged.emit)
source_model.changed.connect(self._apply_sort)
source_model.changed.connect(self.invalidateFilter)
self.sortByRole = sort_by_role
self.filterByRole = filter_by_role
self.sort_func = sort_func
self.filter_func = filter_func
self.reverse = reverse
self._filter = None
@pyqtProperty(str, notify=filterChanged)
def filter(self) -> str:
return self._filter
@filter.setter # type: ignore
def filter(self, pattern: str) -> None:
self._filter = pattern
self.invalidateFilter()
self.filterChanged.emit()
self.countChanged.emit(self.rowCount())
# Sorting/filtering methods override
def lessThan(self, index_left: QModelIndex, index_right: QModelIndex
) -> bool:
left = self.sourceModel()[index_left.row()]
right = self.sourceModel()[index_right.row()]
if self.sort_func:
return self.sort_func(self, left, right)
role = self.sortByRole
try:
return getattr(left, role) < getattr(right, role)
except TypeError: # comparison between the two types not supported
return False
def filterAcceptsRow(self, row_index: int, _: QModelIndex) -> bool:
item = self.sourceModel()[row_index]
if self.filter_func:
return self.filter_func(self, item)
return self.filterMatches(getattr(item, self.filterByRole))
# Implementations
def _apply_sort(self) -> None:
order = Qt.DescendingOrder if self.reverse else Qt.AscendingOrder
self.sort(0, order)
def filterMatches(self, string: str) -> bool:
if not self.filter:
return True
string = string.lower()
return all(word in string for word in self.filter.lower().split())
# The rest
def __repr__(self) -> str:
return \
"%s(sortByRole=%r, filterByRole=%r, filter=%r, sourceModel=%s)" % (
type(self).__name__,
self.sortByRole,
self.filterByRole,
self.filter,
"<%s at %s>" % (
type(self.sourceModel()).__name__,
hex(id(self.sourceModel())),
)
)
@pyqtSlot(result=str)
def repr(self) -> str:
return self.__repr__()
@pyqtProperty(int, notify=countChanged)
def count(self) -> int:
return self.rowCount()
def roleNames(self) -> Dict[int, bytes]:
return self.sourceModel().roleNames()

View File

@ -1,149 +0,0 @@
# Copyright 2019 miruka
# This file is part of harmonyqml, licensed under GPLv3.
import logging
import socket
import ssl
import time
from threading import Lock
from typing import Callable, Optional, Tuple
from uuid import UUID
import nio
OptSock = Optional[ssl.SSLSocket]
NioRequestFunc = Callable[..., Tuple[UUID, bytes]]
class NioErrorResponse(Exception):
def __init__(self, response: nio.ErrorResponse) -> None:
self.response = response
super().__init__(str(response))
class RetrySleeper:
def __init__(self) -> None:
self.current_time: float = 0
self.tries: int = 0
def sleep(self, max_time: float) -> None:
self.current_time = max(
0, min((max_time / 10) * (2 ^ (self.tries - 1)), max_time)
)
time.sleep(self.current_time)
self.tries += 1
class NetworkManager:
http_retry_codes = {408, 429, 500, 502, 503, 504, 507}
def __init__(self, host: str, port: int, nio_client: nio.client.HttpClient
) -> None:
self.host = host
self.port = port
self.nio = nio_client
self._ssl_context: ssl.SSLContext = ssl.create_default_context()
self._ssl_session: Optional[ssl.SSLSession] = None
self._lock: Lock = Lock()
def _get_socket(self) -> ssl.SSLSocket:
sock = self._ssl_context.wrap_socket( # type: ignore
socket.create_connection((self.host, self.port), timeout=16),
server_hostname = self.host,
session = self._ssl_session,
)
self._ssl_session = self._ssl_session or sock.session
return sock
@staticmethod
def _close_socket(sock: Optional[socket.socket]) -> None:
if not sock:
return
try:
sock.shutdown(how=socket.SHUT_RDWR)
except OSError: # Already closer by server
pass
sock.close()
def http_disconnect(self) -> None:
try:
self.write(self.nio.disconnect())
except (OSError, nio.ProtocolError):
pass
def read(self, with_sock: OptSock = None) -> nio.Response:
sock = with_sock or self._get_socket()
response = None
while not response:
left_to_send = self.nio.data_to_send()
if left_to_send:
self.write(left_to_send, sock)
self.nio.receive(sock.recv(4096))
response = self.nio.next_response()
if isinstance(response, nio.ErrorResponse):
raise NioErrorResponse(response)
if not with_sock:
self._close_socket(sock)
return response
def write(self, data: bytes, with_sock: OptSock = None) -> None:
sock = with_sock or self._get_socket()
sock.sendall(data)
if not with_sock:
self._close_socket(sock)
def talk(self,
nio_func: NioRequestFunc,
*args,
**kwargs) -> nio.Response:
with self._lock:
retry = RetrySleeper()
while True:
sock = None
try:
sock = self._get_socket()
if not self.nio.connection:
# Establish HTTP protocol connection:
self.write(self.nio.connect(), sock)
to_send = nio_func(*args, **kwargs)[1]
self.write(to_send, sock)
response = self.read(sock)
except (OSError, nio.RemoteTransportError) as err:
self._close_socket(sock)
self.http_disconnect()
retry.sleep(max_time=2)
except NioErrorResponse as err:
logging.error("Nio response error for %s: %s",
nio_func.__name__, err)
self._close_socket(sock)
if err.response.status_code not in self.http_retry_codes:
raise
retry.sleep(max_time=10)
else:
return response

View File

@ -1,164 +0,0 @@
# Copyright 2019 miruka
# This file is part of harmonyqml, licensed under GPLv3.
import functools
import logging as log
import sys
import time
import traceback
from concurrent.futures import Executor, Future
from typing import Any, Callable, Deque, Optional, Tuple, Union
from PyQt5.QtCore import QObject, pyqtProperty, pyqtSignal, pyqtSlot
class PyQtFuture(QObject):
gotResult = pyqtSignal("QVariant")
def __init__(self,
future: Future,
running_value: Any = None,
parent: Optional[QObject] = None) -> None:
super().__init__(parent)
self.future = future
self.running_value = running_value
self._result = None
self.future.add_done_callback(
lambda future: self.gotResult.emit(future.result())
)
def __repr__(self) -> str:
state = ("canceled" if self.cancelled else
"running" if self.running else
"finished")
return "%s(state=%s, value=%r)" % (
type(self).__name__, state, self.value
)
def __lt__(self, other: "PyQtFuture") -> bool:
# This is to allow sorting, e.g. from SortFilterProxy.lessThan()
return self.value < other.value
@pyqtSlot()
def cancel(self):
self.future.cancel()
@pyqtProperty(bool)
def cancelled(self):
return self.future.cancelled()
@pyqtProperty(bool)
def running(self):
return self.future.running()
@pyqtProperty(bool)
def done(self):
return self.future.done()
@pyqtSlot(result="QVariant")
@pyqtSlot(int, result="QVariant")
@pyqtSlot(float, result="QVariant")
def result(self, timeout: Optional[Union[int, float]] = None):
return self.future.result(timeout)
@pyqtProperty("QVariant", notify=gotResult)
def value(self):
return self.future.result() if self.done else self.running_value
def add_done_callback(self, fn: Callable[[Future], None]) -> None:
self.future.add_done_callback(fn)
_Task = Tuple[Executor, Callable, Optional[tuple], Optional[dict]]
_RUNNING: Deque[_Task] = Deque()
_PENDING: Deque[_Task] = Deque()
def futurize(max_running: Optional[int] = None,
consider_args: bool = False,
discard_if_max_running: bool = False,
pyqt: bool = True,
running_value: Any = None) -> Callable:
def decorator(func: Callable) -> Callable:
@functools.wraps(func)
def wrapper(self, *args, **kws) -> Optional[PyQtFuture]:
task: _Task = (
self.pool,
func,
args if consider_args else None,
kws if consider_args else None,
)
def can_run_now() -> bool:
if max_running is not None and \
_RUNNING.count(task) >= max_running:
log.debug("!! Max %d tasks of this kind running: %r",
max_running, task[1:])
return False
if not consider_args or not _PENDING:
return True
log.debug(".. Pending: %r\n Queue: %r", task[1:], _PENDING)
candidate_task = next((
pending for pending in _PENDING
if pending[0] == self.pool and pending[1] == func
), None)
if candidate_task is None:
log.debug(">> No other candidate, starting: %r", task[1:])
return True
if candidate_task[2] == args and candidate_task[3] == kws:
log.debug(">> Candidate is us: %r", candidate_task[1:])
return True
log.debug("XX Other candidate: %r", candidate_task[1:])
return False
if not can_run_now() and discard_if_max_running:
log.debug("\\/ Discarding task: %r", task[1:])
return None
def run_and_catch_errs():
if not can_run_now():
log.debug("~~ Can't start now: %r", task[1:])
_PENDING.append(task)
while not can_run_now():
time.sleep(0.05)
_RUNNING.append(task)
log.debug("Starting: %r", task[1:])
# Without this, exceptions are silently ignored
try:
return func(self, *args, **kws)
except Exception:
traceback.print_exc()
log.error("Exiting thread/process due to exception.")
sys.exit(1)
finally:
del _RUNNING[_RUNNING.index(task)]
future = self.pool.submit(run_and_catch_errs)
return PyQtFuture(
future=future, running_value=running_value, parent=self
) if pyqt else future
return wrapper
return decorator

View File

@ -1,490 +0,0 @@
# Copyright 2019 miruka
# This file is part of harmonyqml, licensed under GPLv3.
from concurrent.futures import ThreadPoolExecutor
from threading import Lock
from typing import Any, Deque, Dict, List, Optional
from PyQt5.QtCore import QDateTime, QObject, pyqtBoundSignal, pyqtSignal
import nio
from nio.rooms import MatrixRoom
from .backend import Backend
from .client import Client
from .model.items import (
Account, Device, ListModel, Room, RoomCategory, RoomEvent, RoomMember,
User
)
from .model.sort_filter_proxy import SortFilterProxy
from .pyqt_future import futurize
Inviter = Optional[Dict[str, str]]
LeftEvent = Optional[Dict[str, str]]
class SignalManager(QObject):
roomCategoryChanged = pyqtSignal(str, str, str, str)
_lock: Lock = Lock()
def __init__(self, backend: Backend) -> None:
super().__init__(parent=backend)
self.pool: ThreadPoolExecutor = ThreadPoolExecutor(max_workers=6)
self.backend = backend
self.last_room_events: Deque[str] = Deque(maxlen=1000)
self._events_in_transfer: int = 0
self.backend.clients.clientAdded.connect(self.onClientAdded)
self.backend.clients.clientDeleted.connect(self.onClientDeleted)
def onClientAdded(self, client: Client) -> None:
if client.userId in self.backend.accounts:
return
# An user might already exist in the model, e.g. if another account
# was in a room with the account that we just connected to
self.backend.users.upsert(
where_main_key_is = client.userId,
update_with = User(
userId = client.userId,
displayName = self.backend.users[client.userId].displayName,
# Devices are added later, we might need to upload keys before
# but we want to show the accounts ASAP in the client side pane
devices = ListModel(),
)
)
# Backend.accounts
room_categories_kwargs: List[Dict[str, Any]] = [
{"name": "Invites", "rooms": ListModel()},
{"name": "Rooms", "rooms": ListModel()},
{"name": "Left", "rooms": ListModel()},
]
for i, _ in enumerate(room_categories_kwargs):
proxy = SortFilterProxy(
source_model = room_categories_kwargs[i]["rooms"],
sort_by_role = "lastEventDateTime",
filter_by_role = "displayName",
reverse = True,
)
room_categories_kwargs[i]["sortedRooms"] = proxy
self.backend.accounts.append(Account(
userId = client.userId,
roomCategories = ListModel([
RoomCategory(**kws) for kws in room_categories_kwargs
]),
))
# Upload our E2E keys to the matrix server if needed
if not client.nio.olm_account_shared:
client.uploadE2EKeys()
# Add all devices nio knows for this account
store = client.nio.device_store
for user_id in store.users:
user = self.backend.users.get(user_id, None)
if not user:
self.backend.users.append(
User(userId=user_id, devices=ListModel())
)
for device in store.active_user_devices(user_id):
self.backend.users[client.userId].devices.upsert(
where_main_key_is = device.id,
update_with = Device(
deviceId = device.id,
ed25519Key = device.ed25519,
trust = client.getDeviceTrust(device),
)
)
# Finally, connect all client signals
self.connectClient(client)
def onClientDeleted(self, user_id: str) -> None:
del self.backend.accounts[user_id]
def connectClient(self, client: Client) -> None:
for name in dir(client):
attr = getattr(client, name)
if isinstance(attr, pyqtBoundSignal):
def onSignal(*args, name=name) -> None:
func = getattr(self, f"on{name[0].upper()}{name[1:]}")
func(client, *args)
attr.connect(onSignal)
@staticmethod
def _get_room_displayname(nio_room: MatrixRoom) -> Optional[str]:
name = nio_room.name or nio_room.canonical_alias
if name:
return name
name = nio_room.group_name()
return None if name == "Empty room?" else name
def _add_users_from_nio_room(self, room: nio.rooms.MatrixRoom) -> None:
for user in room.users.values():
@futurize(running_value=user.display_name)
def get_displayname(self, user) -> str:
# pylint:disable=unused-argument
return user.display_name
self.backend.users.upsert(
where_main_key_is = user.user_id,
update_with = User(
userId = user.user_id,
displayName = get_displayname(self, user),
devices = ListModel()
),
no_update = ("devices",),
)
def _members_sort_func(self, _, left: RoomMember, right: RoomMember
) -> bool:
users = self.backend.users
return users[left.userId].displayName < users[right.userId].displayName
def _members_filter_func(self, proxy: SortFilterProxy, member: RoomMember
) -> bool:
users = self.backend.users
return proxy.filterMatches(users[member.userId].displayName.value)
def onRoomInvited(self,
client: Client,
room_id: str,
inviter: Inviter = None) -> None:
nio_room = client.nio.invited_rooms[room_id]
self._add_users_from_nio_room(nio_room)
categories = self.backend.accounts[client.userId].roomCategories
previous_room = categories["Rooms"].rooms.pop(room_id, None)
previous_left = categories["Left"].rooms.pop(room_id, None)
members = ListModel()
sorted_members = SortFilterProxy(
source_model = members,
sort_func = self._members_sort_func,
filter_func = self._members_filter_func,
)
categories["Invites"].rooms.upsert(
where_main_key_is = room_id,
update_with = Room(
roomId = room_id,
displayName = self._get_room_displayname(nio_room),
topic = nio_room.topic,
inviter = inviter,
lastEventDateTime = QDateTime.currentDateTime(), # FIXME
members = members,
sortedMembers = sorted_members,
),
no_update = ("typingMembers", "members"),
)
categories["Invites"].rooms[room_id].members.updateAll([
RoomMember(userId=user_id) for user_id in nio_room.users
], delete=True)
signal = self.roomCategoryChanged
if previous_room:
signal.emit(client.userId, room_id, "Rooms", "Invites")
elif previous_left:
signal.emit(client.userId, room_id, "Left", "Invites")
def onRoomJoined(self, client: Client, room_id: str) -> None:
nio_room = client.nio.rooms[room_id]
self._add_users_from_nio_room(nio_room)
categories = self.backend.accounts[client.userId].roomCategories
previous_invite = categories["Invites"].rooms.pop(room_id, None)
previous_left = categories["Left"].rooms.pop(room_id, None)
members = ListModel()
sorted_members = SortFilterProxy(
source_model = members,
sort_func = self._members_sort_func,
filter_func = self._members_filter_func,
)
categories["Rooms"].rooms.upsert(
where_main_key_is = room_id,
update_with = Room(
roomId = room_id,
displayName = self._get_room_displayname(nio_room),
topic = nio_room.topic,
members = members,
sortedMembers = sorted_members,
),
no_update = ("typingMembers", "members", "sortedMembers",
"lastEventDateTime"),
)
categories["Rooms"].rooms[room_id].members.updateAll([
RoomMember(userId=user_id) for user_id in nio_room.users
], delete=True)
signal = self.roomCategoryChanged
if previous_invite:
signal.emit(client.userId, room_id, "Invites", "Rooms")
elif previous_left:
signal.emit(client.userId, room_id, "Left", "Rooms")
def onRoomLeft(self,
client: Client,
room_id: str,
left_event: LeftEvent = None) -> None:
categories = self.backend.accounts[client.userId].roomCategories
previous_room = categories["Rooms"].rooms.pop(room_id, None)
previous_invite = categories["Invites"].rooms.pop(room_id, None)
previous = previous_room or previous_invite or \
categories["Left"].rooms.get(room_id, None)
left_time = left_event.get("server_timestamp") if left_event else None
members = ListModel()
sorted_members = SortFilterProxy(
source_model = members,
sort_func = self._members_sort_func,
filter_func = self._members_filter_func,
)
categories["Left"].rooms.upsert(
where_main_key_is = room_id,
update_with = Room(
roomId = room_id,
displayName = previous.displayName if previous else None,
topic = previous.topic if previous else None,
leftEvent = left_event,
lastEventDateTime = (
QDateTime.fromMSecsSinceEpoch(left_time)
if left_time else QDateTime.currentDateTime()
),
members = members,
sortedMembers = sorted_members,
),
no_update = ("members", "sortedMembers", "lastEventDateTime"),
)
signal = self.roomCategoryChanged
if previous_room:
signal.emit(client.userId, room_id, "Rooms", "Left")
elif previous_invite:
signal.emit(client.userId, room_id, "Invites", "Left")
def onRoomSyncPrevBatchTokenReceived(self,
_: Client,
room_id: str,
token: str) -> None:
if room_id not in self.backend.past_tokens:
self.backend.past_tokens[room_id] = token
def onRoomPastPrevBatchTokenReceived(self,
_: Client,
room_id: str,
token: str) -> None:
if self.backend.past_tokens[room_id] == token:
self.backend.fully_loaded_rooms.add(room_id)
self.backend.past_tokens[room_id] = token
def _set_room_last_event(self, user_id: str, room_id: str, event: RoomEvent
) -> None:
for categ in self.backend.accounts[user_id].roomCategories:
if room_id in categ.rooms:
last = categ.rooms[room_id].lastEventDateTime
if last and last > event.dateTime:
continue
# Use setProperty to make sure to trigger model changed signals
categ.rooms.setProperty(
room_id, "lastEventDateTime", event.dateTime
)
def onRoomEventReceived(self,
client: Client,
room_id: str,
etype: str,
edict: Dict[str, Any]) -> None:
def process() -> Optional[RoomEvent]:
# Prevent duplicate events in models due to multiple accounts
if edict["event_id"] in self.last_room_events:
return None
self.last_room_events.appendleft(edict["event_id"])
model = self.backend.roomEvents[room_id]
date_time = QDateTime\
.fromMSecsSinceEpoch(edict["server_timestamp"])
new_event = RoomEvent(
eventId = edict["event_id"],
type = etype,
dateTime = date_time,
dict = edict,
)
event_is_our_profile_changed = (
etype == "RoomMemberEvent" and
edict.get("sender") in self.backend.clients and
((edict.get("content") or {}).get("membership") ==
(edict.get("prev_content") or {}).get("membership"))
)
if event_is_our_profile_changed:
return None
if etype == "RoomCreateEvent":
self.backend.fully_loaded_rooms.add(room_id)
if self._events_in_transfer:
local_echoes_met: int = 0
update_at: Optional[int] = None
# Find if any locally echoed event corresponds to new_event
for i, event in enumerate(model):
if not event.isLocalEcho:
continue
sb = (event.dict.get("sender"), event.dict.get("body"))
new_sb = (new_event.dict.get("sender"),
new_event.dict.get("body"))
if sb == new_sb:
# The oldest matching local echo shall be replaced
update_at = max(update_at or 0, i)
local_echoes_met += 1
if local_echoes_met >= self._events_in_transfer:
break
if update_at is not None:
model.updateItem(update_at, new_event)
self._events_in_transfer -= 1
return new_event
for i, event in enumerate(model):
if event.isLocalEcho:
continue
# Model is sorted from newest to oldest message
if new_event.dateTime > event.dateTime:
model.insert(i, new_event)
return new_event
model.append(new_event)
return new_event
with self._lock:
new_event = process()
if new_event:
self._set_room_last_event(client.userId, room_id, new_event)
def onRoomTypingMembersUpdated(self,
client: Client,
room_id: str,
users: List[str]) -> None:
categories = self.backend.accounts[client.userId].roomCategories
for categ in categories:
try:
categ.rooms.setProperty(room_id, "typingMembers", users)
break
except ValueError:
pass
def onMessageAboutToBeSent(self,
client: Client,
room_id: str,
content: Dict[str, str]) -> None:
date_time = QDateTime.currentDateTime()
with self._lock:
model = self.backend.roomEvents[room_id]
nio_event = nio.events.RoomMessage.parse_event({
"event_id": "",
"sender": client.userId,
"origin_server_ts": date_time.toMSecsSinceEpoch(),
"content": content,
})
event = RoomEvent(
eventId = f"localEcho.{self._events_in_transfer + 1}",
type = type(nio_event).__name__,
dict = nio_event.__dict__,
dateTime = date_time,
isLocalEcho = True,
)
model.insert(0, event)
self._events_in_transfer += 1
self._set_room_last_event(client.userId, room_id, event)
def onRoomAboutToBeForgotten(self, client: Client, room_id: str) -> None:
categories = self.backend.accounts[client.userId].roomCategories
for categ in categories:
categ.rooms.pop(room_id, None)
self.backend.roomEvents[room_id].clear()
def onDeviceIsPresent(self,
client: Client,
user_id: str,
device_id: str,
ed25519_key: str) -> None:
nio_device = client.nio.device_store[user_id][device_id]
user = self.backend.users.get(user_id, None)
if not user:
self.backend.users.append(
User(userId=user_id, devices=ListModel())
)
self.backend.users[user_id].devices.upsert(
where_main_key_is = device_id,
update_with = Device(
deviceId = device_id,
ed25519Key = ed25519_key,
trust = client.getDeviceTrust(nio_device),
)
)
def onDeviceIsDeleted(self, _: Client, user_id: str, device_id: str
) -> None:
try:
del self.backend.users[user_id].devices[device_id]
except ValueError:
pass

View File

@ -1,20 +0,0 @@
import QtQuick 2.7
import QtQuick.Controls 2.2
import QtQuick.Window 2.7
ApplicationWindow {
id: appWindow
visible: true
width: Math.min(Screen.width, 1152)
height: Math.min(Screen.height, 768)
onClosing: Backend.clients.removeAll()
property int reloadedTimes: 0
Loader {
anchors.fill: parent
source: "UI.qml"
objectName: "UILoader"
}
}

View File

@ -1,81 +0,0 @@
# Copyright 2019 miruka
# This file is part of harmonyqml, licensed under GPLv3.
import signal
from pathlib import Path
from typing import Any, Dict, Generator
from PyQt5.QtCore import QObject, QTimer
from PyQt5.QtQml import QQmlApplicationEngine
class Engine(QQmlApplicationEngine):
def __init__(self, debug: bool = False) -> None:
# Connect UNXI signals to properly exit program
self._original_signal_handlers: Dict[int, Any] = {}
for signame in ("INT" , "HUP", "QUIT", "TERM"):
sig = signal.Signals[f"SIG{signame}"] # pylint: disable=no-member
self._original_signal_handlers[sig] = signal.getsignal(sig)
signal.signal(sig, self.onExitSignal)
# Make SIGINT (ctrl-c) work
self._sigint_timer = QTimer()
self._sigint_timer.timeout.connect(lambda: None)
self._sigint_timer.start(100)
super().__init__()
self.app_dir = Path(__file__).resolve().parent
from .backend.backend import Backend
self.backend = Backend(self)
self.rootContext().setContextProperty("Backend", self.backend)
# Setup UI live-reloading when a file is edited
if debug:
from PyQt5.QtCore import QFileSystemWatcher
self._watcher = QFileSystemWatcher()
self._watcher.directoryChanged.connect(lambda _: self.reloadQml())
self._watcher.addPath(str(self.app_dir))
for _dir in list(self._recursive_dirs_in(self.app_dir)):
self._watcher.addPath(str(_dir))
def onExitSignal(self, *_) -> None:
for sig, handler in self._original_signal_handlers.items():
signal.signal(sig, handler)
self._original_signal_handlers.clear()
self.closeWindow()
def _recursive_dirs_in(self, path: Path) -> Generator[Path, None, None]:
for item in path.iterdir():
if item.is_dir() and item.name != "__pycache__":
yield item
yield from self._recursive_dirs_in(item)
def showWindow(self) -> None:
self.load(str(self.app_dir / "components" / "Window.qml"))
def closeWindow(self) -> None:
try:
self.rootObjects()[0].close()
except IndexError:
pass
def reloadQml(self) -> None:
loader = self.rootObjects()[0].findChild(QObject, "UILoader")
source = loader.property("source")
loader.setProperty("source", None)
self.clearComponentCache()
window = self.rootObjects()[0]
reloaded_times = window.property("reloadedTimes")
window.setProperty("reloadedTimes", reloaded_times + 1)
loader.setProperty("source", source)

5
run.py
View File

@ -1,5 +0,0 @@
#!/usr/bin/env python3
from harmonyqml import run
run()

3
run.sh Executable file
View File

@ -0,0 +1,3 @@
#!/usr/bin/env sh
qml src/qml/Window.qml

8
run_live_reload.sh Executable file
View File

@ -0,0 +1,8 @@
#!/usr/bin/env sh
while true; do
clear
qml src/qml/Window.qml -- --debug
exit_code="$?"
if [ "$exit_code" != 231 ]; then break; fi
done

View File

@ -1,75 +0,0 @@
# Copyright 2019 miruka
# This file is part of harmonyqml, licensed under GPLv3.
"harmonyqml setuptools file"
from setuptools import setup, find_packages
from harmonyqml import __about__
def get_readme():
with open("README.md", "r") as readme:
return readme.read()
setup(
name = __about__.__pkg_name__,
version = __about__.__version__,
author = __about__.__author__,
author_email = __about__.__email__,
license = __about__.__license__,
description = __about__.__doc__,
long_description = get_readme(),
long_description_content_type = "text/markdown",
python_requires = ">=3.6, <4",
install_requires = [
"dataclasses;python_version<'3.7'",
"docopt",
],
include_package_data = True,
packages = find_packages(),
# package_data = {__about__.__pkg_name__: ["*.yaml"]},
entry_points = {
"console_scripts": [
f"{__about__.__pkg_name__}={__about__.__pkg_name__}.cli:main"
]
},
keywords = "<KEYWORDS>",
url = "https://github.com/mirukan/harmonyqml",
classifiers=[
"Development Status :: 3 - Alpha",
# "Development Status :: 4 - Beta",
# "Development Status :: 5 - Production/Stable",
"Intended Audience :: Developers",
"Intended Audience :: End Users/Desktop",
"Environment :: Console",
# "Environment :: Console :: Curses",
# "Environment :: Plugins",
# "Environment :: X11 Applications",
# "Environment :: X11 Applications :: Qt",
# "Topic :: Utilities",
# grep '^Topic' ~/docs/web/pypi-classifiers.txt
("License :: OSI Approved :: "
"GNU General Public License v3 or later (GPLv3+)"),
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.7",
"Natural Language :: English",
"Operating System :: POSIX",
]
)

View File

@ -1,6 +1,3 @@
# Copyright 2019 miruka
# This file is part of harmonyqml, licensed under GPLv3.
"""<SHORTDESC>""" """<SHORTDESC>"""
__pkg_name__ = "harmonyqml" __pkg_name__ = "harmonyqml"

1
src/__init__.py Normal file
View File

@ -0,0 +1 @@
from .app import APP

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

83
src/app.py Normal file
View File

@ -0,0 +1,83 @@
import asyncio
from concurrent.futures import Future
from pathlib import Path
from threading import Thread
from typing import Any, Coroutine, Dict, List, Optional, Sequence
from uuid import uuid4
from appdirs import AppDirs
from . import __about__
from .events.system import CoroutineDone, AppExitRequested
class App:
def __init__(self) -> None:
self.appdirs = AppDirs(appname=__about__.__pkg_name__, roaming=True)
self.backend = None
self.loop = asyncio.get_event_loop()
self.loop_thread = Thread(target=self._loop_starter)
self.loop_thread.start()
def start(self, cli_flags: Sequence[str] = ()) -> bool:
debug = False
if "-d" in cli_flags or "--debug" in cli_flags:
self._run_in_loop(self._exit_on_app_file_change())
debug = True
from .backend import Backend
self.backend = Backend(app=self) # type: ignore
return debug
async def _exit_on_app_file_change(self) -> None:
from watchgod import awatch
async for _ in awatch(Path(__file__).resolve().parent):
AppExitRequested(231)
def _loop_starter(self) -> None:
asyncio.set_event_loop(self.loop)
self.loop.run_forever()
def _run_in_loop(self, coro: Coroutine) -> Future:
return asyncio.run_coroutine_threadsafe(coro, self.loop)
def call_backend_coro(self,
name: str,
args: Optional[List[str]] = None,
kwargs: Optional[Dict[str, Any]] = None) -> str:
# To be used from QML
coro = getattr(self.backend, name)(*args or [], **kwargs or {})
uuid = str(uuid4())
self._run_in_loop(coro).add_done_callback(
lambda future: CoroutineDone(uuid=uuid, result=future.result())
)
return uuid
def pdb(self, additional_data: Sequence = ()) -> None:
# pylint: disable=all
ad = additional_data
ba = self.backend
cl = self.backend.clients # type: ignore
tcl = lambda user: cl[f"@test_{user}:matrix.org"]
import json
jd = lambda obj: print(json.dumps(obj, indent=4, ensure_ascii=False))
import pdb
pdb.set_trace()
APP = App()

122
src/backend.py Normal file
View File

@ -0,0 +1,122 @@
import asyncio
import json
from pathlib import Path
from typing import Dict, Optional, Tuple
from atomicfile import AtomicFile
from .app import App
from .matrix_client import MatrixClient
SavedAccounts = Dict[str, Dict[str, str]]
CONFIG_LOCK = asyncio.Lock()
class Backend:
def __init__(self, app: App) -> None:
self.app = app
self.clients: Dict[str, MatrixClient] = {}
def __repr__(self) -> str:
return f"{type(self).__name__}(clients={self.clients!r})"
# Clients management
async def login_client(self,
user: str,
password: str,
device_id: Optional[str] = None,
homeserver: str = "https://matrix.org") -> None:
client = MatrixClient(
user=user, homeserver=homeserver, device_id=device_id
)
await client.login(password)
self.clients[client.user_id] = client
async def resume_client(self,
user_id: str,
token: str,
device_id: str,
homeserver: str = "https://matrix.org") -> None:
client = MatrixClient(
user=user_id, homeserver=homeserver, device_id=device_id
)
await client.resume(user_id=user_id, token=token, device_id=device_id)
self.clients[client.user_id] = client
async def logout_client(self, user_id: str) -> None:
client = self.clients.pop(user_id, None)
if client:
await client.close()
async def logout_all_clients(self) -> None:
await asyncio.gather(*(
self.logout_client(user_id) for user_id in self.clients.copy()
))
# Saved account operations - TODO: Use aiofiles?
@property
def saved_accounts_path(self) -> Path:
return Path(self.app.appdirs.user_config_dir) / "accounts.json"
@property
def saved_accounts(self) -> SavedAccounts:
try:
return json.loads(self.saved_accounts_path.read_text())
except (json.JSONDecodeError, FileNotFoundError):
return {}
async def has_saved_accounts(self) -> bool:
return bool(self.saved_accounts)
async def load_saved_accounts(self) -> Tuple[str, ...]:
async def resume(user_id: str, info: Dict[str, str]) -> str:
await self.resume_client(
user_id = user_id,
token = info["token"],
device_id = info["device_id"],
homeserver = info["homeserver"],
)
return user_id
return await asyncio.gather(*(
resume(uid, info) for uid, info in self.saved_accounts.items()
))
async def save_account(self, client: MatrixClient) -> None:
await self._write_config({
**self.saved_accounts,
client.userId: {
"hostname": client.nio.host,
"token": client.nio.access_token,
"device_id": client.nio.device_id,
}
})
async def forget_account(self, user_id: str) -> None:
await self._write_config({
uid: info
for uid, info in self.saved_accounts.items() if uid != user_id
})
async def _write_config(self, accounts: SavedAccounts) -> None:
js = json.dumps(accounts, indent=4, ensure_ascii=False, sort_keys=True)
with CONFIG_LOCK:
self.saved_accounts_path.parent.mkdir(parents=True, exist_ok=True)
with AtomicFile(self.saved_accounts_path, "w") as new:
new.write(js)

0
src/events/__init__.py Normal file
View File

Binary file not shown.

Binary file not shown.

Binary file not shown.

23
src/events/event.py Normal file
View File

@ -0,0 +1,23 @@
from enum import Enum
from dataclasses import dataclass
import pyotherside
class AutoStrEnum(Enum):
@staticmethod
def _generate_next_value_(name, *_):
return name
@dataclass
class Event:
def __post_init__(self) -> None:
# CPython >= 3.6 or any Python >= 3.7 needed for correct dict order
args = [
# pylint: disable=no-member
getattr(self, field)
for field in self.__dataclass_fields__ # type: ignore
]
pyotherside.send(type(self).__name__, *args)

36
src/events/rooms.py Normal file
View File

@ -0,0 +1,36 @@
from datetime import datetime
from typing import Dict, Optional
from dataclasses import dataclass, field
from .event import Event
@dataclass
class RoomUpdated(Event):
room_id: str = field()
display_name: Optional[str] = None
avatar_url: Optional[str] = None
topic: Optional[str] = None
last_event_date: Optional[datetime] = None
inviter: Optional[Dict[str, str]] = None
left_event: Optional[Dict[str, str]] = None
@dataclass
class RoomDeleted(Event):
room_id: str = field()
@dataclass
class RoomMemberUpdated(Event):
room_id: str = field()
user_id: str = field()
typing: bool = field()
@dataclass
class RoomMemberDeleted(Event):
room_id: str = field()
user_id: str = field()

View File

@ -0,0 +1,8 @@
from datetime import datetime
from typing import Dict, Optional
from dataclasses import dataclass, field
from .event import Event

16
src/events/system.py Normal file
View File

@ -0,0 +1,16 @@
from typing import Any
from dataclasses import dataclass, field
from .event import Event
@dataclass
class AppExitRequested(Event):
exit_code: int = 0
@dataclass
class CoroutineDone(Event):
uuid: str = field()
result: Any = None

52
src/events/users.py Normal file
View File

@ -0,0 +1,52 @@
from datetime import datetime
from enum import Enum
from typing import Optional
from dataclasses import dataclass, field
from .event import Event
# Logged-in accounts
@dataclass
class AccountUpdated(Event):
user_id: str = field()
@dataclass
class AccountDeleted(Event):
user_id: str = field()
# Accounts and room members details
@dataclass
class UserUpdated(Event):
user_id: str = field()
display_name: Optional[str] = None
avatar_url: Optional[str] = None
# Devices
class Trust(Enum):
blacklisted = -1
undecided = 0
trusted = 1
@dataclass
class DeviceUpdated(Event):
user_id: str = field()
device_id: str = field()
ed25519_key: str = field()
trust: Trust = Trust.undecided
display_name: Optional[str] = None
last_seen_ip: Optional[str] = None
last_seen_date: Optional[datetime] = None
@dataclass
class DeviceDeleted(Event):
user_id: str = field()
device_id: str = field()

View File

Before

Width:  |  Height:  |  Size: 304 B

After

Width:  |  Height:  |  Size: 304 B

View File

Before

Width:  |  Height:  |  Size: 146 B

After

Width:  |  Height:  |  Size: 146 B

View File

Before

Width:  |  Height:  |  Size: 1.7 KiB

After

Width:  |  Height:  |  Size: 1.7 KiB

View File

Before

Width:  |  Height:  |  Size: 1.9 KiB

After

Width:  |  Height:  |  Size: 1.9 KiB

View File

Before

Width:  |  Height:  |  Size: 1.4 KiB

After

Width:  |  Height:  |  Size: 1.4 KiB

View File

Before

Width:  |  Height:  |  Size: 1.5 KiB

After

Width:  |  Height:  |  Size: 1.5 KiB

View File

Before

Width:  |  Height:  |  Size: 339 B

After

Width:  |  Height:  |  Size: 339 B

View File

Before

Width:  |  Height:  |  Size: 86 B

After

Width:  |  Height:  |  Size: 86 B

View File

Before

Width:  |  Height:  |  Size: 378 B

After

Width:  |  Height:  |  Size: 378 B

View File

Before

Width:  |  Height:  |  Size: 250 B

After

Width:  |  Height:  |  Size: 250 B

View File

Before

Width:  |  Height:  |  Size: 153 B

After

Width:  |  Height:  |  Size: 153 B

View File

Before

Width:  |  Height:  |  Size: 288 B

After

Width:  |  Height:  |  Size: 288 B

View File

Before

Width:  |  Height:  |  Size: 347 B

After

Width:  |  Height:  |  Size: 347 B

View File

Before

Width:  |  Height:  |  Size: 364 B

After

Width:  |  Height:  |  Size: 364 B

View File

Before

Width:  |  Height:  |  Size: 344 B

After

Width:  |  Height:  |  Size: 344 B

View File

Before

Width:  |  Height:  |  Size: 471 B

After

Width:  |  Height:  |  Size: 471 B

View File

Before

Width:  |  Height:  |  Size: 235 B

After

Width:  |  Height:  |  Size: 235 B

View File

Before

Width:  |  Height:  |  Size: 446 B

After

Width:  |  Height:  |  Size: 446 B

View File

Before

Width:  |  Height:  |  Size: 585 B

After

Width:  |  Height:  |  Size: 585 B

View File

Before

Width:  |  Height:  |  Size: 811 B

After

Width:  |  Height:  |  Size: 811 B

View File

Before

Width:  |  Height:  |  Size: 121 B

After

Width:  |  Height:  |  Size: 121 B

View File

Before

Width:  |  Height:  |  Size: 1.7 KiB

After

Width:  |  Height:  |  Size: 1.7 KiB

View File

Before

Width:  |  Height:  |  Size: 327 B

After

Width:  |  Height:  |  Size: 327 B

View File

Before

Width:  |  Height:  |  Size: 344 B

After

Width:  |  Height:  |  Size: 344 B

View File

Before

Width:  |  Height:  |  Size: 1.8 MiB

After

Width:  |  Height:  |  Size: 1.8 MiB

22
src/matrix_client.py Normal file
View File

@ -0,0 +1,22 @@
from typing import Optional
import nio
class MatrixClient(nio.AsyncClient):
def __init__(self,
user: str,
homeserver: str = "https://matrix.org",
device_id: Optional[str] = None) -> None:
super().__init__(homeserver=homeserver, user=user, device_id=device_id)
def __repr__(self) -> str:
return "%s(user_id=%r, homeserver=%r, device_id=%r)" % (
type(self).__name__, self.user_id, self.homeserver, self.device_id
)
async def resume(self, user_id: str, token: str, device_id: str) -> None:
self.receive_response(nio.LoginResponse(user_id, device_id, token))

View File

@ -0,0 +1,57 @@
import QtQuick 2.7
ListModel {
// To initialize a HListModel with items,
// use `Component.onCompleted: extend([{"foo": 1, "bar": 2}, ...])`
id: listModel
function extend(new_items) {
for (var i = 0; i < new_items.length; i++) {
listModel.append(new_items[i])
}
}
function getIndices(where_role, is, max) { // max: undefined or int
var results = []
for (var i = 0; i < listModel.count; i++) {
if (listModel.get(i)[where_role] == is) {
results.push(i)
if (max && results.length >= max) {
break
}
}
}
return results
}
function getWhere(where_role, is, max) {
var indices = getIndices(where_role, is, max)
var results = []
for (var i = 0; i < indices.length; i++) {
results.push(listModel.get(indices[i]))
}
return results
}
function upsert(where_role, is, new_item) {
// new_item can contain only the keys we're interested in updating
var indices = getIndices(where_role, is, 1)
if (indices.length == 0) {
listModel.append(new_item)
} else {
listModel.set(indices[0], new_item)
}
}
function pop(index) {
var item = listModel.get(index)
listModel.remove(index)
return item
}
}

Some files were not shown because too many files have changed in this diff Show More