Performance optimization / refactoring (#43)

This replaces #36 and adds some other fixes!

Execution speed should be much faster now, especially when there are not
many changes to synchronize.

* Maintain track cache between different playlists (thanks to @joshrmcdaniel for amazing work on that!)
* Fix incorrect tidal_playlist_is_dirty() implementation
* Remove more redundant API calls
* Avoid unnecessarily spinning up tasks for tracks that were in match failure cache
* Introduce new rate_limit configuration parameter implemented with leaky bucket rate-limiting algorithm
* Where possible, add new tracks to existing playlist instead of erasing the old ones
* Use asyncio multithreading instead of multiprocessing
* When user has large number of spotify playlists, fetch them in parallel instead of one by one
* More typing hints / typing fixes
This commit is contained in:
Tim Rae
2024-06-03 09:11:56 +02:00
committed by GitHub
parent 689637510d
commit 1e8366a0e8
5 changed files with 207 additions and 165 deletions

View File

@@ -14,6 +14,6 @@ spotify:
#excluded_playlists:
# - spotify:playlist:1ABCDEqsABCD6EaABCDa0a
# number of concurrent subprocesses when searching tracks in a playlist.
# increasing this value can improve sync speed, but may increase 429 errors
subprocesses: 25
# increasing these parameters should increase the search speed, while decreasing reduces likelihood of 429 errors
max_concurrency: 10 # max concurrent connections at any given time
rate_limit: 12 # max sustained connections per second