Batching multiple updates to the leaky bucket at a fixed interval
improves the accuracy of the rate limiter. Previously the rate would
drop substantially over the course of the sync operation.
This commit: 1e8366a0e8 broke the loading of playlists. The first results coming back from the Spotify API were not being added to the playlists array.
This replaces #36 and adds some other fixes!
Execution speed should be much faster now, especially when there are not
many changes to synchronize.
* Maintain track cache between different playlists (thanks to @joshrmcdaniel for amazing work on that!)
* Fix incorrect tidal_playlist_is_dirty() implementation
* Remove more redundant API calls
* Avoid unnecessarily spinning up tasks for tracks that were in match failure cache
* Introduce new rate_limit configuration parameter implemented with leaky bucket rate-limiting algorithm
* Where possible, add new tracks to existing playlist instead of erasing the old ones
* Use asyncio multithreading instead of multiprocessing
* When user has large number of spotify playlists, fetch them in parallel instead of one by one
* More typing hints / typing fixes
I've been having an issue where the script hangs for a long time
when starting up. It seems to be caused by the connection request
to Spotify failing on the first request (which uses IPv6) and
having to wait for the retry attempt using IPv4
2s should be plenty for just getting the refresh token and is
much more tolerable in case of IPv6 failure
This change introduces an sqlite database that contains the track_id,
db insertion time, and ttl in the cache. The ttl starts with one week,
and increases exponentially by a factor of 2 each time the same track_id
is added to the database.
This significantly reduces the execution of the time script when there
are a lot of match failures accumulating, which do not need to check
every time.
* Use tidal objects directly when available to avoid some unnecessary http requests
* Only try to repeat a failed sync attempt if HTTPError
* Print the details of the HTTPError to help with debugging
* Use staggered delay times to increase chance of success
I was passing (playlist, None) in the case where the --uri argument was used,
and None means create a new playlist.
I've fixed it to look for the Tidal playlist if it exists like the other
code paths do, and only pass None when it doesn't exist.
Unfortunately this will mean that your Tidal collection might have been polluted with
duplicate playlists. I've added a delete playlist function to tidalapi_patch to help
with deleting these programmatically, which some people might find useful.
Fixes#8
