Batching multiple updates to the leaky bucket at a fixed interval
improves the accuracy of the rate limiter. Previously the rate would
drop substantially over the course of the sync operation.
This commit: 1e8366a0e8 broke the loading of playlists. The first results coming back from the Spotify API were not being added to the playlists array.
This replaces #36 and adds some other fixes!
Execution speed should be much faster now, especially when there are not
many changes to synchronize.
* Maintain track cache between different playlists (thanks to @joshrmcdaniel for amazing work on that!)
* Fix incorrect tidal_playlist_is_dirty() implementation
* Remove more redundant API calls
* Avoid unnecessarily spinning up tasks for tracks that were in match failure cache
* Introduce new rate_limit configuration parameter implemented with leaky bucket rate-limiting algorithm
* Where possible, add new tracks to existing playlist instead of erasing the old ones
* Use asyncio multithreading instead of multiprocessing
* When user has large number of spotify playlists, fetch them in parallel instead of one by one
* More typing hints / typing fixes
This change introduces an sqlite database that contains the track_id,
db insertion time, and ttl in the cache. The ttl starts with one week,
and increases exponentially by a factor of 2 each time the same track_id
is added to the database.
This significantly reduces the execution of the time script when there
are a lot of match failures accumulating, which do not need to check
every time.
