Large feeds and RFC 5005

4 points by fanf


snej

Before a feed is written to disk a lock is acquired so we only have one SQLite writer at a time. This is slow and cannot be changed unless moving away from SQLite.

You solve this by batching the updates, like by having a dedicated goroutine that reads updates from a channel, collects a batch, and inserts them in one transaction.

SQLite is super fast, but you have to play by its rules.