Badger is claimed to be the fastest key-value store available in the Go world in most if not in any circumstances and there’s a temptation to use it in most if not any cases. So did we for the https://way.today backend. And met the condition where the choice was bad. Well, not as bad as “it did not work” or “even we had to go away from it” but still “we decided the switch to boltdb would make our life easier”.
The speed of Badger does not come from nowhere. It is result of memory pre-caching and spare structs. One gets write speed boost with the cost of needing extra RAM, write logs and compact procedures. And this cost is acceptable in case if you have really huge amount of writing. We did not. We got troubles without profit. The first trouble was the backend became memory hog. It is not critical to claim 500Mb of the resident memory nowdays, but what for? MySQL got less in the same conditions. Next we have to add extra go-routine to compact the database files. And finally we’ve still got a huge, very huge dataset of vlog’s and have to monitor the disk space because the problem of notorious
Value log GC attempt didn’t result in any cleanup faced by our team and others does not seem to be solved at the moment.
Yes, everything above might be ignored if there’s a stream of unfiltered IoT messages, but Way.Today backend gets already well-formed, filtered data and have nothing to do with but just store. Moreover it has some internal precaching features where it is needed. So. It was fun and useful experience and i will definitely use Badger in the future, but https://way.today is the case it is overkill.