Full Text Search with IndexedDB
17 points by singpolyma
17 points by singpolyma
Do people actually use IndexedDB at scale/in real world sites?
I remember when it was introduced - in the era of "browsers should provide low level APIs that let people implement the high level interfaces they want rather than then the actual functionality they want". A view that seems to largely have disappeared, though I don't know how much of that is WASM allowing people to compile and many C/C++ libraries that provide this kind of functionality but with the need for a memory model closer to a non-virtualized environment.
In the case if IDB there was another problem: No SQL implementation implemented just the SQL standard, and as a result it was functionally defined as "MySQL", which is not ideal for a standard.
Yes, IndexedDB is used a lot. You can run SQLite in WASM nowadays, but only if you’re OK with a ~1MB download and some awkward file locking behaviors. And it’s overkill for a lot of stuff. IndexedDB has an awful API, but wrappers like Dexie take away a lot of the pain.
it was functionally defined as "MySQL“
I think you mean “SQLite”, which was the case with the earlier WebSQL API.
I mean based on many sites, 1mb is apparently fine :D
And yup, absolutely sqlite, complete brainfart :D
Humerously before IDB we had WebSQL which was more "just the thing you want" but people didn't like being basically married to Sqlite and so we got the lower-level IDB. Which is usually implemented on Sqlite anyway IIRC.
This approach will apply to anything that gives you simple BTree style indexes though.
Right, that's what I was referring to (though brainfarted to mysql). The problem was WebSQL was basically becoming "whatever Sqlite does" rather than any actual standard. e.g. a compatible implementation would have to work out the actual specification and then implement every part of sqlite.
I have a client that uses IndexedDB (via RxDB) somewhat successfully in prod, though it's basically just a read cache / write buffer.
The fact that the user has no visibility or control over the data stored there means that you can't really use it in any more serious way. For example, you couldn't use it as a primary storage for a local-first app.
One of my "dream technologies" is browsers offering a synchronized IndexedDB (or equivalent) just like they offer synchronized bookmarks, passwords, etc.
I'm curious why you think you couldn't use it as primary storage for a local app? That seems to be both what its designed for and what it's best at.
I say that because browsers don't give users any visibility over the databases, or any way to manage them, and actually make it exceedingly easy for users to accidentally delete them (Safari even automatically deletes dbs after 7 days, if you don't visit the website that saved them). So I don't feel like it can be trusted as a primary store.
You can use it as a secondary to cache data stored elsewhere, and as a write buffer while the user is offline. But for the primary database, the source of truth of user data, you need something in the cloud. Or, if you want to offload to the user the responsibility of managing the data, you can build some sort of export to file / import from file functionality into your app. But I'd argue at that point the primary database becomes the file.
If you have cloud storage doesn't that mean the user can even less see the db is manage it? I understand that most users won't open dev tools to do this with indexdedb, but with cloud the data is gone from their computer entirely.
I think safari extends the time if you request persistent storage. I know they extend it if user installs app to home screen.
If you have cloud storage doesn't that mean the user can even less see the db is manage it?
Definitely, but at least their data is saved somewhere. It doesn't vanish if they spill tea on their keyboard and fry their disk, or if they simply clear their browsing data.
So as a developer I can't only use IndexedDb as a primary storage of my app. I either need to set up a companion cloud service, or I need to build a way for my users to export / import their data. (Though again, in that case I'd say the exported file is the primary db.)
As a developer, I would really like to be able to use IndexedDb without having to care about any of that, because (for example) the browser tells the user "this website is saving data to this local database, which will be backed up and synced in Firefox Sync / Chrome Sync / iCloud". (So essentially I'd like browsers to offer a sync engine + cloud storage.)
The user could also back up their computer, which would include the indexeddb data.
In realize in practise no user does this 😔