Bridging Swift with LMDB-backed C code during a large architecture refactor
I helped the company integrate their custom-made embedded LMDB-backed database with the rest of the iOS app. This performance-optimized database made use of memory-mapping to FlatBuffer objects — which meant it did not need memory copying or a deserialization step for its structured data to be accessed, enabling much faster data access speeds.
However, integrating this with a Swift codebase brought several challenges — some of them discovered through deep debugging of crash reports during the migration phase. Here are a few:
- The memory behind items fetched from the database was unowned — its lifetime was tied to its corresponding DB reader transaction object. This does not naturally work well with Swift's memory management, where the lifetime of variables is managed through automatic reference counting — which makes using normal Swift interfaces very error-prone.
- Holding those reader transaction objects open for an extended time (e.g., by storing those variables in a SwiftUI view) could cause large storage usage growth as well as outdated query responses in other parts of the program, which was also error-prone.
- To avoid deadlocks associated with nested queries (which are hard to avoid or check at compile-time), reader transactions could be inherited if one was already open on the same thread, via thread-state dictionaries. Later I found that reader transactions are dangerous to use in async functions, as Swift does not guarantee that a function will resume on the same thread after an async suspension point. This was a third factor that made using this database even more error-prone in Swift code.
- There were several scattered portions of the codebase which were previously getting their data from the network through a different interface from the one provided by the database. All of those would need to be migrated while trying to keep regression risk low.
To address these challenges, I made use of new and advanced Swift 6 language features to perform the refactor in a cost-effective way, while reducing regression risks, and increasing overall interface safety:
- The transaction object types were scoped to only the Swift database wrapper module, to minimize the risk of misuse of such objects and instead provide a safer interface to the layers above it.
- Types related to those unowned database objects were migrated to use Swift's new
~Copyableprotocol, and the functions in the Swift database wrapper module were refactored to only borrow those objects to their callers via Swift's new `borrowing` parameter ownership modifier. Together these ensured safety to the layers above it, by introducing compile-time checks that prevented those three issues related to transaction lifetime and async concerns mentioned above. - To minimize logical regression risk during such a large refactor, I built a universal interface that would work well when fetching data from either the network or the database. That universal interface was built using Swift's
AsyncStream, which models Nostr relay interactions very naturally. This allowed the migration to be done in phases, and to minimize the amount of logical changes needed for each file.