1
0
Fork 0
mirror of https://github.com/dani-garcia/vaultwarden.git synced 2025-06-17 17:00:10 +00:00

Optimize CipherSyncData for very large vaults

As mentioned in #3111, using a very very large vault causes some issues.
Mainly because of a SQLite limit, but, it could also cause issue on
MariaDB/MySQL or PostgreSQL. It also uses a lot of memory, and memory
allocations.

This PR solves this by removing the need of all the cipher_uuid's just
to gather the correct attachments.

It will use the user_uuid and org_uuid's to get all attachments linked
to both, weither the user has access to them or not. This isn't an
issue, since the matching is done per cipher and the attachment data is
only returned if there is a matching cipher to where the user has access to.

I also modified some code to be able to use `::with_capacity(n)` where
possible. This prevents re-allocations if the `Vec` increases size,
which will happen a lot if there are a lot of ciphers.

According to my tests measuring the time it takes to sync, it seems to
have lowered the duration a bit more.

Fixes #3111
This commit is contained in:
BlackDex 2023-01-11 20:23:53 +01:00
parent 367e1ce289
commit 3181e4e96e
No known key found for this signature in database
GPG key ID: 58C80A2AA6C765E1
4 changed files with 35 additions and 26 deletions

View file

@ -187,10 +187,15 @@ impl Attachment {
}}
}
pub async fn find_all_by_ciphers(cipher_uuids: &Vec<String>, conn: &mut DbConn) -> Vec<Self> {
// This will return all attachments linked to the user or org
// There is no filtering done here if the user actually has access!
// It is used to speed up the sync process, and the matching is done in a different part.
pub async fn find_all_by_user_and_orgs(user_uuid: &str, org_uuids: &Vec<String>, conn: &mut DbConn) -> Vec<Self> {
db_run! { conn: {
attachments::table
.filter(attachments::cipher_uuid.eq_any(cipher_uuids))
.left_join(ciphers::table.on(ciphers::uuid.eq(attachments::cipher_uuid)))
.filter(ciphers::user_uuid.eq(user_uuid))
.or_filter(ciphers::organization_uuid.eq_any(org_uuids))
.select(attachments::all_columns)
.load::<AttachmentDb>(conn)
.expect("Error loading attachments")