For example, you have recipients, subject, and sender as JSON fields, when you could have just a headers field with all of them, and even add the rest of the headers in the message.

If it's performance related, you can still have headers as a single json blob and then use generated columns for the specific fields.

For example

  CREATE TABLE IF NOT EXISTS "messages" (
    "id" INTEGER NOT NULL PRIMARY KEY, -- internal id
    "message_id" TEXT NOT NULL, -- Gmail message id
    "thread_id" TEXT NOT NULL, -- Gmail thread id
    "headers" JSON NOT NULL, -- JSON object of { "header": value },
    "subject" TEXT GENERATED ALWAYS AS (json_extract("headers", '$.Subject')) VIRTUAL NOT NULL)
    ...
  );
  CREATE INDEX subjectidx on messages(subject);

  ALTER TABLE messages ADD dkim TEXT GENERATED ALWAYS AS (json_extract("headers", '$."Dkim-Signature"')) VIRTUAL NOT NULL);
  CREATE INDEX dkimidx on messages(dkim);
  SELECT dkim, COUNT(0) FROM messages GROUP BY dkim;

https://github.com/terhechte/postsack

Gmail to SQLite describes 6 steps to get credentials working, but it is not true for me. After 6 steps:

- that Google said that my app was not published, so I published it

- Google said that app cannot be internal, because I am not a workspace user

- for external apps

- then it said I cannot use the app until it is verified

- in verification they wanted to know domain, address, other details

- they wanted to have my justification for scopes

- they wanted to have video explaining how the app is going to be used

- they will take some time to verify the data I provided them

It all looks like a maze of settings, where requiring any of users to go above the hoops required by Google is simply too much.

Links:

[0] https://github.com/rumca-js/Django-link-archive

It is now syncing my messages, but very slowly. Some Async magic could probably be cool :)

The schema from AOX always looked really good to me, but I never have gotten to really giving it a try. I wanted to use it, primarily, to get analytics about my mail and for search (not a daily-driver IMAP server).

It is easy to fix though, since I can get Google Take Out (is that the name?) which I think is free and then parse file files once downloaded.

Still using this tool would be faster from a get it going perspective.

Feature request: parse email content to extract unsubscribe links and allow me to unsubscribe from most frequent senders easily

I admittedly might just not have or understand the use case nor have I thought about how large a Gmail account actually is so feel free to ignore if I'm missing something!

I think it's a big eml file.

To me having to install a tool through Python is a show-stopper.

[0] https://github.com/djipko/gbackup-rs

* Google collects vast amounts of personal data, specifically through receiving all of your email and analyizing it.

* It builds elaborate user profiles and uses them to target you with ads designed to better influence you.

* Its hold on information (from different sources) has made it excessively powerful economically, and thus also politically.

* Google/Alphabet has long started to affect legislation, including through direct registered lobbying: ~15 Million USD in 2024 (opensecrets.org).

* It has been known to pass, and likely still passes, the information it collects - including copies of your email correspondence - on to the US government (Edward Snowden leaks).

and finally:

* There are multiple email providers, many of them quite good - both for pay and gratis. Naturally most of the gratis ones have their own interests, but nothing like Google.