Keep in mind that your descendents probably won’t care about a huge majority of what you leave them. Photos annotated with a date, time, people in them, and an explanation, maybe, but generally my generation hasn’t given a shit about the tonnes of books, music, photos, furniture, knick knacks, and antiquities bequeathed to us. It would be bizarre if our kids didn’t maintain that tradition.
Bear in mind, though, that the technology for dealing with these things are rapidly advancing.
I have an enormous amount of digital archives I’ve collected both from myself and from my now-deceased father. For years I just kept them stashed away. But about a year ago I downloaded the Whisper speech-to-text model from OpenAI and transcribed everything with audio into text form. I now have a Qwen3 LLM in the process of churning through all of those transcripts writing summaries of their contents and tagging them based on subject matter. I expect pretty soon I’ll have something with good enough image recognition that I can turn loose on the piles of photographs to get those sorted out by subject matter too. Eventually I’ll be able to tell my computer “give me a brief biography of Uncle Pete” and get something pretty good out of all that.
Yeah, boo AI, hallucinations, and so forth. This project has given me first-hand experience with what they’re currently capable of and it’s quite a lot. I’d be able to do a ton more if I wasn’t restricting myself to what can run on my local GPU. Give it a few more years.
I agree. I keep loads of shot that I’m hoping one day will just be processed by an AI to pick out the stuff people might want to actually see.
“People” includes me. I don’t delete anything (when it comes to photos, videos, etc) and just assume at some point technology will make it easy to find whatever.
It’s a bit technical, I haven’t found any pre-packaged software to do what I’m doing yet.
First I installed https://github.com/openai/whisper , the speech-to-text model that OpenAI released back when they were less blinded by dollar signs. I wrote a Python script that used it to go through all of the audio files in the directory tree where I’m storing this stuff and produced a transcript that I stored in a .json file alongside it.
For the LLM, I installed https://github.com/LostRuins/koboldcpp/releases/ and used the https://huggingface.co/unsloth/Qwen3-30B-A3B-128K-GGUF model, which is just barely small enough to run smoothly on my RTX 4090. I wrote another Python script that methodically goes through those .json files that Whisper produced, takes the raw text of the transcript, and feeds it to the LLM with a couple of prompts explaining what the transcript is and what I’d like the LLM to do with it (write a summary, or write a bullet-point list of subject tags). Those get saved in the .json file too.
Most recently I’ve been experimenting with creating an index of the transcripts using those LLM results and the Whoosh library in Python, so that I can do local searches of the transcripts based on topics. I’m building towards writing up something where I can literally tell it “Tell me about Uncle Pete” and it’ll first search for the relevant transcripts and then feed those into the LLM with a prompt to extract the relevant information from them.
If you don’t find the idea of writing scripts for that sort of thing literally fun (like me) then you may need to wait a bit for someone more capable and more focused than I am to create a user-friendly application to do all this. In the meantime, though, hoard that data. Storage is cheap.
If you don’t find the idea of writing scripts for that sort of thing literally fun…
I absolutely do. What I find as a potential showstopper for me right now, is that I don’t have a nonintegrated GPU, which makes complex LLMs hard to run. Basically, if I can’t push the processing to CPU, I’m looking at around 2-5 seconds per token; it’s rough. But I like your workflow a lot, and I’m going to try to get something similar going with my incredibly old hardware, and see if CPU-only processing of this would be something feasible (though, I’m not super hopeful there).
And, yes, I, too, am aware of the hallucinations and such that come from the technology. But, honestly, for this non-critical use case, I don’t really care.
I only just recently discovered that my installation of Whisper was completely unaware that I had a GPU, and was running entirely on my CPU. So even if you can’t get a good LLM running locally you might still be able to get everything turned into text transcripts for eventual future processing. :)
It sounds like something similar to RAG (retrieval augmented generation) or a database lookup. Are you storing the transcripts in a SQL like database or noSQL db or doing semantic similarity on any of it?
I was thinking of a similar project and building a knowledge graph for each person.
If you’re interested in “chatting” with your writing there’s a couple of out of the box solutions right now, like Kortex or Reflect Notes. They are AI first note taking apps. I don’t use them out of privacy concerns but if you don’t care that much they might allow you to do what you want. They claim to be E2E encrypted and the AI unable to phone home but these are companies that sprung out of nowhere so I don’t trust they necessarily have done all their homework to actually provide full privacy.
Alternatively there’s an Obsidian plugin that I believe allows you to do such a thing as well with local LLMs if you wanted to which is the privacy first way to this. I’ve just moved to Obsidian from Capacities so I have yet to try it out as I’m still setting up my vault.
Privacy first is my only path. There are a lot of privacyless solutions for this, and they’re all dead to me. The obsidian route is pretty cool. Personally, I don’t care to chat with it, but I like the auto-tags and auto-summaries.
I think it would be interesting to have some kind of global archive. Even if descendants don’t care “now” has the potential to be the beginning of the best documented era in history. Historians would kill for photographs by random average people from any other time.
A lot of people thought that that’s what the Internet would be, but that’s obviously not the case. And I know the “right to be forgotten” is a thing, and deservedly so, but at some point you’re throwing out the wine with the amphora.
Yup. My parents aren’t even in ill health, let alone dead, but we recently took all the old VHS tapes, including a lot of OTA recordings, and a significant number of DVDs, and dumped them. Recordings of talking with relatives got digitized, same way you’d keep family photos.
I have no expectation that people keep my junk. I’ll pass on a handful of stuff like identifying photos of people and places, but nobody wants or needs the 500 photos of my cat. Even I don’t want that many, but storage is cheap enough that I don’t bother to delete the useless ones.
I’m trying to curate a few hundred photos for my kids. I’ve written a couple of bios of relatives. I’d like to record something like a story for them. If they want to trash it, that’s fine, but at least there will be something meaningful for them if they want it.
My wife’s parents recently passed. It took months to slog through their stuff and my wife was over it only weeks in. She dumped so much but constantly fights with herself for both taking more than she wanted/needed to and yet less that what she feels she should have. We’ve told our daughter multiple times “our stuff May mean a lot to us, it doesn’t have to mean anything at all to you. If you don’t want it, never feel bad dumping/selling/letting it go.” Out of all the stuff we all collect in life just by living, barely anything has any sentimental value.
On one hand I’ve got a huge collection of photos and albums I’ve taken and collected. I’m trying to clear some out as I go… but I’m not looking forward to that process when my parents go. My dad’s an avid photographer and I know he has a few hundred thousand photos, most of which are near duplicates and he rarely cleans them up.
Nobody wanted my grandparents collected crap. Or their photos. Or their books. I tried giving them away. I tried consignment. I tried posting them on Facebook. Most ended up in a landfill.
My kids aren’t really interested in the movies I like. They actively avoid the music I listen to. I’ve gotten them copies of the books I love and they give up after a few pages. They get bored with the games I played as a kid.
My dad loves Zen and the Art of Motorcycle Maintenance, the Whole Earth Catalog, and Bruce Springsteen. I do not. If he wills me his copies, I will keep some out of guilt and then my kids will have to throw them away.
Keep in mind that your descendents probably won’t care about a huge majority of what you leave them. Photos annotated with a date, time, people in them, and an explanation, maybe, but generally my generation hasn’t given a shit about the tonnes of books, music, photos, furniture, knick knacks, and antiquities bequeathed to us. It would be bizarre if our kids didn’t maintain that tradition.
Bear in mind, though, that the technology for dealing with these things are rapidly advancing.
I have an enormous amount of digital archives I’ve collected both from myself and from my now-deceased father. For years I just kept them stashed away. But about a year ago I downloaded the Whisper speech-to-text model from OpenAI and transcribed everything with audio into text form. I now have a Qwen3 LLM in the process of churning through all of those transcripts writing summaries of their contents and tagging them based on subject matter. I expect pretty soon I’ll have something with good enough image recognition that I can turn loose on the piles of photographs to get those sorted out by subject matter too. Eventually I’ll be able to tell my computer “give me a brief biography of Uncle Pete” and get something pretty good out of all that.
Yeah, boo AI, hallucinations, and so forth. This project has given me first-hand experience with what they’re currently capable of and it’s quite a lot. I’d be able to do a ton more if I wasn’t restricting myself to what can run on my local GPU. Give it a few more years.
I agree. I keep loads of shot that I’m hoping one day will just be processed by an AI to pick out the stuff people might want to actually see.
“People” includes me. I don’t delete anything (when it comes to photos, videos, etc) and just assume at some point technology will make it easy to find whatever.
You said you released it on your writing. How did you go about doing that? It’s a cool use case, and I’m intrigued.
It’s a bit technical, I haven’t found any pre-packaged software to do what I’m doing yet.
First I installed https://github.com/openai/whisper , the speech-to-text model that OpenAI released back when they were less blinded by dollar signs. I wrote a Python script that used it to go through all of the audio files in the directory tree where I’m storing this stuff and produced a transcript that I stored in a .json file alongside it.
For the LLM, I installed https://github.com/LostRuins/koboldcpp/releases/ and used the https://huggingface.co/unsloth/Qwen3-30B-A3B-128K-GGUF model, which is just barely small enough to run smoothly on my RTX 4090. I wrote another Python script that methodically goes through those .json files that Whisper produced, takes the raw text of the transcript, and feeds it to the LLM with a couple of prompts explaining what the transcript is and what I’d like the LLM to do with it (write a summary, or write a bullet-point list of subject tags). Those get saved in the .json file too.
Most recently I’ve been experimenting with creating an index of the transcripts using those LLM results and the Whoosh library in Python, so that I can do local searches of the transcripts based on topics. I’m building towards writing up something where I can literally tell it “Tell me about Uncle Pete” and it’ll first search for the relevant transcripts and then feed those into the LLM with a prompt to extract the relevant information from them.
If you don’t find the idea of writing scripts for that sort of thing literally fun (like me) then you may need to wait a bit for someone more capable and more focused than I am to create a user-friendly application to do all this. In the meantime, though, hoard that data. Storage is cheap.
That’s awesome! Thank you!
I absolutely do. What I find as a potential showstopper for me right now, is that I don’t have a nonintegrated GPU, which makes complex LLMs hard to run. Basically, if I can’t push the processing to CPU, I’m looking at around 2-5 seconds per token; it’s rough. But I like your workflow a lot, and I’m going to try to get something similar going with my incredibly old hardware, and see if CPU-only processing of this would be something feasible (though, I’m not super hopeful there).
And, yes, I, too, am aware of the hallucinations and such that come from the technology. But, honestly, for this non-critical use case, I don’t really care.
I only just recently discovered that my installation of Whisper was completely unaware that I had a GPU, and was running entirely on my CPU. So even if you can’t get a good LLM running locally you might still be able to get everything turned into text transcripts for eventual future processing. :)
Nicceeeee! Thank you!
It sounds like something similar to RAG (retrieval augmented generation) or a database lookup. Are you storing the transcripts in a SQL like database or noSQL db or doing semantic similarity on any of it?
I was thinking of a similar project and building a knowledge graph for each person.
If you’re interested in “chatting” with your writing there’s a couple of out of the box solutions right now, like Kortex or Reflect Notes. They are AI first note taking apps. I don’t use them out of privacy concerns but if you don’t care that much they might allow you to do what you want. They claim to be E2E encrypted and the AI unable to phone home but these are companies that sprung out of nowhere so I don’t trust they necessarily have done all their homework to actually provide full privacy.
Alternatively there’s an Obsidian plugin that I believe allows you to do such a thing as well with local LLMs if you wanted to which is the privacy first way to this. I’ve just moved to Obsidian from Capacities so I have yet to try it out as I’m still setting up my vault.
Privacy first is my only path. There are a lot of privacyless solutions for this, and they’re all dead to me. The obsidian route is pretty cool. Personally, I don’t care to chat with it, but I like the auto-tags and auto-summaries.
Looked through the plugins and found this: https://github.com/niehu2018/obsidian-ai-tagger-universe
I think it would be interesting to have some kind of global archive. Even if descendants don’t care “now” has the potential to be the beginning of the best documented era in history. Historians would kill for photographs by random average people from any other time.
A lot of people thought that that’s what the Internet would be, but that’s obviously not the case. And I know the “right to be forgotten” is a thing, and deservedly so, but at some point you’re throwing out the wine with the amphora.
Doesn’t archive.org provide that?
In theory yes, but not a lot of people are uploading their family photo albums AFAIK.
No, we do have that. Social media is a gold mine for analysis, both for modern sociology and for future archaeology.
Yup. My parents aren’t even in ill health, let alone dead, but we recently took all the old VHS tapes, including a lot of OTA recordings, and a significant number of DVDs, and dumped them. Recordings of talking with relatives got digitized, same way you’d keep family photos.
I have no expectation that people keep my junk. I’ll pass on a handful of stuff like identifying photos of people and places, but nobody wants or needs the 500 photos of my cat. Even I don’t want that many, but storage is cheap enough that I don’t bother to delete the useless ones.
You only have 500 photos of your cat? Is your camera broken? Got the cat yesterday?
I’m trying to curate a few hundred photos for my kids. I’ve written a couple of bios of relatives. I’d like to record something like a story for them. If they want to trash it, that’s fine, but at least there will be something meaningful for them if they want it.
Assuming it survives the climate wars. 🫠
My wife’s parents recently passed. It took months to slog through their stuff and my wife was over it only weeks in. She dumped so much but constantly fights with herself for both taking more than she wanted/needed to and yet less that what she feels she should have. We’ve told our daughter multiple times “our stuff May mean a lot to us, it doesn’t have to mean anything at all to you. If you don’t want it, never feel bad dumping/selling/letting it go.” Out of all the stuff we all collect in life just by living, barely anything has any sentimental value.
On one hand I’ve got a huge collection of photos and albums I’ve taken and collected. I’m trying to clear some out as I go… but I’m not looking forward to that process when my parents go. My dad’s an avid photographer and I know he has a few hundred thousand photos, most of which are near duplicates and he rarely cleans them up.
Just think. At least you can sell off those nick-nacks. What value is there in digital goods you don’t want?
Nobody wanted my grandparents collected crap. Or their photos. Or their books. I tried giving them away. I tried consignment. I tried posting them on Facebook. Most ended up in a landfill.
Hadn’t thought of it like that. I wish I could at least donate my digital library, though.
Yeah - I’m totally for full, real, actual ownership of digital stuff, and we should be able to give it away.
But I’d be surprised if my kids would be interested in more than a tiny fraction of it. Or anyone else, for that matter.
Pretty sure theyd love a literal metric shit ton of free and cracked content that fits ontop of their pinky nail.
My kids aren’t really interested in the movies I like. They actively avoid the music I listen to. I’ve gotten them copies of the books I love and they give up after a few pages. They get bored with the games I played as a kid.
My dad loves Zen and the Art of Motorcycle Maintenance, the Whole Earth Catalog, and Bruce Springsteen. I do not. If he wills me his copies, I will keep some out of guilt and then my kids will have to throw them away.