• 0 Posts
  • 58 Comments
Joined 2 years ago
cake
Cake day: June 12th, 2023

help-circle
  • Really depends on what data it is and whether you want to search it regularly or just as a one time thing.

    You could load them into an rdbms (MySQL/Postgres) and have it handle the indexing, or use python tools to process the files. Something like elasticsearch could work too.

    If it’s just a one time thing grep is probably fine tho.

    Aleph could work as well but I have no experience with it.

    I guess it depends on how much time you want to invest in setting something up versus how much time you’d lose waiting for grep to finish (if you only need to search a certain column, you can create an index with just that column using awk, search that index file, then extract the full line from the source file based on that result, but at that point you’re basically creating a new database engine).

















  • jonne@infosec.pubtohomeassistant@lemmy.world*Permanently Deleted*
    link
    fedilink
    English
    arrow-up
    7
    ·
    edit-2
    3 months ago

    I personally run it on a MiniPC that runs proxmox. You could run other stuff on it through proxmox as well. I would definitely recommend running always on server stuff on something other than your main daily PC.

    You could probably run all the other stuff on there as well, although transcoding might slow things down a bit. HA itself should be fine as once you’ve set up all your automations it’s not like you need to interact with it directly and you’ll just get a little bit of lag at worst.