• 0 Posts
  • 158 Comments
Joined 2 years ago
cake
Cake day: June 30th, 2023

help-circle










  • I use EndeavourOS (which is arch).

    Most of my programming is web stuff. So it builds to containers and using VS Codes dev containers takes care of all issues relating to arch’s rolling release (IE needing a specific version of a language).
    Ie, I work in containers and I build to container and I run containers for all my code (except ESP32 platformio. Unfortunately I haven’t migrated that away from windows. So I dual boot)

    If I was doing GUI desktop apps, I imagine I would need something other than dev containers.
    But that’s not what I do. Considering all I do is docker, k8s, linux admin, web frontend/backend that is platform agnostic (but ultimately runs on Linux)… I’m not tied to any OS.
    Windows is annoying, I am not a fan of osx nor Apple, I use Linux everyday… So my OS might as well be Linux.
    And Arch & EndeavourOS are nice and just work.
    For a VPS/server, I use Debian (or Talos OS for k8s). But that’s all headless.

    I’m lucky in that I freelance and the companies I work for are good companies.
    I’ve never had to cancel anything because of EndeavourOS, it’s never broken on me (I’ve only had windows break the EFI partition, which it can do to any distro - until you disable fast boot and stuff), it’s never gotten in my way (or if it has, it’s lead to a better solution - like VS Code dev containers). It’s been really really enjoyable.

    I’m sure that I could use any distro in my position, tbh.
    So, probably not helpful overall.






  • Oh, operators are absolutely the way for “released” things.

    But on bigger projects with lots of different pods etc, it’s a lot of work to make all the CRD definitions, hook all the events, and write all the code to deploy the pods etc.
    Similar to helm charts, I don’t see the point for personal projects. I’m not sharing it with anyone, I don’t need helm/operator abstraction for it.
    And something like cdk8s will generate the yaml for you to inspect. So you can easily validate that you are “doing the right thing” before slinging it into k8s.


  • Everyone talks about helm charts.
    I tried them and hate writing them.
    I found garden.io, and it makes a really nice way to consume repos (of helm charts, manifests etc) and apply them in a sensible way to a k8s cluster.
    Only thing is, it seems to be very tailored to a team of developers. I kinda muddled through with it, and it made everything so much easier.
    Although I massively appreciate that helm charts are used for most projects, they make sense for something you are going to share.
    But if it’s a solo project or consuming other people’s projects, I don’t think it really solves a problem.

    Which is why I used garden.io. Designed for deploying kubernetes manifests, I found it had just enough tooling to make things easier.
    Though, if you are used to ansible, it might make more sense to use ansible.
    Pretty sure ansible will be able to do it all in a way you are familiar with.

    As for writing the manifests themselves, I find it rare I need to (unless it’s something I’ve made myself). Most software has a k8s helm chart. So I just reference that in a garden file, set any variables I need to, and all good.
    If there aren’t helm charts or kustomize files, then it’s adapting a docker compose file into manifests. Which is manual.
    Occasionally I have to write some CRDs, config maps or secrets (CMs and secrets are easily made in garden).

    I also prefer to install operators, instead of the raw service. For example, I use Cloudnative Postgres to set up postgres databases.
    I create a CRD that defines the database, and CNPG automatically provisions all the storage, pods, services, config maps and secrets.

    The way I use kubernetes for the projects I do is:
    Apply all the infrastructure stuff (gateways, metallb, storage provisioners etc) from helm files (or similar).
    Then apply all my pods, services, certificates etc from hand written manifests.
    Using garden, I can make sure things are deployed in the correct order: operators are installed before trying to apply a CRD, secrets/cms created before being referenced etc.
    If I ever have to wipe and reinstall a cluster, it takes me 30 minutes or so from a clean TalosOS install to the project up and running, with just 3 or 4 commands.

    Any on-the-fly changes I make, I ensure I back port to the project configs so when I wipe, reset, reinstall I still get what I expect.

    However, I have recently found https://cdk8s.io/ and I’m meaning to investigate that for creating the manifests themselves.
    Write code using a typed language, and have cdk8s create the raw yaml manifests. Seems like a dream!
    I hate writing yaml. Auto complete is useless (the editor has no idea what format the yaml doc should take), auto formatting is useless (mostly because yaml is whitespace sensitive, and the editor has no idea what things are a child or a new parent). It just feels ugly and clunky.


  • So uplink is 500/500.
    LAN speed tests at 1000/1000.
    WAN is 100/400.
    VPN is 8/8.

    I’m guessing the VPN is part of your homelab? Or do you mean a generic commercial VPN (like pia or proton)?

    How does the domain resolve on the LAN? Is it split horizon (so local ip on the lan, public IP on public DNS)?
    Is the homelab on a separate subnet/vlan from the computer you ran the speed test from? Or the same subnet?