Skip to main content

Docker or Proxmox? Something else entirely?

New server has been acquired. Debian 13 has been installed.

GS308EP switches have been acquired and installed.

Now, I'm working to migrate to the new machine. 3 1/2 years ago when I started futzing with Docker, I sorta followed guides and guessed, abused it trying to make it do things it wasn't designed for, and flipped switches I likely shouldn't have flipped, so the set up is more than a little shabby.

As a result, I'll likely end more redeploying than migrating the containers.

So rather than go forward with Docker blindly, I want to reassess whether I shouldn't look into Proxmox, LXC, or Podman instead of Docker, or maybe something else entirely?

Work is just about done dumping ESX for Nutanix, but both of those seem overkill for my needs.

Of course the forums for any of the solutions make their own out to be the best thing since sliced bread and the others useless, so I'm hoping to get a more nuanced answer here.

in reply to JeanValjean

Vendetta9076
Vendetta9076
• •
I use proxmox has the base OS to deploy different virtual machines/LXCs depending on what I need. I have a dedicated docker lxc among those. Say what you will about docker, and man do people have opinions, but proxmox is probably the best way to run a multi lxc/VM setup. Highly recommend regardless of what else you do.
in reply to Vendetta9076

SpikesOtherDog
SpikesOtherDog
• •
I am using proxmox to host my docker machine as well. I'm also using it to host a W11 machine for just-in-casies. Third machine so far is a VM for testing.
in reply to Vendetta9076

abeorch
abeorch
•
I happened to be searching on running docker inside lxc today. There were quite a few suggestions on configs to make them not go kurplunk. I think i might follow them..

Selfhosted reshared this.

in reply to abeorch

uncloaked
uncloaked
• •
I used proxmox helper scripts for portainer lxc and would get my host system kernel panicked from backing up the lxc that had a NFS share mounted with fstab. Solved it with moving to komodo lxc and setting the NFS share being mounted directly to the container with Docker Compose.if you decide portainer you can set that up with stacks feature. Hope that helps.
in reply to abeorch

Scrubbles
Scrubbles
• •
Only pain will you find down that path. I did that for years, but it's a pain. You have to disable so many security features, and I found it to be incredibly brittle. I found myself fearing all proxmox upgrades because each time it would break the lxcs. I wish you luck
in reply to abeorch

felbane
felbane
• •

It's not worth the headache IMO. Just run a docker VM and use lxc for the one-off systems that you want to experiment with.

I have a "production" docker VM and a "sandbox" docker VM and prod only ever runs compose files that I've vetted in sandbox. Super stable, basically bulletproof, and still has the flexibility to experiment and break stuff without affecting my core services.

in reply to Vendetta9076

iturnedintoanewt
iturnedintoanewt
• •
I have a dedicated LXC container called docker where I install all my docker images.
in reply to Vendetta9076

Imacat
Imacat
• •
Same here and it’s worked well. I migrated my arr stack from another machine where it was in docker and just couldn’t see a reason to spend a bunch of time rebuilding it with LXCs.
in reply to JeanValjean

Blastboom Strice
Blastboom Strice
• •
Maybe you could try nixos?
in reply to Blastboom Strice

Creat
Creat
• •
Especially without any additional context is knowledge about their background, directed at someone closely only starting out, this is incredibly bad advice.
in reply to JeanValjean

glizzyguzzler
glizzyguzzler
• •

If you’ve got Debian already installed, I cannot resist advocating for Incus (stable branch from Zabbly repo with web ui blog.simos.info/how-to-install…) in lieu of proxmox. Does the same thing but you don’t have to rip out the kernel Debian uses.

With Debian 13 you have access to podman quadlets, use that for any non-vm needs. The ease of docker compose files easily removes reason for programs in LXC containers, and podman removes reason for docker in an LXC. LXC is left only for programs that aren’t containerized. VMs for security DMZ. Podman for bulk of stuff you want.

Good luck!


How to install and setup the Incus Web UI

Share this: Click to share on X (Opens in new window) X Click to share on Facebook (Opens in new window) Facebook Click to share on Reddit (Opens in new window) Reddit Click to email a link to a friend (Opens i…
Simos Xenitellis (Mi blog lah!)
in reply to glizzyguzzler

tofubl
tofubl
• •
I was in a similar spot not too long ago, setting up a firewall and general network box. I was going to go with Proxmox but a fellow Lemmy guy strongly advocated for Incus on top of vanilla Debian. I was intrigued and ended up going for it. Learned a lot about networking with systemd (bridging, IP assignment and so on) for things I could have gotten for free in Proxmox (literally a few clicks), and had to fight Incus to work with a FreeBSD VM for Opnsense, but I love the setup now. Pure debian with a few Incus VMs and Docker inside of those as needed. So clean!
This entry was edited (4 hours ago)
in reply to tofubl

koala
koala
• •

Came in here to mention Incus if no one had.

I love it. I have three "home production" servers running Proxmox, but mostly because Proxmox is one of very few LTS/comercially-supported ways to run Linux in a supported way with root (and everything else on ZFS). And while its web UI is still a bit clunky in places, it comes in handy some times.

However, Incus automation is just... superior. incus launch --vm images:debian/13 foo, wait a few seconds then incus exec foo -- bash and I'm root on a console of a ready-to-go Debian VM. Without --vm, it's a lightweight LXC container. And Ansible supports running commands through incus exec, so you can provision stuff WITHOUT BOTHERING TO SET UP ANYTHING.

AND, it works remotely without fuss, so I can set up an Incus remote on a beefy server and spawn VMs nearly transparently. + incus file pull|push to transfer files.

I'm kinda pondering scripting removal of the Proxmox bits from a Proxmox install, so that I just keep their ZFS support and run Incus on top.

in reply to JeanValjean

Nico198X
Nico198X
• •

i just came into containerization recently and skipped Docker completely and went straight to podman. it's been fantastic and integrates well with systemd. i would recommend anyone new to do the same.

since you're not new, and are looking for something different, i would say do podman instead of docker.

in reply to Nico198X

reluctant_squidd
reluctant_squidd
• •

I concur. Podman is superior in my opinion. It’s more secure by default (rootless containers) and can do pretty much everything docker can do naively (you can literally alias docker to podman in your shell and it will work)

It’s not as easy to find info on some of the systemd specific stuff (Quadlets), but once you figure that out, it’s pretty amazing.

I ended up making up my own scripts to allow me to create new system users, pre-loaded with aliases and shortcut functions to make my life easier ( automatic quadlet container file generation, pre-set network rules, etc), but it is not required.

All the info is there, but starting out it can be a bit overwhelming.

My containers are pretty much self sufficient now. I just intervene when something needs major updating or config changes

in reply to JeanValjean

chocrates
chocrates
• •

Proxmox is a hypervisor. I would install that as your os if that is something you want.

If you just need containers, podman is good. Docker is fine, but podman is open source.

That being said, I run proxmox on my server and have various vms running. I have a laptop that I will use to play with containers and if it turns into a service I want I'll make it run on proxmox, either as a container itself or as something running on a VM on the hypervisor

This entry was edited (3 hours ago)
in reply to JeanValjean

Creat
Creat
• •

Proxmox and Docker don't really do the same thing. They live in the same area, but the coverage is very different. You can always use docker when your host is running proxmox: either individually or in groups inside of an lxc, or all in w dedicated VM, or even natively on the same house if you prefer chaos. But you can't do the opposite: Sometimes you just need a VM. Maybe you only need a couple of devices, and you know they run on or are even designed for docker, then that's the better option. In all other cases, and when just getting started, proxmox is just the way more universal solution if you're only planning on having a single host (for now).

The management tools in proxmox are great. The community scripts are a fantastic resource and only work with proxmox. I would suggest you set it up natively, not on top of Debian though, even if that's already installed. Not the least of the reasons are to be able to use ZFS easily, including on on the boot partition (select that in the installer).

Finally, if you're gonna stick with docker, like others said: consider podman. That really does the same thing docker does, but it's fully open source. Arguably it's better in some areas, but on the flip side might, in occasion, require fiddling with something intended specifically for docker and using advanced setups.

Also there really is no wrong answer, either. And you can always change whatever you choose.

in reply to Creat

abeorch
abeorch
•
Alot of apps seem to come with Docker images - can you use those with Podman?

Selfhosted reshared this.

in reply to abeorch

Creat
Creat
• •
Yes. That's basically the point. They call it a "drop in replacement", but last I used it manually there were some extra steps for what I wanted to do. To be clear: not for every thing you want to setup, just one if the things I read don't up required extra steps. But I also hear that those things have changed since then and it's mostly seamless now.
in reply to JeanValjean

Lka1988
Lka1988
• •

I run Proxmox in my small cluster. Proxmox runs multiple VMs that each run various groups of docker containers.

They're on different levels.

This entry was edited (3 hours ago)
in reply to Lka1988

tazeycrazy
tazeycrazy
• •
Yea I'm the same. planning on making some VMS/docker nodes, Highly available while others can be kicked to the kerb if there are issues.
in reply to JeanValjean

frongt
frongt
• •

Proxmox can run lxc containers natively.

Personally I keep a Debian VM for docker, a holdover from before hypervisors supported containers natively. I use docker compose and it Just Works™.

in reply to frongt

tazeycrazy
tazeycrazy
• •
I have three docker nodes on three Debean VMs on three proxmox servers. I should play around with lxcs and native containers on proxmox but it would mess up with my tri lateral system.
in reply to JeanValjean

abeorch
abeorch
•

I am not in a position to judge but Id be interested to hear comments on this position.

From the Proxmox Discussion Forum

Linux Containers (= LXCs) and docker containers are something completely different. LXCs are system containers. These contain a full OS except for the kernel which is shared with the host. You will have to administrate them like you would do it with a VM.
Docker containers are application containers where you containerize a single appliance. You don't individualize/upgrade them. You just throw them away and create a new one. So you are quite limited what you can do with them, as all you can configure is what the container creator wants you to be able to change.

So if you want a service as secure and independent as possible, or if you want to use Win/Mac/FreeBSD, use a VM.
If you want something like a VM running Linux, but you want less overhead, at the cost of security, use a LXC.
If you just want to run some services but you don't want to invest time on learning how they work or administrating them, then running a VM with docker would be a good choice.

Selfhosted reshared this.

⇧