ECS is really the only way I'd go with Docker - So far in my experience, Docker in production is really a lot harder than the casual developer / devops person realizes based on the "getting started with docker" tutorials.
My conclusion with Docker is that, in general™, you really need to have a justifiable reason to go whole-hog into Docker, especially if you're not on AWS / considering ECS.
I'm glad the article covers ECS, as it makes a lot of the scheduling / config issues simpler!
ECS is AWS-specific, which is perfectly fine for some. But Kubernetes has been amazing for us. It abstracts many of the differences between AWS/Google Cloud, it's open source, and is far more powerful and flexible.
The only issue right now is that setting the cluster up involves running some shell scripts (yuck). We use Google Container Engine (hosted Kubernetes on Google Cloud), so we don't have to deal with that, but the option is there should we ever need to go multi-cloud.
Figured I'd toss that out there for anyone struggling with ECS (it can be a bit rigid) or keeping an eye on things beyond AWS. Kubernetes is still young and rough in areas, but it is a nice, opinionated way to orchestrate containers.
wow so instead of running my configuration management stuff I know need to either:
- be on aws
- be on gcloud
- any other cloud that has docker support out of the box
OR:
I need docker + a configuration management to setup my environment, or I need to somehow manage my coreos configs.
Oh and finally don't forget to run a network over a network cause it's dockerish so for most clouds this means we run a network on a network which runs a network.
docker adds so much complexity. people just don't see this right upfront and use most of their time into these stuff, but there are easier ways to deploy.
YES if you are really big and if your servers needs to scale way beyond the most than you need it probably since configuration management won't help you and setting up servers even in the cloud take some time looking at some netflix articles.
however i just don't get it why people use their time for docker when there are other things to do in their programs.
But it doesn't really, it is just another tool to learn like your configuration management software. Honestly, learning Docker to a high level was WAY easier than getting my Chef knowledge to a mediocre level. Further, it has other benefits like being MUCH easier to set up well isolated dev environments with Docker than with Chef. Since we switched we can usually have someone running the dev environment and tests within an hour or so of unboxing their laptop. With chef running the whole thing from scratch took that long by itself (after you installed all the software) and usually failed. And the idea that it is too complicated for something simple- I built a sample blogging app that you can deploy to a digital ocean or ec2 machine with like 3 or 4 commands (https://github.com/pbecotte/devblog). The system we have built at work has 8 separate services between a bunch of data stores, background workers, and a couple different pieces of our app- but Docker allows you to run the entire environment on your local machine and then deploy that same setup to our cluster, without any differences to account for making all those apps run on one VM for your local environment (which can get bad when one service requires a different version of a ruby gem than another for example).
How do you do the same thing without a cloud Environment?
Do you install the software manually?
Do you configure your network manually?
Do you configure your os manually?
Do you install the docker daemon manually?
What about installations behind firewalls?
Or about code that shouldn't belong to the docker registry since it shouldn't be pushed over the internet ?
etc...
Why do you have 8 separted services anyway?
How many people does your company have? for 8 services you should at least have 8 * 3 people.
I second that. Docker makes it very easy to separate the whole application into a bunch of isolated services and then reproduce the whole production environment on a laptop for a very convenient development.
> wow so instead of running my configuration management stuff I know need to either: - be on aws - be on gcloud - any other cloud that has docker support out of the box
No. You need a networked computer with a quasi-recent kernel. There are no requirements to use a cloud provider or to use AWS/GCP.
> OR: I need docker + a configuration management to setup my environment, or I need to somehow manage my coreos configs. Oh and finally don't forget to run a network over a network cause it's dockerish so for most clouds this means we run a network on a network which runs a network.
This sounds like aimless rambling. You don't have to use any of this. You can, of course, but I can do some nasty stuff without Docker as well. Like I mentioned with Container Engine, setting up a Kubernetes cluster is like two mouse clicks. You can get as simple as ECS, or you can roll your own from the ground up. You have the option to pick a point on a spectrum. With ECS, it's Amazon's way or the highway.
> however i just don't get it why people use their time for docker when there are other things to do in their programs.
Because it saves us loads of time developing, testing, and deploying our systems. There is more initial setup work, but after that we are deploying images seamlessly, have a great rolling upgrade and rollback story out of the box, and get a lot of other bonuses like service discovery, auto-scaling (vertically and horizontally), and much better (higher) resource usage levels. And we're far from a mega-corp.
To me, it sounds like you may have skimmed some, perhaps even played a bit. But you ran into a snag, threw up your hands, and have summarily dismissed an entire ecosystem after your experience(s). I see enough un-informed or 100% incorrect things above to think you're missing some details.
> Because it saves us loads of time developing, testing, and deploying our systems. There is more initial setup work, but after that we are deploying images seamlessly, have a great rolling upgrade and rollback story out of the box, and get a lot of other bonuses like service discovery, auto-scaling (vertically and horizontally), and much better (higher) resource usage levels. And we're far from a mega-corp.
Question what benefits do you get with docker than without?
Without I have Cgroups, my software is versioned i.e.:
programm-X.X.X.jar / programm-X.X.X.rpm / whatever.
Why do you get better / higher resource usage I mean you use an additional layer you could do the same with just CGroups.
How do you do service discovery? Etcd could be installed without docker?! How do you auto-scaling without a cloud? I mean even without docker auto-scaling is trivial.
> Question what benefits do you get with docker than without? Without I have Cgroups, my software is versioned i.e.: programm-X.X.X.jar / programm-X.X.X.rpm / whatever. Why do you get better / higher resource usage I mean you use an additional layer you could do the same with just CGroups. How do you do service discovery? Etcd could be installed without docker?! How do you auto-scaling without a cloud? I mean even without docker auto-scaling is trivial.
Not to be dismissive, but why would we want to use cgroups? Docker works great for us. There are indeed tons of alternatives to everything. Why cgroups? Why not BSD and jails? Why not build on unikernels and VM images?
> Why do you get better / higher resource usage I mean you use an additional layer you could do the same with just CGroups.
Kubernetes (and most other Docker orchestration systems) come with a scheduler. Containers are allocated based on desired characteristics to the machines with the most spare capacity. You can cut down on idle capacity by intelligently spreading the load without having to think very much about it.
You could do the same with cgroups, but you're going to need to write an orchestration system, a scheduler, and you probably won't have the massive communities that some of the alternatives have already built.
> How do you do service discovery? Etcd could be installed without docker?!
Yes, it can be installed without Docker if you'd like. We run some things in Docker, and others outside ourselves. It's no silver bullet.
> How do you auto-scaling without a cloud? I mean even without docker auto-scaling is trivial.
Docker is just one piece of the puzzle, which is something you seem to be missing. In our case, Kubernetes brings it all together into an easy-to-use package.
Life is full of few absolutes. It's great if your cgroup deploy is working for you, it's just not a great fit for everyone. The comparison between Docker and cgroups is a bit apples to oranges, too. In reality, you need to compare a lot more than just the tech, the Docker ecosystem is the biggest appeal to us.
I can't speak for the commenter, but for me the ... concern ... with using shell scripts is the complete lack of understanding of what they are going to do. It's not exactly a lack of trust for the Kubernetes team as much as when it goes toes-up (not if), then one needs to be able to understand what it did in order to understand what it didn't.
I haven't used their cluster provisioning in a while, but back when I did it made a lot of modifications to AWS before falling over, at which point I need to either undo those changes or the scripts need to be smart enough to resume where they left off. Shell scripts are not well suited to that purpose, in my experience.
Yes, I am aware of their salt stack procedure, but it doesn't hold a candle to the simplicity of Kubernetes on CoreOS, which unlike the 23+ directories worth of salt things to read is something you would probably fit on 3 A4 sheets of paper: https://github.com/coreos/coreos-kubernetes/tree/master/mult... I provisioned a Vagrant cluster using the Kubernetes salt mechanism just for this comment, and the "Cockpit" username and password didn't work after it was finished. Which salt config contains that information? Beats me. I'm thankful that `vagrant ssh master` did as expected.
I recognize that the previous paragraph is just _my_ experience and _my_ preferences, but the anecdote is the reason why (in my experience) one must understand what the "magic" deployment process is doing, and only after that voluntarily cede control to the scripts merely as a labor-saving tactic. It does me no good to have a cluster brought to life that I then have zero idea how to maintain.
Totally agree! To be honest I'm still a bit wary of using ECS via the web console. Too many (new) terms to understand! Thankfully the folks at AWS were smart enough to make a CLI that understands docker-compose. So that's how I ended up using it.
My conclusion with Docker is that, in general™, you really need to have a justifiable reason to go whole-hog into Docker, especially if you're not on AWS / considering ECS.
I'm glad the article covers ECS, as it makes a lot of the scheduling / config issues simpler!