Re: [orchestration] AWS public cloud images

[Mark Meyer]

What does composable mean? Let's assume we build three options into Guix :1) 
use the AWS mechanism to distribute public keys, 2) use some centralized 
authentication system like LDAP, 3) pre-bake your keys into the image. I think 
giving the uset a choice how to maintain their systems leads to reuse and 
ultimately to better infrastructure. You're not constantly re-inventing the 
wheel, because there are already pre-baked recipes that enable your use-case. 
And Guix enables this method of working, because the entire system lives in one 
repo, easily configurable from a single config file.

On Tue, Mar 27, 2018, at 23:33, Mark Meyer wrote:
> Hi David,
> I think most of the AWS build processes feel imperative and hacky. I 
> always equate this to a big pot of soup which you stir and pass on to 
> the next guy, who builds another image/stirs it again. In general I 
> think you should not start with an AMI, but with an empty volume and 
> then deploy to that. Your AMI definition process should be composable, 
> so if you have different teams working on it, i.e. systems engineering 
> builds the base AMI and an application group customizes it, you should 
> only run one AMI build and not pass down the artifact. To put it more 
> extreme: it  looks like some junkies passing around a dirty needle.
> 
> That's one of the reasons the code to build the AMIs is currently the 
> main deliverable. My proposition is not to refrain from providing a 
> Golden Master image, it will be worthwhile for some purposes, my 
> suggestion is to make the AMI build process fully automated and easily 
> extensible.
> 
> For the moment I don't mind having Packer in the mix. I don't however 
> hold it in high regards, I think it's essentially glue code. But when 
> you want to replace it, please consider that there are use cases like 
> different partitions (GovCloud and AWS China) that have very different 
> implementations of the AWS API. Speaking from my experience these can 
> lead to huge headaches, and in general I think it's not worth to work 
> around the same set of bugs another time.
> 
> Cheers, Mark
> 
> On Tue, Mar 27, 2018, at 20:08, Thompson, David wrote:
> > Hi Mark,
> > 
> > On Mon, Mar 26, 2018 at 4:18 PM, Mark Meyer <address@hidden> wrote:
> > > Hi,
> > > I've the beginning of Guix cloud images available over at Github at
> > >
> > >   https://github.com/ofosos/guix-packer/
> > >
> > > There's a small writeup of what has been done and what's still missing 
> > > over here:
> > >
> > >   https://ofosos.org/2018/03/26/guix-images-01/
> > >
> > > All in all, I split the heavy lifting between Packer (AWS API calls) and 
> > > `guix system` and that worked remarkably well. There's some integration 
> > > with EC2 (You can inject a pubkey into the image via the console), but 
> > > there's also a lot of stuff missing. In the end, I would like to provide 
> > > public cloud images in some weeks time. Of course you'll have all the 
> > > tools to rebuild your own images (surprisingly simple).
> > >
> > > I think there's still a lot of polish we can apply, but at some point 
> > > we'll need some help from AWS. I do have AWS support access at work, but 
> > > am not really comfortable to use company resources for this yet, but I'll 
> > > probably ask if there's some  avenue to get some 'official' help as a 
> > > free software project, when I run across our technical account manager.
> > 
> > First of all: Thanks! This is a great start! I've wanted to run GuixSD
> > EC2 instances for some time but haven't gotten around to it. The more
> > I think about it, though, the more I wonder how useful an official
> > GuixSD image is vs. providing software to create AMIs for any given
> > system configuration. The use-case I'm particularly interested in is
> > using GuixSD instances in auto scaling groups.  I'm wondering if
> > there's any other way to create GuixSD AMIs than starting from some
> > official image, updating Guix, running 'guix system reconfigure', and
> > using the result as the basis for the AMI. While this ought to work,
> > it feels very imperative and hacky.  In the future it would also be
> > great to eliminate the need for Packer entirely and replace it with
> > Guile.
> > 
> > Anyway, just some food for thought.  Awesome work!
> > 
> > - Dave
> 
> 
> -- 
>   Mark Meyer
>   address@hidden
> 


-- 
  Mark Meyer
  address@hidden