ansible and cloud instances

October 31, 2012

A few days ago I posted about using ansible to provision persistent ip instances in a public or private cloud.

Today I finished up the next piece I needed to make this work for copr builders w/transient ips/instances.

I needed a way to create a new instance, provision it as if it was one of the normal inventory of our systems, and return back the ip of that was given to it via eucalyptus (or ec2) automatically. And when I was done with it – I didn’t want any record of it preserved in our inventory system. I wanted it gone.

The problem was ansible wants to act only on the hosts it knows about in its inventory. This is quite understandable. But since I’m not specifying an ip to this host and I don’t know it in advance I wanted a way, in a single playbook to do this.

So I wrote add_host an action_plugin for ansible. These let you interact with the internal state of the inventory/playbook/etc while executing a playbook.

All it does is to add the host to the in-memory inventory the ansible playbook object has. And it adds that host to a group you specify. This is so in the second play in the playbook you can say ‘oh operate on hosts in this special group name’ and that will be the only host in that group.

I’ve sent in a pull request for it but it’s not been accepted quite yet. 🙂 However, if you want to try it out you can just toss it into your  action_plugins dir in ansible and call it.

Here’s an example playbook. It’s very similar to the one for creating a persistent instance. In the fedora public ansible repo we are, in fact, importing the same set of tasks from another file to set them up the same.

It just means if anyone wants an instance in our private cloud running f17 or el6 it is incredibly trivial to make one available for you.



You’ve just spun up a new instance and you need to give additional people access to the system as root. You have a common IDMS that houses ssh pub keys for your users. You want to trivially specify a list of those users and have their keys show up in root’s .ssh/authorized_keys file.

Here’s what you do:


– name: add root keys for other allowed users
action: authorized_key user=root key=’$PIPE(/path/to/script/for/keys ${root_auth_users})’
only_if: is_set(‘${root_auth_users}’}


In our infrastructure FAS houses all the pubkeys. So Toshio wrote this script:

So if you define a hostvar in your ansible inventory for this host – then the above will automatically populate your root authorized_keys with the right pub keys.

Kinda awesome, I think.

Since we’ve been working more and more with our private cloud setups on eucalyptus and openstack I’ve been trying to come up with a semi-sane way of having persistent hosts in the cloudlets.

So that if we shut things down or an instance dies – it can be brought back up automatically without someone having to explicitly say ‘go start this up’. I’ve also been working on setting up ansible such that we can run it via cron on our central command host to maintain our systems.

So, of course, I’ve combined those two things into one task 🙂

We want certain cloud instances to:

  1. always exist
  2. have the same ip each time they come up.

These are for things like the build master for COPRs/PPAs or for the jenkins-master server or for any number of random services where they need a persistent ip throughout recreations. For the ips we’re using euca-associate-address after allocating a number to our account.

So I started on a mechanism to create those instances and give me results I could use in ansible for other actions. I came up with ec2 which lets you do that from a local_action call in ansible.

Once you have that module in your ansible library path you just need to:

  • add the host to your ansible inventory file
  • take  this playbook example
  • modify it for your host
  • run it like: ansible-playbook thatplaybook.yml

you’ll need euca2ools installed along with the deps for ansible.

Here’s what the playbook does:

  1. it looks to see if the host is running (by a simple nc connect to port 22)
  2. if it is not running then it will run ec2 with your args
  3. then associate the ip
  4. wait for the host to get the ip and become available
  5. then it goes on to the next play which is normal ansible provisioning

which is perfect, for the next time you run – steps 2-4 won’t happen – it just goes directly to provisioning. Since ansible playbooks are idempotent you can run them as many times as you want w/o maligning your server.

Now the example playbook is all merged together – but if you use host_vars in your ansible inventory and you setup the check/create tasks as an includable task file – then the playbook for creation and provisioning of any new cloud instance you want becomes VERY short.

More to come as I finish it.

ansible and cloud notes

October 18, 2012

I’ve been working with ansible a lot recently. Fedora has a new public repo:

That’s just the beginnings of our repo for managing systems. There’s a number of details to work out, though and I have to merge over all the non-private things from our builder repo. I’ve started that process it’s just a matter of making reasonable decisions on where the tasks should live.

Recently I’ve been looking at:

and figuring out how a euca2ools-based local_action module would work. I can spin up new instances, if they don’t already exist, and then provision them then I can create an entire network from more or less nothing.

Also working out how best to use euca-allocate-address and and associate-address in our infrastructure to provide some persistent and consistent servers/apps in the cloud. I think I have that bit hashed out but we’re discussing some tracking for cloud instances so we can sensibly shut them down when we need to w/o making everyone angry 🙂

I’m positive I can do all these things with ansible and euca/openstack. The question now is how long it will take to make them all happen. Anyone interested in helping out, come by #fedora-admin and talk to me.



What I want from schwab:

October 12, 2012

What I want from schwab:

I had told my partner that schwab had always treated me well. That
they had never made me jump through any ridiculous hoops. That I
could do everything via the web. No need for phone calls.

I told her this b/c I thought it was true. I told her this b/c
she’s deaf and it means a lot to her to be able to do things
entirely on her own w/o having to involve anyone else.

So it was a bit shocking when she was asked to call schwab to verify
the deposit amounts and complete the moneylink setup to your
brokerage service.

Worse yet, when I called to deal with the problem, with her at my
side, I was told no one could speak with me, despite having all
the requisite security information. Furthermore, I was told that
if she wanted to complete the process she would need to call in
via TDD or Relay operator.

This was all done in the name of security. Let’s explain a
bit about how security works.

A person is asked a series of questions that only they should know the
answers to. As a result that person is authenticated as actual. Whether
you ask these questions over the web, in person or on the phone, it
provides you with proof that the person you were speaking with is
authentic. Yes, sometimes that data is compromised by people intending
to defraud but if the person presents all the valid information,
it is impossible to know that.

This is the very basics of how it works:
Question, Answer, Authenticated.

In this case the questions were:
1. ssn
2. mother’s maiden name
3. account number
4. the precise amounts deposited into the other account
to moneylink with

Schwab requires that this information be presented ONLY over the phone.
They claim that this is to make it more secure. How or why a telephone
conversation is more secure is never offered. But I will let it go
for the moment their claim that it is. For a hearing person.

For a deaf person who is forced to use a Relay operator it means
connecting to a service, having them dial another number, then giving
all of the above information to a random person with whom you have
no contractual nor social relationship of any kind and virtually no
way to track them down, especially if you use the service more than

Please explain to me how that is MORE secure than having my partner
submit the information over an encrypted network connection?

If this were all done over the web at the end of the day the same
data exchange results but instead of making my partner angry
and resentful, you’ve made her feel happy. And you’ve not made me
into a liar when I told her that schwab wouldn’t make her do all that.

I want this corrected.

Here’s how you correct it:
1. Make the service entirely web-based and web-driven.

2. Provide a mechanism on your website where if a customer
needs to interact with a representative they can do so
in an encrypted chat window.

3. If you are going to try to force people to use the TTY and
or relay operators. Test it out for yourselves and make
your security audit and customer service people use it
with their own information. See how much trust they feel
in that system.