Xcode Server + Ansible: Scalable and Programmable iOS CI/CD Infrastructure

gowebing

One of the complaints about Xcode Server is it's not scalable to be used at a large-scale or enterprise level. However, if you have the skills to understand and manage an infrastructure for iOS deployment, then scalability can be easily achieved for small, medium, or large iOS teams using a mixture of iOS and DevOps toolkits. We have achieved it in the past in my previous role at Photobox Group using the combination of tools Ansible + Fastlane + TeamCity. You can read my blog post on the Moonpig Engineering blog here to learn how we achieved this.

In this post, we will see how learning a little bit of DevOps tools like Ansible and Apple's CI solution Xcode Server can help to manage large-scale iOS CI/CD infrastructure without any pain.

Background

Continuous Integration practices for iOS development aren't great as compared to the web world because of some unique challenges specific to iOS development. It's essential to understand the entire iOS ecosystem to setup and manage CI/CD infrastructure for the team. Some Senior/Lead iOS developers or iOS Tech Architects/Lead are great at the understanding whole stack of Apple developer tools, technologies, and infrastructure and implementing it in their projects, but most of the iOS developers in the industry at the moment are just iOS developers. They are good at writing Swift or Objective-C to build UI for applications inside Xcode; however, when it comes to understanding the infrastructure, they suck. They don't properly understand the underlying tools and technologies used by Xcode under the hood and that's why they became totally incapable of setting up or managing CI/CD practices for iOS apps. As an effect of this, they always have to depend on local Xcode or someone from the DevOps team to manage the infrastructure stack for them, or CI/CD is handled by highly expensive third-party services. It means developers are not full-stack and do not understand their own tools and technologies. Unfortunately, the world of iOS development is full of these kinds of amateurs and half-arsed developers. Continuous Integration won't be successful without skilled engineers on the team. In my previous post, I wrote about the top five reasons for CI failures. Those are wrong selection of CI server, CI amature engineers, lack of Infrastructure as Code, poor quality build servers, and lack of support from the team.

iOS Infrastructure Automation Challenges

On top of iOS engineers' inability to understand the entire iOS ecosystem, there are some traditional challenges in automation of iOS ecosystem. Some of them are imposed by Apple's proprietary software, and others are due to lack of tooling to automate iOS and macOS infrastructure. Some of the common challenges are

As a result of these challenges, nobody wanted to spend the time to explore a clean and automated solution to set up iOS infrastructure for smooth Continuous Delivery, which results in the following situations:

These symptoms are obstacles to the best application development practices. We can solve some of these issues by knowing a little bit about configuration management tools and proper continuous integration tools. Now, we will see how the combination of Xcode Server and Ansible can help to set up large-scale continuous integration and manage it effectively.

Xcode Server

Xcode Server is a native Continuous Integration service developed by Apple. Xcode Server is one of the easiest ways to get started with Continuous Integration for iOS applications. It just needs Xcode installation and permission to access the Apple developer portal on macOS servers. If you use other solutions like Jenkins or TeamCity, you will need to install other dependencies like Java and other related software which have nothing to do with iOS development. Apple still has old documentation for Xcode Server at the time of writing this post, but I wrote the setup instructions in my previous blog post here. In order to set up the Continuous Integration server, we just need to install the following on the build machines:

We can manually install that software on each machine, but it will time-consuming if we have multiple macOS machines. We need to make sure that Xcode and other tool versions are consistent on all the servers. Now, we need to have configuration management to manage servers and software versions on those servers.

Provisioning Xcode Server With Ansible

There are various configuration management tools on the market, like Chef, Puppet, and Salt, but Ansible is very easy to learn and use. Although Ansible is designed for Linux, there are enough modules that can be used with macOS as well. This is not an Ansible tutorial, but you need to read a bit of documentation to get started with Ansible here.

In order to provision Xcode Server, we need the following things:

Additionally, we can install other tools like Ruby and Homebrew using Ansible, if needed.

Provisioning Xcode Installation

In order to install Xcode, we need to have the .xip file downloaded from Apple Developer Portal and put it on the Ansible remote machine to copy over to ssh machines. There are various other things to be installed as part of the Xcode installation:

We can achieve all these tasks using Ansible.

---
- name: Check if Xcode is already installed
  stat: path=~/Library/Developer/Xcode/
  register: xcode_dir

- name: test whether xcode was uploaded earlier
  stat: path=/Users/{{ ansible_ssh_user }}/{{ xcode_src }}
  register: xcode_dl_path

- name: upload xcode
  copy: src={{ xcode_src }} dest=/Users/{{ ansible_ssh_user }}/
  when: xcode_dl_path.stat.exists == False

- name: check that the xcode archive is valid
  command: "pkgutil --check-signature ~/{{ xcode_src }} | grep \"Status: signed Apple Software\""
  when: xcode_dir.stat.exists == False

- name: Install Xcode from XIP file Location
  command: bash -c 'open -FWga "Archive Utility" --args /Users/{{ ansible_ssh_user }}/{{ xcode_src }}'
  when: xcode_dir.stat.exists == False

- name: Move Xcode To Application
  command: bash -c 'mv ~/Xcode*.app /Applications/'
  when: xcode_dir.stat.exists == False


- name: accept license agreement
  command: bash -c "sudo /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -license accept"

- name: install mobile device package
  command: bash -c " sudo installer -pkg /Applications/Xcode.app/Contents/Resources/Packages/MobileDevice.pkg -target /"

- name: install mobile device development package
  command: bash -c "sudo installer -pkg /Applications/Xcode.app/Contents/Resources/Packages/MobileDeviceDevelopment.pkg -target /"

- name: install Xcode System Resources package
  command: bash -c "sudo installer -pkg /Applications/Xcode.app/Contents/Resources/Packages/XcodeSystemResources.pkg -target /"

- name: install additional xcode components
  command: bash -c "sudo installer -pkg /Applications/Xcode.app/Contents/Resources/Packages/XcodeSystemResources.pkg -target /"

- name: check on the status of developer mode
  command: DevToolsSecurity -status
  become: yes
  become_method: sudo
  register: dev_tools_security_status


- name: enable developer mode
  command: DevToolsSecurity -enable
  become: yes
  become_method: sudo
  when:
    - "'disabled' in dev_tools_security_status.stdout"
    - xcode_dir.stat.exists == False

- name: Set Xcode Developer Directory
  command: bash -c "sudo xcode-select -s /Applications/Xcode.app/Contents/Developer/"

Note that the code above is the example of an Ansible task; we have to set up some variables to configure users and directories.

Provisioning Xcode Server

Now we can install Xcode as part of the task mentioned above. The next thing is to control the Xcode Server using Ansible. Xcode Server has command line tools, xcscontrol, to manage the Xcode Server. We can start, stop, reset, and restart Xcode Server using this utility. We can write an Ansible task like this:

---

  - name:  Start Xcode Server with given user
    command: bash -c 'sudo xcrun xcscontrol --initialize --build-service-user {{xcode_server_user}}'

  - name:  COnfigure Xcode Server Certificate
    command: bash -c 'sudo xcrun xcscontrol --configure-ota-certificate'

  - name:  Configure Xcode Server timeout
    command: bash -c 'sudo xcrun xcscontrol --configure-integration-timeout 3600'

  - name:  Check Xcode Server Health
    command: bash -c 'sudo xcrun xcscontrol --health'

With this simple configuration, we can easily manage Xcode Servers on multiple macOS servers.

Xcode Server: Ansible Role

I have created an Ansible role for Xcode Server and published to Ansible Galaxy. You can find this role here.

This role is very configurable and can be used for setting up multiple macOS servers as per your needs. You can find detailed information in the README file.

You can see all the variables used for this role and turn them ON/OFF or set them as per your needs. You can optionally set up Ruby and Homebrew if needed.

vars:
    clean_xcode: yes
    clean_rvm: no
    clean_homebrew: no

    configure_xcode: yes
    configure_xcodeserver: yes
    configure_macos_defaults: yes
    configure_ruby_rvm: no
    configure_homebrew: no

    xcode_src: Xcode_9.1.xip

    xcode_server_user: shashi
    ansible_ssh_user: shashi

How to Use an Xcode Server Ansible Role

Imagine, you have a fresh Mac with fresh macOS installed. You can set up all your Xcode Server for CI by creating a Playbook for this role. You can set up config variables as per your needs. Assuming you have installed Ansible, we can download the role by running this command:

$ ansible-galaxy install Shashikant86.XcodeServer

Now we have to create our own playbook for this role by setting variables. We can use the default variables here. Now make a new directory called xcode_server and create a files directory and xcs_playbook.yml file.

$ mkdir xcode_server
$ mkdir XcodeServer/files
$ touch xcs_playbook.yml

Now place your Xcode_9.x.xip file inside the files directory and insert the following into the playbook file:

---
- hosts: localhost
  connection: local

  vars:
    clean_xcode: yes
    clean_rvm: no
    clean_homebrew: no

    configure_xcode: yes
    configure_xcodeserver: yes
    configure_macos_defaults: yes
    configure_ruby_rvm: no
    configure_homebrew: no

    xcode_src: Xcode_9.1.xip

    xcode_server_user: shashi
    ansible_ssh_user: shashi

  roles:
    - Shashikant86.XcodeServer

Change ansible_ssh_user and xcode_server_user to your username and Feel free to set variables as per your need. You can also specify your inventory to enable this playbook to run on multiple hosts. Now execute this playbook using the following command:

$ ansible-playbook xcs_playbook.yml

Watch your Mac Mini servers getting set up with Xcode Server for iOS Continuous Integration.

When to Use Xcode Server and Ansible

There are situations when we want to run Ansible playbook on servers; some common situations are:

Ideally, we should run the Ansible playbook regularly to check if anything has changed.

Benefits

Some of the benefits of using configuration management tools with Xcode Server are

You might see several other benefits and increased confidence and trust in Continuous Integration.

Conclusion

Using the combination of DevOps tools and Apple's native CI/CD tools, we can achieve scalable and programmable infrastructure for our iOS apps. The Xcode Server and Ansible combination can work well together to achieve Continuous Delivery of iOS apps without any pain. Are you using any configuration management tools for iOS CI/CD infrastructure? What are your experiences with Xcode Server? Please let me know what you think about this approach- any ideas or suggestions would be great.

 

 

 

 

Top