Mining Camp

Easy automated configuration and deployment of Minecraft servers on AWS spot instances, with features such as instance shutdown monitoring and automatic backups and restorations using S3.

Table of Contents

• Introduction
  • Sample Cost Breakdowns
    • i3.large
    • t2.medium
• Setup
  • Requirements
  • Configuration
    • AWS
    • Virtual Environment & Requirements
    • Minecraft Server Archive
    • Settings
    • Terraform
• Spot Instance Interactions
  • Server Launch & Provision
  • Server Shutdown
• Tests

Introduction

Amazon sells spare EC2 instances that aren’t currently reserved, called "spot instances", for a fraction of the cost of a reserving them. If you can mitigate their volatility risk, they’re an excellent deal and powerful enough to host demanding game servers.

Advantages:

• More control over everything. SSH into the box, troubleshoot, adjust anything you like on the fly.

• Better bang for your buck. More powerful systems than most hosting services offer.

• Pay for exactly what you use. Spot instances are billed by the second, so if your server isn’t going to be used for a few days, or even overnight, just turn it off.

• Linux-based. This might be a disadvantage, depending on your comfort level. Windows instances are not only more of a pain, they’re also far more expensive.

• Works with any AWS region, so you can minimize latency for your player base.

Disadvantages:

• Spot instances can be terminated at any time. This is mitigated by both the emergency shutdown monitoring script, but in practice reasonable bids for spot instances result in excellent uptime.

• Requires initial work to setup. This isn’t a push-button solution, but once you’re done with the setup day-to-day interactions are extremely easy.

Sample Cost Breakdowns

I’ve provided a couple samples of pricing based on different instance types below. When deciding what to use, cross reference your requirements against the spot instance pricing and the EC2 instance types. Be aware that spot instance prices can differ not only between AWS regions, but between availability zones as well.

All calculations below are based on an instance in us-east-1, and assume a 31-day month.

Note	AWS charges for absolutely everything. It’s critical that you monitor your bill and be aware of where you’re incurring charges.

i3.large

An i3.large is a quite powerful instance with ~16GB of RAM and a 500GB NVMe drive, meaning an EBS (beyond the 8GB root volume) isn’t required. Price estimates for the various moving pieces:

• Current spot instance price for an i3.large is $0.0371 per hour at present.

• An 8GB EBS is used as the root volume for the instance, at $0.10 per GB-month, or roughly $0.00013 an hour. This is deleted on instance shutdown, so no charges are incurred when the server is off.

• Elastic IPs are free when assigned to a running instance, and $0.005 per hour when unassigned.

• AWS data transfer is pretty expensive, at $0.09 per GB. I use about 1.4GB per day, but my server has a lot of time when it’s completely idle. For the sake of round numbers, 2GB of bandwidth a day winds up being $0.0075 an hour. This cost is very subjective.

• S3 is actually quite cheap, especially with lifecycle management rules cleaning up old backups. 20GB of combined server archives and rotating backups is $0.023 per GB, $0.46 a month, or $0.0006 an hour.

From these numbers, when my server is running, I’m paying roughly $0.045 an hour or $33.73 a month to run it full-time. When my server is off, but I want to preserved my backups and elastic IP address, I’m paying $4.17 a month. Just preserving backups is only $0.46, which is very reasonable.

t2.medium

Options with less RAM more be suited to smaller loads. One example is a t2.medium, which has only 4GB of RAM and no SSD, which means it requires an EBS volume. The upside is that it only costs $0.0134 an hour. If we use a 30GB ephemeral EBS volume that’s deleted on instance termination, that adds roughly $0.0004 an hour.

Using this less powerful, less expensive instance brings considerable savings: $0.0255 an hour, or $18.97 when run for an entire month.

Setup

Start by checking out this repository, and navigating to your checkout-out directory. Sample bash commands provided are from the root directory unless otherwise noted.

Requirements

You’ll need to download and install the following requirements (versions I build and tested with are in parenthesis):

• pip (20.1)

• terraform (0.12.24)

You’ll need an appropriate version of the JDK/JVM to run the server locally at least once, allowing it to generate the necessary configuration files.

You’ll also need a fully configured AWS developer account, as you’ll be using the console extensively in the next step.

Configuration

AWS

Most of the heavy lifting in AWS is done by terraform. However, a few steps need to be taken before using terraform.

Credentials

You’ll need your AWS credentials available for most of these operations, under the minecraft profile. ~/.aws/credentials will look like:

[minecraft]
aws_access_key_id = <your_access_key_here>
aws_secret_access_key = <your_secret_key_here>

If you have more than one AWS profile, you’ll need to set the AWS_PROFILE environment variable with export AWS_PROFILE=minecraft for the aws commands below to work.

Key Pair

You’ll need a key pair for accessing your instance. Generate a public-private key pair. As an example, you can do this with ssh-keygen:

ssh-keygen -t rsa -b 4096 -C "AWS"

In the EC2 console, select Import Key Pair on the NETWORK & SECURITY → Key Pairs page. Upload your public key, and name it "aws-public". The launch configuration Terraform creates includes this key, allowing SSH access to Ansible (and for troubleshooting!)

Elastic IP Creation

You’ll need to create an elastic IP for association with your instance, providing a convenient public-facing IP. In the AWS console, do the following:

1. Enter the EC2 service.

2. Click on Elastic IPs, under the NETWORK & SECURITY menu on the left-hand side of the screen.

3. Click Allocate new address.

4. Leave the scope as "VPC", and click close.

5. You should see your new elastic IP in the list. Save the Allocation ID for later use during the setup.

Once a server has been spun up, this elastic IP will be attached to it. Note that allocated elastic IPs are included in the price of a running instance, but you will be billed for any un-assigned EIPs by the hour. For this reason, if you plan to stop your Minecraft server for long periods of time, be sure to delete your EIPs and create new ones when you’re ready to begin hosting again.

Virtual Environment & Requirements

Using pip, install the necessary Python 2.7 requirements. I recommend using virtualenv and virtualenvwrapper. Running the following installs Ansible, the AWS command-line interface, and libraries required for interacting with AWS programmatically.

$ mkvirtualenv minecraft
(minecraft) $ pip install -r requirements.txt

Minecraft Server Archive

You’ll need to create a Minecraft server archive to be pulled onto your instance each time the box is spun up. In this example, I’ll be creating an archive for my Feed the Beast server named daftcyborg.

$ # Create a base directory named after your server name
$ mkdir daftcyborg
$ cd daftcyborg

$ # Get your base server pack. In my case, I've already downloaded the FTB server
$ ls
FTBRevelationServer_1.0.0.zip
$ unzip FTBRevelationServer_1.0.0.zip

$ # Install the server requirements
$ sh ./FTBInstall.sh

$ # Launch the server. You'll need to do this twice, once to create the
$ # eula.txt and once to generate the base
$ sh ./ServerStart.sh
Missing eula.txt. Startup will fail and eula.txt will be created
Make sure to read eula.txt before playing!
To continue press <enter>

Open eula.txt, and agree (or don’t) to the terms and conditions.

Launch the server again, and wait for it to complete. This will generate the world base, and any settings and properties files necessary. Quit the server, and do the following as desired:

• Remove the world directory, which is the world directory name used by default and which will (assuming you update the server.properties file) be named differently when your server is run.

• Edit server.properties as desired. It is important that the server-port be left as 25565, otherwise you’ll need to adjust the Terraform configuration. Fields I recommend changing are level-name, level-seed, and motd.

• Add yourself and any other players desired to ops.json.

• Update server-icon.png to a custom icon.

Copy server.properties to ansible/files/server.properties, which Ansible will install every time over the top of the properties file in the archive, allowing easy configuration changes.

Now, clean up your leftover base archive, since you don’t need it anymore:

$ rm FTBRevelationServer_1.0.0.zip

Navigate up a level, and create a gzipped tarball:

$ cd ..
$ tar -cvzf daftcyborg-server-12-20-2017.tgz daftcyborg/

Lastly, push the archive to S3:

$ # The parameterized command is 'aws s3 cp <server_file> s3://<bucket_name>/<server_name>/'
$ # My version looks like:
$ aws s3 cp daftcyborg-server-12-20-2017.tgz s3://josh-minecraft/daftcyborg/

Lastly, save the full name, including file extension, of the archive you generated; it will be required when you run the setup wizard.

Settings

The recommended way to configure the system is to run the setup wizard from the root repo directory, like so:

$ ./utilities/setup.py

This guides you through each required setting, offering default values if available. It then takes your input and renders out terraform/variables.tf and ansible/group_vars/all from corresponding *.j2 templates. If you like, you can populate those templates by hand.

• It’s important you choose the right aws_availability_zone, since spot prices can vary substantially from zone to zone.

• Maximum spot price determines the maximum price you’re willing to pay per hour. Setting this wisely will prevent you from being surprised by a large bill at the end of the month.

Terraform

Terraform allows you to easily setup EC2 and S3 to match your needs. To apply the terraform configuration, run:

terraform apply terraform/

Once this has successfully completed, your AWS configuration is done. Unless you change your configuration, you won’t need to run this again.

Spot Instance Interactions

Server Launch & Provision

Jump to the ansible directory, and run the start.yml playbook to configure the instance and launch the minecraft server:

cd ansible
ansible-playbook -i ec2.py --private-key=~/.ssh/aws -u ubuntu start.yml

Note	If the auto-inventory script is taking too long, you can update ansible/ec2.ini's regions entry with the particular AWS region you’re using.

Note	If using an older version of Ansible, the Paramiko library used by default may run into errors when gathering facts from the remote host. If this happens, add -c ssh to the ansible-playbook command above.

Server Shutdown

Shutting down your server is just as easy as starting it:

cd ansible
ansible-playbook -i ec2.py --private-key=~/.ssh/aws -u ubuntu -c ssh stop.yml

When this playbook finishes, your instance will be gone, but the state of the server will have been preserved and pushed to S3, ready for the next time you launch it.

Tests

Tests are currently available for the Prospector tool. You’ll need to install the requirements in the test directory in order to run them. From the root, with your virtual environment active:

(minecraft) $ pip install -r utilities/tests/requirements.txt

Now you can launch the test suite:

(minecraft) $ python -m unittest -v utilities.tests.test_prospector