Reflections on software development from anywhere on an iPad

I feel like I'm living in the future, yet simultaneously living in the dark ages as Australia remains divided between those who can work remotely (the laptop class) and those who cannot. It has been 16 weeks since my children last saw their father because the state borders are closed. Fortunately, this moment in time will be soon over.

living in the dark ages

I feel like I'm living in the dark ages. Here in Australia, the complete exclusion of Australians with technical knowledge from decision-making and development of public technology is so normalised that "digital holograms" (JPGs) can be passed off as security.

Fortunately, the community that has grown up around the public examination of COVIDSafe is now shining the spotlight on this bullshit. Digital holograms are not a thing. It's not a hologram, it's just an animated picture (more animated when you wobble the phone, but who cares?), which can be copied as easily as anything else on electronic devices.

Politicians of Australia, please for the love of god, cease with these statements that JPGs are secure, as you are making Australia look like clowns in the broader international technology community.

JabMaker 💉 | Real Fake Vaccination Certificates

We have real fake certificates like you wouldn’t believe! What are you worried about? Come get real fake vaccination certificates. Don’t even hesitate, Don’t even worry and don’t even give it a second thought. What are you worried about?

JabMaker 💉real actual rex patrick

Join the AusOpenTech Discord Server!

Check out the AusOpenTech community on Discord - hang out with 576 other members and enjoy free voice and text chat.

Discord

Yet, at the same time, I feel like I'm living in the future because I can work from anywhere, from any device and 10 years from now I'll likely camping somewhere in a forest, surrounded by younger digital nomads who are doing exactly what I am to hack Australia's housing affordability crisis.  

Anyway, due to a series of unfortunate events related to bricking my Macbook Pro M1 laptop by un-enrolling the device from the macOS beta program then re-installing macOS I'm now stuck in a circular situation where users cannot be created on the device after installation because the device isn't enrolled and there's no way to roll back to stable without access to another Macbook Pro M1 to restore the IPSW.

Considering that it is a seven-hour return trip drive to the nearest Apple store I'm now all in with no other choice.  An iPad Pro M1 with LTE has become my primary software development device instead of being used as a portable monitor for my Macbook Pro M1 and is my only option for backend, frontend and operating system software development for the foreseeable future.

The Macbook Pro M1 is the software development laptop of choice yet I love my iPad considerably more. I'm all in.

lessons learned and recommendations

If you head down this path make sure you get the 12.9-inch iPad Pro with LTE as I think screen sizes less than 13" are unpleasant to work with and that extra LTE modem enables internet multiplexing scenarios between different mobile providers.

peripherals

The Magic Keyboard + Trackpad is lovely and my favourite keyboard to type on. It doesn't have an ESC key but pressing Command + <period> is the shortcut. The shortcut gets old fast so my #1 recommendation is to head to iPadOS Settings > Keyboard > Hardware Keyboard and rename the capslock key to escape!

If you don't want to go for the Magic Keyboard then there are... many... options...

as standard USB devices can be plugged into it an iPad, or, of course, one can go wireless via Bluetooth.

software

I highly recommend purchasing http://blink.sh as it is worth every single dollar. Blink is an SSH/MOSH terminal for the iPad that provides standard UNIX command-line utilities. Make sure you pair Blink with https://mosh.org on your server to achieve terminal nirvana...

I highly recommend purchasing https://shiftscreen.app which provides a virtual desktop manager and allows you to use your iPad on your external display in "desktop-like ways".

I highly recommend purchasing https://www.duetdisplay.com. Duet when paired with another computer turns your iPad into a second monitor with full gesture support.

internet connectivity

As a bare minimum configuration get an iPad with LTE connectivity, carry a 4G hotspot around in your backpack from a different mobile carrier and use Speedify to bond the multiple networks.

Speedify performs channel bonding to combine multiple Internet connections into a faster, more reliable connection and automatically prioritizes streams over other network traffic so you avoid stuttering, buffering and disconnects.

Speedify - Use all of your Internet connections at the same time

Speedify is the only app that can combine multiple internet sources into a bonded super-connection for more stable & secure live streaming, video calling & web browsing.

Speedify

When I'm near the van then my internet connectivity has faster internet and is six times more redundant than the typical Australian house. See this blog post for insights into how you can put together your own vehicle with the best internet connectivity in Australia.

There are six internet links on my office on wheels. Seven when Starlink arrives.

Is this the best internet in Australia?

Geoffrey Huntleyghuntley

previously in the series

In this blog post, I'll be sharing recommendations formed from the last 30-days of using an iPad for software development, upcoming industry trends, insights as to how you can move towards iPad first software development, and how you can roll your own personal develop from anywhere cloud on bare-metal hardware.

This post is the sixth in the series about remote work from a van. If this is your first time reading my blog and you have questions about living (including toilets or showering) and working from a van then check out these blog posts:

There are six internet links on my office on wheels. Seven when Starlink arrives.

Is this the best internet in Australia?

Geoffrey Huntleyghuntley

my office is a tent that my employer purchased for me

If you look up on Instagram, you’ll see jaw-dropping pictures under the hashtag #vanlife. It’s true. The big blue world is freaking beautiful, but beauty comes at a cost.

Geoffrey Huntleyghuntley

how long

People routinely ask me how long do you see yourself living in a van for so let’s unpack that question shall we?

Geoffrey Huntleyghuntley

a new chapter: full-time working from a van in a forest

For many people, the year 2020 will go down as a moment in time of hardship in their lives but for me, the year 2019 was dramatically harder as it was the realization that a long-term relationship wasn’t going to work out...

Geoffrey Huntleyghuntley

living in the future

By day I work at Gitpod, a product (and open-source project) that provides ephemeral software dev environments in the cloud and by night I'm participating in mangopdf's "do not get arrested challenge" (thank god for Electronic Frontiers Australia Inc).

The pandemic has permanently changed how the world functions. In the tiny niche of software engineering, all the top companies are offering remote employment. 3 years ago, remote gigs were very hard to find. Today, right now, they are the norm.

If are your employer does not offer remote or is not firmly and openly discussing keeping remote as an option it is very important that you make changes before the pendulum swings back. The opportunities that are open right now might not be here in another year.

I strongly advocate for people to optimise for remote over money because remote provides unique ways to reduce living expenses, reduce stress and unlock huge quality of life improvements. However, having said that it is important to know your worth and be aware of the unique income disparities in our industry.

Apart from topics of income, one of the biggest ways that Apple / Twitter / Google / Microsoft style companies are different from the rest of the software industry is that they have entire teams dedicated to the creation of internal developer tools that make their software engineers more productive.

When these employees leave the top-tier companies they experience deep culture shocks about the state of public developer tooling which often results in the formation of startups that recreate and offer these internal tools to the public.

What does Google’s web IDE look like?

Answer: It’s called Cider — it looks and works effectively the same as Atom. Its main strengths is that it has integrations for Google-specific functionalities — for example, direct linking between bugs and TODO lines, direct management of multiple commits at the same time (Google uses a proprie...

Quora

The Uber Engineering Tech Stack, Part I: The Foundation

Uber’s mission is transportation as reliable as running water, everywhere, for everyone. Here’s the first of a two-part series on the tech stack that Uber Engineering uses to make this happen.

Uber Engineering BlogLucie Lozinski

private technology becomes public

Phillip Carter was right on the money when he coined the following words of wisdom:

Cloud-based, reproducible developer environments are a sleeper technology that’s going to ramp up for a decade in usage until, one day, everyone will be “behind the times” if they’re not already using them. I see it as similar to git (in the era of widespread CVS, TFS, SVN, etc. usage in industry), where the possibilities are endless, the model is superior, work evolves around it as an ecosystem builds up around it, and then it’s the new normal.

Reproducible developer environments center around one central premise: you just create an environment in the cloud to build stuff when you need it.

When you need to write some code, you spin up an environment and you write your code, debug it, do whatever. You then push your changes to a branch. Then the next day, you just create another environment against that branch with the same context. Critically, you are not:

• Going back to that same environment from the day before (no finding your environment in some dashboard somewhere)
• Managing a complex, stateful set of tools on your local machine
• Re-running build scripts or docker build to re-setup your environment
• Constantly switching things like runtime and SDK versions based on the codebase you’re working on
• Massaging some kind of build tool or runtime or set of packages so that it stops acting funny

You’re just spinning up a new environment in the cloud and solving problems with code. This has huge implications!

For starters, the nightmare of onboarding on engineering teams (especially if they’re remote) is more or less solved. New developers just click a button on GitHub or Gitlab (or some other source control host) and they can start writing code. Or if they prefer, maybe it’s a button on the local IDE of choice. It doesn’t matter, the point is that:

• The code didn’t have to be downloaded
• No tools had to be downloaded
• No huge swath of packages had to be downloaded
• No finicky environment had to get set up
• No tool had to get pointed at the right finicky environment
• Nobody had to wait for their build tool and bespoke scripts to finish doing their thing

This is a serious time saver when onboarding onto teams. But it’s not just for onboarding!

Do you work at a place where you program against a particular environment and everything, from your build tool to your packages and your runtime environment, is consistent across all of your codebases? Well congratulations, like me you’re in developer nirvana.

Most people have a cobbled-together set of build tools, packages, runtime environments, and IDEs that they all try desperately to maintain as they work on different projects, and they waste incredible amounts of time on that.

What if you worked on different Python apps with different dependencies running against different Python environments, and you didn’t have to remember to run the right commands and click the right buttons in your editor to get everything working correctly? Instead, you just clicked a button and spun up an environment isolated from everything else, with the right tools and packages and runtime environment for that codebase?

That isn’t some dream. It’s actually possible today, and it's only going to become more capable and widespread over time:

GitHub’s Engineering Team has moved to Codespaces | The GitHub Blog

Over the past months, we’ve left our macOS model behind and moved to Codespaces for the majority of GitHub.com development.

The GitHub BlogCory Wilkerson

Gitpod shipped GitHub a launch cake for Codespaces

This is a story about a cake. Following a long-standing tradition in technology, we decided to send an edible Gitpod workspace to GitHub’s San Francisco office.

thin clients for hipsters

One of the repeated cycles I have seen in my 26+ years in the software industry is that we constantly go through this "thin client / thick client" cycle.

Citrix WinFrame back in 1995 is where my experience with this topic begins. Citrix WinFrame was the first product to offer remote desktop and application tunneling on a customized version of Windows NT 3.51. This is the same technology that was later evolved into Windows NT 4.0 Terminal Services edition and is now known as Windows Remote Desktop.

It's now 2021, the cycle is in the early stages of repeating, both GitHub Codespaces and Gitpod are uniquely positioned because of advancements in technology, internet connectivity, and a world that has been permanently reshaped by a pandemic.

Unlike Virtual Desktop Infrastructure which is all about unburdening system administrators at the expense of developer happiness. Cloud-based, reproducible developer environments are all about developer experience and sparking joy.

By 2023 working with ephemeral cloud-based dev environments will be the standard. Just like CI/CD is today.

github codespaces vs gitpod

I use both products 45/45 throughout my work week and it is very easy to switch between both of them!

GitHub - gitpod-io/template-gitpod-sxs-codespaces: A template that demonstrates how to have isomorphic developer environments between Gitpod and GitHub Codespaces

A template that demonstrates how to have isomorphic developer environments between Gitpod and GitHub Codespaces - GitHub - gitpod-io/template-gitpod-sxs-codespaces: A template that demonstrates how...

GitHubgitpod-io

Both products work exceedingly well out-of-the-box on an iPad and offer terminal-based experiences (vim / emacs ):

[feature] gitpod integration · Issue #2845 · emacs-lsp/lsp-mode

It will be interesting to explore the posibility of running emacs+lsp-mode+ in gitpod environment. This will allow the new users to explore the possibilities of Emacs with one click. https://www.gi...

GitHubemacs-lsp

There are some rough edges from time to time in VSCode but both products aggressively follow the upstream GitHub > Microsoft > VSCode repository to the point that bugs appear in both platforms and the engineers of Gitpod / Codespaces work side-by-side in harmony to resolve bugs within VSCode.

The main difference between either product is as follows:

platform

• GitHub Codespaces is a product feature of a proprietary platform.
• Gitpod is an open-source product that accepts pull requests.

ecosystem

• GitHub Codespaces only works with GitHub.
• Gitpod works with GitLab, GitHub, and/or Bitbucket accounts.

availability

• GitHub Codespaces is only available to teams.
• Gitpod's free tier is open to every developer with a GitLab, GitHub, and/or Bitbucket account.

pricing

• GitHub Codespaces costs money upfront before you can use test it out and charges start from the very first minute of usage. An eight-core machine used 24x7 will set you back $525.6 USD / month.
• Gitpod provides developers with access to both public and private repositories for 50 hours per month on similar hardware for free then $9 USD /month for 100 hours with the option of $25 USD / month for unlimited hours.

Both platforms have the following commonalities:

• Thin client development (all-day battery life, compilation on beefy servers)
• Multi-track development (develop and review complicated features with ease)
• Local machine limits (larger workloads, more data, more dependencies, more testing)
• Security (no source code on laptops policies)
• Consistency (end of all “works on my machine” problems)
• Onboarding (shave off a few initial days of tedious workspace setups)
• Remote work (fully distributed or hybrid companies)

roll your own dev from anywhere cloud

If we look at GitHub Codespaces & Gitpod critcally then in essence they are services which host a Dockerfile (which can contain anything, including qemu virtual machiens or kubernetes clusters), a virtual filesystem (the contents of your git repository) and provide a workbench (vscode / terminal pty).

Thanks to the recent release of OpenVSCode Server you can now run what is essentially exact same version of Visual Studio Code that is used on GitHub Codespaces and Gitpod on your own infrastructure.

From Theia to OpenVSCode Server - A history of Cloud IDEs from Gitpod’s IDE architect

Background story about the last four years improving the editing experience of Cloud IDEs.

A history of Cloud IDEs from Gitpod's IDE architect

Which is exactly what I do. On an average day at Gitpod I find myself working with developers in the open-source community, hoping between programming language and operating system (windows, ubuntu, nixos, arch, appliance and kubernetes) ecosystems. After this blog post ships I'm switching over to provisioning a TrueNAS virtual machine to explore what would be involved in packaging up OpenVSCode Server for that ecosystem. If you have experience in that area please get in contact!

backups

Now, if you head down this path then the responsible thing to do is talk about backups because unlike Gitpod where workspaces are ephemeral/disposable workspaces and there's a workflow of starting fresh for each new task from here on out you need to think about protecting your data, monitor that the backups are working and test that the backups work!

I use the following software and services:

• https://restic.net
• https://www.backblaze.com/b2/cloud-storage-pricing.html
• https://help.backblaze.com/hc/en-us/articles/1260804565710-How-to-use-rclone-with-Backblaze-B2-Cloud-Storage

Here is an example configuration of how to tie them together on NixOS to do hourly backups (via systemd), encrypted backups (via restic), to B2 (via rclone):

# /etc/nixos/configuration.nix

 environment.systemPackages = with pkgs; [
    vim
    rclone restic
  ];


  services.restic.backups = {
    b2 = {
      user = "root";
      repository = "rclone:b2-homelab:ghuntley-homelab";
      passwordFile = "/etc/resticPasswd";
      extraBackupArgs = [ "" ];
      paths = [
        "/home/ghuntley"
      ];
    };
  };

  systemd.services.prunebackups = {
    serviceConfig.User = "root";
    serviceConfig.Type = "oneshot";

    path = [
      pkgs.restic
      pkgs.rclone
    ];

    script = ''
      ${pkgs.restic}/bin/restic unlock
      ${pkgs.restic}/bin/restic forget --keep-hourly 48 --keep-daily 7 --keep-weekly 8 --keep-monthly 6 -r rclone:b2-homelab:ghuntley-homelab --password-file /etc/resticPasswd
      ${pkgs.restic}/bin/restic prune -r rclone:b2-homelab:ghuntley-homelab --password-file /etc/resticPasswd
    '';
  };

  systemd.timers.prunebackups = {
      wantedBy = [ "timers.target" ];
      partOf = [ "prunebackups.service" ];
      timerConfig.OnCalendar = "daily";
  };

network access and firewall

Same story again. As this is 24/7 running infrastructure you'll need to take steps to make it harder for the ABBA singing hax0rs...

Configure your server firewall to DROP all inbound connectivity, enable fail2ban and don't expose any services (ever). The way you'll access your homelab in the sky is via Tailscale:

Tailscale

Tailscale is a zero config VPN for building secure networks. Install on any device in minutes. Remote access from any network or physical location.

TailscaleTailscale

What firewall ports should I open to use Tailscale?

Nearly all of the time, you don’t need to open any firewall ports for Tailscale. Tailscale uses various NAT traversal techniques to safely connect to other Tailscale nodes without manual intervention — it “just works.”

TailscaleTailscale

On NixOS Tailscale can be enabled via

# /etc/nixos/configuration.nix

<snip>
programs.tailscale.enable = true;
<snip>

$ zt login

If you need to access to the console of the server to decrypt your filesystem storage or to troubleshoot things then use IPMI or log a support ticket for an engineer to connect a KVM console.

KVM Console - Hetzner Docs

Hetzner Docs

filesystem encryption

You need to encrypt the contents of the filesystem on your bare metal machine. Hard drives in your server fail. Hax0rs aren't the concern here. Dumpster divers are the ones you need to protect yourself against. Whilst your hosting company might state that drives are properly decomissioned, but do you trust them? The best of processes fail and humans have bad days where mistakes happen...

Australia: At home in Crunchy Town | one man, one bike, one world

This post is about the treasure at the end of the rainbow as mentioned in the last article June – October 2015 “ A gathering place for travellers to feel at home. An utopia where friendships are born daily and creative minds work together to build, inspire and bring out ones inner self. Surroundings…

one man, one bike, one world | discovering the world on a bicycleadmin

Fortunately on NixOS configuring filesystem encryption with remote unlock via SSH is a couple of lines of code.

ZFS - NixOS Wiki

NixOS Wiki

use mosh with ssh

If you don't know about https://mosh.org then stop what you are doing right now, head to the homepage of the project and send regrets my way about how you wished you knew about this software sooner.

Using Mosh on an iPad is amazing. It enables you to hop between networks, work on networks with high network latency or high packet loss and well, it just works. No disconnects. You can turn off your iPad and when it powers back on your session is right there waiting for you with the state before powering off the iPad.

On NixOS mosh and OpenSSH can be enabled as follows:

# /etc/nixos/configuration.nix
services.openssh.enable = true;
programs.mosh.enable = true;

simple homelab starter pack

The simplest way to build your home-lab in the sky is to get a virtual private server from a company such as BinaryLane. For $48 USD /month you can get a machine that would cost you $ 525.6 USD / month on GitHub Codespaces.

Download the NixOS installation ISO, create the encrypted ZFS filesystem and mount it under /mnt then run the following commands:

$ nixos-generate-config --root /mnt
$ vim /mnt/nixos/configuration.nix
$ nixos-install --root /mnt

Reboot your server, when it comes up adjust /etc/nixos/configuration.nix to enable OpenVSCode-Server.

# /etc/nixos/configuration.nix

virtualisation.oci-containers.containers = {

    openvscode-server = {
      image = "gitpod/openvscode-server:latest";
      volumes = [
        "/srv:/home/workspace"
        # enable docker in docker
        # nb. careful this allows root access
        "/var/run/docker.sock:/var/run/docker.sock"
        "/nix:/nix"
      ];
      ports = ["3000:3000"];
      environment = {};
      cmd = [];
    };
  };

If NixOS isn't your thing then you can use Ubuntu, Arch, or whatever and run the docker image from a bash terminal after installing Docker (or podman).

$ docker run -it --init -p 3000:3000 -v "/srv:/home/workspace:cached" gitpod/openvscode-server

you can have it all starter pack

Now, this is where it gets interesting. If you are a technology CEO on a sabbatical rediscovering the joys of effect systems and getting back on the tools or someone like myself who needs endless compute possibilities without restraint at a fixed cost then a dedicated bare metal server is how you do it:

Instead of spending $525.6 USD / month on an eight-core virtual machine (where the CPU threads / cache is shared between other people) for $256.99 USD / month, you can get the best money can buy:

• AMD EPYC™ 7502P 32 Core CPU
• 128GB DDR ECC of memory
• 2 x 4TB of NVME storage.

If that's too much then $74.59 a month will get you:

• AMD Ryzen™ 7 3700X Octa-Core CPU
• 64 GB DDR4 ECC of memory.
• 2 x 1 TB of NVME storage.

I highly recommend that you install a hypervisor to partition up your server. Skip VMWare ESXi because it requires hardware RAID for your storage and hardware raid inferior to ZFS + only available on more expensive servers.

Instead of VMWare ESXi what you want to install is Proxmox and purchase a software update subscription for $104.48 USD / year.  

Once you have Proxmox up and running you can do literally anything. For each virtual machine that you provision install Tailscale on the virtual machine so that you can access the virtual machine from your iPad.

If you need access to a Ubuntu machine use the Blink app to mosh into it. If you need access to a Windows desktop use the Microsoft Remote Desktop App.

When the baker of the cake I sent to GitHub Codespaces sent across the video below it was multiple gigabytes in size and I only had 2G / edge network connectivity. Instead of downloading the video to a local computer I did all of the video editing on my homelab via remote desktop.

Anyway, thanks for reading. This post is the sixth in the series. I'm blogging more and tweeting less, so if you want to learn about sweet places to visit in Australia, working remotely from a van enter your email address to be notified when future blog posts ship.