The road to Nix, a functional package manager to rule them all


#61

Daily progress update:

  • Yesterday’s changes to create a shell.nix, while very beneficial to create a cache reliably, brought some issues, notably:
    • with fastlane (some issue building the unf_ext gem). Today was spent mostly bringing Nix’s fastlane package to status-react, so we don’t use it from outside and don’t have to build the Gems;
    • the iOS build started failing, which turned out to be due to the fact that mkShell brings stdenv by default, instead of the stdenvNoCC that we need in iOS.

#62

Daily progress update:

  • The fastlane migration will take more work than expected, so I’m splitting that into a separate PR, so that the status-go PR can be submitted for review later today.
  • Created PR to upstream gomobile nix package.

#63

Daily progress update:

  • With the help of @jakubgs, the Makefile now has a clean syntax for implicitly entering a temp Nix shell when needed.

I’ve also done away with the make setup command, so now when you clone the repo you can run something like make release-android and it will:

  1. install Nix;
  2. pull all required dependencies;
  3. build/deploy the Android app.

Result when running make release-android after git clone without further user intervention:

real    13m5.198s
user    3m38.569s
sys     0m27.528s

#64

Daily progress update:

  • the Nix status-go branch has been merged! :sparkler::tada::fireworks: No more downloading artifacts from Digital Ocean.
  • investigated Nix fastlane some more, but it looks like it is too dependent on the host macOS system for it to be reliable, so no real advantage on macOS systems. Will likely drop this branch.
  • pushing two other branches for completion which should make Nix much easier to use (basically not any worse than pre-Nix usability, due to implicit calls to nix-shell and even implicit call to make setup if needed).

#65

Daily progress update:

  • Babysitting the outstanding Nix PRs and merging them;
  • Looking into yarn2nix, a promising tool to integrate our node dependencies into Nix and avoid having to symlink things depending on whether we’re building for desktop or mobile (would like it to be done automatically depending on which Makefile target is being executed).

#66

Daily progress update:

  • Looked into yarn2nix, and although it worked perfectly for normal node packages (it would create a symlinked node_modules into the Nix store cached node_modules), React Native Android gradle tasks didn’t like it, because they want to be able to write to node_modules. So I’m abandoning that investigation;
  • Currently looking into leveraging androidenv.androidsdk instead of our own download of the Android SDK. Some preliminary findings:
    • Even though we can easily point the env vars to the SDK in the nix store, it will not be usable by the gradle tasks, since the SDK licenses are not accepted in nixpkgs (which makes sense, each individual developer needs to accept the license and create his own ID). We also can’t use sdkmanager to accept the licenses since it would try to write to the immutable nix store directory.
    • The next best approach is to copy the nixpkgs Android SDK to our managed location (~/.status/Android/Sdk) any time we detect it is not present. This has the advantage that it creates a single source of truth in terms of SDK sub-package versions in the nix expression, and is much faster to recreate after we clean the home directory folder.

As we can see, Nix can be a painful medicine, but it does force us to recognize which parts of our setup are impure (such as the gradle tasks or the Android SDK) and manage it accordingly.


#67

Daily progress update:

  • Merged the remaining PRs (optimizing the Makefile so it causes Nix to only install/build the bare minimum required, copy Android SDK from nix store and license it instead of downloading from internet, migrate Node.js to 10.15, and shell.nix refactor);
  • Started working on getting our builds to pass with nix-shell --pure flag, which forbids access to network, directories outside the nix store/repo and environment variables, so that we can be guaranteed as much as possible that my build is identical to the CI build and your build.
    Known impediments:

I’ll be on vacations the next couple of weeks, so don’t expect a lot of progress in this area during that time, but there’s plenty of cool stuff coming down the pipe once I get back!