The road to Nix, a functional package manager to rule them all


#48

I’d like to thank you Pedro for this amazing work. This should in the long term make our builds(both manual and CI) more reliable and predictable. As with any big change this might cause some issues initially, but all in all this is definitely a move in the right direction. I owe you a beer, or probably several.


#49

I’ve had a call this morning with @volodymyr.kozieiev and the nix branch is working for him, so it won’t be an immediate blocker for him. It will be a nuisance for new developers wanting to debug Status Desktop, but I feel like that will be greatly outweighed by the number of external contributors now able to install and run the mobile app from the first attempt.


#50

I had managed to build from develop and able to successfully create Status-x86_64.AppImage however I was unable to execute and successfully run the AppImage

NixOS isn’t really designed to run arbitrary binaries, so I am using appimage-run utility, further discussion here.

$ appimage-run Status-x86_64.AppImage
./AppRun: error while loading shared libraries: libharfbuzz.so.0: cannot open shared object file: No such file or directory

It’s unclear to me if harfbuzz is a missing dependency that appimage-run is missing, or something our AppImage is missing?

I’ve had success with appimage-run on other AppImages, such as Turtl.

Alternatively it would be great to have a status nixpkg to install.


#51

I just confirmed that libharfbuzz.so.0 is not included in the AppImage. In fact, linuxdeployqt blacklists some libraries which it assumes are present in all supported distros. We could modify linuxdeployqt to not blacklist those DLLs (at the expense of the size of the AppImage), however for NixOS, I think the right thing to do is to just publish a package on nixpkgs and avoid patching things back and forth.


#52

Daily progress update:

  • In an effort to make the whole pipeline reproducible, I’ve started looking into writing a Nix expression to build status-go. This should make it easier to test status-react with user status-go branches, since Nix will know how to build it if necessary (if it is not yet on the Nix cache). It will also simplify the distribution model and eventually make it easy to have a single distribution model for Desktop and Mobile.
  • In view of this, I’ve determined that in order to build Mobile status-go, we’ll also need to build and package the gomobile project. This is something that in time we can contribute back to the Nixpkgs community.
  • Today I’ll look into building the gomobile package, which seems to require some improvements in the android-ndk package.

#53

Daily progress update:

  • After fixing the Android NDK package and building a gomobile one, I’ve been able to use that to generate a status-go Android artifact, but only from the nix-shell. If It try to run the same command in the status-go expression’s buildCommand attribute, I get some failures in cgo. Currently trying to figure out the cause.

    # runtime/cgo
    fork/exec /nix/store/ik293nv1ynswd6q97laz02bys598spgy-android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/armv7a-linux-androideabi16-clang: no such file or directory
    

#54

Daily progress update:

  • Yesterday was a very productive day, marked by being able for the first time to produce a statusgo.aar artifact in the /nix/store. After much spelunking in the source code of gomobile and nixpkgs, and with the help of @jakubgs with his suggestion to employ strace to figure out the missing file, I was able to patch gomobile (it wasn’t able to use our downloaded Android SDK and needed patching to deal with the symlinking of nixpkgs’s Android SDK).
  • The road ahead is now much clearer and allows me to establish a plan:
    • Expose the generated .aar file to the Gradle files, and replace code that downloads the artifact from Digital Ocean;
    • Figure out why libgojni.so files in statusgo.aar are 50% larger than existing ones. Probably strip is not being run on them. Update: Unable to strip library '/home/pedro/src/github.com/status-im/status-react/android/app/build/intermediates/transforms/mergeJniLibs/release/0/lib/armeabi/libconceal.so' due to missing strip tool for ABI 'ARMEABI'. Packaging it as is.
    • Try to use Android NDK package from nixpkgs. This will inform the same improvement for the rest of status-react;
    • Replace bundle-status-go.sh with a convenient way to build a local status-go;
    • Fine tune the Android SDK component versions installed by Nix using androidenv.composeAndroidPackages.
    • Build status-go for iOS.
      • Replicate necessary fixes on Android NDK for Darwin platform;
      • The problem here is how to expose XCode to the status-go recipe, which already uses a derivation specific to Go compilation;

Link to new Pivotal Tracker story: https://www.pivotaltracker.com/story/show/164984148


#55

Good news, I’ve just installed my first Status app building status-react and status-go from scratch with a single make shell / make release-android sequence! Nix will build gomobile and use it to build the status-go version specified by the STATUS_GO_VERSION file if that hasn’t been built before.

CI logs here for the mind-blowing experience of seeing status-go and status-react building on the same go.


#56

Daily progress update:

  • Built status-go package consisting of Android libraries (/lib)and Linux/macOS binaries (in /bin, for statusd, bootnode, node-canary, etc);
  • Replaced usage of React Native’s Gradle wrapper with Nix’s Gradle package. That way we avoid one additional download;
  • Finished gomobile package, leveraging Nix’s Android NDK and SDK packages instead of our custom ones.

Next I’ll pair with @jakubgs to do the same for iOS.


#57

Hey @zahary, just so you’re aware of the current CI status, we built a custom Docker image based on NixOS’s which has the cache prepopulated. We also run a nightly Jenkins job which incrementally populates https://nix-cache.status.im. I’m hoping to work with @jakubgs soon to find a way for our NixOS docker image to be able to share the Nix store with other builds so we don’t need a fat prepopulated Docker image.


#58

Daily progress update:

  • Fixed iOS build by wrapping system XCode in a Nix package (with version-check). Looks like the nixpkgs xcode doesn’t contain the mobile toolchain;
  • Adapted Desktop CMake file to leverage the Nix status-go package. This means that the only platform which builds a custom status-go now is Windows. All the others rely on the Nix package;
  • Did a 1-1 with Jakub to go over the structure of the upcoming PR.

#59

Daily progress update:

  • Worked with Jakub to get nix build command to work properly on both macOS and Linux, and to fetch/build all packages. It previously was only fetching some, so that wasn’t ideal to e.g. populate the Nix cache.
  • Met with Jakub and Hlöðver Sigurðsson (https://github.com/hlolli) who’s versed Nix/NixOS so he could look into how we are doing things, and apparently we’re on a good path.

#60

oh, cool!

I still have some problems with running the app for Genymotion (react-native can’t find the server), iOS works just fine now.


#61

Daily progress update:

  • Yesterday’s changes to create a shell.nix, while very beneficial to create a cache reliably, brought some issues, notably:
    • with fastlane (some issue building the unf_ext gem). Today was spent mostly bringing Nix’s fastlane package to status-react, so we don’t use it from outside and don’t have to build the Gems;
    • the iOS build started failing, which turned out to be due to the fact that mkShell brings stdenv by default, instead of the stdenvNoCC that we need in iOS.

#62

Daily progress update:

  • The fastlane migration will take more work than expected, so I’m splitting that into a separate PR, so that the status-go PR can be submitted for review later today.
  • Created PR to upstream gomobile nix package.

#63

Daily progress update:

  • With the help of @jakubgs, the Makefile now has a clean syntax for implicitly entering a temp Nix shell when needed.

I’ve also done away with the make setup command, so now when you clone the repo you can run something like make release-android and it will:

  1. install Nix;
  2. pull all required dependencies;
  3. build/deploy the Android app.

Result when running make release-android after git clone without further user intervention:

real    13m5.198s
user    3m38.569s
sys     0m27.528s

#64

Daily progress update:

  • the Nix status-go branch has been merged! :sparkler::tada::fireworks: No more downloading artifacts from Digital Ocean.
  • investigated Nix fastlane some more, but it looks like it is too dependent on the host macOS system for it to be reliable, so no real advantage on macOS systems. Will likely drop this branch.
  • pushing two other branches for completion which should make Nix much easier to use (basically not any worse than pre-Nix usability, due to implicit calls to nix-shell and even implicit call to make setup if needed).

#65

Daily progress update:

  • Babysitting the outstanding Nix PRs and merging them;
  • Looking into yarn2nix, a promising tool to integrate our node dependencies into Nix and avoid having to symlink things depending on whether we’re building for desktop or mobile (would like it to be done automatically depending on which Makefile target is being executed).

#66

Daily progress update:

  • Looked into yarn2nix, and although it worked perfectly for normal node packages (it would create a symlinked node_modules into the Nix store cached node_modules), React Native Android gradle tasks didn’t like it, because they want to be able to write to node_modules. So I’m abandoning that investigation;
  • Currently looking into leveraging androidenv.androidsdk instead of our own download of the Android SDK. Some preliminary findings:
    • Even though we can easily point the env vars to the SDK in the nix store, it will not be usable by the gradle tasks, since the SDK licenses are not accepted in nixpkgs (which makes sense, each individual developer needs to accept the license and create his own ID). We also can’t use sdkmanager to accept the licenses since it would try to write to the immutable nix store directory.
    • The next best approach is to copy the nixpkgs Android SDK to our managed location (~/.status/Android/Sdk) any time we detect it is not present. This has the advantage that it creates a single source of truth in terms of SDK sub-package versions in the nix expression, and is much faster to recreate after we clean the home directory folder.

As we can see, Nix can be a painful medicine, but it does force us to recognize which parts of our setup are impure (such as the gradle tasks or the Android SDK) and manage it accordingly.


#67

Daily progress update:

  • Merged the remaining PRs (optimizing the Makefile so it causes Nix to only install/build the bare minimum required, copy Android SDK from nix store and license it instead of downloading from internet, migrate Node.js to 10.15, and shell.nix refactor);
  • Started working on getting our builds to pass with nix-shell --pure flag, which forbids access to network, directories outside the nix store/repo and environment variables, so that we can be guaranteed as much as possible that my build is identical to the CI build and your build.
    Known impediments:

I’ll be on vacations the next couple of weeks, so don’t expect a lot of progress in this area during that time, but there’s plenty of cool stuff coming down the pipe once I get back!