Nimbus on mobile

Yes, seems like there is some confusion on what exactly hybrid and clean slate means, the way I see it:

The end result in both cases is identical (only nimbus will be there, no status-go/status-protocol-go), it’s only about the process, i.e how to get there.

  • Clean slate approach: No Nimbus - Go integration. Clojure/RN interacts directly with Nimbus. Thus at the level of status-react API.
  • Hybrid approach: There is a Nimbus - Go integration. Go sits in-between Nimbus and Clojure/RN. Where exactly this integration happens is to decide. But it would probably start as low as e.g. Whisper API level and gradually increase.

This sounds exactly how I understood it, I am advocating for the hybrid approach, slowly chocking status-go, and eventually get rid of it completely, so when you say:

However, I think that the clean slate approach is not necessarily restricted to not having any intermediate solutions with both Nimbus & status-go / geth.

That sounds like the hybrid approach to me, so probably we are not far off in views, but maybe there are some fundamental differences in the way we see it.
Whether we do it vertically (one endpoint is fully using nimbus, while the rest will be using status-go), or horizontally (one endpoint calls status-go, which in turns uses nimbus), I guess depends on what we will be tackling and in which order, both seems reasonable approaches and we might mix and match.
The end-result in both approaches will be the same (no status-go, only nimbus), which I believe we are all in agreement and it’s the company strategy.

Would this be a good place to start?

Yes, plus https://github.com/status-im/status-react/blob/develop/src/status_im/native_module/core.cljs which are some endpoint that are only exported natively, on top of that there are the asynchronous signals https://github.com/status-im/status-react/blob/ebc2ff04cd9d27269729e3208b1fdeac46bd2cb3/src/status_im/signals/core.cljs#L47 which the app is reacting to.

1 Like

Me after writing this post

image

(+1 for hybrid approach)

6 Likes

We’ll be using this thread to share regular updates on the progress of moving the app to rely on Nimbus. Today we had our first status-nim meeting (notes available here, thanks @mamy!) and the next will be tentatively scheduled for Oct 14th.

Next step will be to do some janitorial work on status-protocol-go to ensure that geth types/implementation details do not leak out of the very lowest layers, making it simpler to later implement a Nimbus transport layer. I’ll be scheduling a quick meeting with @cammellos and @adam to go over that.

cc @kim

Coming a bit late to the discussion :wink:

Very good point.

geth sucks in being a library. I guess it has never been a goal for them to provide a Go library for the Ethereum stack but rather provide a command line Ethereum client.

Not sure how nimbus is built but in geth there are plenty of places where we don’t have as much control as we needed to efficiently implement business logic.

The end result in both cases is identical (only nimbus will be there, no status-go/status-protocol-go)

I saw this statement a few times in the posts above and I wonder why is it so*? Why do we need to choose one instead of plan for interfaces, modularity and encapsulation?

I think that a vision where we have a few components like p2p stack, storage, chat logic etc. where some are written in Nim and some in Go is totally fine. The important part is how the client can interact with these components.

This is the mistake we have been repeating over and over – building these giant monoliths where to run a stupid unit test you need an Ethereum node and business logic is tangled with low-level p2p bullshit. Nimbus on mobile effort seems like a good opportunity to break out from this loop.

*One valid argument is to simplify the build process. That makes sense however, (1) we already figure out that and (2) having just one output skews the perspective. Also, using this argument, I could say that we should get rid of React Native – that would simplify a lot of things :slight_smile:

Next step will be to do some janitorial work on status-protocol-go to ensure that geth types/implementation details do not leak out of the very lowest layers, making it simpler to later implement a Nimbus transport layer.

That sounds fine. One more thing that I mentioned during the call is to figure out how Go and Nim (and Java and Objective-C) can communicate. So far, we have been relying on C bindings. Maybe there is a better way?

Not quite the same, but we just dusted off stratus to make sure it still builds (thanks @stefantalpalaru!) - if you’re looking for more prototype-level whisper code to experiment with, there’s some in there.

Hopefully it should work well with the 1.9.5 geth/whisper implementation.

Daily update:

  • After having made a first pass at wrapping Geth data structures/APIs in status-protocol-go yesterday, I’m working today in updating status-go and status-console-client to use the updated API. Lost a bit of time since I started on recent Geth 1.9.x branches, only to realize that out-of-the-box it is incompatible with Geth 1.8.x due to PoW changes, so messages would have no chance of arriving to the destination. Rebased on existing Geth 1.8.x branches.

Important note on those PoW changes: Whisper in Nimbus has always used that new PoW calculation (To be fully correct: there is still a small difference on size calculation, which I could easily alter. But it does do the same hashing).

Is there a plan on moving to that version?

Interesting,
so is this POW calculation backward compatible? i.e if I send a message using the new POW calculation (1.9) will it be accepted by 1.8 ?

The reason I am asking is that when we did our experiments we would only get through 8 out 2000 messages for example, from nimbus to geth, wondering if maybe that’s the reason.

I’m pretty sure that they are not backward compatible. 8 out of 2000 sounds statistical in terms of randomly getting the right PoW. IIRC it had to do with what part of the hash was being checked for PoW (front or back) so to be backwards compatible we should check both.

This should be fixed asap in status-go or status-protocol-go, wherever it lives.

I think some work is already underway, https://github.com/status-im/status-go/pull/1617

I guess we’re getting close to the point where we’ll need everyone to move to the new desktop and mobile clients, with new chat identities. And upgrade the bridge bots too, of course. cc @jakubgs

No they are not backward compatible.
It was not just the part of the hash that is being checked (the leading zeroes counted), but also the hashing done itself. They would actually do a keccak256 hashing twice (whisper v5 leftover). If you’re lucky to have enough leading zeroes also with the other calculation, your message will not be dropped.

If you want to remain compatible you have to either lower the PoW requirement even more (practically 0) or (temporarily) implement and use both calculations at receiving end.

@pedro I was waiting with the switch to new chat identities until Keycard initialization works properly. But now that I think about I’m actually not sure how that would work with desktop. If I create an identity on mobile with my Keycard I assume I’d need a usb card reader for my desktop to recover it there, right? Is that something we support?

Hmm good point, I’d hope so. Maybe @gravityblast knows something about that?

There’s no Keycard integration for desktop for now, so you won’t be able to use it. Unless you enter your mnemonic on desktop, which is probably something you don’t want to do if you have a Keycard.

Why is that? Would that work? Wouldn’t that just give me a different address without the Keycard?

It would give you the same chat key, but negates a lot of the security benefits of using a keycard, as the keys have left the hardware wallet and live on a very buggy alpha desktop connected to the dirty internet.

Right, so we(core contributors) might as well not use Keycard at all if we want to use desktop app for work in the future. Unless support for keycard for desktop is planned as well.

If that is the case then we might as well bite the bullet and pick a day to switch to V1 chat identities soon, right?

1 Like

I’ve been forgetting to post the daily progress here (I’ve been doing it in #status-standups), but where we stand currently is that we have a refactored status-protocol-go that is ready to receive Nimbus under a facade, and updated status-go/status-console-client that consume that facade. I’m ready to start working with Kim to bring Nimbus to status-protocol-go. Our bi-weekly meeting will happen today to coordinate efforts.

1 Like

Daily progress:

  • After fixing several issues with C <-> Go marshaling yesterday, I started investigating more complex scenarios like using the subscription to a filter to emulate the arrival of envelope events, but this requires ugly changes to the Nimbus API and is not really required for the console app PoC, so I’ve decided to focus on getting a minimal implementation working and work from there.