Signed messages: Protocol change proposal


#1

Problem

Currently we rely on whisper for signing the messages.
It works by taking anything you send though whisper, signing the whole payload S(payload), and sending both payload and signature to other peers.

A client will use that signature and the payload to extract the public key of the sender, which we then use to:

  1. Identify which encryption key to use in case of PFS encrypted messages
  2. Identify the author of the message

This works fine for our use case, but has currently a few drawbacks:

  1. We rely on out-of-band information (whisper) for our own protocol (anything above whisper) for authentication
  2. It does not support relay messages to third parties, which is currently needed for MVDS, as currently it does not forward the whole whisper payload, but only the content of it, which means that current messages are unauthenticated.

It would also be beneficial for paired devices (whereby syncing anything would be just a matter of forwarding that message, effectively getting rid of the need of having a separate protocol and allowing to have more redundancy when a device has not been included, but that’s sci-fi programming for now).

Where we want to be

Ideally in the best case scenario, we would like our protocol to support both plausible deniability and relaying messages (the two things are not compatible), and let the client specify which one to use.

As it stands, messages are not plausibly deniable because of whisper (there’s a plausibly deniable mode in whisper, but it works by signing only the encryption key and not the content of the message, so not sure what kind of guarantees we can make).

How to get there

Public messages

Public messages needs to be signed, as there’s no such thing as a plausibly deniable public message (that would just be an anonymous message). They are also not encrypted (in our layer), so here the solution is pretty straightforward.

We can add to each message a signature of the raw payload (in our case of the transit encoding), so that we have:

bytes signature
bytes payload

Upon receiving a message a client will calculate the PK from the payload and the signature.
In this case we will not be using the whisper signature at all, therefore the message can be relayed to third parties, which will understand the original author of the message.

Backward compatibility

Backward compatibility is a bit tricky here, but it’s doable.

There are two cases that we need to consider:

  1. The message is not relayed, so the whisper key == key extracted from the signature. This is not a problem, old clients will use the whisper key, new clients will use the signature.
  2. The message is relayed, so the whisper key != key extracted from the signature. In this case, old clients will wrongly use the whisper key, resulting in the message being attributed to the relaying party and not the original author. To avoid this scenario, we can have the relayed message in a different field, and when relaying a message clients will leave the original payload empty. Effectively old clients will see an empty message, while new clients will understand it’s relayed message.

So the format of the message would be:

bytes signature
bytes payload // Empty for relayed messages
bytes relayedPayload // Empty for non-relayed messages

Another important property we would like is that we should be able to relay public messages over private channels.
For example:

Alice sends a message to the public chat #status
Bob receives the message
Bob & Donald do their dirty datasync stuff, and agree that Donald should be sent Alice message.

It is important that this message can be PFS encrypted. We will make sure that is the case once we explorer 1-to-1 communication.

One to one messages

One to one messages are a bit more complex, as we also need to correctly identify which encryption key we will use for decrypting the message (currently we use the key extracted by whisper).

Also, we have some negotiation over the protocol, so we just need to be careful not to make any assumptions about the content which we are encrypting/relaying and the peer that is sending it (i.e if the relaying node is running v2 , it does not mean that the message relayed is also running v2).

The crucial thing here is that we will not compute the signature over the encrypted payload, but always over the unencrypted payload. That’s because otherwise relaying would not work as the other peer might not be able to decrypt the same message. So the format would be:

bytes signature // Signature of the unencrypted payload
bytes payload // Empty for relayed messages
bytes relayedPayload // Empty for non-relayed messages

In this case we could also just have payload instead of separating between relayedPayload and payload, as this are direct messages and we can just send only to those clients that we know will understand this, but we could keep it the same for consistency.

In order to be able to know which encryption key should be used to decrypt the message, we can simply add that to the payload:

bytes identity // The pk of the user sending the message
bytes signature // Signature of the unencrypted payload
bytes payload // Empty for relayed messages
bytes relayedPayload // Empty for non-relayed messages

So we can get rid of the dependency on whisper.

It is clear that this method works both for relaying publc chats or one to one messages (there’s no difference between them, as we always sign the unencrypted payload).

Plausible deniability

In plausible deniability mode, signature will not be there.
In that case it can’t be a relayed message.

bytes identity will still be there (but not signed, so anyone could “lie” about it)

We will use that identity to pull a double ratchet from the database, if the messages decrypts, we know bytes identity has authenticated with us using X3DH, so we have effectively validated identity.

This will not work for initial non-PFS messages, which will still need to be signed, although those are increasingly rarer as we are getting better at propagating the bundle, but necessary until ux changes or swarm/ipfs integration.

If a signature is not present, that message will be plausibly deniable (not counting whisper), and we won’t be able to relay it.

Backward compatibility

This change is backward compatible

TLDR

We need to add in-protocol signature of messages to be able to use MVDS effectively and get rid of whisper dependency.
I propose changing the payload of messages to include two extra pieces of information:

For both public and one to one:

bytes signature The signature of the unencrypted payload

For one to one:

bytes identity The identity of the user sending the message (not of the user that authored the message)

The change is backward compatible by separating the payload from the relayedPayload:

bytes payload // Original payload, empty if relayed message bytes relayedPayload // Relayed payload, empty if not relayed

If we don’t want to be backward compatible (say v1 breaks), then we can simply have a single field:

bytes payload

And we know there’s not going to be any problem of wrong authentication.

What do people think? feedback is welcome


#2

When you say “relay” does that refer to a network-wide rebroadcast, or a relay to a user’s other devices?

How specific is the relay?


#3

Both are possible, not sure what we want to do with datasync, it’s not going to be a network-wide broadcast, but for public chats might be a public-chat-wide broadcast (as any public message), while for one to one messages it would be just a direct-one.
Basically is as specifically as we want to be (you can send a public message to a single user/device, or send a public message to anyone in the public chat)


#4

In the case of keeping backward compatibility, does that mean old clients will get spurious messages from relayedMessages that look empty?

If so, how do they handle them? throw them away or be a dumb relayer of seemingly empty messages?

if relay, is that a potential source of spam?


#5

old clients will get empty messages, yes, and they will just be discarded.

Potentially relay can happen on a separate topic to avoid this, say clients joining #status will relay on #status-relay , so only never clients will listen to that topic.

In terms of spam it should not be any more spammy then sending a public chat message, if that makes sense :slight_smile: , basically a spammer will not be advantaged by using relayed messages (of course we need to make sure data sync clients are not coherced into spamming, but that’s different)


#6

What do we mean by payload is that a Status Message, a Whisper Message or an MVDS Payload?


#7

That would be the encoded version of a Status Message (but could be anything),
it could also be an MVDS encoded message (not just the payload), if that can be relayed, so effectively you would have:

WHISPER

ENCRYPTION

MVDS

STATUS MESSAGE

or another option is

WHISPER

MVDS

ENCRYPTION

STATUS MESSAGE

1 is probably better and closer to briars


#8

@cammellos so its important we start cleaning up language for this to reduce confusion. In MVDS we have a payload, which contains multiple records. one of those record types is Message which contains a byte payload that is a Status Message. In my opinion a signature key should be added to the Status Message.


#9

I agree with the language.

There are two status message that we are possibly referring to:

In the console client, there is StatusMessage and it’s an internal construction, never sent across the wire, so not part of the protocol, https://github.com/status-im/status-console-client/blob/695bdfb61c6119a97df74f41c341e59013a18484/protocol/v1/message.go#L65 (The one in your PR).

I was referring to what actually is sent through the wire which is basically https://github.com/status-im/status-console-client/blob/695bdfb61c6119a97df74f41c341e59013a18484/protocol/v1/message.go#L66 , i.e the encoded version, this one can’t contain a signature as it’s the message that needs signing, any signature will either need to be appended/prepended/put in a wrapper, which is likely to be a breaking change (for public chats at least), but worth considering.

Eventually it would end up in StatusMessage in the console client, as that keeps the metadata, but that’s not what is being sent over the wire, which is this https://github.com/status-im/status-console-client/blob/695bdfb61c6119a97df74f41c341e59013a18484/protocol/v1/message.go#L73 or https://github.com/status-im/status-console-client/blob/695bdfb61c6119a97df74f41c341e59013a18484/protocol/v1/pairinstallationmessage.go#L10 (transit encoded).

You bring a good point about mvds dispatching multiple messages, so I will need to adjust the proposal to accommodate that, thanks.


#10

Ok so we add the signature to https://github.com/status-im/status-console-client/blob/695bdfb61c6119a97df74f41c341e59013a18484/protocol/v1/message.go#L73? That to me seems like the cleanest solution, how do you feel about this?


#11

It comes with some issues,
basically first it would need to be added to any of the messages https://github.com/status-im/status-console-client/blob/695bdfb61c6119a97df74f41c341e59013a18484/protocol/v1/message.go#L73 https://github.com/status-im/status-console-client/blob/695bdfb61c6119a97df74f41c341e59013a18484/protocol/v1/pairinstallationmessage.go#L10 and other that the status-client does not support yet), but that’s probably ok.

The main issue is that we are adding the signature to the same message that needs signing, so upon receiving the payload the client would have to:

  1. Deserialize the message
  2. Extract signature

And then we need to check the signature against the signed message to extract the pk, here it depends on how the signature is computed:
a) Signature is computed by using some specific fields (i.e S([]byte(Text, Content…)):
This is easier to compute but will result in a breaking change when adding new fields, as the client verifying the signature will not know about the new fields, so this is not an option.
b) Signature is computed over the serialized message without the signature:
This has the advantage of not breaking on new fields, but it means we would have to double encode the message. (Deserialize the message, take signature, serialize with the signature, verify), so that’s something we don’t probably want to do.

So basically it’s not advisable to have the signature inside the message that we are encoding.

The simplest solution is to append/prepend/wrap the signature to the bytes payload of the serialized message, so the actual payload is always:

[32]byte + []byte

or

[]byte + [32]byte

Because in our case the signature is optional (for plausible deniability), makes appending/prepending a a bit trickier, and we would have to either fingerprint or have some codes to specify whether the signature is included.

So at that point is probably just easier to wrap the message in a protobuf structure:

type MessageWithSignature {
bytes signature
bytes payload
}

(that’s basically the solution above, although does not support multiple messages, and therefore we would have to have another wrapper to accomodate that).
In that case verifying the signature is just extracting the payload & signature and verifying, without having to encode twice or breaking when new fields are added.

Roughly that’s what I am thinking, it’s a matter to see how we can make it work with the current stack and how much overhead it adds to use protobuf (I would think little, but it’s to verify), let me know if you have any comments


#12

I looked at Whisper and it has a flag to detect whether a signature is present or not (first byte is for flags) and append the signature. It looks efficient in terms of space and encoding/decoding speed but it’s a bit more tricky to work with.

@cammellos when you said that it is backward compatible, you mean our current “backward compatibility” definition which is two versions back? Because wrapping of the payload (after transit encoding) is a breaking change by itself.

Also, I have a question about relay mode. In this case, the relayedPayload would be { bytes originalSignature, bytes originalPayload }?

And finally regarding the layers, I though that MVDS would be between Whisper and Encryption so the second option. It seems more natural as otherwise we would need to keep an adapter between Encryption and Whisper, i.e. decide which Whisper.NewMessage properties should be set based on the message spec. I imagine that MVDS could play that role while from the Encryption layer perspective the interaction with MVDS would be a lot simpler.


#13

I looked at Whisper and it has a flag to detect whether a signature is present or not (first byte is for flags) and append the signature. It looks efficient in terms of space and encoding/decoding speed but it’s a bit more tricky to work with.

I guess it’s an option, and we could use a similar approach, but at at that point I would consider protobuf instead of rolling out or own, unless there’s a big difference in size, what do you reckon?

@cammellos when you said that it is backward compatible, you mean our current “backward compatibility” definition which is two versions back? Because wrapping of the payload (after transit encoding) is a breaking change by itself.

In the initial post I was referring to that, in the last post I was referring specifically to adding fields, wrapping would be a breaking change, so it’s only on the table if we decide to break in v1, can’t remember what’s the state of that conversation, otherwise the signature will have to go somewhere else, which ties with the last point you made.

Also, I have a question about relay mode. In this case, the relayedPayload would be { bytes originalSignature, bytes originalPayload } ?

It could be, although not necessary, as when relaying messages I don’t think we need any signature other than the original signature, as there’s no data authored by the relayer. (unless we want to authenticate the relayer for some reason, but don’t see why).

And finally regarding the layers, I though that MVDS would be between Whisper and Encryption

This has some consequences, it depends on what we want. If MVDS sits between whisper & then encryption layer:

  1. MVDS needs to propagate PFS messages (currently it does not, but it should as we can’t lower our security guarantees when using mvds)
  2. Any mvds interaction will not be protected by PFS (i.e the topic/group id the messages you are interested in, the id of the messages etc)

The thing I am most unclear about is:

  1. MVDS alone will not know whether a message can be decoded by other peers, making relaying to a 3rd party less flexible

Let me explain that, also this is only for private group chats/1-to-1, public chats work fine:

Say you have 3 people listening to the same group-id (currently is the whisper topic, but might very well be the group-chat id for example).

A sends a message (M1) to B & C, but only the message to B actually gets sent and received (mind that we use pairwise encryption here). This message is a pfs encrypted message, so only B has access to it.
This message will be added by the data sync layer in it’s PFS encrypted form. B & C will datasync over the groupID , and C will request M1, but won’t be able to decode it.
So effectively we are in a place where not all the messages can be meaningfully data-synced, effectively we can’t relay to third parties messages that were not originally encrypted for the given recipient (so we can’t use data-sync only for making sure all of our devices have all the messages, for example).

The advantage of this is that relaying messages to their intended recipient is easier.

The other method is to make it sit between Encryption and application layer.
In this case we can more easily relay to third parties, as the problem above does not apply. It is consistent with what briar does “The transport layer security protocol is responsible for ensuring the confidentiality, integrity, authenticity and forward secrecy of the data it carries.”.
Messages also can be always be relayed by MVDS, the only matter is whether we want to relay this messages, as per Briar docs “The client is responsible for deciding which peers the group should be shared with, what constitutes a valid message, which messages should be shared, and which messages should be deleted”.

The fundamental differences in my view is that:

if mvds sits between whisper an encryption, the responsability of deciding who gets to see what lies in the sender of the message. If I don’t encrypt a message for you, then you won’t be receiving. On the flip side, I can more easily relay messages that are not for myself, but that’s not a behavior we currently use (only mailservers do that, and also client would be limited by whisper).

if mvds sits between encryption and application, each client can make decisions on which messages are shared with whom, as long as the original message targeted us, so effectively when sending a message you also lose some control over it and trust the other peer on sharing it only to peers who should have access to it, which makes it more flexible at the cost of some risks. Also here you will only be able to relay messages that targeted you (or that have been relayed to you specifically). Essentially this is more viral.

I don’t have a strong opinion myself, both offer pro & cons, I lean towards having Whisper->encryption->MVDS->Application as that’s what briar does, so we have less unknowns


#14

+1. This layout seems more natural to me.


#15

Re where MVDS fits into layers:

From https://github.com/status-im/bigbrother-specs/blob/master/data_sync/mvds.md#secure-transport

This specification does not define anything related to the transport of packets. It is assumed that this is abstracted in such a way that any secure transport protocol could be easily implemented. Likewise, properties such as confidentiality, integrity, authenticity and forward secrecy are assumed to be provided by a layer below.


#16

@oskarth I guess provided by below isn’t fully accurate always. Message deniability, aka signatures could be provided by a layer above? Assuming we receive an encoded message that provides a signature.


#17

@decanus I think you are correct, and might be that they belongs on both places, Briar I believe has it at the application layer (so above), as far as messaging is concerned, but might also be in the layer below (such in our case, with whisper providing integrity).