IPFS alternatives (SNT utility)


#1

As I mentioned on core dev call #18, we’re having issues with IPFS storage for SNT utility features. Chu’s had trouble getting Tribute to Talk to work in 90% of cases.

I learned from @barry that he’s also having issues publishing to IPFS, so am creating this thread to track how we fix this.

Problem
The IPFS gateway we use is unreliable.* Any feature for which we’re storing data on IPFS is affected—Stickers, Tribute to Talk and the liquid funding DApp.

*Could be something to do with Infura API blocking certain IPs?

Potential solutions

  1. Host our own IPFS gateway
    Risks are that anyone could upload a large file and bring this down. Accessing data via Cloudflare may lead to slow load times. And also IPFS’ hashing algorithm is not deterministic.
  2. Use Swarm feeds instead
    In their latest release, Swarm has fixed bugs that made it an unviable option in the past. @yenda and @jarradhope will be with the team in Madrid for their hackathon this weekend and can test/ask questions.

Current plan
The TtT team is going to experiment with Swarm further (@yenda) and check in with @jakubgs about the set-up of our own IPFS gateway—this sounds like ongoing work. TtT team will discuss both options again after the Swarm hackathon this weekend.

Notes from our call here; updates to come.


#2

I can definitely look into deploying our own IPFS gateway, but I assume it would have to scale.

Could you give me a rough idea of how much traffic it would have to handle initially? Just a ballpark figure for numbers of requests would be nice.


#3

It would say it should be similar to mailservers in terms of numbers of requests, except they will be much smaller. However we might also or instead want to deploy a swarm gateway with our own swarm cluster.


#4

It should only need to scale if we expect to upload a lot of content frequently to the nodes. In theory content that is frequently called will start to cache around the network on other IPFS nodes and will even reduce demand on our nodes as other nodes that are closer to requesting clients will deliver the content.

There should be a good amount of info on setting up a cluster here: https://cluster.ipfs.io/


#5

I also want to point out that IPFS multihashing function is non-deterministic depending on the DAG generation when chunking. So if we verify content downloaded (and we should [by running the hashing function locally and seeing if it matches with the query, combined with filesize in manifest]) then we’ll get false negatives.

This for me is a another strike against ipfs as you can’t really be sure what your asking for is what your getting, which to me almost defeats the point, will be asking how swarm tackles this (I suspect it probably isn’t an issue because not many implementations, but I’m curious if it is well specified)


#6

There already is one:

Thanks!


#7

link? Post must be at least 20 characters, imagine that.


#8

You could just embed some fitting meme image, it counts as text.


#9

Is it updated to swarm 0.4 already?


#10

Is there a working browser client that can upload to swarm and then disappear from the network without having the content garbage collected?


#11

The content is garbage collected by removing the chunks that have not been requested for the longest.

For feeds it should be safe because they would be checked everytime the user logs in or someone checks the user profile, and even more if we use it for things like public profile in public chats.


#12

We use the ethdevops/swarm:latest image:
https://github.com/status-im/infra-swarm/blob/master/ansible/roles/swarm/defaults/main.yml#L2

Let me verify, for example the container on node-01.gc-us-central1-a.swarm.test was created 13 days ago, and is:

 $ docker run --rm -e PASSWORD=123qwe ethdevops/swarm:latest --version 
...
Running Swarm:
Swarm Version: 0.3.5-stable Git Commit: 8bbe72075 Go Version: go1.11.1 OS: linux
localhost swarm version 0.3.5-8bbe7207

Nope, looks like latest points at 0.3.5: https://hub.docker.com/r/ethdevops/swarm/tags
There is an explicit v0.4.0 tag, but using it would stop automatic updates.
I guess we could use edge if we want to be edgy.


#13

Oh wait, let me retract that. I verified this on node-01.do-ams3.swarm.testand it’s 4.0 there:

[email protected]:~ % d exec -it swarm-node /swarm --version
swarm version 0.4.0-c3b317a4

That one was updated 22 hours ago. It appears the GC host in the swarm has run out of space, despite having --store.size specified, which probably blocked the container updated. I’ll fix that.


#14

Okay, now all hosts are running 0.4:

 $  ansible all -o -a 'docker exec swarm-node /swarm --version' 
node-01.gc-us-central1-a.swarm.test | CHANGED | rc=0 | (stdout) swarm version 0.4.0-c3b317a4
node-01.do-ams3.swarm.test | CHANGED | rc=0 | (stdout) swarm version 0.4.0-c3b317a4
node-01.ac-cn-hongkong-c.swarm.test | CHANGED | rc=0 | (stdout) swarm version 0.4.0-c3b317a4

#15

@rachel Do we have a Github issue for the IPFS problems? I can’t find a simple problem description or debugging trail for what we tried, if it is a specific gateway problem or IPFS general problem or whatever.

Update: No issue exists, conversation continued in #status-core-ui and https://github.com/status-im/status-react/pull/8129. Issue created here https://github.com/status-im/status-react/issues/8251


#16

DAG Builder isn’t specified, multi dag builders for same content! hooray!, but client doesn’t know what dag builder was used so have fun verifying


#17

This seems like such an oversight…


#18

Could someone on go-team push the timeout for the feed chunk exploration to 5 sec and make a docker image that we can use for our swarm cluster? @igor
I discussed with the person that developed that algorithm and the current 1 sec is arbitrary and has an error rate that is too high


#19

@adam or @pedro can you take a look, it looks like a super small thing to do?


#20

Update on the situation regarding Tribute to Talk:

To ensure stability for the MVP, we’ll revert back to using the “dumb” version of this feature and storing tribute amount in the contract (at some cost to user: Tx required for setting + privacy sacrifice).

Eric’s feedback on using Swarm for a public profile proof of concept is here.

We agreed it’s best to experiment with allowing users to store ‘lesser’ elements of their public profiles on Swarm, such as their photo. Pending how that goes, we could then start incorporating SNT utility features such as ENS name and TtT setting in the public profile as well.

@jakubgs this means that we won’t need the IPFS gateway for this particular feature. From my POV you can pause work on that, but as IPFS issues have also affected @barry and @cryptowanderer and will probably impact sticker market as well, I don’t believe it’s a wasted effort. Following up on how we’ll ensure reliability of stickers in #status-core-ui.