Step by step guide to deploy a private IPFS cluster on Ubuntu
In this article, we will be creating a private IPFS network. It means that it won't connect to the public IPFS network and the data you put on the nodes of this cluster will be localised to your private network. So let's start.
IPFS stands for InterPlanetary File System. It is a protocol and network somewhat similar to BitTorrent. If you want to know what IPFS is, go to their official website for a demo - https://ipfs.io/Step 1 - Install IPFS using the command line
sudo apt-get update
sudo apt-get install golang-go -y
wget https://dist.ipfs.io/go-ipfs/v0.4.13/go-ipfs_v0.4.13_linux-amd64.tar.gz
tar xvfz go-ipfs_v0.4.13_linux-amd64.tar.gz
sudo mv go-ipfs/ipfs /usr/local/bin/ipfsankit@ankit-K54C:~$ ipfs version
ipfs version 0.4.13Step 2 - Initialize the first and second private node
The participants of the public network of IPFS usually have just one IPFS node on their machines since that is all they require to participate. However, in our case, we will deploy 2 IPFS nodes on the same machine. You can do it on 2 different machines too which are on the same network and are able to ping each other. By default, IPFS is initialized in a hidden directory in your user home directory - ~/.ipfs. Since we will be deploying 2 nodes on the same machine, we will be doing so in two difference directories - ~/.ipfs1 and ~/.ipfs2. You can do it in the default ~/.ipfs and at any other location or if you are using different machines then you don't need to worry about the directories. Let us start the deployment:ankit@ankit-K54C:~$ IPFS_PATH=~/.ipfs1 ipfs init
initializing IPFS node at /home/ankit/.ipfs1
generating 2048-bit RSA keypair...done
peer identity: QmQVvZEmvjhYgsyEC7NvMn8EWf131EcgTXFFJQYGSz4Y83
to get started, enter:
ipfs cat /ipfs/QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv/readme
ankit@ankit-K54C:~$ IPFS_PATH=~/.ipfs2 ipfs init
initializing IPFS node at /home/ankit/.ipfs2
generating 2048-bit RSA keypair...done
peer identity: QmToxF88PoBFEkbSLzZ3WLePJn6tvGAM8ZDrJuPrhGhj7P
to get started, enter:
ipfs cat /ipfs/QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv/readme
You should be able to see two directories ~/.ipfs1 and ~/.ipfs2
ankit@ankit-K54C:~$ ls .ipfs*
.ipfs1:
blocks config datastore datastore_spec Keystore version
.ipfs2:
blocks config datastore datastore_spec keystore version
If you are getting the same output then you have successfully initialized 2 IPFS nodes. You can initialize any number of nodes you want.Creating a Private network
Till now we have just created your IPFS nodes which are one step behind in joining the public IPFS network. However, since we are creating a private IPFS cluster we need to use the still-in-experimental-stage features of IPFS to create a private network.
Create swarm.key file to enable private network feature of IPFS
We will start by adding a key called the swarm.key that tells the IPFS nodes that they will be a part of a private network which all will share this swarm.key file.
There is an application which generates this swarm file but for that, you need to have Go language installed on your system. Install Go before proceeding with the next step.
Once you have Go installed, run the following command to install the swarm.key generation utility:go get -u github.com/Kubuxu/go-ipfs-swarm-key-gen/ipfs-swarm-key-gen
Now run this utility in one of your node like this:ipfs-swarm-key-gen > ~/.ipfs1/swarm.key
Copy the file generated to the IPFS directory of each node.Bootstraping IPFS node
IPFS requires one or more bootstrap nodes which are used by the IPFS daemon to learn about the other nodes that are present in the network. In the case of a private network, we need to set up our own bootstrap node since we won't be connecting to the public network node and hence won't have access to the bootstrap nodes of IPFS developers. A bootstrap node is basically the same IPFS node just with the added feature of acting as the bootstrap. There is no separate installation required for this but just a config file entry.
First of all, you need to remove the default entries of bootstrap nodes from all the nodes you have created. Do this by using this command:ankit@ankit-K54C:~$ IPFS_PATH=~/.ipfs1 ipfs bootstrap rm --all
Now add the hash address of your bootnode to each of the nodes including the bootnode.ankit@ankit-K54C:~$ IPFS_PATH=~/.ipfs1 ipfs bootstrap add /ip4/127.0.0.1/tcp/4001/ipfs/QmQVvZEmvjhYgsyEC7NvMn8EWf131EcgTXFFJQYGSz4Y83
The IP part - 127.0.0.1 will be changed to your machine IP in case you are using different machines. The last part is the node hash key which is generated when you create your node. You can see it above where it shows "peer identity: QmQVvZEmvjhYgsyEC7NvMn8EWf131EcgTXFFJQYGSz4Y83". Run this for all of your nodes.Assigning port number for gateway and configuring IP for communication
Inside the .ipfs folder, like in my case .ipfs1 folder, there is a "config" file. It contains a lot of details including the network details on which our IPFS nodes will work on.
Open this config file and find "Addresses". It will look like this:"Addresses": {
"API": "/ip4/127.0.0.1/tcp/5001",
"Announce": [],
"Gateway": "/ip4/127.0.0.1/tcp/8080",
"NoAnnounce": [],
"Swarm": [
"/ip4/0.0.0.0/tcp/4001",
"/ip6/::/tcp/4001"
]
},
The IP mentioned in the API key is the one on which IPFS will bind on for communication. If you are using different machines then mention the IP address of your computer. In our case, since we are using localhost/127.0.0.1, we will just select different ports for each of the machines and will leave IP as it is. For the first node there is no need to change anything, but for the second, we will increment each of the port values like below:"Addresses": {
"API": "/ip4/127.0.0.1/tcp/5002",
"Announce": [],
"Gateway": "/ip4/127.0.0.1/tcp/8081",
"NoAnnounce": [],
"Swarm": [
"/ip4/0.0.0.0/tcp/4002",
"/ip6/::/tcp/4002"
]
},Start the node and test!
We are done with all the configurations and now it is time to start both the nodes to see if everything went fine. To do this, open two consoles since we need to keep both the nodes running. You can do this by sending the process background too if you prefer that.
We will use an environment variable to make sure that just in case if there is some mistake in our configuration and the private network is not fully configured, the nodes don't connect to the public IPFS network and the daemons just fail.
The environment variable for the same is "LIBP2P_FORCE_PNET" and to start the IPFS nodes you just need to start the daemon using "ipfs daemon" command. In our case the command and the output look like this:ankit@ankit-K54C:~$ export LIBP2P_FORCE_PNET=1 && IPFS_PATH=~/.ipfs1 ipfs daemon
ankit@ankit-K54C:~$ export LIBP2P_FORCE_PNET=1 && IPFS_PATH=~/.ipfs2 ipfs daemon
Do note the message log stating "Swarm is limited to the private network of peers with the swarm key" which means that our private network is working perfectly. Now let's add the file from ipfs1 and try to access it from ipfs2.ankit@ankit-K54C:~$ mkdir ankit
ankit@ankit-K54C:~$ cd ankit/
ankit@ankit-K54C:~/ankit$ echo hello > file1.txt
ankit@ankit-K54C:~/ankit$ IPFS_PATH=~/.ipfs1 ipfs add file1.txt
added QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN file1.txt
ankit@ankit-K54C:~/ankit$ IPFS_PATH=~/.ipfs2 ipfs cat file1.txt
Error: invalid 'ipfs ref' path
ankit@ankit-K54C:~/ankit$ IPFS_PATH=~/.ipfs2 ipfs cat QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN
hello
Note that we have added the file to ipfs1 which gave us back a hash. Now when you try to access the same file from ipfs2 with the file name, it won't work, but replace that with the hash and you can see the contents of the file. Now we will try to access the file from the browser.
You can access the file by using the gateway address like this:
http://127.0.0.1:8080/ipfs/QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN
OR
http://127.0.0.1:8081/ipfs/QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN
You should be able to see the content of the file on the browser.
You can similarly add a whole directory too.ankit@ankit-K54C:~$ IPFS_PATH=~/.ipfs1 ipfs add ankit/ -r
added QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN ankit/file1.txt
added QmUs71TZY6Vy47vYrgu5baJwfGQNpMQKeHE2eAn3k2r5kv ankit
and when you try to access it you will see the hash of the file inside if any along with the file name:ankit@ankit-K54C:~$ IPFS_PATH=~/.ipfs2 ipfs ls QmUs71TZY6Vy47vYrgu5baJwfGQNpMQKeHE2eAn3k2r5kv
QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN 14 file1.txt
If you face any problems in any of the steps please do feel free to communicate with me through the comments section.
When I do IPFS_PATH=~/.ipfs1 ipfs swarm peers, It doesn't show anything, can you tell me what's wrong?