Manipulating directories with IPFS Files CLI

IPFS includes tools to manipulate directories without fetching the entire content. This is especially useful in the case of huge datasets or enormous files.

Let’s take a look at how these tools can be used to add new proof parameters files (around 100GB) to an existing directory at https://proofs.filecoin.io/ without fetching either of the two.

IPFS Files toolkit

IPFS exposes a set of commands under the files command space to enable manipulation of directories and files. Our task is to take files from the directory referenced by /ipfs/Qmd5MWnbCt4ceyqfDvPc553gsugdWGs6qS5CgJQKddBLZb and place them into a directory linked under /ipns/proofs.filecoin.io.

Let’s first resolve the domain name to the CID:

> ipfs resolve /ipns/proofs.filecoin.io
/ipfs/QmaCj28DnmCxvd5Sinc1YScxypibmZfSQPiZPMJuCfsJmT

Now we will copy both the origin and the new files into MFS:

ipfs files cp /ipfs/Qmd5MWnbCt4ceyqfDvPc553gsugdWGs6qS5CgJQKddBLZb /snap
ipfs files cp /ipfs/QmaCj28DnmCxvd5Sinc1YScxypibmZfSQPiZPMJuCfsJmT /origin

Now we can copy files out of /snap into the /origin.

> ipfs files ls /snap | grep -v -e '\.contribs$' -e '\.info$' \
	| while read n; do ipfs files cp "/snap/$n" "/origin/"; done

We just copied references to files without copying or even fetching files themselves. We are done; the last step is to look at the new CID, pin it somewhere, and update the domain.

> ipfs files stat /origin
QmcP5rEtRNHAS1GDKoY3WJTW8XEiVY8tkhYyEXaH8g4Ke4
Size: 0
CumulativeSize: 344662471447
ChildBlocks: 67
Type: directory