MurAll: Under the hood

10 min readJan 13, 2021

To find out more about MurAll see our first article: What is MurAll?

In this article we’re going to deep dive into the inner workings of the MurAll smart contracts so fair warning: this is going to be quite technical! The main things that artists will need to consider when painting on MurAll are:

images on MurAll have a maximum of 256 colours which the artist is free to choose
images on MurAll use the RGB565 colour space
the canvas size of MurAll is 2048 pixels wide by 1024 pixels high, though images can take up as little or as much of the canvas as the artist likes

We’ve created a proprietary image encoder/decoder tailored to getting the most image data into an Ethereum transaction while saving gas (and therefore money) for the artist, allowing for class leading visuals for an on-chain art experience. And when you’re pushing the boundaries, there aren’t a lot of examples to follow; in our case this has never been done, so we didn’t have any examples to follow! Read on to see how we achieved it…

Choices for storage

There are 2 main choices of data storage when it comes to blockchain technology:

On-chain — all data held on the blockchain itself
Off-chain — using a separate storage solution with references to the files kept on chain

In terms of which data storage solution is the most secure and most distributed, the Ethereum blockchain cannot be beaten, but comes at a very high cost. Some other blockchain file storage options exist (such as Arweave) however they are still fairly small, and fairly new thus do not have the same decentralisation, security and integrity as Ethereum. The Interplanetary File System (IPFS) is fairly well established and thus makes a fair solution for storing large pieces of data such as images, however IPFS isn’t without it’s flaws: it is unreliable, it doesn’t have anywhere near the same kind of uptime as the Ethereum blockchain itself, and is also no way near as decentralised and spread as the Ethereum blockchain. Of course it is a lot cheaper to store data on IPFS but we believe “you get what you pay for”, i.e. storing data on the Ethereum blockchain puts it on one of the most secure, most distributed and most reliable storage media available. This is why we chose to store both the image data and metadata on Ethereum.

Where Ethereum stores data

Ethereum has 2 main places it stores its data: log storage and contract storage. Contract storage is the storage of the Ethereum smart contracts, holding the state of the blockchain; the most expensive storage, costing 20,000 gas for every 32 bytes, and is the most useful in that all economic/transactional activity happens within this storage. Log storage costs 375 gas for each topic plus 8 gas per byte of the LOG operations data, so to store 32 bytes would be 375 gas + (8 gas * 32 bytes) = 631 gas, which is a lot cheaper, but storing on log storage comes with the caveat that the log operation is immutable and not accessible from within contract storage.

We thought hard about the image data and came to the question: “does the image data itself need to be read by contract storage?” Surely any place able to display the image would have access to the log storage too; thus we chose the log storage for storing the image data (even with the caveat that the image data cannot be accessed from contract storage), for the following reasons:

the data is immutable — perfect for our use case where the image data is not to be altered
log storage is part of every node on the network — it is as decentralised and as secure as any data on the contract storage, thus will be as accessible as the contract storage is
it consumes a lot less gas —not only does this reduce the cost of the transactions, it also means we can fit larger amounts of image data than what is possible in contract storage due to the gas limits of Ethereum transactions

How Ethereum deals with data

The Ethereum Virtual Machine (EVM) deals with 32-byte objects; objects smaller than 32-bytes are converted initially from 32-bytes objects when dealt with by the EVM, and that conversion costs gas, e.g. the number 1234 only takes 2 bytes and could be represented as a uint16:

0x3039

But it starts as a uint256 and would be converted from a 32-byte object in the EVM to reduce it to uint16:

0x0000000000000000000000000000000000000000000000000000000000003039

Hence 32-byte objects are the cheapest, most gas efficient objects to use for our transaction data. Obviously there is a lot of wasted space only using a few bytes in the entire 32-byte object, however it is possible to pack several individual pieces of data smaller than 32 bytes into a 32-byte object using a technique called “bit packing”, for example the number 1234, which is 2 bytes in length can be repeated 16 times to fit inside a single 32-byte object:

0x3039303930393039303930393039303930393039303930393039303930393039
  ^   ^   ^   ^   ^   ^   ^   ^   ^   ^   ^   ^   ^   ^   ^   ^

This approach allows us to get the most data into a single transaction and in turn save gas.

Processing the image data

Considerations

The main things that needed to be taken into consideration in terms of the image data were:

data validity: maintaining data integrity, ensuring that only valid data can go into the smart contracts, increasing efficiency by reducing or eliminating the work done to validate the data
keeping images rich: making the right decisions to allow as large/colour-rich images as possible to be placed into the on chain storage
maintaining the MurAll core functionality: allowing for the data to follow the main concepts of MurAll, allowing for painting over the canvas to replace what is underneath, and also enforce the proper burning of the PAINT used in the transaction, which means only charging for painted pixel and not charging for any transparent pixels sent in the transaction
keeping things as cheap as possible: getting the most bang for your buck, squeezing in as much data as possible into every transaction, and reduce or avoid work done on the smart contract where possible to save gas

Breaking down the image

Digital images at their core are collections of pixels, which can be broken down into 2 main components: colour and position. Thus, our main approach was to get as much information about colour and position of the pixels into the transaction data.

Considering we were dealing with 32-byte objects in terms of the Ethereum transaction data, it’s worth thinking about the possibilities of what can be packed into each byte too: 1 byte holds up to 256 possible values, 2 bytes holds 65536 possible values (256²), 3 bytes holds 16,777,216 possible values (256³) etc.

After researching existing image compression techniques/standards, one that really took our eye was the GIF standard: GIF uses a palette of 256 predefined colours, and constructs the image using pixels that are just references to the index of each colour instead of holding the full colour of each pixel separately. What really drew our minds to this approach is that only 1 byte will allow for 1 pixel referencing these possible 256 colours, thus 32 pixels can fit in a 32-byte object! The only things that need ensuring are that the 32 pixel group is correctly referenced and the pixels are in the correct order in the group.

Regarding the colour index: colour is typically RGB888 in most digital images, meaning 8 bits per colour channel (i.e. 8 bits for red, 8 bits for green and 8 bits for blue) totalling 24 bits/3 bytes for a single colour. So we looked at ways to reduce that data too, and found the RGB565 standard: a 16 bit/2 byte colour space (i.e. 5 bits for red, 6 bits for green as human eyes see greens better, and 5 bits for blue). As you can see in the following example images the reduction in the colour data is barely noticeable (even with a 16 bit colour palette and only 256 colours per image, you can still get a great image!) which allows us to free up more room for those all important pixels:

Original image in RGB888 colour space with 686,388 unique colours — https://gateway.pinata.cloud/ipfs/QmQ1MN2LYCSjZKbEyAFnb8wUfDRTMRJtedPY4nMe5eVJeh

The same image reduced to RGB565 colour space with only 42,350 unique colours — https://gateway.pinata.cloud/ipfs/QmdJUcmFEQBadD9kwcVKrmnNjwRXs8bjTEmrDXYEB2iepn

The same image reduced further using the RGB565 colour space and only 256 unique colours — https://gateway.pinata.cloud/ipfs/QmW8n8qcPRcoWphfQV1xnoDibUekkVZ53AcBGQQSQLajtZ

As RGB565 colours only take 2 bytes, this means we can pack 16 colours per 32-byte object, and create the colour index of 256 colours as an array of 8 32-byte objects. We also allow for transparency by sacrificing the colour at the first index as representing the alpha channel of the image if the artist chooses.

With these in mind we divided the canvas into groups of 32 pixels (ordered left to right, top to bottom), and settled on a resolution of 2048 pixels by 1024 pixels as it is close to the standard 1920 x 1080 pixel resolution, but more importantly it would give us 65536 groups of 32 pixels, and the number 65536 has particular significance as we mentioned above that 2 bytes holds 65536 possible values; this means that not only do each of our group indexes only take up only 2 bytes, but also no validation of the index is required on the smart contract as the entire range of all possible values in 2 bytes is covered, and no other positional information is needed as long as the pixels are in the correct order inside their groups!

Thus for the transaction data of the image we have an array that represents any 32 pixel groups containing no transparency, twinned with another array representing the positions of each of these groups on the canvas (each group index being 2 bytes meaning 16 indexes can fit into a single 32-byte object), a separate array of any 32 pixel groups containing transparency (which allows the smart contract to only process these groups looking for transparent pixels to ensure the proper amount of PAINT is charged, while saving as much gas as possible). But what if 1 group only contains 1 pixel for example; isn’t that wasteful? with that in mind we created a separate packed array for individual pixels, consisting of the colour (1 bytes) and their group positions (2 bytes) and their position in that group (1 byte) allowing for up to 8 colour/position pairs in 32 bytes, e.g.:

0x4995CDD14995CDD14995CDD14995CDD14995CDD14995CDD14995CDD14995CDD1
  ^ ^   ^|^ ^   ^|^ ^   ^|^ ^   ^|^ ^   ^|^ ^   ^|^ ^   ^|^ ^   ^|
  1 2   3|1 2   3|1 2   3|1 2   3|1 2   3|1 2   3|1 2   3|1 2   3|1. colour index for pixel (1 byte)
2. group position index that pixel belongs to (2 bytes)
3. position of the pixel within the group (1 byte)

In terms of other data validation, given that all the data sent uses the full range of bytes available for each of their purposes there is no way any data sent will be out of range, thus no checking is required (with the exception of the 1 byte in the individual pixel array representing the pixels position in the group of 32; considering 1 byte has 256 possible values but the position range is only between 0–31, thus we round any value higher than 31 to 31), which cuts the work done by the smart contract down to just 1 job: ensuring the correct amount of PAINT is used in the transaction. This results in the most gas savings.

In summary, the full transaction data sent as part of the drawing onto MurAll consists of:

the colour index (up to 256 RGB565 colours, 2 bytes each, packing up to 16 colours into each 32 bytes)
the whole groups without transparency (32 pixels, 1 byte each, referencing the colour from the index 0–255)
the group indexes where these groups are located on MurAll (2 bytes per index, up to 16 indexes packed into 32 bytes)
the groups containing transparency (same as the whole groups: 32 pixels, 1 byte each, referencing the colour from the index 0–255)
the group indexes where these groups with transparency are located on MurAll (also the same: 2 bytes per index, up to 16 indexes packed into 32 bytes)
The image metadata, consisting of name (up to 32 characters, hence 32 bytes), number, series id, and whether the image contains alpha (3 bytes, 3 bytes and 1 bit respectivily, packed into a 32-byte object).

The minted MURALL NFT consists of this image metadata, plus the keccak256 hash of the image data sent in the transaction (i.e. the colour index array plus all other pixel position arrays) ensuring that when fetching the image data from log storage the metadata can validate it.

However we knew that some people may really want to push the image data onto smart contract storage no matter the cost, and we wanted to cater to those artists, so we designed a “write once” mechanism into our NFT to allow it to be “filled” with the image data.

The “write once” NFT

The NFT contains a mechanism to push the image data from the original transaction onto smart contract storage, if the artist is inclined to do so and does not mind paying the high fees for the storage. As previously mentioned, we have a way to validate the image data inside the NFT, so the data is validated to ensure it is the original data, then the data is written piece by piece until the process runs out of gas. The next fill transaction will continue writing the data where it left off from the previous transaction. It may take quite a few transactions depending on the size of the image, and thus it may cost a lot, but if the artist is truly driven to do it no matter the cost, then it is entirely possible.

To conclude, our approach with the data is non-standard, and fairly complex, however it has allowed us to tailor a solution to the Ethereum transaction model to allow artists to fit as much data into every transaction and to save as much gas in the process.

Stay tuned for more information about the release of MurAll to mainnet coming this month!